When calculating the centre of mass for a small body compared to the earth, the equation $$\vec r_c = \frac {\sum_i m_i \vec r_i} {\sum_i m_i}$$ is used, where $\vec r_c$ is the position vector for the centre of mass from a reference point $O$, $\vec r_i$ is the position vector for each particle in the body, $m_i$ is the mass for each particle and $\sum_i$ is the sum that extends over all particles comprising the body.
My question is, when you calculate the centre of mass of a body, what do you choose as the reference point $O$? $\vec r_c$ should be the point where as if the whole weight of the body acts there and the point that gives the correct torque of the body when the weight of the body is applied there.
Since there is a consideration of torque involved, should the reference point $O$ be a pivot point? But how do you define a pivot point for a body, for example a stationary book on a table or a floating balloon?