Inside the entire Earth as a whole, and throughout our Solar System and galaxy, etc., magnesium is at least 14 times (or more) as common as calcium, atom-per-atom.
But, on Earth's crust and oceans, calcium is at least as abundant, atom-per-atom, as magnesium....
Why is this? Calcium IS denser/heavier.... They're both Group-2....
Two factors need to be considered here. Magnesium ions are smaller, thus forming more compact formula units, and they are also often mixed with iron ions which make the formula unit weight heavier than you expect. Magnesium-rich mafic rock, being also iron-bearing and with compact ions, literally packs on the pounds and tends to sink deeper into the crust and the mantle compared with rock (even mafic rock) containing more calcium.
A false bargain
Magnesium may form lighter atoms than calcium, but it also forms smaller atoms--especially when the metal forms cations as in the most common minerals where the metal is bonded to oxygen. So the magnesium compounds have less formula weight but pack that mass into less volume. If we compare densities of corresponding calcium and magnesium compounds we see that these effects almost cancel out. Data for these examples are taken from Wikipedia:
$\text{CaO}$: 3.34 g/cm³
$\text{MgO}$: 3.6 g/cm³
$\text{Ca}_2\text{SiO}_4$: 3.28-3.33 g/cm³ (larnite)
$\text{Mg}_2\text{SiO}_4$: 3.21-3.33 g/cm³ (forsterite)
So there is only a little density difference overall -- in a world where mafic rocks do not contain iron. Ah, but they do.
Going green, but packing heavy
Why is jade usually green? The most common jade mineral is nephrite, which is mainly a slightly hydrated calcium-magnesium silicate -- with a little iron mixed in to get that green color, at least at low levels of iron. The formula for this mineral is a mouthful: $\text{Ca}_2\color{green}{\text{(Mg,Fe)}}_5\text{Si}_8\text{O}_{22}\text{(OH)}_2$. As indicated by the text in that suitable green color, the iron is intermixed with the magnesium, not with the calcium. The same holds true with the simpler, and also green-colored, olivine, $\text{(Mg,Fe)}_2\text{SiO}_{4}$, except that olivine can contain more iron than is usually found in nephrite.
Magnesium and calcium both lie in Group 2 and both form ions with +2 charges. But that alone does not mean they are intermixed in mineral structures. To fit nicely into the precise, tightly packed solid phase ions must not only have consistent charges at a lattice site, they must also have consistent sizes. Calcium ions, with an ionic radius of 114 pm (ionic radius data from Wikipedia), are simply too big to fit where there should be magnesium ions with a radius of 86 pm. Iron with +2 charge in its high-spin state (oxides of transition metals usually have high-spin ions) comes in at 92 pm, not perfect but a much more acceptable charge/size doppelganger for magnesium. While nephrite in green jade has only a little iron, olivine can have a full range of iron/magnesium ratio from forsterite ($\text{Mg}_2\text{SiO}_4$) to the dark fayalite $\text{Fe}_2\text{SiO}_4$, with a wide range of densities to match -- all the way from 3.2 to 4.5 g/cm³. This range overwhelms the small difference we saw earlier between calcium and magnesium compounds without iron noted earlier.
So the combination of a common, small ionic size for iron and magnesium with a relatively heavy atomic weight for iron makes magnesium-rich rocks sufficiently iron-rich and closely-packed to weigh them down. Calcium adds extra bulk from its own ions and remains iron free in its lattice sites of mineral lattices, so calcium-bearing minerals are less dense and tend to float into the crust (and even continental crust).
Earth's Cream of the Crop
This answer may be compared with the one here regarding the Martian crust. There, we find more magnesium relative to calcium. While the calcium vs magnesium/iron differentiation described above occurs there, lack of a specific continental crust means the differentiation is less. Accordingly, the composition of the Martian crust is closer to that of the magnesium- and iron-rich mantle than that of the terrestrial crust.
