It is because perfect octahedral symmetry is normally assumed; the two $e_g$$\mathrm{e_g}$ levels and three $t_{2g}$$\mathrm{t_{2g}}$ are degenerate. If there was a distortion, say by lengthening both z$z$-axis ligand positions then the $e_g$$\mathrm{e_g}$ degeneracy would be removed as the $d_{z^2}$$\mathrm d_{z^2}$ becomes more stable than $d_{x^2-y^2}$$\mathrm d_{x^2-y^2}$. This happens because the z$z$-axis ligand has more effect on $d_{z^2}$$\mathrm d_{z^2}$ than on $d_{x^2-y^2}$$\mathrm d_{x^2-y^2}$ orbitals simply due to its position. The $d_{xy}$$\mathrm d_{xy}$ orbital also increases in energy removing the degeneracy of the $t_{2g}$$\mathrm{t_{2g}}$ and the $d_{yz},d_{zx}$$\mathrm d_{yz},$ $\mathrm d_{zx}$ lowered.
