I'm learning optics and have been told that when light enters a medium (e.g. glass) and slows down the frequency of the light stays constant while it is the wavelength which is reduced. The explanation for this is that there is no energy loss from the beam of light passing through the glass and since $E=h\nu$ we have the same frequency and that we can easily verify this by looking at a sigle frequency light source through glass and noting the colour doesn't change.
I've also been told that the reason that light slows down in glass is because the Electromagnetic field of a beam of light interacts with and accelerates the charged electrons in the glass molecules which then generate their own photons and the combined wave resulting from the beam and these photons have a slower group velocity, hence the light travels slower.
Both of these explanations make sense on their own, however since the second explanation requires the wave to accelerate electrons that means the wave must give up some of its own energy to these electrons. That would then imply the loss of energy in the wave itself and since $E=h\nu$ we'd see the frequency of the wave shift and thus the colour of the wave would look different when we observed it. However that's very much not the case empirically so where is my intuition going wrong?