First of all, your gedankenexperiment is actually fairly realistic (if we consider spheres instead of shells). Imagine for example a star in the center of which occurs nuclear fusion. This is literally the conversion of part of the (rest) mass of the "fuel", e.g. hydrogen, into energy in the form of radiation. As this happens at the center of the star, the radiation can't escape immediately and exerts pressure. Usually, a star is in equilibrium, meaning that the radiation pressure together with the gas pressure stabilizes the star against gravitational collapse. However, in some situations (through very complicated processes), the internal pressure gets higher than the gravitational attraction such that the star expands, for example at the end of its life when it turns into a red giant. Some stars even change their radius periodically.
The solution to your problem lies in the fact that we have to be very careful which mass we are actually talking about. For one thing, there is the so called baryonic mass, which is just the sum of all the masses of the constituents of the star. Imagine taking the star apart into its atoms, measuring their individual masses and adding them. But this is not equal to the gravitational mass of the star, which is used in Newton's gravitational law to calculate the gravitational field that an outside observer measures. The discrepancy between those two values stems in fact again from mass-energy equivalence. The gravitational mass of the star corresponds to its total energy, which includes the rest mass of its constituents, but also its binding energy (and other forms of energy like the heat trapped inside and all other kinds of interaction energy between the constituents, but I will neglect this here). To put this into an equation:
\begin{equation}
M = M_B + E_B
\end{equation}
where $M$ is the gravitational mass, $M_B$ is the baryonic mass and $E_B$ is the binding energy (note that I choose the signs such that the binding energy is negative). In our example, the star expanded, which just means that it raised its binding energy. In other words, it converted part of its baryonic mass into binding energy. However, as we neglected any incoming or outgoing energy flux, the gravitational mass of the star (which, again, corresponds to its total energy content) stays constant! So an outside observer won't feel a difference, he will still measure the same gravitational mass.
Finally, let me note that we're talking about small effects here, at least considering "normal" stars. They stem from general relativity, which makes sense, as we used mass-energy equivalence, which is a relativistic effect. The situation we discussed is described by the Tolman-Oppenheimer-Volkoff equation. You can find a more mathematical discussion of this topic, including the different "kinds" of masses in the referenced Wikipedia article.