This is about qualia, experiences which cannot be known or communicated beyond an individual subjectivity.
"good for me" is a good example of a language game, because there are massive amounts of contextual cues involved here to make sense of it. A whole nested cosmology interprets identity, valuing & evaluating, wellbeing, & so on. A baby with no language has instincts, and they get feedback, about how to regulate themselves directly from sensations and experiences. But the abstractions in the sentence happen at a higher level, where they have been organised intersubjectively towards sharing strategies through engagement with shared-modes-of-life. Good, preferences, depend on your entire conceptual cosmology applied to your personal evaluation of your needs and aims. That can just be the body if you want, or whatever else - it implies what moves you to act, to enact that you have a given preferences, to which preference we add you have labelled it 'good'.
You can know your personal qualia, and wellbeing may be fundamentally built on that knowing, you just can't communicate what it really feels like subjectively (the 'wellbeingness of wellbeing' like 'the redness of red'), only the tags we append to it to talk about what we can communicate, which is founded on what we can observe and share. The Private Language Argument doesn't deny subjectivity, it just says communication requires intersubjectivity.
Yes this is a powerful argument against egoism, because it says the conceptual structures involved like self-vs-other are not at the level of qualia, but are part of the shared-intelligence embodied in language. A pre-linguistic baby simply cannot reflect on the issue. Reflecting on the validity of Egoism already implicitly requires the existence of the language and concepts involved, and so the existence of the intersubjective community which developed these and is necessary to make sense of them.
