Let's just look at the first situation. What you're asking is, in effect, what happens when I run the situation in reverse because for every orientation that looks like the first scenario, there is one that matches the reverse.
But look at the effect down the axis from both sides. In both situations, the plane of polarization has shifted about 45° counterclockwise. So no matter the direction of light in this situation, the sense of rotation is the same, so there is no canceling of the net rotation due to symmetry; the symmetry reinforces the sense of rotation.
This makes perfect sense if you were to look at screw. Regardless of which end you look at it from, for a regular right-handed screw, you always turn to the right (clockwise) to drive it in (away from you).