You are working with a spin-1 gauge field theory, which means you are working with the Proca equation with a fourvector $A^{\mu}$.
Having 4 entries, you start with 4 degrees of freedom, i.e. you have to make up 4 numbers in order to fully determine $A^{\mu}$.
You can reduce the number of degrees of freedom by finding equations that relate them among themselves.
The first you know is the gauge condition. $A^{\mu}$ is not the physical observable quantity, but it is related to the electric and magnetic fields via derivatives. So, $A^{\mu}$ is subject to a larger class of transformations than $E$ and $B$, i.e. you can do more stuff with it while leaving the physics unchanged. This means that you can “fix the gauge”, that is requiring $A^{\mu}$ so satisfy some condition in order to get rid of this redundancy. Common “gauges” are the Coulomb, Weyl or Lorenz gauges. For example $\nabla \cdot \mathbf{A} = 0 $ is the Coulomb gauge.
This reduces the degrees of freedom from 4 to 3.
You can get rid of one more degree of freedom because the field is massless. Some maths can should you that the product $k_{\mu} A^{\mu}$ should be proportional to the mass $m$. Having $m=0$ entails you have yet another equation that relates the entries of $A$ among themselves.
So for a spin-1 massless gauge field theory, you only have 2 degrees of freedom. This makes sense in the context of light in free space, since we know it can be described as L and R polarisation, or H and V.
This doesn’t mean that the polarisation is always perpendicular to the direction of propagation. In a medium, there will be a component of the electric field along $z$. But you can still re-cast the problem as a 2 degree of freedom problem.
Addendum:
To answer specifically to your question.
The position in space and the direction of travel affect the dynamics (the time evolution) of your state. For example if you are in free space rather than in a material, or if you’re travelling along the strong axis of a polarisation maintaining fibre rather than at an angle.
But the **number ** of non-redundant equations, that is the number of degrees of freedoms, I.e. the minimum number of parameters to fully describe the system - is still 2.
In the special case of free space, you can interpret these 2 as two orthogonal polarisations. But in a waveguide, for instance, there is also $E_{\perp}$ so this intuitive correspondence does not apply.