Trouble reconciling these two views on gauge theory

Question

Very generally speaking, I view gauge theory as asking what local symmetries leave our theory invariant and then seeing the consequences. Thus, taking a look at the Lagrangian for electromagnetism, we can act on each point by a $U(1)$ action, i.e. multiply by $e^{i\theta(x)}$, and this would not change the equations of motion. The generalizations of this idea to other symmetry groups and other field theories also make sense to me.

What is confusing me is reconciling this local symmetry idea with how gauge theory is presented in classical electromagnetism books. For example, in E&M you may modify a vector potential $A$ and scalar potential $V$ by the following gauge transformation: $$ A \rightarrow A + \nabla \lambda\\ V \rightarrow V - \frac{\partial \lambda}{\partial t} \tag{1}$$ where $\lambda$ is any scalar function and this has no effect on the physical theory. We can then use the above transformations to conveniently fix a gauge such as the Coulomb gauge or the Lorenz gauge.

I am aware that this can be made rigorous by seeing how the local sections of a principal bundle over spacetime change when acted on by the transition functions of the bundle. But on a more intuitive or physical level, what is the relationship between the above gauge transformations and the idea of a symmetry at every point in space-time?

For example, I don't see what the $U(1)$ local symmetry of the Maxwell theory has anything to do with the gauge transformations (1) or specific gauges such as the Coulomb or Lorenz gauge (or more generally, what it has to do with gauge fixing at all). What does the function $\lambda$ have to do with the value we choose for $\theta(x)$ in the $U(1)$ action?

The two seem like different ideas that I'm sure are related but I cannot see the connection. How can one tie these ideas together?

Answer 1

Gauge theory resembles general relativity applied to extra dimensions beyond the big four. Whether that's the true nature of gauge fields or just a convenient mental picture isn't clear, but it's at least possible that what I'm about to say is close to the literal truth.

For $U(1)$ gauge theory, you can imagine that at every point of 4D spacetime, there is a circle. We can't see this curled-up fifth dimension because all known fields (particles) are fixed, low-degree harmonics of it, i.e. they're spread around it and can't be localized.

The fields are really functions of a 5D position, but because they are all fixed harmonics of the fifth dimension, you can represent them more efficiently as $f(\mathbf x,θ) = f(\mathbf x)e^{iqθ}$ where $\mathbf x$ is the 4D coordinate, $θ$ is the fifth coordinate, and $q$ is a small integer that is constant for each field. The choice of zero point for $θ$ at each $\mathbf x$ is a kind of coordinate system, and the physics of the system obviously shouldn't depend on it.

There is no symmetry mixing the big four dimensions with the fifth dimension, and the circles have a specific connectivity such that once you've picked a zero point on a circle, you can pick the zero point on a nearby circle to be the same, so you might think that you could fix the fifth coordinate choice up to an uninteresting global phase this way. That doesn't work because the connectivity around closed loops in 4D (toruses in 5D) can be nontrivial, i.e. the torus may have a net twist. You can extend an untwisted coordinate system almost all of the way around any torus, but it generally won't meet up at the ends.

The total twists around loops/toruses are the only observable aspect of the connectivity of the circles. These are called Wilson loops. The four-vector potential $A^μ$ encodes the amount of twist in your arbitrary zero points between infinitesimally separated spacetime points, and therefore the net twist around any Wilson loop is the integral of the vector potential around it. The field tensor $F^{μν}$ encodes the net twist around infinitesimal Wilson loops, i.e., it's the curl of the vector potential.

Under a change of the fifth coordinate, it follows from the definitions above that you need to add the gradient of $Δθ(\mathbf x)$ to $A^μ$, and you need to multiply $f(\mathbf x)$ by $e^{iqΔθ(\mathbf x)}$, where $Δθ(\mathbf x)$ is the angle between the old and new zero points (old minus new). This is the same as the functions you called $θ(\mathbf x)$ and $λ(\mathbf x)$.

Answer 2

A local $U(1)$ transformation alone does not leave the original Lagrangian invariant.

Since the laws of physics must apply locally, we want the Lagrangian to be invariant under a local $U(1)$ transformation $$\mathcal L=\mathcal L',\tag1$$

It's straightforward to show that a global transformation of the fields $$\psi' \to \mathrm e^{\mathrm i \theta} \psi$$ (where $\theta$ is a constant) satisfy condition (1) by direct substitution into the original Lagrangian.

But when we consider a local $U(1)$ transformation, where $\theta\to\theta(x)$ is now a function of local spacetime coordinates, that is the fields now transform as $$\psi' \to \mathrm e^{\mathrm i \theta(x)} \psi,$$ we find that the Lagrangian does not resemble its initial form, and that condition (1) is no longer satisfied.

But again, the fact that we require the laws of physics to be valid locally should not alter the physics, and so we need to restore this invariance buy adding terms that will restore our Lagrangian. We find that this is accomplished via the gauge transformation $$A_\mu' \to A_\mu + \frac{\partial \lambda}{\partial x^{\mu}}.$$

By direct substitution of $A_\mu'$ into the original Lagrangian, you will find that you get back the original Lagrangian. That is, the condition (1) is now satisfied$^1$. You should try this as an exercise to convince yourself.

$^1$ In the language of QED we say that gauge invariance is retained if there was an additional field $A$ that couples to the field $\psi$ and is of the form $(A_\mu + \frac{\partial \lambda}{\partial x^{\mu}})$. Mathematicians, like yourself, called this new vector field a connection (physicists will usually call this a covariant derivative).