When we work in path integral formalism in some field theory (or even in QM), we usually look on the action, find its symmetries and than treat them as a gauge symmetry - we sum over all possible fields up to this symmetry.

Now let's say that we happand to have a system which have some physically symmetry - for simplicity let's look on free particle propagating from on point on the X axis to another so that we have clear symmetry for rotation around the x axis. The action will also be symmetric but this is not gauge symmetry so we still want to make the path integral over different paths

So my question is how do we know what is gauge symmetry and what is physical symmetry of we start from the action (for example how do we know that the full $weyl$ X $diff$ of the polyakov action is indeed gauge)?