Upon learning about the Lagrangian formulation of GR, where varying an action with respect to a metric (in order to, for instance, arrive at the Einstein field equations) is common, I can't help but wonder how this operation is even well defined. The reason is that you can write down a tensor in a number of different ways by raising and lowering its indices (or contracting them) with the metric itself, so that you can write it with any number of metric tensors multiplied in front of another tensor with the appropriate combination of upstairs/downstairs indices. This means that varying a tensor with respect to a metric can give a variety of different answers depending on how you write down your tensor (in particular on how many metric tensors that have been extracted out in the expression for the tensor).
Let me give a concrete example. Consider a rank (0,4) tensor $C_{\mu\nu\rho\sigma}$. We assume it satisfies the relation $$C_{\mu\nu\rho\sigma}=C_{\nu\mu\sigma\rho}\tag{1}$$ We can contract the first and third indices: $$C_{\nu\sigma}=g^{\mu\rho}C_{\mu\nu\rho\sigma}\tag{2}.$$ We can also form a scalar by contracting once more: $$C=g^{\nu\sigma}C_{\nu\sigma}\tag{3}.$$ Hence we also have $$C=g^{\nu\sigma}g^{\mu\rho}C_{\mu\nu\rho\sigma}\tag{4}$$
Now say we wish to calculate the expression $\frac{{\delta}C}{{\delta}g^{\alpha\beta}}$. If we substitute eq. 3 for $C$, and vary with respect to $g^{\alpha\beta}$ while keeping $C_{\nu\sigma}$ fixed, we would get $$\frac{{\delta}C}{{\delta}g^{\alpha\beta}}=C_{(\alpha\beta)}\tag{5},$$ where parentheses denote that the tensor is symmetrized. However, consider what happens if we substitute eq. 4 for $C$ and vary it while keeping $C_{\mu\nu\rho\sigma}$ fixed. Then using the product rule (since we now have two factors of the metric with $g^{\nu\sigma}$ and $g^{\mu\rho}$), exploiting eq. 1 and contracting using eq. 2, we now get $$\frac{{\delta}C}{{\delta}g^{\alpha\beta}}=2C_{(\alpha\beta)}\tag{6}.$$ Clearly, a contradiction between eq. 5 and 6. What's going on? How are you supposed to properly vary a tensor with respect to the metric? The key would seem to be to figure out what stays fixed during the variation. But the tensor I chose is a very general tensor that may not have any explicit dependence on the metric at all (you may have noticed that the Riemann tensor could easily be a candidate for $C_{\mu\nu\rho\sigma}$, but since the Riemann tensor in GR typically has an explicit dependence on the metric due to our choice of affine connection, I'll ignore that easy option). If such an explicit dependence existed, then we could simply use it to vary everything directly like we do when computing the variation of the Riemann tensor. But in general, how do you determine what should be fixed during the variation of a tensor, even when the tensor has no explicit dependence on the metric?