Mathematically, it just gives a duality with vector fields and scalar fields in multivariable calculus, associated with conservative vector fields and line integrals. As such, the $\pm$ is irrelevant, because it can be absorbed into the force vector. For physics, we take the sign convention to be negative, so that it agrees with the fact that the force is restoring the object it acts on to a lower energy configuration. Note that we could alternatively absorb the negative sign into the potential! It is all a matter of sign convention, and when you define potential and force in physics (as stated above), the negative sign appears in your equation.
