The usual defination of force in terms of potential energy is $$\vec F=-\nabla U$$
This definition leads to
$$K_1+P_1=K_2+P_2\rightarrow \Delta K + \Delta P =0$$
Where $K$ and $P$ are kinetic and potential energies.
However the minus sign over here seems just a convention to me rather than something physically meaningful. That is, if we define
$$\vec F=\nabla U$$
Then the energy equation becomes
$$K_1-P_1=K_2-P_2\rightarrow \Delta K = \Delta P $$
Except for this change, nothing else really changes.
Now, some people like to argue that the flow of energy is from higher to lower potential, as an analogy to pressure, but again that is just a convention. One might as well say that energy flows from low to high potential.
Thus is minus sign really just a convention or something else that I am missing?