In the generality that you stated it, this statement is false. What is true is that on an object always acts a force $F = - \nabla \Phi$ that is the gradient of the Potential. For an object at point $\vec{p}$, the negative gradient points towards a local minimum of the potential. This minimum isn't neccesarily the nearest minimum of the potential, and it's not even given that the potential has a minimum.
However, there is a case in which the statement is true: If you have a dissipative system, that means, the kinetic energy of your object can not only be transfered to potential energy as your object moves in a conservative force-field, but it can also dissipate and just dissappear (due to friction with the ground or surrounding objects or whatever).
In all this cases, your object will eventually come to halt at the position with the lowest potential energy: For example, your pendulum is able to move away from the lowest position, but if you let it swing, it will somewhen stop exactly at the position of minimum potential.
In that sense you could say that "the motion is toward the position of minimum potential energy".