Examples from Principles of Physics (by Walker,Resnick,Halliday) will say it better:
Let us throw a tomato upward. . .as the tomato rises, the work $\mathbf{W_g}$ done on the tomato by the gravitational force is negative as the force transfers energy from the $E_K$ of the tomato. We can now finish the story by saying that this energy is transferred by gravitational force to the gravitational potential energy of the tomato-earth system. The tomato slows down, stops & then begins to fall back down. During the fall, the transfer is reversed: the work $\mathbf{W_g}$ done on tomato by the gravitational force is now positive - that force transfers energy from the gravitational potential energy of the tomato-Earth system to the $E_K$ of the tomato.
If we abruptly shove a block to send it moving rightward, the spring force acts leftward & thus does negative work on the block, transferring energy from the $E_K$ of the block to the elastic potential energy of the spring-block system. . . .
And many,many other examples. Now, why is the potential energy attributed to the system and the kinetic energy to the object itself? What is the logic/cause?