It doesn't matter whether or not the thing you are connecting is a battery. It doesn't matter whether or not the other thing is the Earth. If you have two conductive objects with no conductive path between them, and if there is a difference of electrical potential between them, then when you create a connection between them (e.g., by attaching a wire) there will be a brief flow of electric current until the potential is equalized.
See also, https://en.wikipedia.org/wiki/Capacitance
A battery is special though because it has two terminals, and it maintains a constant potential between them. Suppose you have a 1.5V battery whose '-' terminal is connected to Earth, and whose '+' terminal is not connected to anything. The '+' terminal must be 1.5V above Earth potential. Now disconnect the wire from the '-' terminal, and connect it to the '+' terminal. A tiny pulse of current must flow to bring the '+' terminal down to Earth potential. At the same time, because of the "constant voltage" property of the battery, that must drive the '-' terminal to be 1.5V below Earth potential.
Now, disconnect the wire once more, and reconnect it to the '-' terminal. Again, a tiny current must flow, in the opposite direction this time, to bring the '-' terminal back up to Earth potential; and that drives the '+' terminal to be 1.5V above Earth potential, bringing the system back to the state where we started.
Repeat as many times as you like, and a tiny pulse of current flows each time.
Something similar happens here, except the Earth is not part of the picture: https://en.wikipedia.org/wiki/Oxford_Electric_Bell