It is indeed correct that only the difference between two potential energies is physically meaningful. An in-depth explanation follows. For the rest of this answer, forget everything you know about potential energy.
I suppose you know that when you have a conservative force $\vec{F}$ acting on an object to move it from an initial point $\vec{x}_i$ to a final point $\vec{x}_f$, the integral $\int_{\vec{x}_i}^{\vec{x}_f}\vec{F}\cdot\mathrm{d}\vec{s}$ depends only on the endpoints $\vec{x}_i$ and $\vec{x}_f$, not on the path. So imagine doing this procedure:
- Pick some particular starting point $\vec{x}_0$
Define a function $U(\vec{x})$ for any point $\vec{x}$ by the equation
$$U(\vec{x}) \equiv -\int_{\vec{x}_0}^{\vec{x}} \vec{F}\cdot\mathrm{d}\vec{s}$$
This function $U(\vec{x})$ is the definition of the potential energy - relative to $\vec{x}_0$. It's very important to remember that the potential energy function $U$ depends on that starting point $\vec{x}_0$.
Note that the potential energy function necessarily satisfies $U(\vec{x}_0) = 0$. So you can write
$$U(\vec{x}) - U(\vec{x}_0) = -\int_{\vec{x}_0}^{\vec{x}} \vec{F}\cdot\mathrm{d}\vec{s}$$
Now why would you do that? Well, suppose you choose a different starting point, say $\vec{x}'_0$, and define a different potential energy function
$$U'(\vec{x}) \equiv -\int_{\vec{x}'_0}^{\vec{x}} \vec{F}\cdot\mathrm{d}\vec{s}$$
(Here I'm using the prime to indicate the different choice of reference point.) Just like the original potential energy function, this one is equal to zero at the starting point, $U'(\vec{x}'_0) = 0$. So you can also write this one as a difference,
$$U'(\vec{x}) - U'(\vec{x}'_0) = -\int_{\vec{x}'_0}^{\vec{x}} \vec{F}\cdot\mathrm{d}\vec{s}$$
The neat thing about this definition is that even though the potential energy itself depends on the starting point,
$$U(\vec{x}) \neq U'(\vec{x})$$
the difference does not:
$$U(\vec{x}_1) - U(\vec{x}_2) = U'(\vec{x}_1) - U'(\vec{x}_2)$$
Check this yourself by plugging in the integrals. You'll notice that anything depending on the starting point cancels out; it's completely irrelevant.
This is good because the choice of the starting point is not physically meaningful. There's no particular reason to choose one point over another as the starting point, just as if you're on a hilly landscape, there's no particular reason to choose any one level to be zero height. And that's why potential energy itself is not physically meaningful; only the difference is.
Now, there is a convention in (very) common use in physics which says that when possible, unless specified otherwise, the starting point is at infinity. This allows you to get away without saying "difference of potential energy" and explicitly defining a starting point every time. So when you see some formula for potential energy, like
$$U(\vec{r}) = -\frac{k q_1 q_2}{r}$$
unless specified otherwise it is actually a difference in potential energy relative to infinity. That is, you should read it like this:
$$U(\vec{r}) - U(\infty) = -\frac{k q_1 q_2}{r}$$
Note that the function $\frac{k q_1 q_2}{r}$ goes to zero as $r\to\infty$. That's not a coincidence. It was chosen that way to ensure that $U(\infty) = 0$, so that you could insert it the same way I inserted $U(\vec{x}_0)$ in the calculations above. (This is just another way of saying it was chosen to make the $\frac{1}{a}$ term in the integral you did go away, so you don't have to write it.)
There are some situations in which you can't choose the reference point to be at infinity. For example, a point charge with an infinite charged wire has an electrical potential energy of
$$U = -2kq\lambda\ln\frac{r}{r_0}$$
where $r$ is the distance between the point charge and the wire. This potential energy function decreases without bound as you go to infinite distance ($r\to\infty$), it doesn't converge to zero, so you can't use infinity as your starting point. Instead you have to pick some point at a finite distance from the wire to be your starting point. The distance of that point from the wire goes into that formula in place of $r_0$.
By the way, electrical potential (not potential energy) is something a little different: it's just the potential energy per unit charge of the test particle. For a given test particle, it's proportional to electrical potential energy. So everything I've said about applies equally well to electrical potential.
