Under the assumptions of large-scale homogeneity and isotropy, cosmic evolution can be described by the scale factor $a(t)$ that describes by what ratio the large-scale structure of the universe has expanded or contracted. The scale factor is unitless, and by convention taken to be $1$ in the present epoch, and the rate of change of $a$ can be denoted $\dot{a}$, so the the universe is expanding whenever $\dot{a}>0$ and contracting whenever $\dot{a}<0$. The existence of a past singularity is then the claim that $a\to 0$ for some finite amount of time in the past.
The scale factor is related to the Hubble parameter by $H \equiv \dot{a}/a$. The Hubble parameter is very frequently and erroneously also called the Hubble constant, which is probably the source of your confusion. It is true that if $H$ is taken to be constant, then trivially $a = a_0e^{Ht}$ is the only possible solution, which only asymptotically tends to $0$ in the infinite past. If spatially flat, as our universe is, this is the de Sitter solution describing a universe dominated by a cosmological constant and negligible matter or radiation.
But that's simply not our universe. In the past, matter was dominant (and prior to that, radiation was dominant), and the de Sitter model simply isn't applicable. Simply put, the Hubble constant is not constant, and so should properly be called Hubble parameter.
In general, the evolution of the scale factor or the Hubble parameter is described by the Friedmann equations:
$$\begin{eqnarray*}
H^2\equiv\left(\frac{\dot{a}}{a}\right)^2 &=& \frac{8\pi G}{3}\rho - \frac{kc^2}{a^2}\text{,}\\
\dot{H}+H^2\equiv\frac{\ddot{a}}{a} &=& -\frac{4\pi G}{3}\left(\rho+\frac{3p}{c^2}\right)\text{,}
\end{eqnarray*}$$
where $\ddot{a}$ denotes the second time-derivative of $a$, the cosmological constant (dark energy) is included in the density $\rho$ and pressure $p$ of the universe, and for our spatially flat universe, $k=0$.
You may also wish to look up the deceleration parameter $q\equiv -\ddot{a}a/\dot{a}^2$ (so defined because the universe was once thought to be decelerating). But the main point is that $H$ is in fact not a constant, so it is consistent for the scale factor to become $0$ a finite amount of time in the past. How much time depends on the changing density $\rho$ and pressure $p$, which is studied in detail in the standard ΛCDM model.
