(not a solution as yet)
Let $[n]$ denote the expected number of rounds it takes to end the game when we have a value of $n$.
By linearity of expectation, we have
$[0] = 0$
$[1] = 1 + \frac{1}{2} [2] + \frac{1}{2} [ 0] $
$[2] = 1 + \frac{1}{3} [ 3] + \frac{2}{3} [ 1]$
$\vdots $
$[n] = 1 + \frac{1}{n+1} [n+1] + \frac{n}{n+1} [n-1]$
Letting $[1] = \alpha$, we can calculate that
$\begin{array} {l|l}
[1] & \alpha\\
[2] & 2 \alpha - 2 \\
[3] & 4 \alpha - 9 \\
[4] & 10 \alpha - 34 \\
[5] & 34 \alpha - 139 \\
\vdots & \vdots
\end{array}$
Let $[n] = f_n \alpha - g_n $. Then, since $[n] \geq 0$, this tells us that $\alpha \geq \frac{g_n}{f_n}$.
Claim: $f_n, g_n$ grow very fast (to be quantified).
(I think we'd have to figure out the sequences.)
We have $f_{n+1} = (n+1) f_n - n f_{n-1}$ and $g_{n+1} = (n+1) g_n - n g_{n-1}+ (n+1)$.
$f_n$ is A003422, known as the left factorials, and given by $f_n = \sum_{i=1}^n i!$.
Claim: $\frac{g_n}{f_n}$ is an increasing sequence, that is bounded above.
(No idea why, but if we can solve the recurrence, that might follow.)
Claim: $\alpha = \lim_{n\rightarrow \infty} \frac{g_n}{f_n}$.
We already have $ \alpha \geq \lim_{n\rightarrow \infty} \frac{g_n}{f_n}$. If $\alpha > \lim_{n\rightarrow \infty} \frac{g_n}{f_n}$, then we will have $[n+1] - [n] > 2 $ for large $n$, which doesn't make sense.