This is not a full answer but too long for a comment. I just solve the cases $k = 1$ and $k = 2$ and show how induction could possibly be applied for a full solution.
We first want to compute $S$ exactly as an expression of $X$ and $Y$ digits. To do that, we define $S_k(X, Y)$ as the number of decimal numbers $0.z_1z_2 \ldots z_k$ that satisfy $0.z_1z_2 \ldots z_k < X$ and $0.z_kz_{k-1} \ldots z_1 < Y$.
We then compute $S_1$:
$$ S_1(0.x_1, 0.y_1) = \min\{x_1, y_1\} $$
assuming e.g. $\min\{x_1, y_1\} = x_1$ we have:
$$ S_1(0.x_1, 0.y_1) - 10XY = x_1 - \frac{x_1y_1}{10} = x_1\left(1 - \frac{y_1}{10} \right) \le 9\left(1 - \frac{y_1}{10} \right) \le 9$$
Then, let's compute $S_2$. We can note that all $0.z_1z_2$ with $z_1 < x_1$ and $z_2 < y_1$ satisfy the requirements: the number of these is $x_1y_1$ ($z_1 = 0,1, \ldots ,x_1-1$ and $z_2 = 0,1, \ldots ,y_1-1$). We have left out the cases $z_1 = x_1$ and $z_2 = y_1$. When $z_1 = x_1$ the problem reduces to $0.x_1z_2 \lt 0.x_1x_2$ and $0.z_2x_1 \lt 0.y_1y_2$. The first one is equivalent to $z_2 \lt x_2$ while the second one is equivalent to $z_2 \lt y_1$ when $x_1 \ge y_2$ and $z_2 \lt y_1+1$ when $x_1 < y_2$. When $z_2 = y_1$ the problem reduces to $0.z_1y_1 \lt 0.x_1x_2$ and $0.y_1z_1 \lt 0.y_1y_2$. The second one is equivalent to $z_1 \lt y_2$ while the first one is equivalent to $z_1 \lt x_1$ when $y_1 \ge x_2$ and $z_1 \lt x_1+1$ when $y_1 < x_2$. We can count these solutions with the function:
$$ f(u_1,u_2,v_1,v_2) =
\begin{cases}
\min\{u_2,v_1\}, & \text{if $u_1 \ge v_2$} \\
\min\{u_2,v_1+1\}, & \text{if $u_1 \lt v_2$}
\end{cases} $$
So the solutions when $z_1 = x_1$ are $f(x_1,x_2,y_1,y_2)$ and when $z_2=y_1$ they are $f(y_1,y_2,x_1,x_2)$. For the inclusion-exclusion principle, we must then subtract the possible solution with both $z_1 = x_1$ and $z_2 = y_1$:
$$ g(x_1,x_2,y_1,y_2) =
\begin{cases}
1, & \text{if $0.x_1y_1 \lt 0.x_1x_2$ and $0.y_1x_1 \lt 0.y_1y_2$} \\
0, & \text{otherwise}
\end{cases} $$
Combining the above we have:
$$S_2(0.x_1x_2, 0.y_1y_2) = x_1y_1 + f(x_1,x_2,y_1,y_2) + f(y_1,y_2,x_1,x_2) - g(x_1,x_2,y_1,y_2)$$
and
$$S_2(0.x_1x_2, 0.y_1y_2) - 100XY = x_1y_1 + f(x_1,x_2,y_1,y_2) + f(y_1,y_2,x_1,x_2) - g(x_1,x_2,y_1,y_2) - \frac{(10x_1 + x_2)(10y_1 + y_2)}{100} = x_1y_1 + f(x_1,x_2,y_1,y_2) + f(y_1,y_2,x_1,x_2) - g(x_1,x_2,y_1,y_2) - x_1y_1 - \frac{x_1y_2}{10} - \frac{x_2y_1}{10}
-\frac{x_2y_2}{100} \le f(x_1,x_2,y_1,y_2) + f(y_1,y_2,x_1,x_2) \le 9 + 9 = 18$$
We can continue by induction, with a step like this:
$$ S_k(0.x_1x_2 \ldots x_k, 0.y_1y_2 \ldots y_k) \approx 10^{k-2}x_1y_1 + S_{k-1}(0.x_2 \ldots x_k, 0.y_1 \ldots y_{k-1}) + S_{k-1}(0.x_1 \ldots x_{k-1}, 0.y_2 \ldots y_k) - S_{k-2}(0.x_2 \ldots x_{k-1}, 0.y_2 \ldots y_{k-1})$$
where the first addendum is $10^{k-2}x_1y_1$ because we have $z_2 \ldots z_{k-1}$ digits that can range from $0$ to $9$ when $z_1 < x_1$ and $z_k < y_1$.
It think it can be seen with some algebra that $S_k$ is near to $10^{k}XY$. The problem is corner cases that we must handle replacing $S_{k-1}$ and $S_{k-2}$ in the induction step with a function like the above $f$ that we have used instead of $S_1$. We cannot use an approximation e.g. like $S_{k-2} \le \ldots$ because $S_{k-2}$ is subtracted. Also, we must verify that $S - 10^kXY \ge 0$ always or handle cases when $S - 10^kXY \lt 0$.
