I am curious as to how I can calculate the probability distribution of a first passage time for a particular DTMC. The ultimate goal is to find the probability of this distribution time.
The DTMC looks like (the random variable denoting the state is Y(n)):
Let $T := inf\{n : Y(n) = 3\}$. What is a starting point for finding $\mathbb{P}(T = n)$ for $n$ $\epsilon$ $\mathbb{N}$? Is there a way to estimate $\mathbb{P}(T < n)$?
Because $\{Y(n)\}$ is an absorbing Markov chain, $T$ has a discrete-phase type distribtion. The transition matrix for $Y(n)$ can be written as $$ P = \begin{bmatrix} S & \mathbf S^0\\ \mathbf 0 & 1\end{bmatrix}, $$ where $S$ is the substochastic matrix corresponding to transitions between transient states and $\mathbf S^0+S\mathbf 1=\mathbf 1$. The density for $F$ is $f(k) = S^{k-1}\mathbf S^0$ for $k=1,2,\ldots.$ Here $$ S=\begin{bmatrix} \frac58&\frac38\\0&\frac78 \end{bmatrix},\quad \mathbf S^0 = \begin{bmatrix}0\\\frac18\end{bmatrix}. $$ The $(k-1)^{\mathrm{th}}$ power of $S$ is given by $$ S^{k-1} = \frac1{8^{k-1}}\begin{bmatrix}5^{k-1}& \frac32(7^{k-1}-5^{k-1})\\0&7^{k-1} \end{bmatrix}, $$ and so $$ f(k) =\frac1{8^{k-1}}\begin{bmatrix}5^{k-1}& \frac32(7^{k-1}-5^{k-1})\\0&7^{k-1} \end{bmatrix}\begin{bmatrix}0\\8^{-1}\end{bmatrix} = \frac3{8^k}\begin{bmatrix} 7^{k-1}-5^{k-1}\\7^{k-1}\end{bmatrix}. $$ Assuming $\mathbb P(Y(0) = 1)=1$, we have $$ \mathbb P(T = n) = \frac3{2\cdot8^k}(7^{k-1}-5^{k-1}), k=2,3,\ldots. $$
