All Questions
21
questions
3
votes
0
answers
48
views
Random Walk Problem in Feller Probability Theory vol 2 Chapter XII page 425
Let $X_{i}$ be iid discrete random variable with distribution $P(X_{1}=-1)=q$ and $P(X_{1}=i)=f_{i}$ for all $i\geq 0$.
Then consider the random walk $S_{n}=\sum_{i=1}^{n}X_{i}$ and $S_{0}=0$. Let $\...
- 1,285
0
votes
0
answers
65
views
How to prove null recurrence when there are infinitely many states?
I have a random walk on a chessboard, where each square is a state/vertex.
I understand that the location of pieces on the board describes a Markov chain, and one way to look at this is through a ...
- 235
0
votes
1
answer
341
views
How to prove that a Markov chain is transient?
I have a Markov chain $\{Y_n: n\geqslant 0\}$ where the $Y_n$ are integer-valued.
The probability of going from any state $i$ to its right (i.e., from state $i$ to state $i+1$) is $p$,
and the ...
- 235
0
votes
0
answers
283
views
Relationship between Markov chains and i.i.d. random variables
I am studying Markov chains. I understand that a sequence of i.i.d. random variables is a special type of Markov chain. However, I am trying to prove that a finite-valued Markov chain is a sequence ...
- 235
0
votes
0
answers
89
views
Prove that the first return time is finite a.s. for a Markov chain
Suppose $X$ is an irreducible Markov Chain on a discrete state space $E$. I would like to prove that
$$
P_x[\tau_x^1 < \infty]=1
$$
where $\tau_x^1=\inf\{n>0: X_n=x\}$.
Is it necessary to know ...
- 563
0
votes
1
answer
51
views
Random walk and inequality involving the Green function on N step
I would like to check if my arguments are the correct ones in proving
$$
G_N(x,y):=\sum_{k=0}^N P_x[X_k=y]\leq G_N(y,y)
$$
where $X$ is a random walk on $\mathcal{Z}$ with $E_0[|X_1|]<\infty$ and $...
- 563
0
votes
0
answers
53
views
Markov Chains Concatenation Prohibited?
Assuming $X_1,...,X_4$ are discrete random variables, consider the following 2 Markov Chains:
$C_1:X_1 \rightarrow X_2 \rightarrow X_3$
$C_2:X_2 \rightarrow X_3 \rightarrow X_4$
I was trying to ...
- 108
0
votes
1
answer
71
views
How do I prove that $\Bbb{P}_x(\tau_1<\infty)=1$, where $\tau=\inf\{n\geq 1:X_n\neq X_0\}$?
Let $(X_n)$ be a Markov chain on a countable space $E$ and let $T$ be it's transition matrix. We assume $T(x,x)<1$ for all $x\in E$. Let $\tau=\inf\{n\geq 1:X_n\neq X_0\}$. I want to show that $\...
- 2,018
1
vote
0
answers
225
views
Markov chain on a subset of state space
Problem : Let $(X_n)_{n\ge 0}$ be an irreducible Markov chain on $I$ having an invariant distribution $\pi$ . For $J \subseteq I$ let $(Y_m)_{m\ge 0}$ be a Markov chain such that $Y_m = X_{T_m}$ ...
- 910
1
vote
0
answers
61
views
Finding recurrent and transient states with first hitting probabilities (Norris exercise 1.5.1)
I want to see why the following attempt won't work .
Problem : In a discrete time markov chain (DTMC) $(X_n)_{n\ge 0}$ with transition matrix P and state space I , which states are recurrent and ...
- 910
1
vote
1
answer
225
views
State classification of Markov chains
Consider the Markov chain on $S=\{1,2,3,4,5,6\}$ with transition matrix
$$P=\begin{pmatrix}0&1/2&1/4&1/4&0&0\\1&0&0&0&0&0\\0&1/3&0&1/3&0&1/3\...
user1016931
2
votes
1
answer
267
views
Finding eigenvalues of transition matrix of a given Markov process (solution verification)
Consider sampling the uniform measure $\mathbb{Z}_L$ by the Markov chain $X^{(k)}$ with
$$P_i[X^{(i)}=(i+1)\mod L]=1$$
with initial condition $X^{(0)}=L-1$. Here $P_i$ means conditioning on $X_0=i$.
$$...
- 1,050
1
vote
0
answers
18
views
the probability that the system remains at transient states forever and the probability that the system goes into absorbing states
I am trying to justify the last statement, but I have failed to do it.
If $\sum \epsilon_k < \infty$, then $\epsilon_k \to 0$. Let $a_j = 1-\epsilon_j$. Then, this implies that for $\varepsilon >...
- 3,659
0
votes
1
answer
174
views
Is counting the dice throws needed to get $n$ times 6 a Markov chain?
I am working on the followign exercise and I would be glad if you could have a look at my solution attempt:
For $n \ge 1$, let $Z_n$ be the RV representing the number of rolls of a fair die until 6 ...
- 4,222
1
vote
0
answers
137
views
Markov chain, periodicity implies no equilibrium
Given an irreducible Markov chain with period $k$, I would like to show that $p^{(n)}(x,x)$ does not converge to a limit for a recurrent state $x$.
For the proof, I will find two subsequences that ...
- 9,586