A counterexample of reducible Markov chains about Birkhoff's ergodic theorem

Question

In R.Durrett's boos Probability Theory and Examples(5th). There is an example 6.2.4(CH6.2 P301),

"Let $X_{n}$ be an $\textbf{irreducible}$ Markov chain on a countable state space that has a stationary distribution $\pi$..., and applying the ergodic theorem, $$ \frac{1}{n} \sum_{m=0}^{n-1} f(X_{m}) \longrightarrow \sum_{x}f(x)\pi(x)."$$

I want to know if there exists a counterexample removing the condition "irreducible", in other words, how about the reducible Markov chain? In this counterexample, the Birkhoff's ergodic theorem does not hold.

Answer 1

Take the example of a Markov chain with three states $0$, $1$ and $2$, with transition probabilities

This Markov chain has precisely two irreducible components corresponding to the two absorbing states $1$ and $2$. Every distribution $\pi$ with $\pi(1)+\pi(2)=1$ is stationary. But depending on the initial state, the Markov chain will eventually be absorbed to one of the two absorbing states. Therefore, the ergodic averages $(1/n)\sum_{m=0}^{n-1}f(X_m)$ will converge almost surely to a random variable with two possible values $f(1)$ and $f(2)$, and in particular will not converge to a constant unless $f(1)=f(2)$.

In general, if a Markov chain is not irreducible, then it may have $0$, $1$ or more irreducible components, and each of these components may or may not support a stationary distribution. The convergence of the ergodic averages will then depend on whether or not the chain ends up in an irreducible component and whether or not that irreducible component supports a stationary distribution ($\equiv$ is positively recurrent). If $C_i$ is an irreducible component with stationary distribution $\pi_i$, then conditioned on the event that the Markov chain ends up in $C_i$, the ergodic averages will converge almost surely to the mean $\sum_x\pi_i(x)f(x)$.

Just a final remark that for the convergence of the ergodic averages, aperiodicity is irrelevant.