HMM - Difference between forward and backward case

Question

In the HMM formulation where z is hidden state and x is observed

In the forward case, I see it represented as such:

$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1},x_{1:k})$

but in the backward case I see it as such

$\beta_{k}(z_{k})=P(x_{k+1:n}|z_{k})=\sum_{z_{k+1}}P(x_{k+1:n},z_{k+1}|z_{k})$

---

I thought that in the probability rules, we can factor in a new term based on this expression:

$P(A,B)=\sum_{C}P(A,C|B)$

If so, how are both of those expressions even expanded? Shoudn't the forward rule then be:

$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1}|x_{1:k})$

and the backward rule be not possible to factorize based on the above rule? Is my rule wrong?

Answer 1

The probability rule is not right. I think you are thinking of the sum rule: \begin{align} P( A ) = \sum_B P( A, B) \end{align} or, for three variables \begin{align} P(A, B) = \sum_C P(A, B, C) \end{align} You can think of the sum rule as either introducing another variable (from the left side of the equal sign to the right side) or removing a variable (from the right side of the equal sign to the left side). For this reason, we sometimes talk about the sum rule as "summing out" a variable (or "integrating out" for continuous variables). In the above equation, we're "summing out" $C$.

The forward algorithm relies on the sum rule to sum out the previous state: \begin{align} \alpha_k(z_k) & = P(z_k, x_{1:k}) \\ & = \sum_{z_{k-1}} P( z_k, x_{1:k}, z_{k-1}) \end{align} and the backward algorithm relies on the sum rule to sum out the next state: \begin{align} \beta_k(z_k) & = P(x_{k+1:n} | z_k) \\ & = \sum_{z_{k+1}} P(x_{k+1:n}, z_{k+1} | z_k) \end{align} Notice that both algorithms use the sum rule in exactly the same way; they just choose a different variable to sum out. Notice also that the backward algorithm does not introduce conditioning information; we're conditioning on $z_k$ in both lines.