0
$\begingroup$

$z_i=f+a_i+\epsilon_i$ ,where $f\sim N(\bar{f},\sigma_{f}^2)$ ; $a_i\sim N(\bar{a_{i}},\sigma_{a}^2)$; $\epsilon_i\sim N(0,\sigma_{\epsilon}^2)$. We can see the signals $\{z_i\}$ where $i\subseteq {1,2,……,M}$, and note that $f$ is a common term for all signals. The question is we want to know the Posterior mean of $a_i$ for any $i$.

The outcome in that paper is as follows but I don't understand it. I want a more detailed process.

\begin{array}{l} E\left[a_{i} \mid\left\{z_{i}\right\}_{i=1, \ldots, M}\right] \\ =\bar{a}_{i}+\frac{\sigma_{a}^{2}\left(\sigma_{a}^{2}+\sigma_{\varepsilon}^{2}+(M-1) \sigma_{f}^{2}\right)}{\left(\sigma_{a}^{2}+\sigma_{\varepsilon}^{2}\right)\left(\sigma_{a}^{2}+\sigma_{\varepsilon}^{2}+M \sigma_{f}^{2}\right)}\left(z_{i}-\bar{z}_{i}\right)-\frac{\sigma_{a}^{2} \sigma_{f}^{2}}{\left(\sigma_{a}^{2}+\sigma_{\varepsilon}^{2}\right)\left(\sigma_{a}^{2}+\sigma_{\varepsilon}^{2}+M \sigma_{f}^{2}\right)} \sum_{j \neq i}\left(z_{j}-\bar{z}_{j}\right) \end{array}

$\endgroup$
2

1 Answer 1

1
$\begingroup$

$\newcommand{\bR}{\mathbb{R}} \newcommand{\one}{\mathbf{1}} \newcommand{\diag}{\textrm{diag}} \newcommand{\Id}{\textrm{Id}}$ I guess you assume independence of the random variables $f,a,\varepsilon$, which I am going to use.

General strategy for Gaussian conditioning in linear models:

Let $a\sim N(\bar{a},Q_{a})$ on $\bR^{n}$ and $z = Aa+\eta$, where $A\in\bR^{m \times n}$ and $\eta \sim N(\bar{\eta},Q_{\eta})$ on $\bR^{m}$ is independent of $a$. Then consider the joint distribution of $a$ and $z$, $$ \begin{pmatrix} a\\ z \end{pmatrix} \sim N\left( \begin{pmatrix} \bar{a}\\ A\bar{a}+\bar{\eta} \end{pmatrix} , \begin{pmatrix} Q_{a} & Q_{a}A^{\top} \\ A Q_{a} & Q_{\eta} + A Q_{a} A^{\top} \end{pmatrix} \right) $$ and apply the Gaussian conditioning formula (see e.g. here, scroll down to Conditional distributions) to obtain the conditional mean $$ (\ast)\qquad \bar{a}^{z} = \bar{a} + (Q_{a}A^{\top}) (Q_{\eta} + A Q_{a} A^{\top})^{-1} (z-A\bar{a}-\bar{\eta}). $$

Answer to your question:

Now let us consider your particular case and denote $\one_{m} = (1,\dots,1)^{\top} \in \bR^{m}$ and a diagonal matrix with the values $s_{1},\dots,s_{m}$ on the diagonal by $\diag(s_{1},\dots,s_{m})$ and by $\diag(s)$ if all $s_{j}$ equal $s$.

The first nontrivial step is to identify $\eta$ correctly: $z = A a + \eta$, where $A = \mathrm{Id_{M}}$ and $$ \eta = \underbrace{(\one \ \Id_{M})}_{=: B} \begin{pmatrix} f\\ \varepsilon_{1}\\ \vdots\\ \varepsilon_{M} \end{pmatrix} \sim N\left( \bar{f}\one , B \, \diag(\sigma_{f}^2,\sigma_{\varepsilon}^{2},\dots,\sigma_{\varepsilon}^{2}) \, B^{\top} \right). $$ (Here we used the fact that $y = Bx$, $x\sim N(\bar{x},Q_{x})$ implies $y\sim N(B\bar{x},BQ_{x}B^{\top})$; I can provide details on this if required.) Then, a simple (but tedious) computation shows that \begin{align*} Q_{\eta} + AQ_{a}A^{\top} &= \diag(\sigma_{a}^{2} + \sigma_{\varepsilon}^{2}) + \sigma_{f}^{2} \one \one^{\top}, \\ (Q_{\eta} + AQ_{a}A^{\top})^{-1} &= \frac{\diag(\sigma_{a}^{2} + \sigma_{\varepsilon}^{2} + M \sigma_{f}^{2}) - \sigma_{f}^{2} \one \one^{\top}}{(\sigma_{a}^{2} + \sigma_{\varepsilon}^{2})(\sigma_{a}^{2} + \sigma_{\varepsilon}^{2} + M \sigma_{f}^{2})} \end{align*} Note that finding the inverse might be hard, but showing that its the inverse, once you have it, is easy (but tedious): Simply multiply the two matrices above to obtain $\Id_{M}$.

If you plug this into $(\ast)$ and write it out componentwise you obtain the desired result.

Hope this helps. If you have any questions, don't hesitate to ask in the comments.

Remarks:

  • $(\ast)$ holds similarly if $\bR^{m}$ and $\bR^{n}$ are replaced by arbitrary separable Hilbert space (possibly infinite-dimensional). In this case replace "transpose" $A^{\top}$ by "adjoint" $A^{\ast}$.
  • There is also a formula for the conditional covariance matrix/covariance operator, which, somewhat surprisingly, does not dependent of $z$ (a particular property of Gaussian conditioning): $$ Q_{a}^{z} = Q_{a} - Q_{a}A^{\top} (Q_{\eta} + AQ_{a}A^{\top})^{-1} A Q_{a}. $$ $a|z \sim N(\bar{a}^{z},Q_{a}^{z})$ has again a normal distribution.
$\endgroup$
1
  • $\begingroup$ Thank you very much. It's really helpful, and it's broadly applicable. $\endgroup$ Commented Mar 1, 2023 at 9:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.