Using transformations to solve question surrounding correlation of vectors

Question

While studying statistical inference and data-analysis, I came across this question. I have been able to show part (a) of this question already myself, but I don't know how to solve part (b). Can anyone help me with that?

(a) Show that $$ \text{Cor}(\mathbf{a}^\top \mathbf{X}_1, \mathbf{b}^\top \mathbf{X}_2) = \frac{\mathbf{a}^\top \boldsymbol{\Sigma}_{12} \mathbf{b}}{\sqrt{\mathbf{a}^\top \boldsymbol{\Sigma}_{11} \mathbf{a}} \sqrt{\mathbf{b}^\top \boldsymbol{\Sigma}_{22} \mathbf{b}}} $$

(b) Use appropriate transformations to show that $$\max_{\mathbf{a}\neq\mathbf{0}, \mathbf{b}\neq\mathbf{0}} \text{Cor}(\mathbf{a}^\top \mathbf{X}_1, \mathbf{b}^\top \mathbf{X}_2) = \max_{\mathbf{u}\neq\mathbf{0}, \mathbf{v}\neq\mathbf{0}} \frac{(\mathbf{u}^\top \mathbf{M} \mathbf{v})^2}{(\mathbf{u}^\top \mathbf{u})(\mathbf{v}^\top \mathbf{v})} $$

Thanks in advance.

Answer 1

We have that $$ \operatorname{Var}(X) = E[XX^\top] = E \left[ \begin{pmatrix} X_1 \\ X_2\end{pmatrix} \begin{pmatrix} X_1^\top & X_2^\top \end{pmatrix} \right] = E \left[\begin{pmatrix} X_1X_1^\top & X_1X_2^\top \\ X_2X_1^\top & X_2X_2^\top \end{pmatrix} \right] $$ so $$ \operatorname{Cov}(a^\top X_1, b^\top X_2) = E[a^\top X_1 (b^\top X_2)^\top] = E[a^\top X_1X_2^\top b] = a^\top\Sigma_{12}b $$ as well as $$ \operatorname{Var}(a^\top X_1) = E[a^\top X_1X_1^\top a] = a^\top \Sigma_{11} a $$ and $$ \operatorname{Var}(b^\top X_2) = E[b^\top X_1X_1^\top b] = b^\top \Sigma_{22} b $$ so substituting into $$ \operatorname{Cor}(a^\top X_1, b^\top X_2) = \frac{\operatorname{Cov}(a^\top X)}{\sqrt{\operatorname{Var}(a^\top X_1)\operatorname{Var}(b^\top X_2)}} $$ gives part (a). For part (b), assuming that $X$ is non-degenerate normal (e.g. every component has positive variance) then we have that $\Sigma_{11}$ and $\Sigma_{22}$ are positive-definite and thus there are some (symmetric, invertible) matrices $\Sigma_{11}^{1/2}, \Sigma_{22}^{1/2}$ that square to $\Sigma_{11}$ and $\Sigma_{22}$. Put $u = \Sigma_{11}^{1/2} a$ and $v = \Sigma_{22}^{1/2}b$, so that $$ \begin{align*} \max_{a \neq 0, b \neq 0}\operatorname{Cor}(a^\top X_1, b^\top X_2) &= \max_{a \neq 0, b \neq 0} \frac{a^\top \Sigma_{12} b}{\sqrt{a^\top \Sigma_{11} a} \sqrt{b^\top \Sigma_{22} b}} \\ &= \max_{a \neq 0, b \neq 0} \frac{a^\top \Sigma_{11}^{1/2}\Sigma_{11}^{-1/2} \Sigma_{12} \Sigma_{22}^{1/2}\Sigma_{22}^{-1/2} b}{\sqrt{a^\top \Sigma_{11}^{1/2}\Sigma_{11}^{1/2} a} \sqrt{b^\top \Sigma_{22}^{1/2} \Sigma_{22}^{1/2} b}} \\ &= \max_{u \neq 0, v \neq 0} \frac{u^\top M v}{\sqrt{u^\top u}\sqrt{v^\top v}} \end{align*} $$ and since maximizing the square is the same maximization problem, this reduces to $$ \max_{u \neq 0, v \neq 0} \frac{(u^\top M v)^2}{(u^\top u)(v^\top v)} $$ as desired.

Edit: changed Cov to Cor