Eigenvectors of real symmetric matrices are orthogonal

Question

Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = Q\Lambda Q^{-1} = Q\Lambda Q^{T}$ where $\Lambda$ is diagonal.

Answer 1

For any real matrix $A$ and any vectors $\mathbf{x}$ and $\mathbf{y}$, we have $$\langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle.$$ Now assume that $A$ is symmetric, and $\mathbf{x}$ and $\mathbf{y}$ are eigenvectors of $A$ corresponding to distinct eigenvalues $\lambda$ and $\mu$. Then $$\lambda\langle\mathbf{x},\mathbf{y}\rangle = \langle\lambda\mathbf{x},\mathbf{y}\rangle = \langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle = \langle\mathbf{x},A\mathbf{y}\rangle = \langle\mathbf{x},\mu\mathbf{y}\rangle = \mu\langle\mathbf{x},\mathbf{y}\rangle.$$ Therefore, $(\lambda-\mu)\langle\mathbf{x},\mathbf{y}\rangle = 0$. Since $\lambda-\mu\neq 0$, then $\langle\mathbf{x},\mathbf{y}\rangle = 0$, i.e., $\mathbf{x}\perp\mathbf{y}$.

Now find an orthonormal basis for each eigenspace; since the eigenspaces are mutually orthogonal, these vectors together give an orthonormal subset of $\mathbb{R}^n$. Finally, since symmetric matrices are diagonalizable, this set will be a basis (just count dimensions). The result you want now follows.

Answer 2

Since being symmetric is the property of an operator, not just its associated matrix, let me use $\mathcal{A}$ for the linear operator whose associated matrix in the standard basis is $A$. Arturo and Will proved that a real symmetric operator $\mathcal{A}$ has real eigenvalues (thus real eigenvectors) and that eigenvectors corresponding to different eigenvalues are orthogonal. One question still stands: how do we know that there are no generalized eigenvectors of rank more than 1, that is, all Jordan blocks are one-dimensional? Indeed, by referencing the theorem that any symmetric matrix is diagonalizable, Arturo effectively threw the baby out with the bathwater: showing that a matrix is diagonalizable is tautologically equivalent to showing that it has a full set of eigenvectors. Assuming this as a given dismisses half of the question: we were asked to show that $\Lambda$ is diagonal, and not just a generic Jordan form. Here I will untangle this bit of circular logic.

We prove by induction in the number of eigenvectors, namely it turns out that finding an eigenvector (and at least one exists for any matrix) of a symmetric matrix always allows us to generate another eigenvector. So we will run out of dimensions before we run out of eigenvectors, making the matrix diagonalizable.

Suppose $\lambda_1$ is an eigenvalue of $A$ and there exists at least one eigenvector $\boldsymbol{v}_1$ such that $A\boldsymbol{v}_1=\lambda_1 \boldsymbol{v}_1$. Choose an orthonormal basis $\boldsymbol{e}_i$ so that $\boldsymbol{e}_1=\boldsymbol{v}_1$. The change of basis is represented by an orthogonal matrix $V$. In this new basis the matrix associated with $\mathcal{A}$ is $$A_1=V^TAV.$$ It is easy to check that $\left(A_1\right)_{11}=\lambda_1$ and all the rest of the numbers $\left(A_1\right)_{1i}$ and $\left(A_1\right)_{i1}$ are zero. In other words, $A_1$ looks like this: $$\left( \begin{array}{c|ccc} \lambda_1 & \\ \hline & & \\ & & B_1 & \\ & & \end{array} \right)$$ Thus the operator $\mathcal{A}$ breaks down into a direct sum of two operators: $\lambda_1$ in the subspace $\mathcal{L}\left(\boldsymbol{v}_1\right)$ ($\mathcal{L}$ stands for linear span) and a symmetric operator $\mathcal{A}_1=\mathcal{A}\mid_{\mathcal{L}\left(\boldsymbol{v}_1\right)^{\bot}}$ whose associated $(n-1)\times (n-1)$ matrix is $B_1=\left(A_1\right)_{i > 1,j > 1}$. $B_1$ is symmetric thus it has an eigenvector $\boldsymbol{v}_2$ which has to be orthogonal to $\boldsymbol{v}_1$ and the same procedure applies: change the basis again so that $\boldsymbol{e}_1=\boldsymbol{v}_1$ and $\boldsymbol{e}_2=\boldsymbol{v}_2$ and consider $\mathcal{A}_2=\mathcal{A}\mid_{\mathcal{L}\left(\boldsymbol{v}_1,\boldsymbol{v}_2\right)^{\bot}}$, etc. After $n$ steps we will get a diagonal matrix $A_n$.

There is a slightly more elegant proof that does not involve the associated matrices: let $\boldsymbol{v}_1$ be an eigenvector of $\mathcal{A}$ and $\boldsymbol{v}$ be any vector such that $\boldsymbol{v}_1\bot \boldsymbol{v}$. Then $$\left(\mathcal{A}\boldsymbol{v},\boldsymbol{v}_1\right)=\left(\boldsymbol{v},\mathcal{A}\boldsymbol{v}_1\right)=\lambda_1\left(\boldsymbol{v},\boldsymbol{v}_1\right)=0.$$ This means that the restriction $\mathcal{A}_1=\mathcal{A}\mid_{\mathcal{L}\left(\boldsymbol{v}_1\right)^{\bot}}$ is an operator of rank $n-1$ which maps ${\mathcal{L}\left(\boldsymbol{v}_1\right)^{\bot}}$ into itself. $\mathcal{A}_1$ is symmetric for obvious reasons and thus has an eigenvector $\boldsymbol{v}_2$ which will be orthogonal to $\boldsymbol{v}_1$.

Answer 3

It would appear that you want to write vectors as rows, so your preferred multiplication will be on the left side, as in $v \mapsto v A.$

The ordinary dot product is then $ v \cdot w = v w^T = w v^T = w \cdot v.$ Note that $v w^T$ is a number, or a 1 by 1 matrix, and is equal to its transpose.

In the same way, $v A \cdot w = v A w^T.$ However, $v A w^T$ is again a 1 by 1 matrix and is equal to its transpose, and $A^T = A,$ so we get $$ v A \cdot w = v A w^T = (v A w^T)^T = (w^T)^T A^T v^T = w A v^T = w A \cdot v$$

First suppose $v,w$ are eigenvectors with distinct eigenvalues $\lambda, \mu.$ We have $$ v A \cdot w = \lambda v \cdot w = w A \cdot v = \mu w \cdot v.$$ Or, $\lambda v \cdot w = \mu v \cdot w,$ finally $$ (\lambda - \mu) v \cdot w = 0.$$ So, eigenvectors with distinct eigenvalues are orthogonal.

It is possible that an eigenvalue may have larger multiplicity. However, for a fixed eigenvalue $\lambda,$ the set of vectors $v$ for which $ v A = \lambda v$ is a subspace, of full dimension (meaning the Jacobi form has no off-diagonal elements), and we may simply choose an orthonormal basis for this subspace. Choosing, in this way, all basis vectors to be length 1 and orthogonal, we get an orthonormal basis of eigenvalues of $A.$ Write those as rows of a matrix $P,$ we get $P A P^T = \Lambda.$

The only difficult aspect here is this: if an eigenvalue has algebraic multiplicity larger than one, that is the characteristic polynmial has a factor of $(x-\lambda)^k$ for some $k \geq 2,$ how can I be sure that the geometric multiplicity is also $k?$ That is, with $A$ symmetric, how do I know that $$ v (A - \lambda I)^k = 0 \; \; \Rightarrow \; \; v (A - \lambda I) = 0?$$ Working on it. It appears that this is, at heart, induction on $k,$ and takes many pages. Give me some time.

Alright, this works. An induction on dimension shows that every matrix is orthogonal similar to an upper triangular matrix, with the eigenvalues on the diagonal (the precise statement is unitary similar). How do we know the eigenvalues are real? We have an eigenvalue $\lambda$ with an eigenvector $v,$ perhaps both with complex entries. As is traditional, for a vector or matrix define $v^\ast = \bar{v}^T$ and $A^\ast = \bar{A}^T.$ It is easy to see that $v v^\ast$ is a positive real number unless $v = 0.$ In any case $A^\ast = A.$ So, given $v A = \lambda v,$ $$ ( v A v^\ast)^\ast = (v^\ast)^\ast A^\ast v^\ast = v A v^\ast.$$ As a result, the complex number $v A v^\ast$ is actually a real number. At the same time, $v A v^\ast = \lambda v v^\ast,$ and since both $v A v^\ast$ and $v v^\ast$ are real numbers, the latter nonzero, it follows that $\lambda$ is real.

Put these together, we get that each real matrix with real characteristic values is orthogonal similar to an upper triangular real matrix. However, as $A$ is symmetric, this upper triangular matrix is actually diagonal.

Answer 4

Let's assume that $x$ is an eigenvector of $A$ corresponding to the eigenvalue $\lambda_1$ and $y$ an eigenvector of $A$ corresponding to the eigenvalue $\lambda_2$, with $\lambda_1 \neq \lambda_2$.

$Ax=\lambda_1x \\ Ay=\lambda_2y$

After taking into account the fact that A is symmetric ($A=A^*$):

$y^{\intercal}Ax=\lambda_1y^{\intercal}x \\ x^{\intercal}A^{\intercal}y=\lambda_2x^{\intercal}y$

Now subtract the second equation from the first one and use the commutativity of the scalar product:

$y^{\intercal}Ax-x^{\intercal}A^{\intercal}y=\lambda_1y^{\intercal}x - \lambda_2x^{\intercal}y \\ 0 = (\lambda_1 - \lambda_2)y^{\intercal}x$

Hence $x$ and $y$ are orthogonal.

Answer 5

How about Let $A$ be symmetric, then there exists a matrix $D$ such that $A=QDQ^T$, taking the transpose of $A$, namely

$$\left(A\right)^T = \left(QDQ^T\right)^T $$ $$A^T = \left(Q^T\right)^TD^TQ^T$$ $$A^T = QDQ^T$$

thus $A^T = A$ if and only if $A$ is symmetric.

It is noteworthy that $D^T = D$ since $D$ is diagonal and $Q$ is the matrix of normed eigenvectors of $A$, Thus $Q^T = Q^{-1}$

Answer 6

Note a real symmetric matrix is a linear operator on Euclidean space with respect standard basis (orthonormal). So the fact that it equals to its conjugate transpose implies it is self-adjoint.

For two distinct eigenvalues $\lambda_1, \lambda_2$ and corresponding eigenvectors $v_2, v_2$, $$(\lambda_1-\lambda_2)\langle v_1, v_2\rangle=\langle \lambda_1 v_1,v_2\rangle-\langle v_1,\bar{\lambda_2} v_2\rangle=\langle Tv_1,v_2\rangle-\langle v_1,T^* v_2\rangle=0$$ where the 2nd last equality follows from properties of self-adjoint (thus normal) linear operator (Lemma below).

Lemma: Assume $T$ is normal. If $(\lambda, v)$ is eigenvalue and eigenvector of $T$, $(\bar{\lambda}, v)$ is eigenvalue and eigenvector of the adjoint $T^*$. (pf.) Trivial from definition of normality.

Answer 7

You ask `Can someone point me to a paper...'. This is standard elementary linear algebra that can be found in introductory text books and courses so try looking in your local library or bookstore.