Does a PSD matrix only have non-negative main diagonal entries?

Question

This is regarding the answer by AAL in https://stats.stackexchange.com/questions/312518/why-variance-of-ols-estimate-decreases-as-sample-size-increases.

I am stuck at the italicized parts in:

Because $(X'X)^{-1}x'x(X'X)^{-1}$ is positive semi-definite (it is the multiplication of a matrix with its transpose) and $1+x'x>0$, the diagonal elements of the subtracting term are greater than or equal to zero

How does $(X'X)^{-1}x'x(X'X)^{-1}$ being PSD and $1+x'x>0$ mean that $$ \operatorname{diag}\left( \frac{(X'X)^{-1}x'x(X'X)^{-1}}{1+x'x>0} \right) > 0 $$ ?

This seems to imply that PSD matrices have only non-negative main diagonal elements. Is this true? If so, why?

I know that PSD means the eigenvalues of the matrix are $\geq 0$, and that the sum of eigenvalues equals the sum of the trace of the matrix, but this doesn't guarantee that there aren't negative elements along the diagonal.

Answer 1

Let $\{e_{1},\dots,e_{n}\}$ be the standard orthonormal basis for $\mathbb{R}^{n}$. It is a fact of life that if $A$ is a $n \times n$ matrix, then $A_{ii} = A e_{i} \cdot e_{i}$. (Here $\cdot$ denotes the Euclidean inner product.) More generally, $A_{ij} = Ae_{j} \cdot e_{i}$.

From this, it follows that if $A$ is a symmetric positive semi-definite matrix, then its diagonal elements are non-negative.