What do you think of this proof for Fisher information?

Question

I want to prove This formula:

The score function is basically the derivative of the maximum likelihood's log, so to get the information I make another derivative of that:

$$ -E[∂/∂θ s(X;θ)] = -E[∂/∂θ (∂log f(X;θ) / ∂θ)] = -E[∂^2/∂θ^2 log f(X;θ)] $$

Now I did $$ -E[∂^2/∂θ^2 log f(X;θ)] = -E[(∂^2/∂θ^2 f(X;θ)) / f(X;θ)] + E[(∂/∂θ f(X;θ)/f(X;θ))^2] $$

I did integral of $-E[(∂^2/∂θ^2 f(X;θ)) / f(X;θ)]$ which aventualy equals to zero, which leaves us with

$$ -E[∂^2/∂θ^2 log f(X;θ)] = 0 + E[(∂/∂θ f(X;θ)/f(X;θ))^2] $$

And since $$ E[(∂/∂θ f(X;θ)/f(X;θ))^2] = E[s(X;θ)^2] $$

This solves the proof.

What do you think?

Answer 1

Assuming the regularity conditions, let $\mathbf X\sim f_\theta(\mathbf x)$ w.r.t. a sigma-finite measure $\mu$ on $(\mathscr X,\mathfrak B(\mathscr X)).$

Now

\begin{align}0 &= \frac{\partial}{\partial \theta}\int_\mathscr X f_\theta(\mathbf x) ~\mathrm d\mu(\mathbf x) \\ &=\int_\mathscr X \frac{\partial}{\partial \theta}f_\theta(\mathbf x) ~\mathrm d\mu(\mathbf x)\\&=\int_\mathscr X \frac{\partial}{\partial\theta}\ln f_\theta(\mathbf x)\cdot f_\theta(\mathbf x) ~\mathrm d\mu(\mathbf x) \\\implies 0&=\int_\mathscr X\frac{\partial}{\partial\theta}\left[\frac{\partial}{\partial\theta}\ln f_\theta(\mathbf x)\cdot f_\theta(\mathbf x) \right]~\mathrm d\mu(\mathbf x)\\ 0&= \int_\mathscr X\frac{\partial^2}{\partial\theta^2}\ln f_\theta(\mathbf x)\cdot f_\theta(\mathbf x) ~\mathrm d\mu(\mathbf x)+ \int_\mathscr X\left\{\frac{\partial}{\partial\theta} \ln f_\theta(\mathbf x)\right\}^2 f_\theta(\mathbf x) ~\mathrm d\mu(\mathbf x)\\ &= \mathbb E_\theta\left\{\frac{\partial^2}{\partial\theta^2}\ln f_\theta(\mathbf X) \right\}+ \mathbb E_\theta\left\{\left(\frac{\partial}{\partial\theta}\ln f_\theta(\mathbf X) \right)^2\right\}.\tag 1\label 1\end{align}

Then, Fisher information

\begin{align}\mathsf I(\theta)&:=\mathbb E_\theta\left\{\left(\frac{\partial}{\partial\theta}\ln f_\theta(\mathbf X) \right)^2\right\} \\&\overset{\eqref{1}}{=} -\mathbb E_\theta\left\{\frac{\partial^2}{\partial\theta^2}\ln f_\theta(\mathbf X) \right\}\\&= -\mathbb E_\theta\left\{\frac{\partial}{\partial\theta}s_\theta(\mathbf X) \right\}.\end{align}