1
$\begingroup$

The score test says that we take the derivative of the log-likelihood at $H_0$ and divide it by the fisher information at $H_0$.

$U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.$

$I(\theta )=-\operatorname {E} \left[\left.{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log f(X;\theta )\,\right|\,\theta \right]\,$

The intuition here mostly makes sense. If $H_0$ is near the most likely point in the data, then the derivative of the log-likelihood will be 0, at the peak of the pdf. If $H_0$ is further away from the peak -- say along the sharp upwards curve of the pdf, as it approaches its maximum -- then the derivative will be large. Therefore, a larger derivative indicates the value is farther from the maximum, and the probability of rejecting the null increases.

But what about farther away from the peak, still.

bell curve

If we are out three or four sigma from the peak of the curve, then our derivative of the log-likelihood under the null will be very small. However, this contradicts the logic of the prior intuition.

So how does the score test actually work in practice? I'm not clear on how the Fisher information could be counteracting this effect.

Edit:

x <- seq(from=-100, to=100, by=0.10)
y <- sapply(x, function (z) log(dnorm(z, 0, 1)))
plot(x, y)

We can see by simulation that with the $L(x)$, that the general principle holds up.

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

Remember that we're differentiating the log-likelihood. If $L(\theta;x)$ goes to 0 as $\theta$ moves away from the peak, then $\log L(\theta;x)$ will go to $-\infty$.

Try sketching the log-likelihood and its derivative for the simple case where we draw a single observation $X$ from a $N(\theta,1)$ distribution.

$\endgroup$
1
  • 1
    $\begingroup$ I tested by simulation and updated my post. Thanks for the tip! $\endgroup$ Commented Sep 26, 2023 at 16:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.