6
$\begingroup$

So, in the usual case, one can prove from the asymptotic normality of a maximum likelihood estimator that the corresponding log-likehood surface is quadratic near the MLE (e.g. in the proof of the correspondence between Fisher information and the variance of the MLE).

My question is, does the opposite hold necessarily? I.e. suppose I know that my log-likelihood surface is quadratic around the best fit: can I then infer that the corresponding MLE is (approximately) normally distributed?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
4
$\begingroup$

If you are not working with the asymptotic case, that "around the best fit" is key; if functions are twice differentiable, they are "locally linear" and also "locally quadratic", which latter implies that the quadratic approximation is arbitrarily good as you shrink the region over which you are approximating towards any given point.

This means that all twice-differentiable log likelihood surfaces are approximately quadratic in a sufficiently small region around the best fit. Naturally this does not imply that the MLE is approximately (to any given degree) normally distributed, because, writing loosely, that "sufficiently small region" can be much smaller than the region in which the MLE might plausibly fall.

If you are working with the asymptotic case, then things are different. If, asymptotically, your log-likelihood surface becomes quadratic around the best fit, then the corresponding MLE is, as you suspect, asymptotically Normally distributed. To see this, note that (one of the) standard proofs of asymptotic Normality of the MLE involves taking a Taylor expansion of the log likelihood function and ignoring all terms above the second order term; see for example http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf, page 8. Obviously the validity of doing so requires that those terms actually be ignorable, i.e., that the log-likelihood surface becomes quadratic around the true parameter value as the sample size goes to $\infty$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hmm I guess I suspected that much, but I wonder if there is any sort of guarantee? For example say I know the likelihood surface is extremely close to quadratic over some range of parameter value near the MLE. If this "known quadratic" range is a certain size, do I then know that the MLE is very close to normal out to some distance in the tails? For example if the quadratic region could be used to construct 95% confidence intervals, do I also know that the MLE is normal at least that far into the tails? Something like that? I guess the Taylor expansion you mention probably implies this. $\endgroup$
    – Ben Farmer
    Commented May 29, 2018 at 20:33
  • 2
    $\begingroup$ Well, now, that's a very interesting question indeed, and I'll think about that. I imagine you could do something approximate using the asymptotic $\chi^2$ distribution of the log likelihood ratio, say the maximum of the log likelihood was -10 and the 99% (asymptotic) lower bound was -20, perhaps if you could show that the quadratic approximation was reasonably good for values of the likelihood function $> - 20$... $\endgroup$
    – jbowman
    Commented May 29, 2018 at 22:00

Not the answer you're looking for? Browse other questions tagged or ask your own question.