All Questions
Tagged with bias maximum-likelihood
27
questions
3
votes
0
answers
29
views
Is there a likelihood penalization or (im)proper prior to remove estimation bias for gamma parameters?
So I am learning that maximum likelihood estimation of the parameters for a gamma distribution are biased. As far as I understand there is no guarantee in general that there exists a prior (or base ...
- 9,401
2
votes
0
answers
61
views
Is this a correct explanation of the asymptotic bias of maximum likelihood?
I want to be sure I understand, so please critique the following:
In regular parametric statistical models, the non-linear maximum likelihood estimator is biased. Given some data, $y_i$, parameters, $...
- 41
2
votes
2
answers
147
views
Why is the asymptotic bias of the maximum likelihood estimate $b(\theta) = \frac{b_1(\theta)}{n}+\frac{b_2(\theta)}{n^2}+...$?
Firth (1993) states in his introduction that for a $p$-dimensional parameter $\theta$ the asymptotic bias of the maximum likelihood estimate $\hat{\theta}$ may be written as:
$b(\theta) = \frac{b_1(\...
- 41
0
votes
0
answers
27
views
bias is large relative to the variance when we pool information over small samples or when data is highly stratified?
I am reading Yudi Pawitan's In All Likelihood Chapter 5. The book makes the following assertion on MLE bias.
"...bias is large relative to the variance when we pool information over small samples ...
- 1,445
1
vote
1
answer
112
views
Bias of an estimator depends on whether you take expectation of the estimator or its inverse
(Please read until the end)
Consider two ways of writing the exponential distribution:
(A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and
(B) $\theta e^{-x\theta}$
If I estimate $\beta$ or $\theta$ ...
- 11
1
vote
1
answer
819
views
Bias correction for MLE of mean of geometric random variable
Parameter estimation [ edit]
For both variants of the geometric distribution, the parameter $p$ can be estimated by equating the expected value with the sample mean. This is the method of moments, ...
user318514
2
votes
0
answers
40
views
Bias of MLEs increases rather than decreases in n
In the context of conducting simulations to assess the performance of MLE point estimates for truncated data, I am encountering surprising settings in which the bias of MLEs is clearly non-monotonic ...
- 3,750
2
votes
1
answer
88
views
Bias of MLE scales with $1/N$?
I was reading this paper (link) and it gave me some confusion.
$P(r|\theta)$ is a distribution that generates sample $r$ based on some Poisson distribution, whose mean and variance are defined as some ...
- 281
1
vote
1
answer
190
views
Bias corrrection for MLE when dealing with normally distributed small samples
When estimating the standard-deviation for samples of normally distributed data, it is sometimes necessary to account for bias in whatever estimator one chooses -- which is usually related to the ...
- 328
3
votes
1
answer
2k
views
Showing bias of MLE for exponential distribution is $\frac{\lambda}{n-1}$
I want to show that the bias of $\hat \lambda = \frac{N}{\sum\limits_{i=1}^N x_i}$ is $\lambda/(n-1)$. There's a good chance that I'm too mathematically illiterate to understand the answer here ...
1
vote
2
answers
227
views
How to find MLE, probability distribution, and bias?
Suppose the data consist of a single number $X$, from the following probability density:
$$f(x|θ) = \begin{cases}
\frac{1+xθ}{2} & & \text{for } -1 \leqslant x \leqslant 1, \\[6pt]
0 & &...
- 19
2
votes
2
answers
44
views
What exactly is meant by bias in this context?
I'm working through an example of survival-time analysis with censored and un-censored data.
We're given the survival times of 94 patients.
Some of these survival times are censored i.e.in this ...
- 1,617
4
votes
0
answers
213
views
Bias correcting penalized maximum likelihood / maximum a posteriori estimates
Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing:
$$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$
where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\...
- 535
2
votes
0
answers
325
views
Bias and over-fitting in Maximum Likelihood estimation
In his book, "Pattern recognition and Machine learning", Bishop talks about the influence of the bias and overfitting in the MLE framework. Here is a quote from p.28, just before he has shown that the ...
- 931
2
votes
0
answers
71
views
Biasedness of ML estimators for an AR(p) process
Do you know any derivations (or references) which quantify the biasedness of ML estimators of an AR(p) process?
- 681