Questions tagged [estimators]
A rule for calculating an estimate of a given quantity based on observed data [Wikipedia].
106
questions
198
votes
3
answers
247k
views
What is the difference between a consistent estimator and an unbiased estimator?
What is the difference between a consistent estimator and an unbiased estimator?
The precise technical definitions of these terms are fairly complicated, and it's difficult to get an intuitive feel ...
- 2,760
25
votes
5
answers
51k
views
What is the relation between estimator and estimate?
What is the relation between estimator and estimate?
user3412
12
votes
3
answers
2k
views
T-consistency vs. P-consistency
Francis Diebold has a blog post "Causality and T-Consistency vs. Correlation and P-Consistency" where he presents the notion of P-consistency, or presistency:
Consider a standard linear regression ...
- 68.6k
58
votes
5
answers
21k
views
When is a biased estimator preferable to unbiased one?
It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
- 4,423
29
votes
2
answers
12k
views
Correlation between OLS estimators for intercept and slope
In a simple regression model,
$$ y = \beta_0 + \beta_1 x + \varepsilon, $$
the OLS estimators $\hat{\beta}_0^{OLS}$ and $\hat{\beta}_1^{OLS}$ are correlated.
The formula for the correlation ...
- 68.6k
16
votes
3
answers
22k
views
Why is OLS estimator of AR(1) coefficient biased?
I am trying to understand why OLS gives a biased estimator of an AR(1) process. Consider
$$
\begin{aligned}
y_{t} &= \alpha + \beta y_{t-1} + \epsilon_{t}, \\
\epsilon_{t} &\stackrel{iid}{\...
- 163
18
votes
1
answer
12k
views
root-n consistent estimator, but root-n doesn't converge?
I've heard the term "root-n" consistent estimator' used many times. From the resources I've been instructed by, I thought that a "root-n" consistent estimator meant that:
the estimator converges on ...
- 2,289
13
votes
3
answers
2k
views
Revisiting the Rule of Three
The rule of three is a method for calculating a 95% confidence interval when estimating $p$ from a set of $n$ IID Bernoulli trials with no successes.
My understanding from its derivation is that the ...
- 1,463
16
votes
1
answer
7k
views
What's the difference between asymptotic unbiasedness and consistency?
Does each imply the other? If not, does one imply the other? Why/why not?
This issue came up in response to a comment on an answer I posted here.
Although google searching the relevant terms didn't ...
2
votes
2
answers
510
views
How to include the observed values, not just their probabilities, in information entropy?
Shannon entropy measures the unpredictability in a random variable's outcome as the weighted average of the probabilities of that variable's outcomes or observed values. However, it discards the ...
- 4,025
43
votes
9
answers
55k
views
What is the difference between an estimator and a statistic?
I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the ...
- 499
30
votes
1
answer
3k
views
Is there a statistical application that requires strong consistency?
I was wondering if someone knows or if there exists an application in statistics in which strong consistency of an estimator is required instead of weak consistency. That is, strong consistency is ...
- 431
14
votes
4
answers
4k
views
How does one explain what an unbiased estimator is to a layperson?
Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$.
How does one explain this to a layperson? In the past, what I have said ...
- 5,077
5
votes
1
answer
647
views
Estimator that is optimal under all sensible loss (evaluation) functions
Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
- 68.6k
54
votes
3
answers
98k
views
Maximum Likelihood Estimators - Multivariate Gaussian
Context
The Multivariate Gaussian appears frequently in Machine Learning and the following results are used in many ML books and courses without the derivations.
Given data in form of a matrix $\...
- 9,816