All Questions
153
questions
1
vote
0
answers
49
views
Standard practice to show Biased CRBs
I have a problem with four-parameter estimation. I have derived the variances for the estimated parameters using Monte Carlo simulations (numerical ones) and theoretical ones using the inverse of the ...
- 235
0
votes
1
answer
22
views
Does the intuitive sense of overfitting in this mechanism design context exemplify bias-variance tradeoff?
Suppose the (we can say unanimous) preference of each individual in a society is to select roads for travel by placing 95% weight on the objective of minimizing travel time, and the remaining 5% ...
- 123
0
votes
0
answers
51
views
Why the MSE of the fitted data is not equal to the sum of the bias and the variance in R?
I use simple linear regression and I want to find the decomposition of MSE, that is as a sum of the bias, the variance and the variance of the error terms. I have the following code:
...
0
votes
1
answer
70
views
How to compare two multivariate distribution (of distances) to zero in terms of mean and variance in R?
We have N 3D coordinates estimated with two methods and want to compare them with a reference set of N 3D coordinates which is the ground truth, so in notations:
...
- 303
2
votes
0
answers
60
views
Bias and Variance of a Honest Random Forest
I am trying to read the paper Estimation and Inference of Heterogeneous Treatment
Effects using Random Forests. In the section 3.1(Theoretical Background), page 13 paragraph 2, The authors have ...
- 137
2
votes
2
answers
172
views
Proof of the bias-variance decomposition in Bishop's book
I am trying to rewrite the demonstration given in Bishop's book: Pattern Recognition and
Machine Learning (2009)
I reproduce the figure (page 149) in which I am unclear about the step leading from (3....
- 153
1
vote
0
answers
21
views
What is the exact relationship between variance and bias in regression?
I know about the bias variance tradeoff such that with higher variance we have less bias and vice versa. But what is the relationship of this tradeoff? Is it just an exact inverse proportionality i.e $...
- 171
0
votes
0
answers
252
views
Leave One Subject Out Cross Validation: mean vs median
Assume we have a dataset with n subjects and m labels and train a classifier. To ensure that there is no subject bias in the ...
- 121
2
votes
1
answer
371
views
Philosophical insight of Bias Variance Decomposition
As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:
$$\operatorname{MSE}(\hat{\theta}) = \operatorname{tr}(\operatorname{...
- 31
0
votes
0
answers
35
views
Bias and variance of estimators - Normal Sample
If we consider the two following estimators $$\hat{\mu_1} = \frac{\bar{X_1}+\bar{X_2}}{2}$$ $$\hat{\mu_2} = \frac{n_1\bar{X_1}+n_2\bar{X_2}}{n_1+n_2}$$ where $X_1, X_2$ are samples from a normal ...
- 197
1
vote
0
answers
51
views
Problem in showing $\rm MSE = Var + Bias^2.$ [duplicate]
I am trying to prove the equality of $$\rm MSE(\langle I\rangle)=Var(\langle I \rangle)+Bias(\langle I \rangle)^2$$ but obviously I got something wrong as they don't equal in my calculation:
So here ...
- 143
3
votes
3
answers
2k
views
If we reduce size of training dataset does it decreases bias?
I'm a newbie and learning ML. I've a doubt, normally we know we should increase the size of training dataset or should add more data to reduce variance (fairly understood why). Now variance has ...
- 133
1
vote
1
answer
179
views
Reasons to prefer low bias with higher variance over the alternative (and vice versa)
I am trying to understand the bias-variance tradeoff in practice. I have read several related questions and answers, but still have a few questions:
Assume we are estimating a structural equation ...
- 115
3
votes
0
answers
415
views
Apart from the Bias-Variance "Decomposition" - is there a Bias-Variance "Proof"?
I am sure at some point, many of us have come across the "Bias-Variance Tradeoff" : The "error" of any "estimator" (e.g an estimator can be considered as a linear ...
14
votes
5
answers
4k
views
Does the biased estimator always have less variance than unbiased one?
Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always.
...
- 299