Questions tagged [bias-variance-tradeoff]
In predictive modeling, unbiased models can have higher variance, & thus be less accurate. Modelers may prefer some bias to maximize accuracy. Use this tag also for questions about the bias-variance decomposition.
261
questions
0
votes
1
answer
28
views
Why does increasing model complexity reduce bias over the entire data distribution?
In ML, we often talk about the bias-variance tradeoff, and how increasing model complexity both reduces bias and increases variance. I understand why increasing model complexity reduces bias at first, ...
- 406
0
votes
0
answers
28
views
time series squared forecast evaluation
I have a time series with very weak autocorrelations- mostly unforecastable. However, its squared values have stronger autocorrelations. Something like this:
...
- 643
1
vote
0
answers
18
views
Optimal estimate under altered MSE loss function
Suppose I am interested in estimating $\theta \in \mathbb{R}$ and I observe a noisy data point $\tilde{\theta}=\theta + N(0,\sigma^2)$ where $\sigma^2$ is known. I am interested in constructing an ...
0
votes
1
answer
22
views
Does the intuitive sense of overfitting in this mechanism design context exemplify bias-variance tradeoff?
Suppose the (we can say unanimous) preference of each individual in a society is to select roads for travel by placing 95% weight on the objective of minimizing travel time, and the remaining 5% ...
- 123
5
votes
2
answers
363
views
Expected loss function from bias variance trade off (integral help)
I have a hard time understanding this formula. It's from bias-variance trade-off proof. and the expected loss function is as follows:
$$L(\hat f) := \mathbb E_D\mathbb E_{(x,y)}[(y-\hat f(x))^2]=\...
- 89
0
votes
0
answers
20
views
Variance Bias Tradeoff decomposition of Linear Regression with a twist
Normally, for a linear regression problem with fixed observations, we have the variance and bias tradeoff as:
$$Var(Y) + Bias^2 (\hat{\beta_x}) + Var(\hat{\beta_x})$$.
My question is what happens to ...
3
votes
1
answer
55
views
What's the relationship between "bias-variance tradeoff" and "consistent model selection"?
I'm very confused about the relationship between "bias-variance tradeoff" and "consistent model selection". Based on my current interpretation, the ultimate goal of taking care of ...
- 2,966
0
votes
0
answers
15
views
Formally state the bias-variance tradeoff & intuition [duplicate]
The following states the bias-variance dilemma formally:
$$\hat{\epsilon}_i = E[\epsilon^2] + \left[f - E[\hat{f}] \right]^2 + E\left[ \left(\hat{f} - E[\hat{f}] \right)^2 \right]$$
where this is ...
- 623
1
vote
1
answer
56
views
Does the Bias of the Model Only Depend on Model Class?
In machine learning with statistical approach, does the bias of the model solely depend on the selection of the model class without considering the training data? There is a claim regarding the bias-...
4
votes
1
answer
105
views
Mean Squared Error for point estimation
I am attempting to understand Mean Squared Error when evaluating point estimators for particular parameters of interest. The book we are reading for class states the following:
The mean squared error (...
- 345
0
votes
0
answers
46
views
Derivation of bias variance trade-off with or without conditional expectation?
I found this nice lecture here where the bias variance trade-off is explained using conditional expectation - using e.g. $E_{y|X}[...]$
In this lecture here I found another proof of the formula ...
- 209
1
vote
0
answers
59
views
Who was the first to notice that the bias can be decomposed into model bias and estimation bias?
As the title says, who was the first to notice that the bias can be decomposed into model bias and estimation bias?
For reference, I'm talking about the quantities here at page 224 eq. (7.14)
https://...
- 11
1
vote
0
answers
26
views
Bias-Variance Tradeoff, computing bias theoretically
Bias, in machine learning, is mathematically defined as $f-E(\hat{f})$, where $f$ is the true model and $\hat{f}$ is the estimate.
I was wondering how we can compute theoretically $E(\hat{f})$, given ...
- 9,217
3
votes
1
answer
34
views
Explanation for the success of bagging
I'm reading Machine Learning - A First Course for Engineers and Scientists. On page 168 they give a rough explanation for why bagging works. I'm a little confused by their explanation.
They consider ...
- 235
6
votes
1
answer
51
views
Bias-variance trade-off for a specific fitted model vs. a class of models: terminology
Consider a data generating process
$$Y=f(X)+\varepsilon$$
where $\varepsilon$ is independent of $x$ with $\mathbb E(\varepsilon)=0$ and $\text{Var}(\varepsilon)=\sigma^2_\varepsilon$. According to ...
- 68.6k