All Questions
Tagged with bias cross-validation
37
questions
0
votes
1
answer
147
views
How to avoid bias/avoid overfitting when choosing a machine learning model? [closed]
My typical workflow in the past, when creating machine learning models, has been to do the following:
Decide on some candidate model families for the task at hand.
Divide dataset into train and test ...
anon
1
vote
0
answers
36
views
Cross-validation as strategy for training
Let's say I have a classification model to be trained and a relatively small dataset. The data is splited in k-folds (eg. k=5), in such a way that:
A, B and C are used for training, D for testing and ...
- 11
0
votes
0
answers
252
views
Leave One Subject Out Cross Validation: mean vs median
Assume we have a dataset with n subjects and m labels and train a classifier. To ensure that there is no subject bias in the ...
- 121
3
votes
1
answer
189
views
How to handle outcome variables during imputation of missing data in model building and assessment process?
Der community
I have a question about the appropriate handling of the imputation of missing data to get an unbiased estimate of prediction accuracy during model building and assessment. While ...
- 31
2
votes
1
answer
79
views
Cross-validation: error estimation and bias
When obtaining the error estimation of a model over a dataset using k-fold cross-validation, lower values of the error estimation necessarily imply a lower bias? Are both concepts, error estimation ...
- 65
1
vote
1
answer
324
views
Why will the estimates of prediction error typically be biased upward with Cross-Validation?
Why the estimates of prediction error will typically be biased upward with Cross-Validation?
Is it like with decisions tree? Using a stopping criterion will increase a little the bias but will ...
- 33
2
votes
1
answer
128
views
Which "bias in research" when splitting the dataset into training / testing set where k-fold cross-validation reached its max validation accuracy?
If you run k-fold cross-validation, and you do not just take the mean of the accuracies but instead, you take the dataset split with the best validation accuracy to use this split as a static split of ...
- 334
0
votes
1
answer
861
views
Why does LOOCV have a higher bias than a single validation set? [duplicate]
In An Introduction to Statistical Learning, the following statement is made comparing leave-one-out cross validation to using a single validation set:
LOOCV has a couple of major advantages over the ...
- 135
16
votes
2
answers
2k
views
Why cross-validation gives biased estimates of error?
I came across many posts on CrossValidated discussing cross-validation and nested cross-validation as an alternative (e.g. here or here). I don't quite understand why 'ordinary' K-fold cross-...
- 540
3
votes
0
answers
68
views
The bias of the bootstrap estimator
From what I gather, a Bootstrap estimation of the generalization error for a ML procedure is optimistically biased, e.g.:
What is the .632+ rule in bootstrapping?
Why is bootstrapping called an "...
- 4,518
2
votes
0
answers
134
views
What is the Bias Variance Tradeoff from a Bayesian perspective
How do Bayesian's treat the Bias Variance Tradeoff?
Typically the Bias Variance Tradeoff is expressed as $Bias^2 + Variance + irreducible\_error$, however wouldn't choosing a prior introduce bias and ...
- 372
1
vote
0
answers
62
views
Biased bootstrap: how to construct confidence intervals?
For a project I am trying to evaluate a set of predictors using logistic regression. So I have defined a model selection procedure that I applied on the data, and estimate an AUC to assess model ...
- 240
2
votes
0
answers
199
views
K-Fold Cross Validation: Bias in cross-validated effect size measures?
As pointed out by various authors (e.g., Hastie, 2011), K-fold cross-validation has an upward bias of prediction error. I wonder whether the same holds for cross-validated effect size measures such as ...
- 393
0
votes
1
answer
37
views
How are variance and bias interpreted in relation to data sets
To interpret the bias we just need the training data and the test data, since it is the measure of how far off the predicted values are from the true values(test data). But, to understand the variance ...
- 111
2
votes
0
answers
955
views
Cross validation and the Bias Variance trade-off
So I know that there have been a lot of questions about this topic but I try to understand it from a bit more theoretical/mathematical point of view.
I have some basic questions of how cross-...
- 931