Linked Questions
15 questions linked to/from AIC versus cross validation in time series: the small sample case
291
votes
13
answers
258k
views
Is there any reason to prefer the AIC or BIC over the other?
The AIC and BIC are both methods of assessing model fit penalized for the number of estimated parameters. As I understand it, BIC penalizes models more for free parameters than does AIC. Beyond a ...
- 18.9k
133
votes
5
answers
137k
views
Using k-fold cross-validation for time-series model selection
Question:
I want to be sure of something, is the use of k-fold cross-validation with time series is straightforward, or does one need to pay special attention before using it?
Background:
I'm ...
- 1,488
40
votes
5
answers
13k
views
Cross-validating time-series analysis
I've been using the caret package in R to build predictive models for classification and regression. Caret provides a unified interface to tune model hyper-parameters by cross validation or boot ...
- 24.1k
7
votes
3
answers
4k
views
Why is AIC or BIC commonly used in model selections for time series forecasting?
On scikit-learn documentation, I found the following comments about AIC:
Information-criterion based model selection is very fast, but it relies on a proper estimation of degrees of freedom, are ...
- 455
13
votes
1
answer
4k
views
Calculating forecast error with time series cross-validation
I have a forecasting model for a time series and I want to calculate its out-of-sample prediction error. At the moment the strategy I'm following is the one suggested on Rob Hyndman's blog (near the ...
- 3,712
6
votes
2
answers
912
views
Is the Cross Validation Error more "Informative" compared to AIC, BIC and the Likelihood Test?
Is the Cross Validation Error more "Informative" compared to AIC, BIC and the Likelihood Test?
As far as I understand:
The Likelihood Test is used to determine : Given some data, is some ...
5
votes
1
answer
5k
views
ARIMA Cross Validation
I work with R and have got some questions regarding my ARIMA model. In specific, I have yearly data ranging from 1946 to 2019 and would like to do a basic two-step ahead ARIMA forecast for 2020 and ...
- 53
9
votes
0
answers
2k
views
Selecting regularization penalty: cross validation or information criteria?
I will use an elastic net to estimate a regression model which will later be used for forecasting.
I have a grid of $\alpha$ values within [0,1] representing the proportion of $L_1$ versus $L_2$ ...
- 68.6k
1
vote
1
answer
492
views
ARIMA accuracy measures, rolling forecast
Regarding ARIMA model selection and especially accuracy measures several questions came into my mind. To shortly summarize, in my understanding, after necessary transformations/differencing, p and q ...
- 59
3
votes
0
answers
468
views
Forecasting: AIC, AICc and BIC VS Cross Validation for model selection (cases for different horizons)
The majority of the automatic model selection algorithms like auto.arima and ets (https://robjhyndman.com/publications/automatic-...
- 31
3
votes
1
answer
303
views
LASSO vs AIC for submodel selection via nonzero coefficient variable selection
Suppose you have a linear model which you believe has too many variables -- a cubic in 10 lags, for example. You believe, without being certain, that it is probably quadratic, and maybe linear, and ...
- 3,157
1
vote
1
answer
165
views
What is the purpose of the model AIC if you can iteratively check the forecast RMSE at each lag?
If the purpose of the time series modelling is to build one that gives the most accurate forecast, may I ask if it necessary to check the model AIC to determine the optimal lag when you can ...
- 13
0
votes
0
answers
89
views
Should I use AIC / BIC or rather cross validation for discovering gov. equations through linear regression (SINDy)?
I want to use linear regression with very large design matrix for discovery of governing equations to i.e. physical systems. The design matrix would include potential terms that can be part of the ...
- 11
0
votes
0
answers
53
views
ISLR Chapter 6 : Choosing the Optimal Model
I had a question regarding the "choosing the optimal model" section of chapter 6 of ISLR (pg. 232). The book states that
"In order to select the best model with respect to test error, ...
1
vote
0
answers
29
views
Appropriate test set/training set for small sample time-series model?
So variations of this question have been asked a few times, but I think my case is somewhat different than the previous questions as they either have larger sample sizes and/or cross-sectional data.
I ...
user281754