Questions tagged [model-selection]
Model selection is a problem of judging which model from some set performs best. Popular methods include $R^2$, AIC and BIC criteria, test sets, and cross-validation. To some extent, feature selection is a subproblem of model selection.
2,003
questions
0
votes
1
answer
228
views
Optimal Lag Selection Indicates Lag 98
I am trying to identify the optimal lag for my multivariate time series and currently I am getting the optimal AIC at lag 98. I have never seen such large optimal lag is this correct?
Note that my ...
CommunityBot
- 1
0
votes
0
answers
13
views
Confused on Bayesian Decision Theory
I am trying to understand what is the right way to pick up an "action", as it is called in Murphy, Machine Learning a Probabilistic Perspective, in the 'chatper 'Bayesian decision theory'.
...
- 1
7
votes
1
answer
756
views
How to argue omitted variable problem is alleviated?
Is there any ways to argue that the omitted variable problem is alleviated after adding a new variable to the model? Right now I'm basically just saying that adding this new variable significantly ...
- 37.3k
0
votes
0
answers
34
views
The function step.lmRob() is not working [closed]
I have a linear model, which i analyzed (in R) through: lmrob_object<-lmrob(diff_mg ~ age + bmi + energy + fiber + ca + phos + iron + potas + supp + uni, data = data), where:
diff_mg is the DV (...
- 151
294
votes
8
answers
220k
views
How to choose a predictive model after k-fold cross-validation?
I am wondering how to choose a predictive model after doing K-fold cross-validation.
This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
- 1
1
vote
2
answers
1k
views
Extracting a particular model from regsubsets in R
I ran regsubsets in r from the leaps library. I have gotten some 16 models in their order of which is best according to certain criterion.
How do I select, say, ...
CommunityBot
- 1
2
votes
2
answers
484
views
Selecting multiple hyper-parameters via successive nested cross-validation
Selecting multiple hyper-parameters via successive nested cross-validation
I am currently working in a classification task on motion data. Each sample to classify is represented by a set of features ...
CommunityBot
- 1
0
votes
0
answers
10
views
CPO, DIC or WAIC, which metric to choose when they don't agree?
I am creating a Bayesian spatiotemporal model with the four type Knorr Held interaction proposal. I am trying the different type of interactions and I want to select the best model based on DIC, WAIC ...
- 23
1
vote
1
answer
1k
views
Vector Autoregression - How do we choose the correct value of p?
I am following this article: https://otexts.com/fpp2/VAR.html#fn24
...
CommunityBot
- 1
0
votes
0
answers
39
views
Questions regarding the definition of the deviance in the context of GLMs
I've been self-studying GLMs and I have some questions regarding the deviance in the context of GLMs. In Generalized Additive Models An Introduction with R, the author defines the deviance of a model ...
- 21
0
votes
0
answers
25
views
Interpret the PACF plot to select the correct lag (AR model order)
I want to select lag (AR model order) for the series Food price inflation.
AIC gives 4.
SIC gives 3.
And also, I print its PACF plot.
How can I interpret the PACF plot to select the correct lag?
- 68.6k
2
votes
1
answer
638
views
GLM Model Selection
I have to fit some data to a glm, family=poisson(link="log").
The response variables are X1, X2, X3 and X4.
I need an algorithm to fit the best possible model (by lowest AIC). All terms must ...
CommunityBot
- 1
10
votes
1
answer
400
views
Bayesian Justification of Cross-validation
If I understand correctly, K-fold cross-validation is supposed to approximate expected log predictive density (ELPD), which is defined as $\mathop{\mathbb{E}}_{D_{new}\sim P(.|M_{true})}\log P(D_{new}|...
- 197
0
votes
0
answers
24
views
Higher order moments to evaluate strength of linear relationship between variables
Let $X_1,\dots,X_n$ be real random variables such that $\alpha_1X_1+\dots+\alpha_nX_n=0$ for some unknown $\alpha_1,\dots,\alpha_n$. If $n=2$, one can study the strength of linear relationship by ...
- 213
10
votes
2
answers
771
views
How to train a model when instead of a target we have a range where it is?
Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
CommunityBot
- 1