2
$\begingroup$

I have forty candidate predictors. They are no colinear. I want to know which ones are related to the DV. Prediction isn't important to me. I want to do this in an exploratory and data-driven way.

What's my best option? I've looked at: multiple regression, stepwise regression (AIC,BIC), best subsets regression, and Adaptive LASSO.

Is one of those better than the others? And if not, what is a better option?

Improve this question
$\endgroup$
1
  • $\begingroup$ Check out articles by Chernozhukov and various other, for high dimensional inference/treatment problems. Double post-lasso may be a good start, check out hdm package of R. $\endgroup$
    – runr
    Commented Sep 14, 2020 at 22:09

1 Answer 1

Reset to default
0
$\begingroup$

There is a battery of problems with stepwise regression, which also includes best subset regression. Multiple regression is fine if $n\gg p$, otherwise you may be better off using regularization.

Namely, a regularized model (like the LASSO you mentioned), restricts the total size of the parameter estimates. Perhaps somewhat surprisingly, introducing this bias can yield better estimates, especially for small $\frac{n}{p}$ ratios.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.