Questions tagged [hyperparameter]
A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.
656
questions
0
votes
0
answers
27
views
A faster way to choose the right number of features and the right parameters [closed]
For a very small dataset with less than a hundred samples, gridsearch does not give us the desired results and has overfit. But when I perform hyperparameter tuning of the model manually and also ...
- 33
1
vote
0
answers
11
views
Assessing Random Search Cross Validation: Tuning in ElasticNet with Large Feature Sets
I'm working on estimating an ElasticNet model for a large dataframe with over 100,000 variables, resulting in a well overidentified scenario. To tune my model, I've set up a grid of hyperparameters (...
- 311
0
votes
0
answers
15
views
Correlation between two Gaussian Processes
I have a space-time series, so it is in 2D. So, the signal model $\mathbf{S}$ is a matrix. If I fix the space, the time series at that point in space is a complex GP:
$$ \mathbf{S}[x, :] \sim \...
- 235
0
votes
1
answer
37
views
Manual selection of parameters and features and bad results by gridsearch
For a very small dataset that I have, when I set the parameters with the help of gridsearch, the test and training results are not acceptable at all and have a huge difference. I have to manually ...
- 33
0
votes
0
answers
27
views
Can I use the Mean Squared Prediction Error to select the prior SD in a CausalImpact model?
I'm using the CausalImpact package (in R), and (as I expect is typical) the findings are very sensitive to the prior being used.
I have an OK understanding, I think, of what the prior is doing in this ...
- 1
0
votes
0
answers
25
views
Avoiding Information Leakage in Backtesting with CPCV-Tuned Hyperparameters
I'm using Combinatorial Purged Cross-Validation to tune hyperparameters for a binary classification model applied in a month-end trading strategy. I have 6 months of data and used CPCV with 15 splits ...
- 1
0
votes
0
answers
12
views
Hyperparameters tuning and Backward feature selection : which one first?
If i have a lot of features and i want to train a light gbm, on the side i want to do the hyperparameter optimisation and on the other side i want to do backward feature selection to reduce the number ...
- 1
0
votes
0
answers
19
views
How to evaluate performance and optimize hyperparameters for clustering algorithms on a dataset with continuous labels?
I'm working on a clustering problem where my dataset's labels are continuous numerical values, not discrete categories. I'm using t-SNE and UMAP to reduce the dimensionality of my dataset's features ...
1
vote
0
answers
90
views
Hyperparameter tuning for small datasets
I have about 10 small imbalanced datasets (some of them only have about 150 samples). I want to try a bunch of balancing techniques on some models. For that, I'm using the repeated stratified cross-...
0
votes
0
answers
14
views
How to develop shared bottom tower serving different tasks
I have two model classes both pyramid architecture.
Let's say first task is predicting user will buy something with architecture [feature_embedding_128, dense_1048, dense_512, dense_128, dense_1]
...
- 314
0
votes
0
answers
11
views
Hyperparameter optimization for CNN
I have a database of defect images on materials, like holes, cuts, and so on. There is not so much information inside the images, I am aware of it.
I am using a CNN, in particular a ResNet50. I know ...
- 151
4
votes
1
answer
57
views
Binary decision boundary requiring 2 hidden layers in neural network with limited neurons
I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer (where number of neurons ...
- 141
0
votes
0
answers
13
views
Determine if the best output of a large number of model fits is due to overtraining
I have a machine learning model (e.g. Gradient Boost Regressor). I use hyperparameter tunning (e.g. random search) in a Cross Validation loop (e.g. 5 folds) in order to recover the optimal parameters ...
- 1
0
votes
0
answers
45
views
SMOTE and Sequential Feature Selection Order
Good morning,
I am doing the following procedure:
Split a Train a Test Dataset
...
0
votes
0
answers
66
views
Advice on Gaussian Process Classifier optimisation best practises? [duplicate]
Hyperparameter Range Determination: My main challenge is in setting effective ranges for hyperparameters such as length_scale, noise_level, and sigma_0. Currently, for length_scale, I've used the ...
