All Questions
Tagged with nonparametric machine-learning
54
questions
4
votes
1
answer
52
views
In what ways is Gaussian Process Regression both parametric and non-parametric?
Gaussian Process Regression is considered a "non-parametric" model. However, the term "non-parametric" is often used imprecisely to mean different things, leading to questions ...
- 847
0
votes
0
answers
4
views
learning guarantees for gaussian weighting of training points
I have my training data for binary classification that consists of $N$ pairs $$(x_i\in R^F, y_i \in {-1, 1})$$ $i\in [1,\dots,N]$. My classification rule of a new point $x$ is simply
$$
\hat{y}(x) = \...
1
vote
0
answers
171
views
Projection pursuit regression
Projection pursuit regression (PPR) is described in Hastie et al.'s The Elements of Statistical Learning in the chapter on neural networks. The algorithm was introduced by Friedman and Stuetzle (1981)....
- 751
5
votes
2
answers
549
views
Is density estimation the same as parameter estimation?
I was studying parameter estimation from Sheldon Ross' probability and statistics book. Here the task of parameter estimation is described as follows:
Is this task the same of density estimation in ...
- 151
4
votes
3
answers
479
views
Perfect Prediction: Why Would We Ever Use a Statistical Model?
Dear statistics experts
I need your help with something that has bothered me for a while now. My problem revolves around perfect prediction and essentially boils down to:
Why would we ever set up and ...
- 69
1
vote
0
answers
39
views
validation and calibration of crop yield data using conditional inference trees
I am trying to validate and calibrate the conditional inference tree model using the crop yield data, and I started by splitting my dataset into training and test sets.
After splitting, I had to ...
- 11
3
votes
0
answers
66
views
Looking for the Holy Grail of nonparametric regression
Unfortunately, to state precisely the question, I need some formal preliminaries.
Let $d \in \mathbb{N}$.
For each $d^* \in \{1,\dots,d\}$, define $\mathcal{M}_{d^*}$ be the set of probability ...
- 193
1
vote
0
answers
159
views
which non parametric test to use for anomalous NN model outputs
Assume I have a bunch of trained NN models for classifying MNIST. All of them except one was trained on the same training set while the one was trianed on a different training set (could have ...
- 403
1
vote
0
answers
436
views
AIPW and Cross-fitting (Stanford stat361)
I am reading lecture note (Stanford stat361: https://web.stanford.edu/~swager/stats361.pdf) written by Stefan Wager. At page 23-24 the author states dependent summands become independent after ...
- 39
3
votes
1
answer
215
views
Random forest with nonnegative dependent variable
I have a modeling framework with an outcome that must necessarily be positive. In the training data, the outcome ranges from close to zero to much higher (approximately 0.05 to 100). Is there a way to ...
- 725
5
votes
1
answer
2k
views
Is it possible to use variational autoencoders with Non-Gaussian data?
I am dealing with two scenarios: 1) Non-Gaussian data distribution and 2) non-stationary data).
First, I am planning to use a variational autoencoder for modeling the probability distribution of the ...
- 333
1
vote
0
answers
135
views
Extraction of modes from a multi-modal density function
I am trying to extract modes from a multi-modal density function and not just peaks. For example, in the two density functions below (images), I would like to extract the curves contained in the black ...
- 333
3
votes
1
answer
337
views
What is the difference between sieve estimation and structural risk minimization?
I was wondering if you could help me out. I am quite confused about the difference between sieve estimators (Ulf Grenander) and structural risk minimization (SRM) (Vladimir Vapnik). Could anyone give ...
- 131
1
vote
0
answers
888
views
what are the main differences between parametric and non-parametric machine learning algorithms?
I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
- 11
2
votes
1
answer
39
views
Why might the functional form of a distribution be "inappropriate" for a particular application?
Working through Bishop's Pattern Recognition and Machine Learning(a great read so far!) and on page 67 he says:
"One limitation of the parametric approach is that it assumes a specific ...
- 1,617
4
votes
0
answers
442
views
Derivation of k nearest neighbor classification rule
One way to derive the k-NN decision rule based on the k-NN density estimation goes as follows:
given $k$ the number of neighbors, $k_i$ the number of neighbors of class $i$ in the bucket, $N$ the ...
- 426
1
vote
2
answers
451
views
1
vote
1
answer
258
views
Does a non-parametric model necessarily have zero bias?
For a parametric model like linear regression, the bias is often interpreted as "the parameters & architecture you chose are inappropriate for the shape of this dataset".
For (one ...
- 1,061
2
votes
1
answer
56
views
Quantifying importance of a parameter in neural networks' prediction
Say I'm given a neural network, parameterized by a $d$-dimensional vector $\theta$, and an input $x$.
Given the prediction of this model $f_{\theta}(x)$, can I somehow quantify importance of each of $...
- 213
1
vote
1
answer
50
views
What are the implications of a nonparametric machine learning algorithm?
I've been looking into the advantages of using a Random Forest classifier and stumbled upon this
random forests are non-parametric
Looking at the definition of what non-parametric statistics mean, ...
- 111
4
votes
1
answer
1k
views
Can someone explain why neural networks are highly parameterized?
I understand that neural networks by definition, are a parametric model.
If I am correct, Parametric methods make an assumption about the functional form, or shape, of f. For a neural network, what ...
- 71
1
vote
1
answer
353
views
How Parzen window density estimate $f_n$ converges to f
I am trying to understand how Parzen window density estimate converges to actual density function f(x).[Actually i am trying to learn machine learning on my own using available free resources. Please ...
0
votes
0
answers
14
views
Doubt in kernel based method - unit hypercube(Parzan window estimate)
I recently started studying pattern recognition on my own. Please clarify me the following.
https://books.google.co.in/books?id=T0S0BgAAQBAJ&pg=PA53&lpg=PA53&dq=hypercube+of+side+h&...
1
vote
0
answers
50
views
Why mixture model with Gibbs sampling works?
I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value?
That is to say,we can sample z by:
\begin{align}
p(z_i=k \,|\, \cdot)
&\...
- 131
2
votes
0
answers
327
views
Is kernalized linear regression parametric or nonparametric?
We know that for linear regression, we can predict:
$$\hat{y} = w^Tx +b$$
Where $w$ is the parameter that minimizes the square loss.
It is easy to prove that for the final solution using gradient ...
- 21
1
vote
0
answers
23
views
Why is "consistent nearest neighbour" Non-parametric? [duplicate]
Definition of "Consistent nearest neighbour", runs our usual KNN classifier
but instead of viewing k as a hyper-parameter it always sets k = ceil[log(n)].
So far, I looked-up many references and ...
- 195
3
votes
0
answers
205
views
Parametric vs non-parametric machine learning methods [duplicate]
I looked-up many references and websites and researched on how to determine if a method is between parametric or non-parametric. I came up with below definitions,
A parametric algorithm has a fixed ...
- 195
11
votes
1
answer
514
views
Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?
The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
1
vote
1
answer
986
views
Estimating conditional probability with many samples
I am confused about the estimation of conditional probabilities. Suppose I want to predict a binary outcome variable $Y = 0,1$ given $n$ categorical features $X = (X_1, \ldots, X_n)$, i.e. to ...
- 63
0
votes
1
answer
184
views
How to know which two hyperparameters are more important in SVM, KNN and MLP?
I am trying to limit myself to a maximum two hyper-parameters that are important in KNN, SVM and ...
- 5