Questions tagged [nonparametric-regression]
Nonparametric regression is a form of regression analysis where the form of the functional dependence of the response on the predictors is not assumed. It subsumes many kinds of models, like spline models, kernel regression, gaussian process regression, regression trees or random forrests, and others.
71
questions
2
votes
1
answer
44
views
How to test whether a nonparametric function is equal to 0?
There is an unknown function $h(x)=E(Y|X=x)$ which I estimated with a nonparametric series estimator (also called sieve estimator) $\widehat{h}(x)$ using data $\{Y_i,X_i\}_{i=1}^n$. I'm interested in ...
- 2,966
0
votes
0
answers
12
views
nonparametric model for longitudinal data analysis
For the longitudinal data provided below, we have the following variables: the response variable 'y', the time variable 'week', 'grp' (with two levels: treatment and control), and 'subject'. My ...
- 1,173
0
votes
0
answers
10
views
Estimation of bivariate function with one variable being constricted
Suppose the following classical supervised regression setting,
$$y_{i} = f(x_{i}) + \epsilon_{i}, \quad i=1,\cdots,n,$$
where $\epsilon_{i}$ are i.i.d. zero mean Gaussian noise.
The above regression ...
- 11
5
votes
1
answer
232
views
Convergence rate of a nonparametric estimator
Optimal rate of convergence for a nonparametric estimator is well-known. This rate is derived for when we don't anything about functional form (expect perhaps degree of smoothness). Suppose we know ...
- 531
0
votes
0
answers
10
views
Variable independece in marginal integration estimator
This is an exercise of the textbook Nonparametric and Semiparametric Models by Wolfgang Hardle
Exercise 8.1. Assume that the regressor variable $X_1$ is independent from $X_2, \cdots, X_d$. How does ...
- 111
0
votes
0
answers
14
views
References: convergence rates of kernel regression, exchangeable data
I have been studying Kernel estimation; in particular, the Nadaraya-Watson estimator. I am interested in studying the rate of convergence in L^p of the NW (or similar) estimators for subgaussian ...
0
votes
0
answers
12
views
Where can I find analysis on the convergence rate of RMSE for various algorithms?
I am looking for convergence rates of the RMSE of various machine learning algorithms and conditions for them. Essentially, I would like to find something of the form
$$E[ |\hat{f}(x) - f(x)|^2] = o(1/...
1
vote
0
answers
69
views
Gaussian Process Regression prior with observations as integrals?
Consider some standard 1d Gaussian Process Regression (GPR). Suppose you are not happy with a typical mean-zero prior away from the data and you actually want something like the derivative of the mean ...
- 312
1
vote
0
answers
38
views
Maximum bias for NW estimator when $r(x)$ is Lipschitz (question 17, chapter 5 All of Non-Parametric Statistics)
The general condition is that $Y_i = r(X_i) + \epsilon_i$, and we want to estimate $r$ using Nadaraya–Watson kernel regression.
We additionally assume $r\colon [0,1] \to \mathbb{R}$ is lipschitz, so $|...
- 636
1
vote
0
answers
103
views
Is Synthetic Control Method a nonparametric estimator?
I'm studying causal inference and I'm struggling to understand how to properly classify an estimator as nonparametric. My colleague argued that the Synthetic Control Method is an example of a ...
- 439
3
votes
0
answers
479
views
Pros and cons of Nadaraya–Watson estimator vs. RKHS method?
Recently I've been reading some materials about nonparametric methods. Two methods related to the word "kernel" rasied my interest-- Nadaraya–Watson estimator and RKHS method.
What's the ...
- 31
2
votes
1
answer
335
views
How well do Multivariate Adaptive Regression Splines work in high dimensional settings?
I have been reading the Hastie and Tibshirani book again lately, and I noticed in Chapter 9 that the mention the MARS algorithm: Multivariate Adaptive Regression Splines, which is a nonparametric ...
- 1,522
4
votes
1
answer
438
views
Calculating local variance
I have some data, and I assume it can be modelled by $y_i = f(x_i) + \epsilon $, where $\epsilon \sim \mathcal{N}(0,\sigma_0^2)$ where $f$ and $\sigma_0^2$ are unknown. I understand that I can ...
- 327
2
votes
1
answer
164
views
Method of Sieves with Data Driven Basis Functions
Consider a nonparametric regression problem with i.i.d. sampled data $(y_1,x_1), (y_2,x_2),\ldots, (y_n,x_n)$ and regression function
$$y_i = g_0(x_i) + \varepsilon_i,\quad \mathbb E[\varepsilon_i | ...
- 2,465
1
vote
1
answer
29
views
Rates of convergence with asymptotically negligibly noisy observations
Apologies in advance if this question is not completely well defined. Suppose that I am estimating a nonparametric model for a conditional expectation function $\mathbb E[Y_i | X_i]$ using some i.i.d. ...
- 2,465