7
$\begingroup$

Consider kernel regression estimation of the mean function $m$ of the process

$$y_t = m(x_t) + \epsilon_t,$$ where $\epsilon_t$' s are correlated with covariance function $R(s,t) = \exp \{-\lambda|s-t|\}$. In my scenario, $x_t$ is a function of a parameter $\alpha$ (for example $\alpha t$) and $R(s,t)$ is a function of $\lambda$.

In a situation where $\alpha$ and $\lambda$ are known, we minimize the mean integrated squared error (MISE) to find an appropriate $h$.

My question is: In my scenario, to find the smoothing parameter $h$ and estimate $\alpha$ and $\lambda$, can I minimize the MISE simultaneously with respect to $h$, $\alpha$, and $\lambda$? Will the minimizer of $\alpha$ and $\lambda$ be consistent (maybe under some additional conditions)?

Another option is: if I minimize MISE with respect to $h$ for $\alpha$ and $ \lambda$ fixed, put back the value of $h$ as an expression of $\alpha$ and $\lambda$ in MISE, and finally minimize it with respect to $\alpha$ and $\lambda$ simultaneously, will the estimators of $\alpha$ and $\lambda$ be consistent (maybe under some additional conditions)?

Or in another way, if I use an iterative algorithm: fix $\alpha = \alpha_0, \lambda = \lambda_0$ to find $h=h_0$, put the value of $h=h_0$ in MISE, minimize it with respect to $\alpha, \lambda$ to get $\alpha = \alpha_1, \lambda = \lambda_1$ and so on, how can I prove theoretically that the algorithm converges?

Note 1: By MISE I mean, $\int\limits_0^1 E \{\hat{m} (x_t) - m(x_t)\}^2 dt.$ I asked the question in a very broad scenario. In data application, all the terms involved in MISE may not be known and we may need to express the MISE by $\frac1n \sum\limits_{j=1}^n E\{\hat{m} (x_{t_j}) - m(x_{t_j})\}^2$, estimate it in terms of residual sum of squares, RSS$=\frac1n \sum\limits_{j=1}^n \{y_{t_j} - m(x_{t_j})\}^2$, and covariances. For details, one may see equation (14) in this pdf.

Note 2: By consistency, I mean the standard definition of it. That is, the estimate will converge to the true parameter in probability.

Any suggestions and/or related articles will be greatly appreciated!

Improve this question
$\endgroup$
6
  • $\begingroup$ A couple questions 1) What does this mean exactly: "$x_t$ is a function of a parameter $\alpha$" 2) MISE suggests you're integrating over some probability density function, but what is it? Can you describe the data you're fitting? $\endgroup$
    – user20160
    Commented Aug 25, 2019 at 1:47
  • $\begingroup$ Please define consistency here. Are you observing 1 trajectory of y(t) over an interval [0,T] of increasing length, or are you observing more and more independent trajectories of y(t) over interval [0,1]?....... Is h() a parameter of the kernel in the kernel regression? Term "smoothing parameter" is too broad. $\endgroup$
    – stans
    Commented Aug 25, 2019 at 8:23
  • $\begingroup$ @user20160 Thanks for your comments. Now I have addressed both of your questions. $\endgroup$
    – Shanks
    Commented Aug 25, 2019 at 8:35
  • $\begingroup$ @stans Thanks to your comment, I have added a clarification. I have only one trajectory over an interval [0,1] at time points $\alpha t_1,\ldots, \alpha t_n$. Yes, $h$ is the parameter of the kernel in the kernel regression. $\endgroup$
    – Shanks
    Commented Aug 25, 2019 at 8:43
  • $\begingroup$ Another question: $\epsilon$ doesn't appear in the expression for the MISE, so $\lambda$ doesn't affect the MISE. What does it mean then to minimize the MISE w.r.t. $\lambda$? How are you proposing to estimate $\lambda$? $\endgroup$
    – user20160
    Commented Aug 27, 2019 at 19:10

0

Reset to default