0
$\begingroup$

Suppose the following classical supervised regression setting, $$y_{i} = f(x_{i}) + \epsilon_{i}, \quad i=1,\cdots,n,$$ where $\epsilon_{i}$ are i.i.d. zero mean Gaussian noise.

The above regression problem can be solved by classical non-parametric approaches, e.g. kernel method.

However, if I suppose that I have a pre-determined (or pre-trained) model $h(x)$, such that $$f(x) = g(x,h(x)).$$ Here, the $g$ is a bivariate function but with the second coordinate restricted in the form of $h(x)$.

When none of the variables is restricted, one can apply the kernel method (like kernel ridge regression) to obtain the estimate $\hat{g}$ in the tensor space of the RKHS. However, I'm wondering if there is any existing literature/research for estimating $g$ given $g$ in such a restricted form.

Improve this question
$\endgroup$
3
  • $\begingroup$ Is there some restriction on $g$? If it's unrestricted, I don't see how learning $g(x, h(x))$ is any different than learning $f(x)$. Or is the function supposed to be $g(x_1, h(x_2))$? $\endgroup$
    – Cliff AB
    Commented Mar 16 at 4:23
  • $\begingroup$ Thanks for your comment. No further restrictions. The general idea is that if $h(x)$ is similar to $f(x)$, then incorporating a learned $h(x)$ in estimating $f(x)$ (thus $g(x,h(x))$) will be better than directly learning $f(x)$ in terms of convergence rate, etc. This is the reason I impose the relationship $f(x) = g(x,h(x))$. There are solutions when $g(x,h(x))$ takes some specific form like additive form. However, here I'm targeting a more general $g(x,h(x))$ which I don't specify its form but a general function (maybe in some functional space like RKHS) induced by $h(x)$. $\endgroup$
    – DoubleL
    Commented Mar 16 at 5:46
  • 1
    $\begingroup$ Without further restrictions on the relation between $g$ and $h$, I don't see how this could be different than just learning $g(x)$ directly? $\endgroup$
    – Cliff AB
    Commented Mar 16 at 16:48

0

Reset to default