I have my training data for binary classification that consists of $N$ pairs $$(x_i\in R^F, y_i \in {-1, 1})$$ $i\in [1,\dots,N]$. My classification rule of a new point $x$ is simply
$$ \hat{y}(x) = \begin{cases} 1 & 0 < \frac{\sum_i y_i\phi(|x_i-x|/r)}{\sum_i \phi(|x_i - x|/r)} \\\\ -1 & otherwise \end{cases} $$ where $\phi(z) = \exp(-z^2/2)$ and $r = \alpha N^{\frac {1} {2F}} $.
I'm unable to find any reference regarding learning guarantees, even if this seems a pretty standard approach.
Bishop writes
It can be shown that both the K-nearest-neighbour density estimator and the kernel density estimator converge to the true probability density in the limit N → ∞ provided V shrinks suitably with N, and K grows with N (Duda and Hart, 1973).
but this is for density estimation and I couldn't find the reference.
