2
$\begingroup$

My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is derived as followed: \begin{align} var(\hat{f}_n(x))&=var\Big(\frac{1}{nh}\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{n^2h^2}var\Big(\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{nh^2}var\Big(k(\frac{X_1-x}{h})\Big)\\ &=\frac{1}{nh^2}(E(k(\frac{X_i-x}{h})^2)-E(k(\frac{X_1-x}{h}))^2)\\ &=\frac{1}{nh^2}\Big(h\int f(x+h*u) k^2(u)du-(h\int f(x+hu)*k(u)du)^2\Big)\\ &=\frac{1}{nh^2}\Big(h\int (f(x)+f^{(1)}(x)hu) k^2(u)du-O(h^2)\Big)\\ &=\frac{1}{nh}\Big(f(x)\int k^2(u)du+O(h\int|u|k^2(u)du)-O(h)\Big)\\ &=\frac{1}{nh}(\kappa f(x)+O(h)) \end{align} , here k is a kernel function with classical assumptions, $X_i,x_1$ realizations, f the true density, h a bandwidth and n the sample size, besides $\kappa=\int k^2(u)du$. What I cannot understand are the last three equalities, i.e. why $\int f^{(1)}(x)hu*k^2(u)du$ results in the bounded term with $O(h\int|u|k^2(u)du)$. The boundedness is obvious since the first derivative is some constant at given x.

  • How does one obtain the particular value for the Big O upper bound (especially in the form where the absolute value of u is used)?
  • And how are the two Big O terms subtracted from each other to obtain the final equality with O(h)?

I appreciate any help!

Improve this question
$\endgroup$
1
  • $\begingroup$ If $ \int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ is assumed to be finite, then $O(h\int|u|k^2(u)du)$ should follow from the triangle inequality for integrals, right? In a textbook of mine, $\mathrm{Var}[k(X)]$ is assumed to be finite, which would imply $ \int_{\mathbb{R}}k^2(x)f(x)\mathrm{d}x$ to be finite, where $f$ is the density of $X$. But I'm unsure if this would also imply $ \int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ to be finite. $\endgroup$
    – psie
    Commented Jul 1, 2021 at 11:13

1 Answer 1

Reset to default
0
$\begingroup$

The $O(\cdot)$ notation can hide away constants and is a slight abuse of notation (you will see people occasionally use set notation instead). Note that for an expression, $a(h)$, we say it is $O(h)$ if there exists a constant $C>0$ such that: $$ |a(h)| \leq C \cdot h \text{ for all } h > 0$$ There are more precise definitions, but this suffices for what you need to prove in this exercise.

So now let us check, what does it mean to say $O(h) - O(h) = O(h)$? The LHS $O(h)$ terms correspond to a specific expression that got summarized earlier on, let us call it $a_1(h) = O(h)$ and the second one $a_2(h) = O(h)$. So we need to show that $a_1(h) - a_2(h) = O(h)$. Take $C_1,C_2$ such that $|a_1(h)| \leq C_1 h$ and $|a_2(h)| \leq C_2 h$ , then:

$$ |a_1(h) - a_2(h)| \leq |a_1(h) + a_2(h)| \leq (C_1 + C_2)h \text { for all }h > 0 $$

So indeed $a_1(h) - a_2(h) = O(h)$, i.e., $O(h) - O(h) = O(h)$. Pause a moment to note that the LHS has a different meaning in this expression than the RHS.

Similarly $a(h) = O(h^2)$ means that:

$$ |a(h)| \leq C \cdot h^2 \text{ for all } h > 0$$

One final remark: Li & Racine derive asymptotics for small $h>0$, so that in all of the above expressions it suffices to check that the $O(\cdot)$ conditions hold for small $h$, say for $0 < h < H$ where $H$ is a small constant.

Can you verify that:

  • $O(h^2) = O(h)$?
  • The other questions you asked?
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks a lot, that makes sense! $O(h^2)$ is therefore $O(h)$ since there is some $C_2>0$ s.t. $|a(h)|\leq C_1*h^2\leq C_2 *h$. Can you elaborate why the integral can be written as $\int|u|k^2(u)du$? The indefinite integral should be zero s.t. the term vanishes by the kernel properties, however using the absolute $|u|$ this will not be the case. $\endgroup$
    – Henry
    Commented May 18, 2020 at 7:47
  • 1
    $\begingroup$ My guess, since the term is symmetric, and $O(h*\int^\infty_{0}vk^2(u)du)+O(h*\int^0_{-\infty}vk^2(u)du))=O(h)$, you simply rewrite the term using the absolutes since the $O(h)$ notation omits the constants anyways? $\endgroup$
    – Henry
    Commented May 18, 2020 at 8:02
  • $\begingroup$ Yes! You will be treating this part as a constant, so it's just an intermediate step putting in the absolute value inside (since I guess the assumptions will have been stated in terms of that quantity). $\endgroup$
    – air
    Commented May 18, 2020 at 8:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.