Covariance of An Empirical Distribution Function Evaluated at Different Points

Question

The problem is extracted from All of Statistics (Exercise 7.5), Larry Wasserman. I don't have a solution manual to the book so I post here the problem together with my attempted answer:

Let $x$ and $y$ be two distinct points. Find $Cov(\hat F_n(x), \hat F_n(y))$.

Here is my attempted answer:

$\hat F_n(x) = \frac{1}{n} \sum I\{ X_i \le x\} $

$\mathop{\mathbb{E}}(\hat F_n(x)) = F(x) $

$\mathop{\mathbb{E}}(\hat F_n(y)) = F(y) $

$Cov(\hat F_n(x), \hat F_n(y)) = \mathop{\mathbb{E}}(\hat F_n(x)\cdot \hat F_n(y)) - \mathop{\mathbb{E}}(F_n(X))\mathop{\mathbb{E}}(F_n(y) $

For (Updated based on Xi'an's answer) \begin{align*} \mathop{\mathbb{E}}(\hat F_n(x)\cdot \hat F_n(y)) &= \frac{1}{n^2} \mathop{\mathbb{E}}(\sum_i I\{X_i \le x\} \sum_j I\{X_j \le y\}) \\&= \frac{1}{n^2} \mathop{\mathbb{E}}(\sum_{i \neq j} I\{X_i \le x\}I\{X_j \le y\} + \sum_{i = j} I\{X_i \le x\}I\{X_j \le y\}) \\&= \frac{1}{n^2}(nF(\min\{x,y\})+n(n-1)F(x)F(y)) \\&= \frac{1}{n}(F(\min\{x,y\}) + (n-1)F(x)F(y))\end{align*}

Combining the above result together, we have (Updated based on Xi'an's answer):

$$ Cov(\hat F_n(x), \hat F_n(y)) = \frac{1}{n}(F(\min\{x,y\}) - F(x)F(y)) $$

I am not sure if my attempt is correct or not. Could anyone verify the answer or point out if there are any flaws in my arguments?

Answer 1

Note that \begin{align*}\text{Cov}(\frac{1}{n} \sum I\{ X_i \le x\},\frac{1}{n} \sum I\{ X_i \le y\}) &=\frac{1}{n^2}\text{Cov}(\sum I\{ X_i \le x\},\sum I\{ X_i \le y\})\\ &=\frac{1}{n^2}\sum_{i=1}^n\text{Cov}(I\{ X_i \le x\},I\{ X_i \le y\})\\ &\qquad\quad{\text{(since the $X_i$'s are independent)}}\\ &=\frac{1}{n}\text{Cov}(I\{ X_1 \le x\},I\{ X_1 \le y\})\end{align*} and \begin{align*} \mathop{\mathbb{E}}[I\{X_1 \le x\}I\{X_1 \le y\}] &= \mathop{\mathbb{E}}[I(X_1 \le \min\{x, y\})] \\&= F(\min\{x,y\}) \end{align*} leading to $$\text{Cov}(\hat F_n(x), \hat F_n(y)) = \frac{1}{n}[F(\min\{x,y\}) - F(x)F(y)]\tag{1}$$ When writing \begin{align*} \mathop{\mathbb{E}}(\hat F_n(x)\cdot \hat F_n(y)) &= \frac{1}{n^2} \mathop{\mathbb{E}}(\sum_i I\{X_i \le x\} \sum_j I\{X_j \le y\}) \\&= \frac{1}{n^2} \mathop{\mathbb{E}}(\underbrace{\sum_{i \neq j}}_{n(n-1)\\\text{distinct}\\\text{pairs}} I\{X_i \le x\}I\{X_j \le y\} + \sum_{i = j} I\{X_i \le x\}I\{X_j \le y\}) \\&\overbrace{=}^\text{wrong!} \frac{1}{n^2}(nF(\min\{x,y\})+nF(x)F(y))\end{align*} the mistake is in not counting the number of distinct pairs $i\ne j$ right: there are $n(n-1)$ of them, rather than $n$. With this correction, $$\mathop{\mathbb{E}}(\hat F_n(x)\cdot \hat F_n(y))=\frac{F(\min\{x,y\})}{n}+\frac{n-1}{n}F(x)F(y)-F(x)F(y)=\frac{1}{n}[F(\min\{x,y\}) - F(x)F(y)]$$ recovering (1). Here is an illustration of the fit between theory and empirical evaluation of $\text{Cov}(I\{ X_1 \le x\},I\{ X_1 \le y\})$

x=rnorm(1e6)
cov((x<a),(x<b))
pnorm(min(c(a,b)))-pnorm(a)*pnorm(b))

based on 10³ random pairs (a,b).