Difficulty understanding the conditional expectation

Question

I'm currently having a confusion dealing with the conditional expectation. Let's recall the definition first:

Let $X$ be an integrable function (or a random variable) defined on a probability space $(\Omega,\mathfrak{M},\mu)$ and let $\mathfrak{A}$ be a $\sigma$-subalgebra of $\mathfrak{M}$. The conditional expectation of $X$ on $\mathfrak{A}$ is then defined to be an $\mathfrak{A}$-measurable function $\mathbb{E}(X|\mathfrak{A})$ which satisfies the identity $$\int_AXd\mu=\int_A\mathbb{E}(X|\mathfrak{A})d\mu $$ for any $A\in\mathfrak{A}$. The existence and uniqueness follow from the Radon-Nikodym theorem. Two special cases are as follows:

(1) If $X=\chi_{\omega}$ for some $\omega\in\mathfrak{M}$, $\mathbb{E}(\chi_{\omega}|\mathfrak{A})$ is called the conditional probability of $\omega$ (relative to $\mathfrak{A}$).

(2) If $\mathfrak{A}$ is the $\sigma$-algebra generated by a function $Y$, we write $\mathbb{E}(X|\mathfrak{A})=\mathbb{E}(X|Y)$.

Q1. Why is the conditional "probability" a function, not a number?

Q2. What is the intuitive meaning of $\mathbb{E}(X|Y)$?

As for Q2, I think that $\mathbb{E}(X|Y)$ should mean an expectation of $X$ provided that $Y$ happened (as in elementary probability theory); for example, the symbol $\mathbb{E}(X|Y=y)$ should indicate an expectation of $X$ when $Y=y$. However, I found it hard to connect this idea and the above definition. I'm not even 100% sure about this; anyway, $\mathbb{E}(X|Y)$ is a function, not an expectation (which is a number).

I'm familiar with measure theory, but not with probability theory. So any help would be appreciated! Thank you.

Answer 1

First, since conditional probabilities are conditional expectations of indicator functions, you can forget about that concept as a separate thing.

Moving on: for intuition, it is helpful to think about finite probability spaces with no nonempty null sets. This makes some statements that are "morally true" in the general case be literally true.

Consider for example $\Omega=\{ 1,2,3 \}$ with the underlying $\sigma$-algebra being $\mathcal{F}=P(\Omega)$ and then think about $\mathcal{G}=\sigma(\{ \{ 1 \} \})=\{ \Omega,\emptyset,\{ 1 \},\{ 2,3 \} \}$. Say $P(\{ i \})=p_i>0$ for all $i$.

Conditional expectation of some rv $X$ with respect to this $\mathcal{G}$ intuitively means you have an instrument that can distinguish whether $\omega=1$ or not, but it can't tell you whether $\omega=2$ or $\omega=3$. Then you're asked to use your instrument to give a "best guess" of $X$ for each $\omega$. This will depend on $\omega$: if $\omega=1$ then it will be $X(1)$ but otherwise it should be somewhere between $X(2)$ and $X(3)$. The measurability requirement forces $E[X \mid \mathcal{G}](2)=E[X \mid \mathcal{G}](3)$. Then the identity tells you $p_2 X(2) + p_3 X(3) = (p_2 + p_3) E[X \mid \mathcal{G}](2)=(p_2 + p_3)E[X \mid \mathcal{G}](3)$, so they are both $\frac{p_2 X(2) + p_3 X(3)}{p_2 + p_3}$.

As for $E[X \mid Y]$, again in the setting of a finite space with no nonempty null sets, you have that $\sigma(Y)$ is the minimal $\sigma$-algebra containing all the preimages of the singletons containing each element of the range of $Y$. Therefore in this setting, $E[X \mid Y](\omega)$ is the same as $E[X \mid Y=Y(\omega)]$ in the sense you have seen previously. Conditional expectation is "the right generalization" of this concept to the situation where $Y=c$ can be nonempty but null.