3
$\begingroup$

Can someone explain this proof to me? I get stuck on the transition from the third line to the last line. Namely:

  • Is the integral being evaluated or not?
  • How does the entire expression reduce to a single Beta function? (I get that this is the idea behind conjugacy, but I'm not "seeing" it.)

Beta-Binomial conjugacy proof

Improve this question
$\endgroup$
1
  • $\begingroup$ It's just notation: nothing is being evaluated. $\endgroup$
    – whuber ♦
    Commented Jul 7, 2020 at 14:44

1 Answer 1

Reset to default
4
$\begingroup$

The key thing is to focus on only those parts in the formula in the third line that contain $\theta$ as a free variable, and to disregard everything that does not depend on $\theta$ any more - because the latter parts are just a multiplicative constant that ensures your probability integrates to one. Then we get an expression of $P(\theta|x)$ as proportional to something:

$$ P(\theta|x) \propto \theta^{x+\alpha-1}(1-\theta)^{n-x+\beta-1}. $$

(Note that the integral in the denominator integrates $\theta$ out, so it again doesn't depend on $\theta$ any more.)

And now we compare this to the PDF of the beta distribution, again up to multiplicative constants - and we see that $P(\theta|x)$ is proportional to the PDF of a $\text{Beta}(x+\alpha,n-x+\beta)$.

So we now have your posterior density $P(\theta|x)$ and the $\text{Beta}(x+\alpha,n-x+\beta)$ density. Both are probability densities, so they both integrate to one. And we just found that they are proportional to each other. But two functions that are proportional to each other and integrate to the same value must be equal.

This way of thinking and arguing is extremely common in Bayesian statistics, especially when looking at conjugates. Think about it and get familiar with it, you will definitely see it again.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the info! I'm not very familiar with CDFs and so I think I have some supplemental reading laid out for me :) $\endgroup$
    – jbuddy_13
    Commented Jul 7, 2020 at 14:54
  • 1
    $\begingroup$ You're welcome. And I should have written about PDFs, not CDFs... Changed now. $\endgroup$
    – Stephan Kolassa
    Commented Jul 7, 2020 at 14:55
  • $\begingroup$ I think I'm close to some key insight; where I'm stuck now is seeing how the integral in the denominator evaluates to B(x+a, n-x+B). I'll give this some thought! $\endgroup$
    – jbuddy_13
    Commented Jul 7, 2020 at 16:04
  • 1
    $\begingroup$ On the one hand, the integral in the denominator is irrelevant, it's only a constant scaling factor, per my answer. On the other hand, the integrand is precisely $B(x+\alpha,n-x+\beta)$ (the beta function evaluated for this pair of arguments) times the PDF of a $\text{Beta}(x+\alpha,n-x+\beta)$ distribution. Since PDFs integrate to $1$, the integral evaluates as $B(x+\alpha,n-x+\beta)$. (Sorry for any confusion between the beta function and the beta distribution.) $\endgroup$
    – Stephan Kolassa
    Commented Jul 7, 2020 at 16:08
  • $\begingroup$ Ah! I see; I wasn't aware that Beta pdf and Euler's Beta functions were different, but related concepts. Crystal clear now; thanks for helping me make the connection :) $\endgroup$
    – jbuddy_13
    Commented Jul 7, 2020 at 16:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.