1
$\begingroup$

If $X$ and $Y$ are random variables distributed as Poisson and binomial with parameters $\lambda$ and ($n$, $p$) respectively, then according to Casella and Berger (of Statistical Inference)

enter image description here

Could you please explain:

  1. How can Poisson probabilities of $X = x$ can be done quickly when we has to calculate probability of $X = x - 1$ first (which is essentially the same as calculate probability of $X = x$)?

  2. Why can the terms $p(y - 1)$ and $py$ be ignored if $p$ is small? What is the value of $p$ to be considered as small? Is there a concrete proof/basis that is the case for those terms to be ignored if $p$ small?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Imagine you had to compute a collection of Poisson probabilities for $x$ in some interval ($x=k, k+1, ..., L$). Then the recursion relationship in (1) would greatly speed the calculation over doing them all indepenently $\endgroup$
    – Glen_b
    Commented Jan 20, 2020 at 1:31
  • $\begingroup$ Oh, I see. Thanks, @Glen_b-ReinstateMonica. $\endgroup$
    – Nemo
    Commented Jan 20, 2020 at 1:42
  • 1
    $\begingroup$ For a derivation of this relation from first principles, see stats.stackexchange.com/questions/214421. For a rigorous account of the approximation see stats.stackexchange.com/questions/167212. $\endgroup$
    – whuber ♦
    Commented Jan 20, 2020 at 18:07

1 Answer 1

Reset to default
1
$\begingroup$

Adding (or subtracting) a small number to (or from) a not-small number results in a number that is roughly the same as the not-small number, hence in many situations can be skipped, meaning the small number can be ignored. As the small number gets even smaller, the approximation obtained by ignoring it and just using the not-small number gets better and better.

In any case, the point of the math isn't that there is some magic number $c$ where everything suddenly works if $p < c$, it's that as $p$ gets smaller, the error in ignoring the two terms referred to goes to zero, and you are left with equation 3.2.8.

Edit (in response to comments):

The approximation of the Binomial by the Poisson, and the rapid calculation of Poisson probabilities using the recursion described in the question, are not typically of computational importance, as computers are more than fast enough to handle the calculations involved in all but extreme cases (which may exist only in my imagination.) Even when building generalized linear models, if you really want to, you can use a log link with the Binomial distribution (the default link for a Poisson distribution) - admittedly risking an invalid $p\geq 1$ at some step of the iteration, which shouldn't occur if $p$ really is "small". Consequently, there's usually little to be gained by using the Poisson in this type of application.

From a modeling perspective, it's going to be application-specific; in most cases, our models are only approximate anyway, and a little extra approximation of the Binomial by the Poisson may well be swamped by the initial approximation of the data generating process by a Binomial. I personally almost never run into situations where I have Binomial variates that I'd like to model, and insofar as the Poisson is concerned, I can only remember one situation in many years of applied work where the Poisson itself was a good choice, w/o any noticeable overdispersion, as a likelihood function in a generalized model. So I can't give any rules of thumb, at least not ones where someone wouldn't be able to provide a counterexample.

From a mathematical perspective, the approximation of the Binomial by the Poisson is a simple, and good, example of a technique of "ignoring (relatively) small numbers that go to zero anyway" that gets used quite a lot, so provides a good introduction to the ideas of limiting distributions and simplifying approximations and how the associated math works.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, @jbowman. I accept that the terms could be ignored if probability $p$ got smaller. But if there wasn't a magic value $c$ as allured to in your answer (on the other hand there was the recommended cutoff value 0.05 for statistically significant test) where the approximation was valid, then how could I make the decision when to use the Poisson instead of binomial? $\endgroup$
    – Nemo
    Commented Jan 20, 2020 at 3:09
  • $\begingroup$ Great exposition! Thanks, @jbowman $\endgroup$
    – Nemo
    Commented Jan 20, 2020 at 23:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.