General rule for finding out a probability distribution

Question

I am trying to learn about how to find the probability distribution for functions of random variables in general.

Suppose we have a random variable $X$ with pdf $f_X$ and a random variable $Y$ with pdf $f_Y$. Assuming there is no correlation between them, the following properties tend to hold:

• If Z = X + Y, then (convolution rule):

$$f_Z(z) = (f_X * f_Y)(z) = \int_{-\infty}^{\infty} f_X(x) f_Y(z - x) dx$$

• If we define $Y = g(X)$ as a new variable, then (change of variable rule):

$$f_Y(y) = f_X(g^{-1}(y)) \left|\frac{d}{dy}g^{-1}(y)\right|$$

My Question: Can the above rules be applied to any situation?

As an example, I wanted to find out the pdf of $X/Y$. I was not sure if the above rules could be applied to this problem.

I tried to work on this problem using general logic without knowledge of the above rules:

1. Define Z:

   $$Z = \frac{X}{Y}$$

2. Write cumulative distribution function (CDF) of Z:

   $$F_Z(z) = P(Z \leq z) = P\left(\frac{X}{Y} \leq z\right)$$

3. Split into two cases based on the sign of Y (i.e. a piecewised approach):

   $$F_Z(z) = P(X \leq zY, Y > 0) + P(X \geq zY, Y < 0)$$

4. Express this using the joint PDF of X and Y:

   $$F_Z(z) = \int_{0}^{\infty} \int_{-\infty}^{zy} f_{X,Y}(x,y) dx dy + \int_{-\infty}^{0} \int_{zy}^{\infty} f_{X,Y}(x,y) dx dy$$

5. differentiate with respect to z (pdf-cdf rule):

$$\begin{align} f_Z(z) &= \frac{d}{dz}F_Z(z) \\ &= \int_{0}^{\infty} y f_{X,Y}(zy,y) dy - \int_{-\infty}^{0} y f_{X,Y}(zy,y) dy \end{align}$$

6. Combine integrals:

   $$f_Z(z) = \int_{-\infty}^{\infty} |y| f_{X,Y}(zy,y) dy$$

7. Since X and Y are independent:

$$f_Z(z) = \int_{-\infty}^{\infty} |y| f_X(zy) f_Y(y) dy$$

Have I done this correctly? Is it possible to apply the two rules I mentioned earlier in this problem? Or is another approach needed?

Thanks!

Answer 1

In my experience, there is in fact only one quite general rule and that is that you need to work with the CDF.

Both convolution and change of variables can be derived from CDF manipulations for continous distributions. However, the CDF approach is more general - as you illustrate with your attempt. The CDF approach also works when the PDF does not exist (e.g. discrete variables) or when the transformation is not nicely behaved or when the variables are not independent.

An important conceptual step is to distinguish between the distribution itself (which is usually understood in terms of measure theory these days) and various descriptions of the distribution (CDF, PDF, quantile function, random number generator, moment generating function](https://en.wikipedia.org/wiki/Moment-generating_function), ...). Each description makes some operations with the distribution easy, while others are hard.

So "finding a distribution" is a different task than "finding a density function" - e.g. the moment-generating function (MGF) of a sum of two independent variables is just the product of their individual MGFs - so we have found the distribution of the sum! But going from an MGF to a density function is usually hard.

Also, as @Masacroso pointed out in the comments, you need independence, not just lack of correlation for the convolution rule (source)

Answer 2

I agree with @Martin Modrák in that the CDF approach is quite general, and with @Masacroso in that the convolution approach may not be of sufficient generality outside the independent world.

But in many easier cases of practical/applied importance, the transformation / Jacobian approach works fairly well if a pdf exists:

In your specific example, this method still works by defining a bivariate 1-1 transformation $g$: $$U:=\left(\begin{array}{c} U_1\\ U_2 \end{array}\right) := g(X,Y):=\left( \begin{array}{c} \frac{X}{Y}\\ X \end{array} \right). $$ Then you can find the pdf of this bivariate $U$ by transformation / Jacobian rules and integrate out the $U_2$ component from the joint pdf to get the pdf of $U_1:=Z$.

Slightly more generally, if the wanted transformation is not 1-1, we may try to break it up into 1-1 pieces, apply the transformation rule to each piece, and glue them back together. Most standard textbooks will introduce such an approach at some point (e.g., at the end of Section 4.6 of Casella & Berger's text).