How to I find the distribution of $\log p(X)$ when $X$ is distributed under $p$?

Question

I have a feeling there's no general solution to this problem, but I'll ask anyway.

I have a multivariate PDF $p$ and, given a random vector $X\sim p$, I'd like to find the the PDF of $\log p(X)$.

For example, if I have a simple 2-dimensional Gaussian,

$$ p(x,y)=\frac{1}{2\pi}\exp\left(-\frac{1}{2}\left(x^2+y^2\right)\right), $$

I can write the distribution of the distance from the peak, $r$, as

$$ p_r(r)=r\exp\left(-\frac{1}{2}r^2\right) $$

and then change variables with $\log p = -\frac{1}{2}r^2 - \log 2\pi$. I find that the PDF of $\log p$ on $(-\infty,-\log 2\pi]$ is then

$$ p_{\log p}(\log p) = 2\pi\exp(\log p). $$

I can use a similar argument for higher-dimensional Gaussians, but is there a general way of getting this result for an arbitrary $p$, without relying on specific symmetries?

Answer 1

In dimension $2$, let $T=\log p(X)$ where $X=(X_1,X_2)$ has density $p$ and define a function $q$ by the implicit identity $\mathrm e^t=p(q(t,s),s)$. Then, for every measurable bounded $u$, $$ E[u(T)]=\iint u(\log p(x_1,x_2))p(x_1,x_2)\mathrm dx_1\mathrm dx_2. $$ Consider the change of variable $(t,s)=(\log p(x_1,x_2),x_2)$, then $(x_1,x_2)=(q(t,s),s)$, hence $\mathrm dx_1\mathrm dx_2=\partial_1q(t,s)\mathrm dt\mathrm ds$ and $$ E[u(T)]=\iint u(t)\mathrm e^t\partial_1q(t,s)\mathrm dt\mathrm ds. $$ Considering $D=\{\log p(x_1,x_2)\mid (x_1,x_2)\in\mathbb R^2,p(x_1,x_2)\ne0\}$, this implies that the distribution of $T$ has density $$ f_T(t)=\mathrm e^t\mathbf 1_{t\in D}\int\partial_1q(t,s)\mathrm ds. $$

---

Example: If $2\pi p(x_1,x_2)=\mathrm e^{-(x_1^2+x_2^2)/2}$, then $D=(-\infty,-\log(2\pi)]$ and $q(t,s)$ is implicitely defined by $q(t,s)^2+s^2+2t+2\log(2\pi)=0$ hence $\partial_1q(t,s)q(t,s)=-1$ and $$ f_T(t)=\mathrm e^t\mathbf 1_{t\leqslant-\log(2\pi)}\int\frac{2\mathrm ds}{\sqrt{-2\log(2\pi)-2t-s^2}}=2\pi\mathrm e^t\mathbf 1_{t\leqslant-\log(2\pi)}. $$ In other words, $T=-\log(2\pi)-U$ where the distribution of $U$ is standard exponential.

---

Likewise, in dimension $n$, using the shorthands $s=(t_2,\ldots,t_n)$ and $y=(x_2,\ldots,x_n)$, define a function $q$ by the implicit identity $\mathrm e^t=p(q(t,s),s)$.

Then the change of variable $(t,s)=(\log p(x_1,y),y)$ yields $(x_1,y)=(q(t,s),s)$, hence $\mathrm dx_1\mathrm dy=\partial_1q(t,s)\mathrm dt\mathrm ds$ and $$ E[u(\log p(X))]=\iint u(t)\mathrm e^t\partial_1q(t,s)\mathrm dt\mathrm ds. $$ Considering $D=\{\log p(x)\mid x\in\mathbb R^n,p(x)\ne0\}$, one sees that the distribution of $T$ has density $$ f_T(t)=\mathrm e^t\mathbf 1_{t\in D}\iint\partial_1q(t,s_2,\ldots,s_n)\mathrm ds_2\cdots\mathrm ds_n. $$

---

Example: If $p$ is the standard normal density on $\mathbb R^n$, then $D=(-\infty,-\frac12n\log(2\pi)]$ and $q(t,s)$ is implicitely defined by $q(t,s)^2+\|s\|^2+2t+n\log(2\pi)=0$ hence $\partial_1q(t,s)q(t,s)=-1$ and $$ f_T(t)=\mathrm e^t\mathbf 1_{t\leqslant-\frac12n\log(2\pi)}\iint\frac{2\mathrm ds}{\sqrt{-n\log(2\pi)-2t-\|s\|^2}}, $$ that is, introducing $t_n=-\frac12n\log(2\pi)$, $$ f_T(t)\propto\mathrm e^t(t_n-t)^{(n-2)/2}\mathbf 1_{t\leqslant t_n}. $$ In other words, $T=t_n-U$ where the distribution of $U$ is gamma $(\frac12n,1)$.

Answer 2

I think this is achieved as follows:

1. augment $\log(p)$ to form a vector of the same dimension of $X$ (call this vector $Y$);

2. partition the support of $Y$ into regions where $Y = g(X)$ is a one-to-one function of $X$ (hence invertible);

3. use the standard results on transformations of multivariate probability distributions (e.g., http://www.statlect.com/subon2/dstfun2.htm), which require to compute the Jacobian of $g^{-1}(y)$.