Suppose we have a two dimensional parameter $\theta=(\mu,\sigma^2)$, and a prior distribution $p(\theta)$. Let our sample come from a normal distribution with mean $\mu$ and variance $\sigma^2$. The prior is such that the posterior is improper, i.e. $$\int p(\theta)p(x|\theta)\;d\theta=\infty.$$ For a sample size larger than one, I am convinced that the posterior remains improper, but are there cases where the posterior is proper for larger samples?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
The posterior is (generally) going to be a function of the sample size. One way of determining propriety of the posterior is to check the same size for which the parameters in the posterior are not defined. The example below is one such construction.
$X_1, \dots, X_n \sim N(\theta, \sigma^2)$. For simplicity, let $\theta = 0$ be fixed (you could put a normal prior on $\theta$) and let $\sigma^2$ have prior $\pi(\sigma^2) \propto \left(\sigma^2 \right)^s$ for some $s$. Then the prior on $\sigma^2$ is improper.
The posterior can be found by \begin{align*} \pi(\sigma^2|\mathbf{x}) & = f(\mathbf{x}|\sigma^2)\pi(\sigma^2)\\ & \propto \left(\sigma^2\right)^s \prod_{i=1}^{n} (\sigma^2)^{-1/2} \exp\left \{-\dfrac{x_i^2}{2\sigma^2} \right \}\\ & = (\sigma^2)^{-n/2 + s+1 - 1} \exp \left\{-\dfrac{\sum x_i^2}{2 \sigma^2} \right \} \end{align*}
Thus the posterior distribution is Inverse Gamma$(n/2 -s - 1, \sum x_i^2/2)$, which is proper only if
$$\dfrac{n}{2} - s - 1 > 0.$$
Thus for $s = 0$, it is improper when $n = 1$, but proper when $n \geq 3$.
answered Apr 5, 2016 at 1:18
-
$\begingroup$ Thanks! I am wondering though, how messy would it be if $\theta\neq0$, and if we left $\pi(\mu,\sigma^2)=(\sigma^2)^s$ as a joint prior? Would we resort to sampling methods to compute the integral of the posterior? It might be that we can use the Inverse-Wishart distribution, so that the conclusion is as you have stated here. $\endgroup$ Commented Apr 6, 2016 at 4:14
$\begingroup$
$\endgroup$
A simple example of a prior remaining improper no matter how large the sample size is goes as follows: take $\pi(\alpha)=\exp\{\alpha^2\}$ and the sample iid $\mathcal{E}(\alpha)$ [which is equivalent to the normal case when the mean is set to zero since exponential and chi-square distributions are almost the same] Then $$\pi(\alpha|x_1,\ldots,x_n) \propto \alpha^n \exp\left\{-\alpha\sum_i x_i+\alpha^2\right\}$$ which does not integrate, no matter what $n$ is.
A more realistic example is provided by mixtures of distributions: for instance, take a sample from $$\frac{4}{5}\mathcal{N}(0,1)+\frac{1}{5}\mathcal{N}(\mu,\sigma^2)$$ and take the improper prior $\pi(\mu,\sigma)=1/\sigma$. For any sample size $n$ the posterior on $(\mu,\sigma)$ remains improper.
