1
$\begingroup$

Suppose I'm given $\theta_0$ and I want to sample data from a density $f(Y|\theta_0)$ and then sample from the posterior of $\theta|Y$ (given, obviously, some prior). I want to do this lots of times, to get the sampling distribution of summaries of the posterior distribution.

If I have one MCMC sample of $M$ realisations, $\theta_1,\ldots,\theta_M$, for a sample $Y_1,\dots,Y_n$ it seems that I should be able to reweight it for another sample $Y'_1,\dots, Y_n$ using the likelihood ratio: the weight for $\theta_i$ would be $$w_i=\frac{f(Y'_1,\dots,Y'_n|\theta_i)}{f(Y_1,\dots,Y_n|\theta_i)}$$ If I then sample from $\theta_1,\dots,\theta_M$ with probabilities proportional to $w_i$, I should get some sort of approximate sample from the posterior of $\theta|Y'$. It's sort of reminiscent of importance sampling and of the particle filter.

This must have been tried. What is known about the effectiveness? How bad is it for well-behaved unimodal low-dimensional posteriors?

Improve this question
$\endgroup$
3
  • $\begingroup$ Something similar was done in Monte carlo simulations for estimating mean values etc., this similar concept is known as (temperature) reweighing. But the focus is on expected values not samples directly ... $\endgroup$
    – Ggjj11
    Commented Jul 3 at 8:06
  • 3
    $\begingroup$ This is a special case of importance sampling and was suggested in one of the earliest papers of Gelfand and Smith in 1990. The performances will degrade with $n$ and with the dimension of $\theta$. $\endgroup$
    – Xi'an
    Commented Jul 3 at 19:19
  • $\begingroup$ Tnanks! Does someone want to put an answer in so I can accept it? $\endgroup$
    – Thomas Lumley
    Commented Jul 3 at 21:55

0

Reset to default