How can I get the probability of a predicted outcome conditional on the posterior in bayesian regression?

Question

Assume that I am running the following regression:

$\hat{y_t} = \beta_0 + \beta_1 \cdot x_t$

where as $\hat{y_t}$ is a continuous variable.

Lets assume a gaussian likelihood and nonconjugate priors such as $\beta_0 \sim N(0, 1)$ and $\beta_1 \sim Beta(1,1)$

Let's assume I have a dataset spanning from timestep $0$ to $t$ and call this dataset $h_t$.

Let's assume I obtain the posterior by some numerical method such as MCMC; let's denote these posterior estimates of our parameters as $\theta$.

Now, I am interested in obtaining the probability of observing a SPECIFIC outcome $z$ given some input $x_t$; thus, what I am interested in obtaining is the following:

$p(y_{t+1} = z | x_{t+1}) = \int p(\theta|h_t)p(y_{t+1} = z | \theta, x_{t+1}) d\theta$

I believe that this expression is equal to zero since we have continuous outcomes; this should lead to $p(y_{t+1} = z | \theta, x_{t+1}) = 0$; thus we have to define intervals to properly define this expression as the following, let's assume three intervals, $z<0$, $0<=z<10$ and $10<z$. Then, we can define, for example:

$p(y_{t+1} \in z_{interval_1} | \theta, x_{t+1}) = \int p(\theta|h_t)p(y_{t+1} \in z_{interval} | \theta, x_{t+1}) d\theta$

where $z_{interval_1}$ denotes one of the intervals.

Is this line of reasoning correct?

Answer 1

In your notation, given the posterior distribution $p(\theta|h_t)$ and the density of a future observation $p(y_{t+1}|\theta,x_{t+1})$, posterior predictive distribution is $$ p(y_{t+1}|h_t,x_{t+1}) = \int_{\theta\in\Theta}p(\theta|h_t)p(y_{t+1}|\theta,x_{t+1})\,\text{d}\theta. $$ Therefore, the posterior predictive distribution doesn't depend on unknown quantities, such as the parameter $\theta$.

If $Y_t$ are continuous random variables, then as for any continuous random variable $P(Y_{t+1} = a|h_t,x_{t+1})=0$, for any scalar $a$. You could undoubtedly compute the posterior predictive probability that $Y_{t+1}$ is within some interval if that's what you are after.

The Bayesian approach equips you with the entire distribution of a future observation $Y_{t+1}$ conditional on the past data and the prior information. What you can achieve with that depends on the purpose of the analysis. For instance, you could compute the expected value of such a prediction, the variance, or you may compute the 95% prediction interval, etc.

Answer 2

so in practise your MCMC parameter samples are fed into your model to generate many realisations of the outcome for your particular $x_t$.see eg stan generated quantities.

Then you are free to use any density estimator to create a smooth density.

Though most use cases work directly with the samples of the outcome... So why do you need the probability density of the outcome?