All Questions
Tagged with prior regression
64
questions
1
vote
1
answer
66
views
Full conditional posteriors
so up to now I dealt with posteriors in the form of:
$$p(\theta|x) \propto p(x|\theta) p(\theta)$$
No we started to model a linear regression with the bayesian approach:
$$Y \sim MVN(X\beta, \sigma^2I)...
- 137
3
votes
1
answer
173
views
Bayesian linear regression: How to enforce constraint on the sum of coefficients?
I have a linear regression problem in which my $X$ matrix is not full rank. Here is a small example:
$$X =
\left[\begin{array}{rrrr}
-1 & 0 & 0 & 1 \\
1 & 0 & -1 & 0 \\
0 &...
- 431
0
votes
0
answers
46
views
Performing a Bayesian (or Bayesian-like) linear regression
I'm trying to model the relationship between two variables. Without going into details on how I get this expectation, my belief on the relationship between the variables looks like this:
However, as ...
- 183
0
votes
0
answers
45
views
Resampling as prior distribution?
Suppose we've got a small dataset that we have no prior knowledge about and we're going to use linear regression on it. I have been wondering whether instead of fitting a normal OLS it would make ...
- 205
3
votes
1
answer
120
views
Parameter distribution of $\theta$ from a rectangular matrix multiplication $C\theta$
I am struggeling to see where this problem fits - i.e. what topics this problem relates to, so I am not able to find the right literature. I want to use some particular information as a prior to a ...
6
votes
3
answers
839
views
How does an ideal prior distribution needs a probability mass on zero to reduce variance, and have fat tails to reduce bias?
I am reading this article about the horseshoe prior and how it is better than lasso and ridge priors. The author makes several points that I don't understand. One of them is "The ideal prior ...
- 308
1
vote
0
answers
53
views
What is the best way to encode the prior of a Gaussian Process model in this application?
I'm using Gaussian Process regression for the first time to model the unknown energy efficiency of a compressor which I know is a smooth, non-linear relationship that looks something like the line in ...
- 165
6
votes
3
answers
700
views
Bayesian Analysis in the Absence of Prior Information?
I have always wondered - how confident do researchers tend to be in their "prior" information when deciding to create statistical models using a Bayesian Approach vs. a Frequentist Approach?
...
1
vote
0
answers
132
views
Popular Methods for Choosing Hyperparameters in Bayesian Statistics
I'm wondering which methods are commonly used to estimate hyperparameters for priors in Bayesian statistics, and how they work? The setting I'm working with is Bayesian linear regression, so I'm ...
- 331
1
vote
0
answers
107
views
Choosing between Gaussian/Laplacian prior distributions for MCMC regression
When doing a linear regression using MCMC, you have to specify prior distributions for the values of the regression coefficients of the independent variables. If all of the priors are Gaussian ...
- 173
0
votes
0
answers
28
views
How to improve the predictions of a model when we have too few predictor variables?
I tried to use a linear model to explain a variable "age" with two variables "x1" and "x2".
I can clearly see a decreasing slope inside my scatterplot for age vs x1, or ...
- 23
1
vote
0
answers
45
views
Is there an implicit independence assumption in Bayesian inference between X and parameters?
I often see things like
$$ p(w|X,y) \propto p(y|X,w) p(w)$$
where $w\in\mathbb R^p$ denotes some parameters, $y\in \mathbb R^n$ denotes some observed outcome values, and $X\in \mathbb R^{n\times d}$ ...
- 11
2
votes
1
answer
185
views
How to find the marginal prior distribution?
Suppose that $\beta$ has the following prior
$$
\beta|\zeta \sim f(\beta,\zeta)
$$
Then I know that the marginal prior distribution of $\beta$ is given by
$$
\int f(\beta,\zeta) d\zeta
$$
However, ...
- 273
0
votes
0
answers
41
views
Incorporating prior evidence of predictor having no effect in bayesian linear regression model
Say we start with a linear regression model of the form
$$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon, \quad \epsilon \sim N(0, \sigma^2)$$
with the conjugate prior
$$
\begin{align*}
&\sigma^...
- 387
1
vote
1
answer
128
views
Partially specified Bayesian prior?
In bayesian linear regression for example, we may specify a model as:
$$y_i \sim N(\beta_0 + \beta_1 x_i, \epsilon^2) \\\\
\beta_0 \sim N(0, \tau_0^2) \\\\
\beta_1 \sim N(0, \tau_1^2) \\\\
\epsilon \...
- 313