Questions tagged [improper-prior]
The improper-prior tag has no usage guidance.
32
questions
9
votes
2
answers
426
views
How is data generated when using an improper prior
Let $X$ be an $\mathcal{X}$ valued random variable. We are doing Bayesian statistics. Suppose that $\theta$ is a $\Theta$ valued random variable with known prior distribution $\Pi$ and that the ...
- 456
1
vote
0
answers
15
views
Unconstrained Biases and Neural Network Regularization
In Bishop's PRML on page 259 he discusses a L2 regularizer for each layer of a 2-layer neural network, given by
$$
\begin{equation}
\frac{\lambda_1}{2}\sum_{w\in W_1}w^2 + \frac{\lambda_2}{2}\...
- 31
7
votes
2
answers
238
views
Is the class of models for which the MLE exists also the one for which flat priors are permissible?
By "permissible" (for lack of a better term) I mean models which despite of a "flat" (improper) prior (i.e., $\int_{\Theta} p(\theta) d \theta = + \infty$) nevertheless produce a ...
- 1,352
2
votes
0
answers
28
views
Distribution families whose likelihoods integrate to $+\infty$ for some sample values
I've recently started learning about Bayesian statistics, and I came across this very nice answer by Xi'an https://stats.stackexchange.com/a/129908/268693, which [in my slight paraphrasing] says the ...
- 121
2
votes
0
answers
108
views
Improper Prior in Logit and Probit Models: Proper Posterior Conditions
Let
$y_i \vert p_i \sim \mathrm{Bernoulli}(p_i)$,
$p_i = F_h(X_i^\prime \beta) \ \ , \ \ h = 1,2 \ ,\ \ X , \beta \in \mathbb R^p$,
where $F_1(x) = (2\pi)^{-1/2}\int_{-\infty}^x \exp(-t^2/2) \ dt \ $ ...
- 31
2
votes
1
answer
209
views
For multivariate normal posterior with improper prior, why posterior is proper only if $n\geq d$
This is related to Gelman's BDA chapter 3 section 5's noninformative prior density for $\mu$.
Let $\Sigma$ be fixed positive definite symmetric matrix of size $d$ by $d$. Let $y_1,\dots, y_n$ be iid ...
- 1,445
1
vote
1
answer
107
views
Is there any strong argument about objective/non-informative improper prior?
Decades ago improper objective priors - e.g. $\pi(\sigma) \propto \sigma^{-1}, \sigma > 0,$ for a scale parameter - were considered problematic because some authors thought they were leading to the ...
- 51
0
votes
1
answer
42
views
Bayesian statistics
Assuming I have that $Y_i\mid \mu$ is an iid ~ $N(\mu,\sigma^2)$, for $i \in (1,\dotsc,n)$ with $\sigma_i$ known and improper prior $\pi(\mu)=1$ for all $\mu$.
i. How can I derive a formula for the ...
1
vote
1
answer
317
views
Prior predictive distribution with an improper prior for a Poisson likelihood
I have recently started exploring some bayesian statistics and I cannot seem to understand something about improper priors. In particular, the set up consists of a Poisson likelihood $ p(X|\theta) = \...
0
votes
1
answer
102
views
Showing that a posterior is Normal given improper prior
I am having difficulty showing the following problem and I suspect it has something to do with my lack of understanding of the question. The question is this:
Suppose we have an improper prior ...
- 125
1
vote
0
answers
183
views
How to obtain a generalized bayes estimator when we have random sample from the uniform distribution with a Pareto prior and a improper hyperprior?
Let $\boldsymbol{X}=\left(X_{1}, \ldots, X_{n}\right)$ be a random sample from the uniform distribution on $(0, \theta),$ where $\theta>0$ is unknown. Let
$$
\pi(\theta)=b a^{b} \theta^{-(b+1)}, a&...
- 699
3
votes
2
answers
2k
views
What is a non-informative choice of parameters for a Dirichlet distribution?
Dirichlet distribution is a conjugate prior for multinomial distribution. I want to impose a non-informative prior over sampling weights $\pi$ for a draw $x=(x_1,…,x_N)$ from a multinomial ...
- 655
2
votes
1
answer
2k
views
Why is this an example of a noninformative prior?
From Bayesian Data Analysis 3rd Edition [Gelman et. al], they give this as an example when introducing non-informative priors:
"We return to the problem of estimating the mean θ of a normal ...
- 25
3
votes
2
answers
223
views
Can an improper prior distribution be informative?
I have just worked through an example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator, leading me to believe that improper priors are uninformative. But ...
- 1,276
1
vote
1
answer
1k
views
Finding the posterior distribution given an improper prior
Let $X \sim N(\theta, \sigma^2)$ where $\sigma^2$ is known. Let the prior density $\pi(\theta) =1, \theta \in \mathbb{R}$ to be the improper uniform density over the real line. Find the posterior ...
- 51