I am using Bayesian probability. In my case, I have an empirical prior probability to use in calculating the posterior probability. But this isn’t the subjective way of doing it, I believe. If I didn’t have any prior empirical information, how could I estimate the prior probability?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Are you asking how to formulate a prior without having seen any data first? $\endgroup$– user20160Commented Jul 16, 2016 at 22:43
-
$\begingroup$ Yes. Say I don't have any historical data for a prior probability to apply or don't what it should be. What way do people do when they don't know the priori like me? $\endgroup$– user122358Commented Jul 16, 2016 at 23:01
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
If you have no strong prior beliefs or assumptions, you can use non-informative priors, which don't impose strong structure or constraints. Rather, they try to express ignorance about the parameters in a principled way. The Jeffreys prior and reference priors are prominent examples. Very general constraints may be imposed (for example, that particular parameters must be non-negative). Non-informative priors are useful when 'stronger' priors would unjustifiably favor some hypotheses in a way that's inconsistent with your actual (lack of) knowledge/beliefs. Along similar lines, see principles of indifference and maximum entropy. Because non-informative priors express a large degree of ignorance about the parameters, the posterior distribution will generally be influenced more strongly by the likelihood function. For more information, see:
Kass and Wasserman (1996). The selection of prior distributions by formal rules.
If you have stronger prior knowledge/beliefs about the system you're modeling (e.g. domain knowledge, previous findings), you can use an informative prior. Your beliefs may not completely specify a full distribution. For example, you might have a notion that certain parameter ranges are more probable, but do your beliefs correspond precisely to some particular parametric form (e.g. a normal distribution, gamma distribution, etc.)? In practice, the choice of prior will reflect some combination of satisfying your (perhaps incomplete) beliefs, convention, and computational convenience.
You can distill knowledge from domain experts into a prior distribution. This is called 'prior elicitation' (e.g. see discusssion here). The general approach is to ask a series of questions to extract the experts' knowledge/beliefs, then use the answers to construct a prior distribution.
answered Jul 17, 2016 at 1:28
-
$\begingroup$ If I have historical data, can I use it for prior? Or beta distribution for it? $\endgroup$ Commented Jul 17, 2016 at 2:12
-
3$\begingroup$ @user122358 If I had historical data and I thought it the world still worked more or less the same way as when that data was collected then I would certainly use it. How to use it is a separate question, but using it seems an obviously good idea. $\endgroup$ Commented Jul 17, 2016 at 2:38
$\begingroup$
$\endgroup$
In my case, I have an empirical prior probability to use in calculating the posterior probability. But this isn’t the subjective way of doing it, I believe.
On the contrary. The 'subjective way of doing it' is to ask what information you want to assume and then construct a prior accordingly. If you're happy to make exchangeability judgements about cases, then that implies some distributional constraints on the prior (see multi-level models). If you have a known constraint e.g. you think some quantity should have a finite variance, without knowing what that variance is, (or even that some quantities should add up to 7), then that implies other distributional constraints (see maximum entropy). If you have historical data, better yet; this may help to pin down some of the parameters for the distributions implied by the previous information.
None of this is required, not least because the subject presupposed by a subjective prior need not be you. However, in ordinary situations you would not want to build a prior around the beliefs of the sort of subject that would, for example, ignore historical evidence. So you'd usually take the historical information into account. What makes the constructed prior 'subjective' is that it could be informed by more than that historical data.
If I didn’t have any prior empirical information, how could I estimate the prior probability?
Neither you, nor anyone else, are ever in this situation, so it doesn't matter.
answered Jul 17, 2016 at 2:35