0
$\begingroup$

For a Bayesian estimation problem that I am working on, where I update the log-posterior (many times based on data) instead of the posterior itself using Bayes rule. I find the following (rather convoluted) form of the log-posterior distribution

$$\text{lpos}(x) := -49949 \log(2) + \log(\exp\{-(1/8) (x-2)^2\}/(2 \sqrt{2 \pi})) + 114148 \log(\cos(x/2))\\ + 86056 \log(\sin(x/2)) + 99898 \log(\sin(x))$$

With the maximum numerically determined (using Mathematica) as -103515. and the minimum -130038.

I am trying to recover the posterior using the The Log-Sum-Exp Trick but due to the large gap between the maximum and minimum I still get the error that "Exp[-26485.] is too small to represent as a normalized machine number; precision may be lost".

Can anyone advise on how to work around this such that I can recover the normalized posterior distribution (as I am interested in the variance thereof) from such a convoluted log-posterior distribution? Many thanks for any assistance.

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ that's a problem of the number representation, not about the method you use. Exp[-26485.] is too small for single precision floating point, thus you might try to see if your language supports double precision numbers, or even more advanced precision numbers (though, you'll see a big increase in memory usage and a decrease in performance, as CPUs might not be very much optimized for such formats) $\endgroup$
    – Alberto
    Commented Mar 31 at 13:05
  • $\begingroup$ @Alberto Okay thanks for the comment. Any recommendations regarding languages? I am currently using Mathematica and Python. $\endgroup$
    – John Doe
    Commented Mar 31 at 13:46
  • $\begingroup$ Try giving a look at docs.python.org/3/library/decimal.html $\endgroup$
    – Alberto
    Commented Mar 31 at 14:15
  • $\begingroup$ It might be a good idea to do sensitivity analysis here. If you have such drastically different numbers in your posterior, perhaps some of them don't actually matter? You could start by making all linear terms to be of roughly the same magnitude, and then check what happens as you approach the values you seek to represent. You may find that irrespective of numerical precision the small values you are worrying about are not important. $\endgroup$
    – Cryo
    Commented Apr 1 at 2:12
  • 1
    $\begingroup$ The first term $-49949 \log(2)$ can be omitted from the unormalised posterior, while all logarithmic transforms of trigonometric functions should involve absolute values or squares. $\endgroup$
    – Xi'an
    Commented Apr 1 at 9:45

0

Reset to default