6
$\begingroup$

I am trying to solve a Lagrange multiplier problem for the following equation

$$ L= - \int_{-\infty}^\infty \rho(x) \ln \frac{\rho(x)}{q(x)} dx + \alpha \left( 1- \int_{-\infty}^\infty \rho(x) dx \right) +\beta \left( \overline{O} - \int_{-\infty}^\infty O(x) \rho(x) dx \right) \tag{1} $$

for $\frac{\partial L}{\partial \rho(w)} =0$.

Where the first term is the relative entropy, the second term is the constraint that it sums to one, and the last term is the constraint of an observable involving its average value. The problem solves for the probability measure that maximizes the relative entropy of a continuous parametrization x.

Eventually, I wish to apply the probability measure to a curved manifold. This is why coordinate invariance is very interesting to me.


An interesting property of the relative entropy is that its equations remains invariant with respect to a change in variable. Indeed,

$$ \int_{-\infty}^\infty \rho(x)\ln \frac{\rho(x)}{q(x)} dx \to \int_{-\infty}^\infty \rho(y(x)) \left|\frac{\partial y}{\partial x}\right|\ln \frac{\rho(y(x))\left|\frac{\partial y}{\partial x}\right|}{q(y(x)) \left|\frac{\partial y}{\partial x}\right|} dx = \int_{-\infty}^\infty \rho(y)\ln \frac{\rho(y)}{q(y)} dy \tag{2} $$

and

$$ \int_{-\infty}^\infty \rho(x)dx \to \int_{-\infty}^\infty \rho(y(x)) \left|\frac{\partial y}{\partial x}\right| dx = \int_{-\infty}^\infty \rho(y) dy \tag{3} $$


However, the last term isn't. Indeed:

$$ \int_{-\infty}^\infty O(x) \rho (x) dx \to \int_{-\infty}^\infty O(y(x)) \left|\frac{\partial y}{\partial x}\right| \rho (y(x)) \left|\frac{\partial y}{\partial x}\right| dx = \int_{-\infty}^\infty O(y) \rho (y(x)) \left|\frac{\partial y}{\partial x}\right| dy \tag{4} $$


How can I modify the integral that contains O so that it is coordinate invariant?

What is the expression for the average of a function O(x) over a continuous parametrization x, such that the average value is coordinate invariant?


edit:

It appears to me that the solution is to consider the observable to be the ratio between two quantities:

$$ \overline{O} = \int_{-\infty}^\infty \frac{A(x)}{B(x)} \rho(x) dx $$

where $O(x) := A(x)/B(x)$. In this case, it is invariant for the same reason that equation (2) is:

$$ \int_{-\infty}^\infty \frac{A(x)}{B(x)} \rho(x) dx \to \int_{-\infty}^\infty \frac{A(y(x)) \left| \frac{dy}{dx} \right| }{B(y(x)) \left| \frac{dy}{dx} \right|} \rho(y(x)) \left| \frac{dy}{dx} \right| dx = \int_{-\infty}^\infty \frac{A(y) }{B(y) } \rho(y) dy $$

Can anyone weight in?

$\endgroup$
8
  • 1
    $\begingroup$ crossposted: physics.stackexchange.com/questions/754961/… $\endgroup$
    – Anon21
    Commented Mar 12, 2023 at 16:38
  • 1
    $\begingroup$ Is that feasible? Does the standard mean, for example, have that property? $\endgroup$ Commented Mar 12, 2023 at 16:39
  • 1
    $\begingroup$ @Aruralreader There has to be a way; for instance, the average height of snow in a given area does not depend on the coordinate system. $\endgroup$
    – Anon21
    Commented Mar 12, 2023 at 16:47
  • 1
    $\begingroup$ @Aruralreader The differential entropy en.wikipedia.org/wiki/Differential_entropy doesn't have the property of coordinate invariance. But the en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence is the modification that cures it. There has to be a modification that cures it to for the observables. $\endgroup$
    – Anon21
    Commented Mar 12, 2023 at 16:47
  • 1
    $\begingroup$ The average depth, correct, the average height, no. $\endgroup$ Commented Mar 12, 2023 at 16:57

0

You must log in to answer this question.