1
$\begingroup$

Let's consider stochastic dynamics with discrete time step $t$ and states $x_t\in\mathbb{R}^d$ that evolve as

$$ \mathrm{p}(x_{t+1}|x_t)=\mathcal{N}(x_t+f(x_t), \Sigma). $$

Moreover, we assume the existence of a unique stationary distribution $\rho(x)$ that satisfies

$$ \int\mathrm{p}(x_t|x_{t-1})\rho(x_{t-1})\mathrm{d}x_{t-1}=\rho(x_t). $$

Can $f$ be written as a gradient descent on $\ln\rho$ transformed by a positive semi-definite matrix $R(x)$,

$$ \mathrm{p}(x_{t+1}|x_t)=\mathcal{N}(x_t+R(x_t)\nabla_{x_t}\ln\rho(x_t),\Sigma)\,? $$

Do I additionally need to assume ergodicity or detailed balance for this to work?

$\endgroup$

1 Answer 1

1
$\begingroup$

I don't think so. Counterexample:

State space = $\{0,1,2\}$

$p_1: x_{t+1}=x_t+1 $ mod $ 3$

$p_2: x_{t+1}=x_t-1 $ mod $ 3$

so $\Sigma=0$.

Then the same $\rho(x_t)$ belongs to both $p_1$ and $p_2$.

$\endgroup$
4
  • $\begingroup$ Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=\delta(x_{t+1}=x_t+1 \,\text{mod}\, 3)$ and $p_1(x_{t+1}|x_t)=\delta(x_{t+1}=x_t-1 \,\text{mod}\, 3)$? I think these systems have no stationary distribution $\rho(x)$. Also, technically $x_t\in\mathbb{R}^d$, but an example with discrete states could already be helpful. $\endgroup$
    – danijar
    Commented Nov 22, 2018 at 20:12
  • $\begingroup$ They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics. $\endgroup$
    – Ian
    Commented Nov 22, 2018 at 23:06
  • $\begingroup$ @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution. $\endgroup$
    – Smind
    Commented Nov 23, 2018 at 11:17
  • $\begingroup$ Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise. $\endgroup$
    – danijar
    Commented Nov 23, 2018 at 15:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .