8
$\begingroup$

From what I gather, a Boltzmann machine (BM) is essentially a spin glass with no applied field evolving under Glauber dynamics (if this is at all mistaken, I don't think it will be off enough to materially affect the following, but I would welcome elaboration anyway). For background, see, e.g. Amit et al., PRA 32, 1007 (1985): their suggestion that the "detailed dynamic properties of the...Hopfield models, e.g. the rate of relaxation" be studied is at the root of my question below.

Now, a restricted BM (RBM) is a bipartite BM, and the maximal independent sets correspond to visible and hidden units/nodes. RBMs can be "stacked" to form a deep BM (DBM), and these DBMs have attracted a lot of interest over the past 5-6 years due to the practicality of a training algorithm for them.

It is well known that spin glasses have very complicated dynamics: in particular, a multitude of separated timescales contribute to their evolution. That said, I have seen recent work that (upon a cursory look) seems to suggest that bipartite spin glasses/RBMs have reasonably well-defined equilibrium (or steady state?) behavior, in which convex combinations of the "visible" and "hidden" glasses characterize the overall situation (Barra et al., J. Phys A 44, 245002 (2011)--NB. I have only skimmed this).

This in turn seems to suggest to me that the relaxation dynamics of these bipartite spin glasses/RBMs is comparatively tractable. So my question is:

Is the preceding sentence justified? And if so, what do the relaxation dynamics of bipartite spin glasses/RBMs look like? Is there a single independent characteristic timescale (e.g., relaxation and mixing times are different in general, but closely related and certainly not independent)?

I would welcome any references within the last 5 years--not, e.g. MacKay, etc.

(NB. A less detailed version of this question was posted at stats.stackexchange a few days ago.)

$\endgroup$
2
  • $\begingroup$ Regarding the sentence: "relaxation dynamics of these bipartite spin glasses/RBMs is comparatively tractable", what physical quantities would you like to compute? Are you trying to sampling from the ground states according to a Boltzmann distribution or trying to compute the partition function (or related quantity)? $\endgroup$ Commented Mar 9, 2012 at 13:17
  • $\begingroup$ I am interested in the mixing or relaxation time. $\endgroup$
    – S Huntsman
    Commented Mar 10, 2012 at 14:46

0