Is the entropy for the classical microcanonical ensemble defined... up to an infinite number?

Question

In the microcanonical ensemble all states $(p, q) \in \Gamma$ (where $\Gamma$ is the phase space of a system with $3N$ coordinates) with the same energy have the same probability density. I would express this as follows:

\begin{gather} \rho(p, q; N, V, E) = \frac{\epsilon \delta(H(p, q) - E)}{\Omega(N, V, E)}\\ \Omega(N, V, E) = h^{-3N} \int_\Gamma d^{3N} p~d^{3N}q~\epsilon \delta(H(p, q) - E) \end{gather}

Where $H(p, q)$ is the Hamiltonian, $\delta$ the Dirac delta function, $E$ the energy of the system, $3N$ the number of degrees of freedom, $\epsilon$ is a "small" constant with units of energy, and $h$ is Planck's constant.

I have never seen this expression written exactly like this anywhere, but I believe it to correctly capture the idea of the ensemble. At the very least, it is dimensionally correct, and it recovers the fact that $\langle H \rangle = E$, but if it has issues please let me know.

From this expression I would like to recover the usual entropy formula for the microcanonical ensemble: $S(N, V, E) = \ln \Omega(N, V, E)$ (I set Boltzmann's constant to 1 for convenience here). My problem is that if I apply the "general" definition of differential entropy: $S = - \int dx~p(x) \ln p(x)$ I get the following:

$$ S(N, V, E) = - h^{-3N} \int_\Gamma d^{3N} p~d^{3N}q~ \rho(p, q; N, V, E) \ln \rho(p, q; N, V, E) \\= - h^{-3N} \int_\Gamma d^{3N} p~d^{3N}q~ \frac{\epsilon \delta(H(p, q) - E)}{\Omega(N, V, E)} \ln \frac{\epsilon \delta(H(p, q) - E)}{\Omega(N, V, E)} $$

This expression has a logarithm of a delta function, which already makes me pretty uncomfortable, but lets say we can apply the usual log properties to get something like:

$$ S(N, V, E) = - h^{-3N} \int_\Gamma d^{3N} p~d^{3N}q~ \frac{\epsilon \delta(H(p, q) - E)}{\Omega(N, V, E)} \ln \epsilon \delta(H(p, q) - E) + h^{-3N} \int_\Gamma d^{3N} p~d^{3N}q~ \frac{\epsilon \delta(H(p, q) - E)}{\Omega(N, V, E)} \ln \Omega(N, V, E) \\= \frac{-h^{-3N}}{\Omega(N, V, E)}\int_\Gamma d^{3N} p~d^{3N}q~ \epsilon \delta(H(p, q) - E) \ln \epsilon \delta(H(p, q) - E) + \ln \Omega(N, V, E) $$

This is almost what I want, since the second term is the expression I needed, but the first term seems to be an... infinite negative number? I can rewrite it using the sifting property: $$ -\frac{\int_\Gamma d^{3N} p~d^{3N}q~ \delta(H(p, q) - E) \ln \epsilon \delta(H(p, q) - E)}{\int_\Gamma d^{3N} p~d^{3N}q~ \delta(H(p, q) - E)} = \frac{-\ln \epsilon\delta(E - E)}{\Omega(N, V, E)} = \frac{-\infty}{\Omega(N, V, E)} = -\infty $$ I get then $S(N, V, E) = -\infty + \ln \Omega(N, V, E)$.

I'm pretty uncomfortable with these delta function manipulations overall, and I feel like this is very ugly, I would like to know which are the issues in this derivation, and whether it is possible to get to $\ln \Omega(N, V, E)$ more cleanly (I know some of my Dirac delta expressions are not well defined).

Answer 1

Yes you are correct. However, this infinite additive constant is “negligible” in the thermodynamic limit. This is why it does not appear in the usual formula. The subtlety is about choosing the right $\delta E$ for the definition of the microcanonical ensemble. You cannot take it arbitrarily small and quantum mechanics gives you an unambiguous lower bound.

One way to see this is by studying the density of states: $$ dN=D(E)dE $$ In a classical system (justified using a semi-classical limit): $$ D(E)=\frac{1}{h^{ND}}\int \delta(H-E)d^{ND}xd^{ND}p $$ For example, for a $D$-dimensional gas in a box of size $L$: $$ D(E)=\left(\frac{2\pi mEL^2}{h^2}\right)^{ND/2}\frac{1}{\Gamma(ND/2)E} $$ Technically, the entropy is taken to be: $$ S=\ln(D(E)\delta E) $$ for a small $\delta E$. As you can see, as long as $\ln\delta E=o(N)$, this will not modify the expression of the entropy in the thermodynamic limit. This is equivalent to taking: $$ S=\ln(D(E)) $$ This makes sense dimensionally as in the thermodynamic limit, you’ll have $S$ is extensive. Therefore you’ll have large powers, and an extra factor morally will not change the dimension. You can see this in the gas example.

Mathematically, you should take $\delta E$ to be arbitrary, independent of $N$. However, the thermodynamic limit is not uniform in $\delta E$, and the limits $0,+\infty$ are problematic. Physically, you want to choose $\delta E$ to be intensive.

You naturally have an upper bound in the continuum limit so that the linear approximation holds: $$ \Omega=D\delta E $$ For example, assuming differentiability: $$ \delta E \ll \frac{1}{\frac{d\ln D}{dE}} $$ Since $\ln D$ is extensive and $E$ as well, you’d expect $\delta E$ to be smaller than an intensive parameter as expected.

In the gas example, this gives the intensive bound: $$ \delta E\ll \frac{2E}{ND} $$

In the classical case, there is no natural lower bound for $\delta E$. You’ll need to appeal to quantum mechanics for that. The lower bound is just the validity of the continuum approximation of the density of states. You can quantify this by estimating the gap between consecutive energy levels.

For the gas in a periodic cubic box, the spacing would typically be: $$ \Delta E=\sqrt\frac{2mE}{N}\frac{h}{L} $$ the first factor being the typical momentum component and the second being the increment when changing one quantum number. Most importantly, $\Delta E$ is intensive and you want: $$ \delta E\gg \Delta E $$ which is consistent with the previous discussion.

Hope this helps.

Answer 2

I'll explain this on the case of one particle moving in 1D. The phase space is two-dimensional, there is one $q$, and one $p$.

Probability density on this phase space has units $q^{-1}p^{-1}$, so your $\Omega$ in the denominator can't be dimensionless - it is actually the phase space volume implied by the energy width $\epsilon$, so the volume has units $qp$. So there is no Planck constant in the formula for the phase space volume.

A better notation with correct units would be

$$ \rho_E(q,p) = \frac{\Delta E}{\Delta \Omega}\delta(H-E) $$ where $\Delta E$ is the energy width chosen and the phase space volume $\Delta \Omega$ is defined as

$$ \Delta \Omega = \int dq \int dp ~\Delta E~\delta(H-E). $$

So the phase volume $\Delta \Omega$ depends on the arbitrary width $\Delta E$. In special simple examples (one free particle in a box), one can find the integral defining the phase volume $\Delta \Omega$ as a function of $E,\Delta E$ and also see that it depends on both $E$ and $\Delta E$. Since the dependence on $\Delta E$ is linear, the probability density $\rho_E$ does not depend on the arbitrary $\Delta E$.

All these results can be motivated and found by assuming normalization of $\rho_E$ to unity and the form $\rho_E = A_E\delta (H-E)$, where $A_E$ is some function of energy $E$.

So your aim of relating the information entropy functional of $\rho_E$, independent of $\Delta E$:

$$ I[\rho_E] = \int dq\int dp~ (-\rho_E \ln \rho_E) $$ to the microcanonical entropy, dependent on $\Delta E$:

$$ S = -\ln \Omega_E, $$ is doomed to failure; one thing is not like the other. These are different concepts of entropy; the information entropy functional does not depend on $\Delta E$, the Boltzmann entropy does (its value is thus arbitrary, just as $\Delta E$ is).

We can define a lower-dimensional measure of the hypersurface implied by the condition $H=E$ - its area, instead of volume of the thin shell:

$$ \mu_E = \lim_{\Delta E\to 0}\frac{\Delta \Omega}{\Delta E}. $$

For typical systems (free particles in a box...) this is finite and does not depend on $\Delta E$, and thus we can use this to define statistical formula estimating thermodynamic entropy, free of the arbitrary $\Delta E$:

$$ S_{hypershell~area}(E) = \ln \mu_E. $$

This still depends on the units used to measure position and momenta, and $E$. However, this still has value different from the information entropy, because for too singular distributions $\rho_E$, such as the one expressed via the delta distribution, the latter functional is infinite. This is a "bug" of the information entropy idea - it works well only in its discrete version, not for distributions in a continuum space.

One can relate thermodynamic entropy to the maximum possible value of the information entropy functional, under the given constraints, in a different mathematical setting. Namely, the probability density must not be allowed to be such a singular function that information entropy comes out infinite - then there is no nice maximum value under the given constraints, only infinity. One way to do this is to define distribution $\rho_E$ not on a continuum space, but on a space consisting of discrete points. These can be associated with "quantum phase space cells" of phase space volume $h^{3N}$, or really, any other arbitrary partitions of the phase space (like when defining coarse grained quantities in a discretized view of the phase space). Then even the most concentrated function $\rho_E$ (1 at a single point and 0 at other points of the discrete space) has finite information entropy.

For an isolated closed system, the appropriate $\rho_E$ maximizing the information entropy is $\rho_E = 1/M$ where $M$ is the number of states compatible with $E$ in this discrete space. Then information entropy

$$ I[\rho_E] = \sum_{i=1}^M - \frac{1}{M}\ln \frac{1}{M} = \ln M, $$

which is compatible with the microcanonical entropy of the discretized system - logarithm of the number of all possible states compatible with the macroscopic constraints.