I am studying David Tong's lecture note on statistical physics, and I have a question regarding the precise definition of pressure. I checked other postings in this community, but was unable to get the answer. Also, apologies for this absurdly long question.
If you feel this question is too long, I guess you can jump directly to $\eqref{e:7}$ and only read the relevant equations.
1. Tong first discusses microcanonical ensemble, a system of fixed energy $E$ in a variable volume $V$. There, he defines the pressure as
$$ P = T \frac{\partial S}{\partial V}. \tag{1}\label{e:1} $$
This relation is then used to derive the relation
$$ \mathrm{d}E = T \, \mathrm{d}S - P \, \mathrm{d}V, $$
which I envision as differential of $E$ when $E$ is understood as a function of $S$ and $V$. Here, my mental picture is that the physical consideration in this context gives rise to a constraint among $T$, $S$, $V$, which in turn defines a 2-dimensional manifold (or foliation). The above relation then implicitly specifies this manifold. So far, everything sounds good.
2. Then Tong moves on to canonical ensemble, where we shift gear to reformulating entropy and related quantities in terms of the Boltzmann distribution. Some of the consequences are the relations
$$ \langle E \rangle - TS = -k_B T \log \mathcal{Z} \qquad\text{and}\qquad S = k_B \frac{\partial}{\partial T}(T \log \mathcal{Z}), \tag{2}\label{e:2} $$
where $\langle E \rangle $ is the average energy of the system and $\mathcal{Z}$ is the partition function. (The two relations are actually equivalent.) Since this quantity is so important, we denote this quantity by $F$ and call it the (Helmholtz) free energy.
Now, again, let us allow the volume $V$ of the system to vary. Then Tong claims that we have
$$ \mathrm{d}F = -S \, \mathrm{d}T - P \, \mathrm{d} V. \tag{3}\label{e:3} $$
However, I failed to derive this formula using the definition $\eqref{e:1}$. Again, I envision that physical constraints on $T$, $S$, $V$ defines a 2-dim manifold. So, any physical quantity that depends on these there variables can be expressed (at least locally) as a function of any two of them.
Now, in order to derive $\eqref{e:3}$, I first noticed that the following relations hold:
The specific heat satisfies $$ T \frac{\partial S}{\partial T} = \frac{\partial(TS)}{\partial T} - S \stackrel{\eqref{e:2}}= \frac{\partial}{\partial T}(\langle E \rangle + k_B T \log \mathcal{Z}) - S \stackrel{\eqref{e:2}}= \frac{\partial\langle E\rangle}{\partial T}. \tag{4}\label{e:4} $$
Regarding $S$ as a function of $T$ and $V$, $$ T \, \mathrm{d}S = T \biggl( \frac{\partial S}{\partial T} \, \mathrm{d}T + \frac{\partial S}{\partial V} \, \mathrm{d}V \biggr) \stackrel{\eqref{e:4}}= \frac{\partial \langle E\rangle}{\partial T} \, \mathrm{d}T + T \frac{\partial S}{\partial V} \, \mathrm{d}V \tag{5}\label{e:5} $$
Then, starting from the definition $F = \langle E \rangle - TS$ and regarding everything as a function of $T$ and $V$, \begin{align*} \mathrm{d}F &= \mathrm{d}\langle E\rangle - \mathrm{d}(TS) \\ &= \biggl( \frac{\partial \langle E\rangle}{\partial T} \, \mathrm{d}T + \frac{\partial \langle E\rangle}{\partial V} \, \mathrm{d}V \biggr) - T \, \mathrm{d}S - S \, \mathrm{d}T \\ &\stackrel{\eqref{e:5}}= \biggl( -T \frac{\partial S}{\partial V} + \frac{\partial \langle E\rangle}{\partial V} \biggr) \, \mathrm{d}V - S \, \mathrm{d}T. \tag{6}\label{e:6} \end{align*}
Finally, comparing $\eqref{e:6}$ and $\eqref{e:3}$, Tong seems suggesting that the pressure $P$ is given by
$$ P = T \frac{\partial S}{\partial V} - \frac{\partial \langle E\rangle}{\partial V}, \tag{7}\label{e:7} $$
which clearly does not match the original definition $\eqref{e:1}$ unless $\frac{\partial \langle E\rangle}{\partial V} = 0$. So here goes my question:
Question. Why do we have the discrepancy between $\eqref{e:1}$ and $\eqref{e:7}$?
I have checked the following posts in this community,
Question about the definition of the pressure in statistical physics
Confusing definition of thermodynamic pressure when calculating the electron degeneracy pressure
but none of them resolved my question. I guess the most relevant one is the last link, in which a statement says that the pressure is actually defined as $P = - \frac{\partial F}{\partial V}$. So it seems like we are actually revising the definition of the pressure so that the formula
$$ \mathrm{d}\langle E \rangle = T \, \mathrm{d}S - P \, \mathrm{d} V$$
continues to hold in a canonical ensemble, but I am not sure if I am correctly interpreting this or I am simply missing some details.
To give you some idea about my background knowledge, I majored in mathematics specialized in probability theory but only tangentially studied physics in my undergraduate. I am now trying to study statistical mechanics as a hobby and to paint some more insight on the mathematical models motivated from statistical mechanics.
which clearly does not match the original definition (1) unless ∂⟨E⟩/∂V=0.
$\endgroup$