Statistical Mechanics

Prerequisites: Entropy. Full path (3 pages)

• Intuitive
• Concrete
• Abstract
• Why is it interesting?

Intuitive

A glass of water contains about ten million billion billion molecules. Nobody can follow all of them, and nobody needs to. To know the pressure, the temperature or how much heat it takes to boil the water, we only need a handful of averaged quantities. Statistical mechanics explains how those averaged quantities follow from the laws for the individual molecules.

The key step is to stop asking which exact arrangement of molecules the system is in, and ask instead how many arrangements are compatible with what we know. A detailed arrangement is called a microstate, and the coarse description (total energy, volume, number of particles) is a macrostate. Some macrostates can be realized by vastly more microstates than others. If you shuffle a deck of cards, almost every order you get looks "random", because there are far more random-looking orders than ordered ones. In the same way, a gas spreads evenly through a box because the even arrangements outnumber the clumped ones by an enormous factor. The logarithm of this count, times a constant, is the entropy.

Temperature enters as the quantity that decides how energy is shared. A small system in contact with a large one at a given temperature is found in states of high energy less often, with a probability that falls off exponentially in the energy. Combined with the microscopic laws of motion (classical or quantum), this rule gives the laws of thermodynamics, the ideal gas law, heat capacities of solids, magnetism and phase transitions such as melting and boiling.

Concrete

Consider a system with microstates labelled by $i$ and energies $E_i$ (for a quantum system, the energy eigenstates). The three standard ensembles assign probabilities $p_i$ to them.

Microcanonical ensemble (isolated system, fixed energy $E$, volume $V$, particle number $N$). All $\Omega(E,V,N)$ accessible microstates are equally likely, and $$S = k_B \ln \Omega, \qquad \frac{1}{T} = \left(\frac{\partial S}{\partial E}\right)_{V,N}.$$

Canonical ensemble (system in contact with a heat bath at temperature $T$). Let the bath have entropy $S_R(E_R)$ and total energy $E_{\text{tot}}$. The probability of a system microstate $i$ is proportional to the number of bath states compatible with it, $$p_i \propto \Omega_R(E_{\text{tot}} - E_i) = e^{S_R(E_{\text{tot}} - E_i)/k_B} \approx e^{S_R(E_{\text{tot}})/k_B}\, e^{-E_i/k_B T},$$ where we expanded $S_R$ to first order because $E_i \ll E_{\text{tot}}$. Normalizing gives the Boltzmann distribution $$p_i = \frac{e^{-\beta E_i}}{Z}, \qquad Z = \sum_i e^{-\beta E_i}, \qquad \beta = \frac{1}{k_B T}.$$ All thermodynamics follows from the partition function $Z(\beta, V, N)$: $$\langle E\rangle = -\frac{\partial \ln Z}{\partial \beta}, \qquad F = -k_B T\ln Z, \qquad S = -k_B\sum_i p_i\ln p_i = \frac{\langle E\rangle - F}{T}, \qquad p = -\left(\frac{\partial F}{\partial V}\right)_{T,N}.$$ For a quantum system $Z = \operatorname{tr} e^{-\beta \hat H}$ and the state is the density operator $\hat\rho = e^{-\beta\hat H}/Z$.

Grand canonical ensemble (exchange of energy and particles with a reservoir at temperature $T$ and chemical potential $\mu$): $p_i \propto e^{-\beta(E_i - \mu N_i)}$, with grand partition function $\mathcal{Z} = \sum_i e^{-\beta(E_i - \mu N_i)}$ and grand potential $\Phi = -k_B T\ln\mathcal{Z}$.

Example: the classical ideal gas. For $N$ identical particles of mass $m$ in a volume $V$ with $H = \sum_k \vec p_k^{\,2}/2m$, the classical partition function is an integral over phase space, $$Z = \frac{1}{N!\,h^{3N}}\int d^{3N}x\, d^{3N}p\; e^{-\beta H} = \frac{1}{N!}\left(\frac{V}{\lambda^3}\right)^N, \qquad \lambda = \frac{h}{\sqrt{2\pi m k_B T}} .$$ The factor $1/N!$ accounts for identical particles and $h^{3N}$ makes $Z$ dimensionless. Then $\ln Z = N\ln V - \frac{3N}{2}\ln\beta + (\text{terms independent of } V, \beta)$, so $$\langle E\rangle = \frac32 N k_B T, \qquad p = k_B T\frac{\partial \ln Z}{\partial V} = \frac{N k_B T}{V},$$ which are the equipartition result and the ideal gas law.

In practice one computes $Z$ exactly only for simple systems (ideal gases, independent spins, harmonic oscillators, the two-dimensional Ising model in zero field). Otherwise one uses expansions in a weak interaction, mean field theory, the renormalization group near critical points, or Monte Carlo sampling of the Boltzmann distribution.

Recommended Resources:

• What is Statistical Mechanics? by Roman Frigg
• The Principles of Statistical Mechanics by Richard C. Tolman
• See also this review, and part 2.
• A great explanation how one can derive the Boltzmann distribution from the principle of maximal entropy is given at page 3 in https://arxiv.org/pdf/1311.0813.pdf
• Fundamentals of Statistical and Thermal Physics by F. Reif
• A crash course in Statistical Mechanics https://scholar.harvard.edu/files/noahmiller/files/statistical_mechanics.pdf
• No-Nonsense Classical Mechanics by Jakob Schwichtenberg, section "Statistical Mechanics"

Abstract

Classical equilibrium states. For a Hamiltonian system on a phase space $(P,\omega)$ with Liouville measure $\mu$, equilibrium states are probability measures invariant under the Hamiltonian flow. The microcanonical measure on the energy surface $\Sigma_E = H^{-1}(E)$ is $\delta(H - E)\,d\mu$, normalized. In canonical coordinates it equals $d\sigma_E/|\nabla H|$, with $d\sigma_E$ the Euclidean surface measure and $|\nabla H|$ the Euclidean norm of the gradient. The canonical state is the Gibbs measure $Z^{-1}e^{-\beta H}d\mu$. The question of when time averages along a single trajectory agree with ensemble averages is the subject of ergodic theory.

Quantum Gibbs states and the variational principle. For a Hamiltonian $H$ with $\operatorname{tr}e^{-\beta H} < \infty$ the Gibbs state is $\rho_\beta = e^{-\beta H}/Z$. It is the unique minimizer of the free energy functional $$F[\rho] = \operatorname{tr}(\rho H) + k_B T\operatorname{tr}(\rho\ln\rho)$$ over density operators, because $$F[\rho] - F[\rho_\beta] = k_B T\, S(\rho\,\|\,\rho_\beta) \ge 0,$$ with $S(\rho\|\sigma) = \operatorname{tr}\rho(\ln\rho - \ln\sigma)$ the relative entropy, which vanishes only for $\rho = \sigma$. The minimum value is $F[\rho_\beta] = -k_B T\ln Z$. Equivalently, $\rho_\beta$ maximizes the von Neumann entropy at fixed $\operatorname{tr}(\rho H)$. The ensembles are related by Legendre transforms: to leading order in the volume, $\ln Z(\beta) = \sup_E\big(S(E)/k_B - \beta E\big)$, and in the thermodynamic limit the ensembles give the same thermodynamics wherever $S(E)$ is strictly concave.

Infinite systems. Phase transitions require the thermodynamic limit, since for a finite system $\ln Z$ is analytic in $\beta$ and the couplings. In the limit one works with the free energy density $f = -\lim_{V\to\infty}\frac{k_BT}{V}\ln Z_V$, which can be nonanalytic. The Gibbs state itself is defined directly in infinite volume. For classical lattice systems, the Dobrushin-Lanford-Ruelle (DLR) equations require that the conditional distribution inside every finite region, given the configuration outside, is the finite-volume Gibbs distribution with those boundary conditions. Coexisting phases show up as several DLR states at the same $\beta$, as in the two-dimensional Ising model below its critical temperature, where the $+$ and $-$ boundary conditions give different infinite-volume states.

In quantum statistical mechanics the analogous notion is the KMS condition. Let $\mathfrak{A}$ be a C*-algebra of observables with time evolution $\tau_t$. With $\hbar = 1$, a state $\omega$ is a $(\tau,\beta)$-KMS state if for all $A,B$ in a dense subalgebra $$\omega\big(A\,\tau_{i\beta}(B)\big) = \omega(BA).$$ For a finite system, $\tau_t(B) = e^{iHt}Be^{-iHt}$ and $\omega(X) = \operatorname{tr}(e^{-\beta H}X)/Z$, and the condition follows from cyclicity of the trace. The KMS condition makes sense when $e^{-\beta H}$ is not trace class, and it characterizes equilibrium in infinite volume.

Relation to field theory. Since $e^{-\beta H}$ is evolution in imaginary time, $Z = \operatorname{tr}e^{-\beta H}$ is a Euclidean path integral over fields on a time circle of circumference $\beta$ (with $\hbar = 1$), with periodic boundary conditions for bosons and antiperiodic ones for fermions. Conversely, the partition function of a classical lattice model near a continuous transition is described at long distances by a Euclidean field theory, which is the basis of statistical field theory and the renormalization group treatment of critical phenomena.

Recommended Resources:

• Statistical Field Theory | David Tong

Why is it interesting?

Statistical mechanics is a framework that allows us to describe systems with many degrees of freedom, although we don't know all the microscopic details. For example, using statistical mechanics we can describe a gas or a block of metal, although we don't know all individual atom configurations.

Path to this page

1. Calculus
2. Statistics
3. Entropy

Leads to: Condensed Matter Physics, Critical Exponents, Ergodicity, Planck's Radiation Law, Spin Models.