In science, most breakthroughs cannot be derived from known physical laws: they are based on inspired conjectures2. Comparison with experiment of the predictions based on such a hypothesis allows us to eliminate conjectures that are clearly wrong. However, there is a distinction between testing the consequences of a conjecture and testing the conjecture itself. A case in point is Edwards’ theory of granular media. In the late 1980s, Edwards and Oakeshott1 proposed that many of the physical properties of granular materials (‘powders’) could be predicted using a theoretical framework that was based on the assumption that all distinct packings of such a material are equally likely to be observed. The logarithm of the number of such packings was postulated to play the same role as entropy does in Gibbs’ statistical-mechanical description of the thermodynamic properties of equilibrium systems. However, statistical-mechanical entropy and granular entropy are very different objects. Until now, the validity of Edwards’ hypothesis could not be tested directly—mainly because the number of packings involved is so large that direct enumeration is utterly infeasible—and, as a consequence, the debate about the Edwards hypothesis has focused on its consequences, rather than on its assumptions. Here we present results that show that now, at last, it is possible to test Edwards’ hypothesis directly by numerical simulation. To our own surprise, we find that the hypothesis appears to be correct precisely at the point where a powder is just at the (un)jamming threshold. However, at higher densities, the hypothesis fails. At the unjamming transition, the configurational entropy of jammed states appears to be at a maximum.

The concept of ‘ensembles’ plays a key role in equilibrium statistical mechanics, as developed by J. Willard Gibbs, well over a century ago3. The crucial assumption that Gibbs made to arrive at a tractable theoretical framework to describe the equilibrium properties of gases, liquid and solids was that, at a fixed total energy, every state of the system is equally likely to be observed. The distinction between, say, a liquid at thermal equilibrium and a granular material is that in a liquid, atoms undergo thermal motion whereas in a granular medium (in the absence of outside perturbations) the system is trapped in one of (very) many local potential energy minima. Gibbsian statistical mechanics cannot be used to describe such a system. The great insight of Edwards was to propose that the collection of all stable packings of a fixed number of particles in a fixed volume might also play the role of an ‘ensemble’ and that a statistical mechanics-like formalism would result if one assumed that all such packings were equally likely to be observed, once the system had settled into a mechanically stable ‘jammed’ state. The nature of this ensemble has been the focus of many studies1,4,5,6.

Jamming is ubiquitous and occurs in materials of practical importance, such as foams, colloids and grains when they solidify in the absence of thermal fluctuations. Decompressing such a solid to the point where it can no longer achieve mechanical equilibrium leads to unjamming. Studies of the unjamming transition in systems of particles interacting via soft, repulsive potentials have shown that this transition is characterized by power-law scaling of many physical properties7,8,9,10,11,12. However, the exact nature of both the ensemble of jammed states and the unjamming transition remains unclear.

Here, we report a direct test of the Edwards conjecture, using a numerical scheme for computing basin volumes of distinct jammed states (energy minima) of N polydisperse, frictionless discs held at a constant packing fraction φ. Uniquely, our numerical scheme allows us to compute Ω, the number of distinct jammed states, and the individual probabilities pi{1, …, Ω} of each observed packing to occur. Figure 1a shows a snapshot of a section of the system, consisting of particles with a hard core and a soft shell. We obtain jammed packings by equilibrating a hard-disc fluid and inflating the particles instantaneously to obtain the desired soft-disc volume fractions (φ), followed by energy minimization (see Methods). The minimization procedure finds individual stable packings with a probability pi proportional to the volume vi of their basin of attraction. Averages computed using this procedure, represented by , would then lead to a bias originating from the different vi values. Recent advances in numerical methods13,14,15,16 have now enabled direct computation of vi, and therefore, an unbiased characterization of the phase space. A summary of the technique is provided in the Methods.

Figure 1: Snapshot of the system studied and illustration of the associated energy landscape at different volume fractions.
figure 1

a, Snapshot of a jammed packing of discs with a hard core (dark shaded regions) plus soft repulsive corona (light shaded regions). b,c, Illustration of configurational space for jammed packings. The hatched regions are inaccessible due to hard-core overlaps. Single-coloured regions with contour lines represent the basins of attraction of distinct minima. The dark blue region with solid dots indicates the coexisting unjammed fluid region and hypothetical marginally stable packings, respectively. The volume occupied by the fluid Vunj is significant only for finite-size systems at or near unjamming. When φ φ (b) the distribution of basin volumes is broad but as φφ (c) the distribution of basin volumes approaches a delta function satisfying Edwards’ hypothesis.

We report a detailed analysis of the distribution of vi for a fixed number of discs N = 64 (all maximum system sizes in our study were set by the current limits on computing power). We compute vi using a thermodynamic integration scheme13,14,15,16, and compute the average basin volume 〈v〉(φ). The number of jammed states is, explicitly, Ω(φ) = VJ(φ)/〈v〉(φ), where VJ(φ) is the total available phase-space volume at a given φ. A convenient way to check equiprobability is to compare the Boltzmann entropy SB = lnΩ − lnN!, which counts all packings with the same weight, and the Gibbs entropy SG = −∑ iΩpi lnpi − lnN! (refs 17,18,19). The Gibbs entropy satisfies SGSB, saturating the bound when all pi are equal: pi{1, …, Ω} = 1/Ω. As shown in Fig. 2a, SG approaches SB from below as φφN=64(S) ≈ 0.82. Fig. 1b, c schematically illustrates the evolution of the basin volumes as the packing fraction is reduced.

Figure 2: Numerical results obtained by basin-volume calculations for jammed packings of N = 64 discs with a hard core and a soft shell.
figure 2

a, Gibbs entropy SG and Boltzmann entropy SB as a function of volume fraction. SB is computed both parametrically by fitting with a generalized Gaussian function (Gauss) and non-parametrically by computing a kernel density estimate (KDE) as in ref. 13. The dashed curves are a second-order polynomial fit. b, Scatter plot of the negative log-probability of observing a packing, −ln pi = Fi + ln VJ(φ), where VJ is the accessible fraction of phase space (see Methods) as a function of log-pressure, Λ. The black solid lines are lines of best fit computed by linear minimum mean square error using a robust covariance estimator and bootstrap (see Methods). c,d, Slopes λ(φ) (c) and intercepts c(φ) (d) of linear fits for equation (16). Solid lines are lines of best fit and error bars refer to the standard error computed by bootstrap28.

To characterize the distribution of basin volumes, we analyse the statistics of vi along with the pressure Pi of each packing. It is convenient to study Fi ≡ − lnvi as a function of Λi ≡ lnPi. As shown in Fig. 2b, we observe a strong correlation between Fi and Λi, which we quantify by fitting the data to a bivariate Gaussian distribution. The conditional expectation of F given Λ then yields a linear relationship (denoted by solid lines in Fig. 2b) such that , where represents the average over all basins at a given Λ. Previous studies at higher packing fractions13 indicate that this relationship is preserved in the thermodynamic limit. Defining f = F/N, we have (see Methods for details)

where . For Edwards’ hypothesis to be valid, we require that in the thermodynamic limit: the distribution of volumes approaches a Dirac delta, which follows immediately from the fact that the variance σf2 1/N (ref. 15) (see Supplementary Information); and Fi needs to be independent of Λi, as well as of all other structural observables (see Supplementary Information), and therefore λ(φ) must necessarily vanish. As can be seen from Fig. 2c, d, within the range of volume fractions studied, λ(φ)decreases but saturates to a minimum as φφN=64(λ). We argue below that the saturation is a finite-size effect. An extrapolation using the linear regime in Fig. 2c indicates that λ → 0 at packing fraction φN=64(λ) = 0.82 ± 0.07, remarkably close to where our extrapolation yields SG = SB. The analysis of basin volumes, therefore, strongly suggests that equiprobability is approached only at a characteristic packing fraction and that the vanishing of λ(φ) can be used to estimate the point of equiprobability.

We next show that λ(φ) does indeed tend to zero in the thermodynamic limit. We use the fluctuations σf2, σΛ2, and the covariance σ2, obtained from the elements of the covariance matrix of the joint distribution of f and Λ (see Methods for details), to define λ and c as:

From Fig. 2b we observe that the decrease of λ is driven by σΛ2 increasing to a maximum, while σf2 and σ2 decrease (see Supplementary Fig. 2). We expect the main features of these distributions to be preserved as the system size N is increased13, which suggests that for larger N, where basin volume calculations are still intractable for multiple densities, the maximum in σΛ2 can be used to identify φN. We have directly measured χΛ = Λ2 using our sampling scheme—which samples packings with probability proportional to the volume of their basin of attraction—for systems of up to N = 128 discs (see inset of Fig. 3a) and finite-size scaling indicates that χΛ diverges as φφN(Λ) = 0.841(3) (see Supplementary Information). The saturation of λ to a minimum as φφN, for small N, is determined by the fact that χΛ diverges only in the thermodynamic limit; a detailed discussion is given in the Methods.

Figure 3: Finite-size scaling analysis.
figure 3

a, and (inset) χΛΛ2, plotted as a function of volume fraction φ. By finite-size scaling (see Supplementary Information) we show that the curves diverge in the thermodynamic limit as φφJ/, implying φN = φNJ = 0.841(3) (see main text for discussion). For φ φNJ, χP approaches a constant value indicating the absence of extensive correlations far from the transition. b, Observed average log-pressure and (inset) probability of obtaining jammed packings by our protocol, as a function of volume fraction φ. By finite-size scaling (see Supplementary Information) we show that as φφNJ(Λ) = 0.841(3) and pJ collapses for φφNJ(pJ) = 0.844(2), thus locating the unjamming point. Error bars, computed by BCa bootstrap29, refer to 1σ confidence intervals. The solid lines are generalized sigmoid fits of the form f(φ) = a − (ab)/(1 + exp(− wΔφ))1/u. We show only values of φ where the probability of finding a jammed packing is at least 1%, so that the observables are computed over sufficiently large sample sizes.

Interestingly, we find evidence that in the thermodynamic limit, the point of equiprobability, φN, coincides with the point at which the system unjams, φNJ. We use two characteristics of the unjamming transition to locate φNJ: the average pressure of the packings goes to zero, and therefore 〈Λ〉 → − (see Fig. 3b); and the probability of finding jammed packings, pJ, goes to zero (see inset of Fig. 3b). A scaling analysis indicates that 〈Λ〉 → − as φNJ(Λ) = 0.841(3), and pJ → 0 as φNJ(pJ) = 0.844(2) (see Supplementary Information). We thus find that φN = φNJ within numerical error and up to corrections to finite-size scaling20. Our simulations therefore lead to the surprising conclusion that the Edwards conjecture appears to hold precisely at the (un)jamming transition. We note that our earlier simulations, which were performed at densities much above jamming13,15, did not support the equiprobability hypothesis. The earlier simulations were in fact far enough away from unjamming that the emergence of equiprobability at this point could not be anticipated. Our earlier findings, therefore, do not contradict the more recent ones and are completely consistent with these.

Why is χΛ related to the unjamming transition? As the particles interact via purely repulsive potentials, the pressure P is strictly positive, which implies that the fluctuations of P have a floor and go to zero at unjamming. The relative fluctuations can be non-zero, and a diverging χP would then imply a diverging χΛ. Because of the bounded nature of P (refs 21,22,23), however, χP can diverge only at the unjamming transition where (see Methods). We find that χP does diverge (Fig. 3a) and finite-size scaling yields φN(P) = 0.841(3), in agreement with what has been found for χΛ. Returning to the N = 64 case that we have analysed using the basin volume statistics, we find that both χP and χΛ saturate to their maximum values over similar ranges of φ and our estimate φN=64 ≈ 0.82, where SG = SB and λ → 0, falls in this region. In addition, the average number of contacts is close to the isostatic value zN=64(iso) ≡ 2d − 2/64 ≈ 3.97 (ref. 10) (see Supplementary Information).

Finally, we note that the states in the generalized Edwards ensemble5,21,24,25 characterized by φ and P have basin volumes that are similar, if not identical, over the full range of φ that we have explored (see scatter plot in Fig. 2b), indicating that equiprobability in the stress–volume ensemble5,24 is a more robust formulation of the Edwards hypothesis. This observation is consistent with recent experiments26.

Although, the equiprobability of jammed states at a given packing fraction was posited by Edwards for jammed packings of hard particles, our analysis shows that for soft particles, the Edwards hypothesis is valid only for the marginally jammed states at φN = φNJ, where the jamming probability vanishes, the entropy is maximized, and relative pressure fluctuations diverge. We have shown not only that there exists a practical ‘Edwardsian’ packing generation protocol, capable of sampling jammed states equiprobably, but we have uncovered an unexpected property of the energy landscape for this class of systems. At this stage, we cannot establish whether the same considerations are valid in three dimensions, although the already proven validity of equation (16) in three dimensions would suggest so13. The exact value of the entropy at unjamming, whether finite or not, also needs to be elucidated. The implications for ‘soft’ structural glasses is apparent: at φJ the uniform size of the basins implies that the system, when thermalized, has the same probability of visiting all of its basins of attraction; hence, there are no preferred inherent structures. This could be a signature of the hard-sphere transition occurring at the same point27. Our approach can, therefore, be extended to spin-glasses and related problems, and it would be clearly very exciting to explore the analogies and differences between ‘jamming’ in various systems for which the configuration space can break up into many distinct basins.


Packing preparation protocol.

In this section we describe the algorithm that we have used to sample the phase space of jammed packings. This procedure samples each configuration proportionally to the volume of its basin of attraction.

Hard-sphere fluid sampling.

We start by equilibrating a fluid of N hard discs (that serve as the cores of the particles with soft outer shells) at a volume fraction φHS in a square box with periodic boundary conditions. The particle radii are sampled from a truncated Gaussian distribution with mean μ = 1 and standard deviation σ = 0.1. We achieve equilibration by performing standard Markov chain Monte Carlo simulations consisting of single-particle displacements and particle–particle swaps, as in ref. 13. To ensure statistical independence, we draw fluid configurations every nMC steps, where nMC is (pre-computed by averaging over multiple simulations) the total number of Markov chain Monte Carlo steps necessary for each individual particle to diffuse at least a distance equal to the largest diameter in the system.

Soft shells and minimization.

We next take each equilibrated hard-disc fluid configuration and inflate the particles (instantaneously) with a WCA-like soft outer shell30, to reach the target soft packing fraction φSS > φHS. Each hard sphere is inflated proportionally to its radius, so that the soft-sphere radius is given by

where d is the dimensionality of the box (2 in our case), and rSS and rHS are the soft- and hard-sphere radii respectively. Clearly, this procedure does not change the polydispersity of the sample. The radii are identical across volume fractions and system sizes, and the hard-disc fluid density is chosen so that the radius ratio of hard to soft discs is (0.88/0.7)1/2 ≈ 1.121.

Next, particle inflation is followed by energy minimization using FIRE31,32, to produce mechanically stable packings at the desired soft volume fractions φ. This protocol has the advantage of generating packings sampled proportionally to the volume of their basin of attraction. In our simulations, we considered all mechanically stable packings, irrespective of the number of ‘rattlers’. To guarantee mechanical stability we required that the total number of contacts is sufficient for the bulk modulus to be strictly positive, Nmin = d(Nnr − 1) + 1 (ref. 33), where Nnr is the number of non-rattlers and d is the dimensionality of the system.

Our implementation of FIRE enforces a maximum step size (set to be equal to the soft-shell thickness) and forbids uphill steps by taking one step back every time the energy increases (and restarts the minimizer in the same fashion as the original FIRE implementation). We use a maximum time step Δtmax = 1, although the maximum step size is directly controlled in our implementation. All other parameters are set as in the original implementation31.

HS-WCA potential.

We define the WCA-like potential around a hard core as follows: consider two spherical particles with a distance between the hard cores rHS, implying a soft-core contact distance rSS = rHS(1 + θ), with θ = (φSS/φHS)1/d − 1. We can then write a horizontally shifted hard-sphere plus WCA (HS-WCA) potential as

where σ(rHS) = (2θ + θ2)rHS2/21/6 guarantees that the potential function and its first derivative go to zero at rSS. For computational convenience (avoidance of square-root evaluations), the potential in equation (4) differs from the WCA form in that the inter-particle distance in the denominator of the WCA potential has been replaced with a difference of squares.

A power series expansion of equation (4) yields

hence, in the limit of no overlap the pair potential is harmonic.

We numerically evaluate this potential, matching the gradient and linearly continuing the function uHS-WCA(r) for rrHS + ɛ, where ɛ > 0 is an arbitrary small constant, such that minimization is still meaningful if hard-core overlaps do occur.

Our choice of potential is based on the fact that: the hard cores greatly reduce the amount of configurational space to explore, replacing expensive energy minimizations (to test whether the random walker has stepped outside the basin) with fast hard-core overlap rejections; and the hard cores exclude high-energy minima (jammed packings) that are not ‘hard-sphere-like’.

Total accessible volume.

The basins of attraction of energy minima tile the ‘accessible’ phase space (schematically shown in Fig. 1b, c). This inaccessible part of the phase space arises due to hard-core constraints and the existence of fluid states (see, for example, ref. 13). The total phase-space volume is equal to VboxN. The inaccessible part of this volume arising from the hard-core constraints (shown as hatched areas in Fig. 1) is denoted by VHS, and the part corresponding to the coexisting unjammed fluid states is denoted by Vunj (shown as blue regions with solid dots in Fig. 1b, c). Vunj is significant only for finite-size systems at or near unjamming. We denote the space tiled by the basins of mechanically stable jammed packings by VJ. We then have VJ = VboxNVHSVunj. In practice we compute VJ using the following equation

where fex(φHS) is the excess free energy, that is, the difference in free energy between the hard-sphere fluid and the ideal gas, computed from the Santos–Yuste–Haro equation of state34 as in ref. 13, and pJ(φSS) is the probability of obtaining a jammed packing at soft volume fraction φSS with our protocol, shown in the inset of Fig. 3b.

Counting by sampling.

We briefly review our approach to computing the number Ω of distinct jammed packings for a system of N soft discs at volume fraction φ. We prepare packings by the protocol described above, which generates jammed structures (energy minima) with probability pi proportional to the volume of their basin of attraction vi. We define the probability of sampling the ith packing as

where VJ is the total accessible phase space, such that

Details of the computation of vi are discussed in refs 13,16. To find Ω, we make the simple observation

from which it follows immediately that

The ‘Boltzmann-like’ entropy, suggested in a similar form by Edwards1, is then

where the lnN! correction ensures that two systems in identical macrostates are in equilibrium under exchange of particles17,18,19.

Note that 〈v〉 is the unbiased average basin volume (the mean of the unbiased distribution of volumes). We distinguish between the biased, (as sampled by the packing protocol), and the unbiased, , basin log-volumes distributions (F = −lnvbasin). Since the configurations were sampled proportionally to the volume of their basin of attraction, we can compute the unbiased distribution as

where is the normalization constant, such that

Since small basins are much more numerous than large ones, and grossly under-sampled, it is not sufficient to perform a weighted average of the sampled basin volumes. Instead, to overcome this problem, one can fit the biased measured basin log-volumes distribution with an analytical (or at least numerically integrable) distribution, and perform the unbiasing via equation (12) on the best fitting distribution. Different approaches to modelling this distribution give rise to different analysis methods, which all yield consistent results as shown in ref. 13. In this work we follow ref. 13 and fit using both a (parametric) generalized Gaussian model35 (see Supplementary Equation 21) and a (non-parametric) kernel density estimate with Gaussian kernels36,37 and bandwidth selection performed by cross-validation13,38, yielding consistent results in agreement with ref. 13. Before performing the fit we remove outliers from the free-energy distribution in an unsupervised manner, as discussed in the ‘Data analysis’ section of the Methods.

No such additional steps are needed to compute the ‘Gibbs-like’ version of the configurational entropy, in fact

is simply the arithmetic average of the observed volumes: the sample mean of F = −lnvbasin is already correctly weighted because our packing generation protocol generates packings with probability pi.

Power-law between pressure and basin volume.

A power-law relationship between the volume of the basin of attraction of a jammed packing and its pressure was first reported in ref. 13. In what follows we provide insight into this expression on the basis of this work’s findings. We observe that distributions of basin negative log-volumes, F = −lnvbasin, and log-pressures, Λ = lnP, are approximately normally distributed (see Supplementary Figs 1 and 9). We therefore expect their joint probability to be well approximated by a bivariate Gaussian distribution (when listing a function’s arguments we place parameters that are held constant before the semicolon), with mean μ = (μF, μΛ) and covariance matrix (ref. 39). This is consistent with the elliptical distribution of points in Fig. 2b. For a given random variable X, with an (observed/biased) marginal distribution , the mean is given by . Similarly, the (biased) conditional expectation of F for a given Λ is then39

This is simply the linear minimum mean square error regression estimator for F, that is, the linear estimator that minimizes . The expectation of the dimensionless free energy (ref. 40) is the average basin negative log-volume at volume fraction φ and log-pressure Λ. Here the average is also taken over all other relevant, but unknown, order parameters Γ, such that . In other words, we write the expectation of F at a given pressure as the (biased) average over the unspecified order parameters Γ. An example of such a parameter would be some topological variable that makes certain topologies more probable than others. Note that F(φ, Λ; Γ) is narrowly distributed around . To simplify the notation we write . We can thus rewrite the power law reported in ref. 13 as

where f = F/N is the basin negative log-volume per particle and λ ≡ 1/κ is the slope of the power-law relation, which depends crucially on the packing fraction φ. The last equality in equation (16) highlights how λ(φ) controls the contributions of the fluctuations of the log-pressures ΔΛΛμΛ(φ) to changes in the basin negative log-volume. Note that one can rewrite the ratio of fluctuations as σ2/σΛ2 = ρσf/σΛ where ρ = σ2/(σfσΛ) is the linear correlation coefficient of f and Λ. Finally, we can gain further insight into the power-law dependence by noting that

Data analysis.

Reduced units. While presenting data from our computations, we express pressure and volume in reduced units as P/P and v/v respectively. The unit of volume is given by vπrHS2〉, where 〈rHS2〉 is the mean squared hard-sphere radius. The unit of pressure is then Pε/v, where ε is the stiffness of the soft-sphere potential, defined in equation (4). The pressure is computed as where is the virial stress tensor and Vbox is the volume of the enclosing box.

Summary of calculations.

For the basin-volume calculations we consider systems of N = 64 discs sampled at a range of 8 volume fractions 0.828 ≤ φ ≤ 0.86 and for each φ we measure the basin volume for about 365 < M < 770 samples.

For the finite-size scaling analysis of the relative pressure fluctuations we study system sizes N = 32, 48, 64, 80, 96 and 128 for 48 volume fractions in the range 0.81 ≤ φ ≤ 0.87. For each system size we generate up to 105 hard-disc fluid configurations and compute the pressure for between approximately 103 and 104 jammed packings (depending on the probability of obtaining a jammed packing at each volume fraction).

Simulations were performed using the open source libraries PELE41 and MCPELE42.

Outlier detection and robust covariance estimation.

Before manipulating the raw data we remove outliers from the joint distribution following the distance-based outlier removal method introduced in ref. 43. This is applied in turn to each dimension, such that we choose to keep only those points for which at least R = 0.5 of the remaining data set is within D = 4σ (compared with the much stricter R = 0.9988, D = 0.13σ required to exclude any points further than |μ − 3σ| for normally distributed data43). On our data sets we find that this procedure removes typically none and at most 0.8% of all data points.

Mean and covariance estimates of are computed using a robust covariance estimator, namely the minimum covariance determinant (MCD) estimator37,44 with support fraction h/nsamples = 0.99. The MCD estimator defines μMCD, the mean of the h observations for which the determinant of the covariance matrix is minimal, and , the corresponding covariance matrix45. We use these robust estimates of the location and of the covariance matrix (computed over 1,000 bootstrap samples28) to fit our observations by linear minimum mean square error39 (see Fig. 2).

Before fitting (required to compute Ω), we perform an additional step of outlier detection based on an elliptic (Gaussian) envelope criterion constructed using the MCD estimator. We assume a support fraction h/nsamples = 0.99 and a contamination equal to 10% (ref. 37). We compute SG and SB from the resulting data sets. The procedure is strictly unsupervised and allows us to achieve robust fits despite the small sample sizes. We fit using both a (parametric) generalized Gaussian model35 and a (non-parametric) kernel density estimate with Gaussian kernels36,37 and bandwidth selection performed by cross-validation13,38.

Error analysis.

Errors were computed analytically where possible and propagated using the ‘uncertainties’ Python package46. Alternatively, intervals of confidence were computed by bootstrap for the covariance estimation28 and by BCa bootstrap otherwise using the ‘scikit-bootstrap’ Python package29,47.

Data availability.

The data that support the plots within this paper and other findings of this study are available from the University of Cambridge data repository doi:

Additional Information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.