Abstract
From thermodynamic origins, the concept of entropy has expanded to a range of statistical measures of uncertainty, which may still be thermodynamically significant^{1,2}. However, laboratory measurements of entropy continue to rely on direct measurements of heat. New technologies that can map out myriads of microscopic degrees of freedom suggest direct determination of configurational entropy by counting in systems where it is thermodynamically inaccessible, such as granular^{3,4,5,6,7,8} and colloidal^{9,10,11,12,13} materials, proteins^{14} and lithographically fabricated nanometrescale arrays. Here, we demonstrate a conditionalprobability technique to calculate entropy densities of translationinvariant states on lattices using limited configuration data on small clusters, and apply it to arrays of interacting nanometrescale magnetic islands (artificial spin ice)^{15}. Models for statistically disordered systems can be assessed by applying the method to relative entropy densities. For artificial spin ice, this analysis shows that nearestneighbour correlations drive longerrange ones.
Main
Our artificial spinice^{15} systems are arrays of lithographically defined singledomain ferromagnetic islands (25 nm thick and 220 nm×80 nm in area) on the links of square and honeycomb lattices (Fig. 1). Shape anisotropy forces island moments to point along the long axes, forming effective Ising spins. The coercive field is about 770 Oe (that is, a barrier of order 10^{5} K), whereas the field from one island on a neighbour is only of order 10 Oe. The arrays are demagnetized by rotating in an inplane external magnetic field H_{ext}, initially strong enough to produce complete polarization, subsequently reduced to zero in small increments^{15,16} ΔH_{ext}, with reversal of the field at each step. For small step sizes, the result is a statistically reproducible macrostate, operationally defined by the demagnetization protocol^{15,16}, which is probed by magnetic force microscopy to obtain the static moments of individual islands. We want the entropy of a single macrostate, but distinct runs might produce distinct macrostates (for example, a residual magnetization at larger step size). In most cases, data are collected in a single run, averting the problem, but the entropy of a macrostate mixture would be relatively unimportant anyway, as is shown in Supplementary Section S3. For large structurally regular systems such as ours, it is more appropriate to work not with total entropy, but with entropy density (see equation (1)) having units of bits per island, a value of 1 corresponding to complete disorder.
The strongest interactions, between islands meeting at a vertex, favour headtotail moment alignment. However, not all of these can be satisfied simultaneously, resulting in a kind of frustration. Still, for the square lattice, the ground state is only twofold degenerate^{15}, because typeI vertices, as defined in Fig. 1, are lowest in energy. That the ordered ground state is never found experimentally^{16,17} suggests that the evolution is kinetically constrained^{18,19}. For instance, one spin flip converts a typeII to a typeIII; flips of two perpendicular islands are required to reach typeI. In contrast to the square lattice, the honeycomb lattice has a macroscopically degenerate ground state when only nearest or nextnearestneighbour interactions are effective (longerrange interactions break the degeneracy^{20} at a much lower energy scale). The interactions prefer a 2in/1out or 1in/2out arrangement at every vertex (quasiice rule). This constraint alone produces a state, ideal quasiice, with an entropy density of 0.724 bits per island. Interactions between (mono)pole strengths Q at neighbouring vertices^{21,22} reduce the groundstate degeneracy to 0.15 bits per island by favouring Q=−1 (2in/1out) next to Q=+1. The contrast between the square and honeycomblattice ground states—twofold degenerate versus macroscopically degenerate—provides an opportunity to investigate the interplay between the strictures of kinetic constraint and the freedom of massive degeneracy.
We now develop a method to extract the entropy densities on our lattices from the measured configurations of the island magnetic moments. Consider a finite cluster Λ of islands, for example, the fiveisland cluster comprising two adjacent vertices (see Fig. 2 legend) and the collection of random variables σ_{Λ} that are the spins belonging to Λ. The Shannon–Gibbs–Boltzmann entropy^{23,24,25} of P_{Λ}(σ_{Λ}), the distribution of σ_{Λ}, is
where the sum runs over all possible values of the random variable(s) σ_{Λ}. Note that S is rendered dimensionless by omitting Boltzmann’s constant, and the base of the logarithm is 2, so that the units are bits. If Λ is taken ever larger while the fraction of islands on the edge tends to zero (van Hove limit), we obtain the bulk entropy density s:
If each island moment independently points either way with probability 1/2, then the entropy density is one bit per island, the largest possible. Lower entropy density indicates correlations in a generic sense. For example, the fully polarized initial state created by a large H_{ext} has zero entropy density.
The obvious approximation to s suggested by equation (1) is simply S(P_{Λ})/Λ for the biggest practicable cluster. However, this ‘simple clusterestimate’ is not very good because the configuration space grows exponentially with cluster size Λ, and boundarycrossing correlations are completely neglected. To understand the last point, suppose the entire lattice covered without gaps or overlap by translates of Λ. The state constructed from the marginals of P on those translates, taking them to be independent, has entropy density exactly S(P_{Λ})/Λ. However, shortrange boundarycrossing correlations are the same as corresponding intracluster correlations, so are reflected in smallcluster data and can be properly counted using conditional entropy. The method resembles one proposed some years ago^{26,27} for Monte Carlo simulations of latticespin models in equilibrium.
One way to think of the total entropy of a given macrostate is as the average uncertainty about the particular microstate at hand. Imagine a microstate of the honeycomb lattice revealed three islands (one vertex) at a time, rowbyrow. One instant in the process looks like this (the grey vertex is about to be revealed):
Neglecting the (far distant) lattice edge, each newly revealed vertex bears the same spatial relation to those already known, so the revelation, on average, reduces the uncertainty by exactly three times the entropy per spin. Cast this way, the entropy density appears as a conditional entropy^{28}. The conditional entropy of σ_{Λ} given σ_{Γ} is
where P(σ_{Λ}σ_{Γ})=P(σ_{Λ},σ_{Γ})/P(σ_{Γ}) is the conditional probability of σ_{Λ} given σ_{Γ}. The joint entropy of σ_{Λ} and σ_{Γ} then has the pleasant decomposition S(σ_{Λ},σ_{Γ})=S(σ_{Λ}σ_{Γ})+S(σ_{Γ}). (Learning σ_{Γ} and σ_{Λ} at once is the same as learning σ_{Γ} and then σ_{Λ}.) Note that if Λ and Γ overlap, common spins contribute zero to S(σ_{Λ}σ_{Γ}).
As a simple illustration, suppose Λ and Γ are single islands, with the probabilities for P(σ_{Λ},σ_{Γ}) being given by P(↓,↑)=0 and P(↑,↑)=P(↑,↓)=P(↓,↓)=1/3. If we know that σ_{Γ}=↑, then the remaining uncertainty about σ_{Λ} is zero, but if we know that σ_{Γ}=↓, then the uncertainty is total: 1 bit. Weighting by the probabilities of σ_{Γ} to be ↑ or ↓ gives the entropy of σ_{Λ} conditioned on σ_{Γ}: P(σ_{Γ}=↑)·(0)+P(σ_{Γ}=↓)·(1)=2/3 bit.
The Methods section explains how conditional entropy and other basic notions of information theory can be used to obtain good approximations to the entropy density s from limited data. The result of applying two such approximations to the experimental data for honeycomb lattices is plotted in Fig. 2 as a function of field step size ΔH_{ext} for each lattice constant, along with one simple clusterestimate for comparison. Dataset sizes are reported in Supplementary Section S1. The simple clusterestimate S(Λ)/Λ using the fiveisland divertex (Fig. 2 legend) provides a very poor bound compared with our conditioning technique. Reducing the lattice constant or step size should lower the entropy because the first leads to stronger interactions, and the second gives interactions a better chance to be the decisive factor for island flips. The expected lattice constant trend is seen but there is an unexpected plateau with respect to ΔH_{ext}.
It makes sense to compare the experimental states with ideal honeycomb quasiice through entropy densities. That of honeycomb quasiice is 0.724 bits per island (Supplementary Section S2). However, proper comparison requires using the same estimates for both systems. The dashed lines in the plot show the estimates for the model system, the upper for the simple clusterestimate (compare crosses) and the lower two (indistinguishable) for the conditioning estimates. Hence, the 520 nm array at ΔH_{ext}=1.6 Oe has lower entropy than ideal quasiice. This can be explained by correlations between net magnetic charge Q=±1 of nearestneighbour vertices. Ideal quasiice has a weak anticorrelation: 〈Q_{i}Q_{j}〉≈−1/9. In some experimental samples, this correlation reaches −0.25, reflected in a small entropy decrease. Complete sublattice ordering, 〈Q_{i}Q_{j}〉=−1, would reduce the entropy to s≈0.15 bits per island (Supplementary Section S2). This extra correlation may explain the slightly better performance of the bound with a complete vertex in the conditioning data.
The entropy of honeycomb artificial spin ice reveals a state close to ideal quasiice, with slight antiferromagnetic vertex charge ordering. The contrasting failure of a.c. demagnetization of the square lattice to approach the completely ordered ground state is precisely quantified by entropy. We use three approximations (upper bounds) for the squarelattice entropy density. They are found by a procedure parallel to that for the honeycomb lattice and are shown in the legend of Fig. 3. The three agree well, and this rough convergence test suggests that they are close to the true entropy densities. As for the honeycomb lattice, we expect smaller entropy for smaller ΔH_{ext} or smaller lattice spacing. In general, this seems to be the case, but the ground state is never approached. Even extrapolations ΔH_{ext}→0 have large entropy densities.
Closer inspection suggests jamming at ΔH_{ext}. A kinetically constrained approach to ground states defines the behaviour of complex systems across many fields^{19}, such as protein folding^{14}, selfassembly, glasses and granular systems^{3,4,5,6,7,8}. Ergodicity is thwarted by both tall energy barriers and configurationspace constrictions, combined into freeenergy barriers. An ergodic system explores all of configuration space, whereas folding proteins live within a ‘folding funnel’. This dynamic constriction of allowed configurations introduces many deep conceptual challenges. a.c.demagnetized artificial spin ice puts the conceptual challenge of kinetic constraint into sharp relief: as the rotating external field weakens, islands onebyone fall out of ‘field following’ mode, driven by interisland interactions that suppress the local depolarization field and lock in the orientation of the fallenaway island. Thus, each spin probably makes only a single decision on how to point on escaping coercion, with no prospect of later surmounting barriers. The system’s approach to the ground state is essentially oneway. Thus, it is not surprising that only a macroscopically degenerate groundstate target can be ‘hit’. Notwithstanding this failure of ergodicity, squarelattice artificial spin ice can still be described by a statistical model based, like thermodynamics, on maximum entropy^{16,17}. As an extreme case of kinetic constraint, rotationally annealed artificial spin ice can afford unique insights into statistical mechanics of complex systems. For example, array topology can control the groundstate degeneracy, as seen here.
Even without detailed knowledge of how the final squarelattice states develop, a concisely descriptive model may be sought. We conjecture that the lattice state is fully determined by correlations between nearestneighbour pairs diagonally or straight across a vertex, and thus model it by a constrained maximumentropy state, which is as random as possible, consistent with those correlations. Adapting the conditioning techniques (see the Methods section), we can efficiently estimate the entropy density of the experimental states relative to the maximumentropy state, s(exptME). This global measure of dissimilarity does not depend on identifying the ‘right’ correlations, and allows an assessment of the model. The results are given in Table 1.
Note that^{29} the probability that a typical experimental state in region Λ is likely to be mistaken for a maximumentropy state decays asymptotically as exp[−Λs(exptME)]. Apparently, the entropy reduction below independent islands in the square lattices is well accounted for by nearestneighbour correlations, and those they entail.
The reduction in the entropy of an interacting system below that of uncoupled degrees of freedom is due mostly to shortrange correlations, even near a critical point. Thus, the efficient conditional entropy technique described here can be applied to a wide variety of resolvable complex systems such as granular media and colloidal systems that can now be spatially resolved in the required detail^{9,10,11,12,13}. Entropy density is a general measure of order that is not tied to preidentified correlations. Hence, it is especially valuable for states such as square ice or other jammed, glassy states that are far from identifiable landmarks.
Methods
According to the discussion around equation (2), the entropy density s of the infinite honeycomb lattice is given by (ignore the colour for now)
Now we find smallcluster approximations to this entropy density, using two principles^{28}. (1) If σ_{Γ} is known there is no uncertainty about it, so S(σ_{Λ},σ_{Γ}σ_{Γ})=S(σ_{Λ}σ_{Γ}) for arbitrary Λ and Γ. Thus, in pictorial equations, visual perspicuity will dictate retention or omission of conditioning variables on the left of the bar. (2) Providing more conditioning information lessens uncertainty: S(σ_{Λ}σ_{Γ})≥S(σ_{Λ}σ_{Γ},σ_{Γ}^{′}). Unlike a simple application of equation (1), our conditional entropy method fully accounts for shortrange correlations without boundary error. Like the simple clusterestimate, it provides upper bounds on the true entropy density. Dropping all but the red islands in equation (3) yields
Alternatively, if we add a vertex in two stages rather than all at once as in equation (3), we immediately arrive at
The red islands again provide a visual cue. Following it, we get the bound
Bounds for a translationinvariant state on the square lattice can be obtained in a similar way. We use the onestep bounds
and
and the twostep bound
A constrained maximumentropy state on the square lattice with given correlations between diagonal and acrossthevertex nearestneighbour correlations coincides with a Gibbs state for an Ising model with effective pair interactions for the two types of nearest neighbour of whatever strength is required to reproduce the required correlations. We have previously^{16} studied specific pair correlations in such maximumentropy states using Monte Carlo simulation. Building on the conditioning techniques developed in this Letter, we compute s(exptME), the relative entropy density of an experimental state to the corresponding constrained maximumentropy state. In the case of two probability measures on the configuration space of a lattice system, the relative entropy of their restrictions to some finite region Λ is
The limiting relative entropy density that we want is
The logarithm of the probability in equation (6) can be expanded in terms of conditionals just as was done for the entropy. For any collection {X_{1},…,X_{N}} of random variables (the m=1 term being read as an unconditioned probability),
parallels exactly the formula
The main difference is that log_{2}P(·) is a random variable, whereas S(·) is a number. Any of the class of approximations for the conditional entropy densities can now be applied to the conditional probabilities to obtain the relative entropy. However, we do not get bounds in this way, just ordinary estimates.
If P_{ME} is a maximumentropy state constrained to have expectations of specified observables match their expectations in P, then the relative entropy density s(PP_{ME}) ought to equal the difference in the absolute densities. We believe that the method we have used is superior to this simple subtraction because it suppresses the unwanted effects of fluctuations in the counting of lowprobability configurations. Owing to limited experimental data, configurations that would be expected to have only a few occurrences may have none at all, which has an anomalously large effect in the subtraction.
References
Landauer, R. Irreversibility and heat generation in the computing process. IBM J. Res. Dev. 5, 183–191 (1961).
Leff, H. S. & Rex, A. F. (eds) Maxwell’s Demon 2: Entropy, Classical and Quantum Information, Computing 2nd edn (Institute of Physics, 2003).
Liu, A. J. & Nagel, S. R. Jamming is not just cool any more. Nature 396, 21–22 (1998).
Coniglio, A. & Nicodemi, M. The jamming transition of granular media. J. Phys. Condens. Matter 12, 6601–6610 (2000).
O’Hern, C. S., Sibert, L. E., Liu, A. J. & Nagel, S. R. Jamming at zero temperature and zero applied stress: The epitome of disorder. Phys. Rev. E 68, 011306 (2003).
Corwin, E. I., Jaeger, H. M. & Nagel, S. R. Structural signatures of jamming in granular media. Nature 435, 1075–1078 (2005).
Majumdar, T. S., Sperl, M., Luding, S. & Behringer, R. P. Jamming transition in granular systems. Phys. Rev. Lett. 98, 058001 (2007).
Behringer, R. P., Daniels, K. E., Majumdar, T. S. & Sperl, M. Fluctuations, correlations and transitions in granular materials: Statistical mechanics for a nonconventional system. Phil. Trans. R. Soc. A 366, 493–504 (2008).
Kegel, W. K. & van Blaaderen, A. Direct observation of dynamical heterogeneities in colloidal hardsphere suspensions. Science 287, 290–293 (2000).
Weeks, E. R., Crocker, J. C., Levitt, A. C., Schofield, A. & Weitz, D. A. Threedimensional direct imaging of structural relaxation near the colloidal glass transition. Science 287, 627–631 (2000).
Cui, B., Lin, B. & Rice, S. A. Dynamical heterogeneity in a dense quasitwodimensional colloidal liquid. J. Chem. Phys. 114, 9142–9155 (2001).
Gasser, U., Weeks, E. R., Schofield, A., Pusey, P. N. & Weitz, D. A. Realspace imaging of nucleation and growth in colloidal crystallization. Science 292, 258–262 (2001).
Alsayed, A. M., Islam, M. F., Zhang, J., Collings, P. J. & Yodh, A. G. Premelting at defects within bulk colloidal crystals. Science 309, 1207–1210 (2005).
Pande, V. S., Grosberg, A. Y. & Tanaka, T. Heteropolymer freezing and design: Towards physical models of protein folding. Rev. Mod. Phys. 72, 259–314 (2000).
Wang, R. et al. Artificial ‘spin ice’ in a geometrically frustrated lattice of nanoscale ferromagnetic islands. Nature 439, 303–306 (2006).
Ke, X. et al. Energy minimization and ac demagnetization in a nanomagnet array. Phys. Rev. Lett. 101, 037205 (2008).
Nisoli, C. et al. Ground state lost but degeneracy found: The effective thermodynamics of artificial spin ice. Phys. Rev. Lett. 98, 217203 (2007).
Ritort, F. & Sollich, P. Glassy dynamics of kinetically constrained models. Adv. Phys. 52, 219–342 (2003).
Janke, W. (ed.) Rugged Free Energy Landscapes (Lecture Notes in Physics, Vol. 736, Springer, 2008).
Möller, G. & Moessner, R. Magnetic multipole analysis of kagome and artificial spinice dipolar arrays. Phys. Rev. B 80, 140409(R) (2009).
Tanaka, M., Saitoh, E. & Miyajima, H. Magnetic interactions in a ferromagnetic honeycomb nanoscale network. Phys. Rev. B 73, 052411 (2006).
Qi, Y., Brintlinger, T. E. & Cumings, J. Direct observation of the ice rule in an artificial kagomé spin ice. Phys. Rev. B 77, 094418 (2008).
Wehrl, A. General properties of entropy. Rev. Mod. Phys. 50, 221–260 (1978).
Penrose, O. Foundations of Statistical Mechanics (Pergamon, 1970).
Jaynes, E. T. Information theory and statistical mechanics. Phys. Rev. 106, 620–630 (1957).
Schlijper, A. G. & Smit, B. Twosided bounds on the free energy from local states in Monte Carlo simulations. J. Stat. Phys. 56, 247–259 (1989).
Schlijper, A. G. & van Bergen, A. R. D. Localstates method for the calculation of free energies in Monte Carlo simulations of lattice models. Phys. Rev. A 41, 1175–1178 (1990).
Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley, 1991).
Dembo, A. & Zeitouni, O. Large Deviations Techniques and Applications 2nd edn (Springer, 1998).
Acknowledgements
We acknowledge the financial support from the Army Research Office and the National Science Foundation MRSEC program (DMR0820404) and the National Nanotechnology Infrastructure Network. We thank C. Leighton and M. Erickson for permalloy growth.
Author information
Authors and Affiliations
Contributions
P.S. and V.H.C. conceived the initial idea of this project and kept it on track. X.K., J.L. and D.M.G. carried out the experiments and collected data. C.N. made theoretical contributions. P.E.L. analysed data and developed theory.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Information
Supplementary Information (PDF 254 kb)
Rights and permissions
About this article
Cite this article
Lammert, P., Ke, X., Li, J. et al. Direct entropy determination and application to artificial spin ice. Nature Phys 6, 786–789 (2010). https://doi.org/10.1038/nphys1728
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nphys1728
Further reading

Advances in artificial spin ice
Nature Reviews Physics (2020)

Realization of ground state in artificial kagome spin ice via topological defectdriven magnetic writing
Nature Nanotechnology (2018)

Interaction modifiers in artificial spin ices
Nature Physics (2018)

Direct observation of deterministic domain wall trajectory in magnetic network structures
Scientific Reports (2016)

Emergent reduced dimensionality by vertex frustration in artificial spin ice
Nature Physics (2016)