What is a pattern? How do we come to recognize patterns never seen before? Quantifying the notion of pattern and formalizing the process of pattern discovery go right to the heart of physical science. Over the past few decades physics’ view of nature’s lack of structure—its unpredictability—underwent a major renovation with the discovery of deterministic chaos, overthrowing two centuries of Laplace’s strict determinism in classical physics. Behind the veil of apparent randomness, though, many processes are highly ordered, following simple rules. Tools adapted from the theories of information and computation have brought physical science to the brink of automatically discovering hidden patterns and quantifying their structural complexity.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Journal of Statistical Physics Open Access 28 March 2022
npj Systems Biology and Applications Open Access 27 January 2022
Scientific Reports Open Access 23 January 2020
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Press, W. H. Flicker noises in astronomy and elsewhere. Comment. Astrophys. 7, 103–119 (1978).
van der Pol, B. & van der Mark, J. Frequency demultiplication. Nature 120, 363–364 (1927).
Goroff, D. (ed.) in H. Poincaré New Methods of Celestial Mechanics, 1: Periodic And Asymptotic Solutions (American Institute of Physics, 1991).
Goroff, D. (ed.) H. Poincaré New Methods Of Celestial Mechanics, 2: Approximations by Series (American Institute of Physics, 1993).
Goroff, D. (ed.) in H. Poincaré New Methods Of Celestial Mechanics, 3: Integral Invariants and Asymptotic Properties of Certain Solutions (American Institute of Physics, 1993).
Crutchfield, J. P., Packard, N. H., Farmer, J. D. & Shaw, R. S. Chaos. Sci. Am. 255, 46–57 (1986).
Binney, J. J., Dowrick, N. J., Fisher, A. J. & Newman, M. E. J. The Theory of Critical Phenomena (Oxford Univ. Press, 1992).
Cross, M. C. & Hohenberg, P. C. Pattern formation outside of equilibrium. Rev. Mod. Phys. 65, 851–1112 (1993).
Manneville, P. Dissipative Structures and Weak Turbulence (Academic, 1990).
Shannon, C. E. A mathematical theory of communication. Bell Syst. Tech. J. 27 379–423; 623–656 (1948).
Cover, T. M. & Thomas, J. A. Elements of Information Theory 2nd edn (Wiley–Interscience, 2006).
Kolmogorov, A. N. Entropy per unit time as a metric invariant of automorphisms. Dokl. Akad. Nauk. SSSR 124, 754–755 (1959).
Sinai, Ja G. On the notion of entropy of a dynamical system. Dokl. Akad. Nauk. SSSR 124, 768–771 (1959).
Anderson, P. W. More is different. Science 177, 393–396 (1972).
Turing, A. M. On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. Math. Soc. 2 42, 230–265 (1936).
Solomonoff, R. J. A formal theory of inductive inference: Part I. Inform. Control 7, 1–24 (1964).
Solomonoff, R. J. A formal theory of inductive inference: Part II. Inform. Control 7, 224–254 (1964).
Minsky, M. L. in Problems in the Biological Sciences Vol. XIV (ed. Bellman, R. E.) (Proceedings of Symposia in Applied Mathematics, American Mathematical Society, 1962).
Chaitin, G. On the length of programs for computing finite binary sequences. J. ACM 13, 145–159 (1966).
Kolmogorov, A. N. Three approaches to the concept of the amount of information. Probab. Inform. Trans. 1, 1–7 (1965).
Martin-Löf, P. The definition of random sequences. Inform. Control 9, 602–619 (1966).
Brudno, A. A. Entropy and the complexity of the trajectories of a dynamical system. Trans. Moscow Math. Soc. 44, 127–151 (1983).
Zvonkin, A. K. & Levin, L. A. The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russ. Math. Survey 25, 83–124 (1970).
Chaitin, G. Algorithmic Information Theory (Cambridge Univ. Press, 1987).
Li, M. & Vitanyi, P. M. B. An Introduction to Kolmogorov Complexity and its Applications (Springer, 1993).
Rissanen, J. Universal coding, information, prediction, and estimation. IEEE Trans. Inform. Theory IT-30, 629–636 (1984).
Rissanen, J. Complexity of strings in the class of Markov sources. IEEE Trans. Inform. Theory IT-32, 526–532 (1986).
Blum, L., Shub, M. & Smale, S. On a theory of computation over the real numbers: NP-completeness, Recursive Functions and Universal Machines. Bull. Am. Math. Soc. 21, 1–46 (1989).
Moore, C. Recursion theory on the reals and continuous-time computation. Theor. Comput. Sci. 162, 23–44 (1996).
Shannon, C. E. Communication theory of secrecy systems. Bell Syst. Tech. J. 28, 656–715 (1949).
Ruelle, D. & Takens, F. On the nature of turbulence. Comm. Math. Phys. 20, 167–192 (1974).
Packard, N. H., Crutchfield, J. P., Farmer, J. D. & Shaw, R. S. Geometry from a time series. Phys. Rev. Lett. 45, 712–716 (1980).
Takens, F. in Symposium on Dynamical Systems and Turbulence, Vol. 898 (eds Rand, D. A. & Young, L. S.) 366–381 (Springer, 1981).
Brandstater, A. et al. Low-dimensional chaos in a hydrodynamic system. Phys. Rev. Lett. 51, 1442–1445 (1983).
Crutchfield, J. P. & McNamara, B. S. Equations of motion from a data series. Complex Syst. 1, 417–452 (1987).
Crutchfield, J. P. & Young, K. Inferring statistical complexity. Phys. Rev. Lett. 63, 105–108 (1989).
Crutchfield, J. P. & Shalizi, C. R. Thermodynamic depth of causal states: Objective complexity via minimal representations. Phys. Rev. E 59, 275–283 (1999).
Shalizi, C. R. & Crutchfield, J. P. Computational mechanics: Pattern and prediction, structure and simplicity. J. Stat. Phys. 104, 817–879 (2001).
Young, K. The Grammar and Statistical Mechanics of Complex Physical Systems. PhD thesis, Univ. California (1991).
Koppel, M. Complexity, depth, and sophistication. Complexity 1, 1087–1091 (1987).
Koppel, M. & Atlan, H. An almost machine-independent theory of program-length complexity, sophistication, and induction. Information Sciences 56, 23–33 (1991).
Crutchfield, J. P. & Young, K. in Entropy, Complexity, and the Physics of Information Vol. VIII (ed. Zurek, W.) 223–269 (SFI Studies in the Sciences of Complexity, Addison-Wesley, 1990).
William of Ockham Philosophical Writings: A Selection, Translated, with an Introduction (ed. Philotheus Boehner, O. F. M.) (Bobbs-Merrill, 1964).
Farmer, J. D. Information dimension and the probabilistic structure of chaos. Z. Naturf. 37a, 1304–1325 (1982).
Crutchfield, J. P. The calculi of emergence: Computation, dynamics, and induction. Physica D 75, 11–54 (1994).
Crutchfield, J. P. in Complexity: Metaphors, Models, and Reality Vol. XIX (eds Cowan, G., Pines, D. & Melzner, D.) 479–497 (Santa Fe Institute Studies in the Sciences of Complexity, Addison-Wesley, 1994).
Crutchfield, J. P. & Feldman, D. P. Regularities unseen, randomness observed: Levels of entropy convergence. Chaos 13, 25–54 (2003).
Mahoney, J. R., Ellison, C. J., James, R. G. & Crutchfield, J. P. How hidden are hidden processes? A primer on crypticity and entropy convergence. Chaos 21, 037112 (2011).
Ellison, C. J., Mahoney, J. R., James, R. G., Crutchfield, J. P. & Reichardt, J. Information symmetries in irreversible processes. Chaos 21, 037107 (2011).
Crutchfield, J. P. in Nonlinear Modeling and Forecasting Vol. XII (eds Casdagli, M. & Eubank, S.) 317–359 (Santa Fe Institute Studies in the Sciences of Complexity, Addison-Wesley, 1992).
Crutchfield, J. P., Ellison, C. J. & Mahoney, J. R. Time’s barbed arrow: Irreversibility, crypticity, and stored information. Phys. Rev. Lett. 103, 094101 (2009).
Ellison, C. J., Mahoney, J. R. & Crutchfield, J. P. Prediction, retrodiction, and the amount of information stored in the present. J. Stat. Phys. 136, 1005–1034 (2009).
Ruelle, D. Do turbulent crystals exist? Physica A 113, 619–623 (1982).
Varn, D. P., Canright, G. S. & Crutchfield, J. P. Discovering planar disorder in close-packed structures from X-ray diffraction: Beyond the fault model. Phys. Rev. B 66, 174110 (2002).
Varn, D. P. & Crutchfield, J. P. From finite to infinite range order via annealing: The causal architecture of deformation faulting in annealed close-packed crystals. Phys. Lett. A 234, 299–307 (2004).
Varn, D. P., Canright, G. S. & Crutchfield, J. P. Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part I: ε-machine Spectral Reconstruction Theory Santa Fe Institute Working Paper 03-03-021 (2002).
Varn, D. P., Canright, G. S. & Crutchfield, J. P. Inferring pattern and disorder in close-packed structures via ε-machine reconstruction theory: Structure and intrinsic computation in Zinc Sulphide. Acta Cryst. B 63, 169–182 (2002).
Welberry, T. R. Diffuse x-ray scattering and models of disorder. Rep. Prog. Phys. 48, 1543–1593 (1985).
Guinier, A. X-Ray Diffraction in Crystals, Imperfect Crystals and Amorphous Bodies (W. H. Freeman, 1963).
Sebastian, M. T. & Krishna, P. Random, Non-Random and Periodic Faulting in Crystals (Gordon and Breach Science Publishers, 1994).
Feldman, D. P., McTague, C. S. & Crutchfield, J. P. The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing. Chaos 18, 043106 (2008).
Mitchell, M., Hraber, P. & Crutchfield, J. P. Revisiting the edge of chaos: Evolving cellular automata to perform computations. Complex Syst. 7, 89–130 (1993).
Johnson, B. D., Crutchfield, J. P., Ellison, C. J. & McTague, C. S. Enumerating Finitary Processes Santa Fe Institute Working Paper 10-11-027 (2010).
Lind, D. & Marcus, B. An Introduction to Symbolic Dynamics and Coding (Cambridge Univ. Press, 1995).
Hopcroft, J. E. & Ullman, J. D. Introduction to Automata Theory, Languages, and Computation (Addison-Wesley, 1979).
Upper, D. R. Theory and Algorithms for Hidden Markov Models and Generalized Hidden Markov Models. Ph.D. thesis, Univ. California (1997).
Kelly, D., Dillingham, M., Hudson, A. & Wiesner, K. Inferring hidden Markov models from noisy time sequences: A method to alleviate degeneracy in molecular dynamics. Preprint at http://arxiv.org/abs/1011.2969 (2010).
Ryabov, V. & Nerukh, D. Computational mechanics of molecular systems: Quantifying high-dimensional dynamics by distribution of Poincaré recurrence times. Chaos 21, 037113 (2011).
Li, C-B., Yang, H. & Komatsuzaki, T. Multiscale complex network of protein conformational fluctuations in single-molecule time series. Proc. Natl Acad. Sci. USA 105, 536–541 (2008).
Crutchfield, J. P. & Wiesner, K. Intrinsic quantum computation. Phys. Lett. A 372, 375–380 (2006).
Goncalves, W. M., Pinto, R. D., Sartorelli, J. C. & de Oliveira, M. J. Inferring statistical complexity in the dripping faucet experiment. Physica A 257, 385–389 (1998).
Clarke, R. W., Freeman, M. P. & Watkins, N. W. The application of computational mechanics to the analysis of geomagnetic data. Phys. Rev. E 67, 160–203 (2003).
Crutchfield, J. P. & Hanson, J. E. Turbulent pattern bases for cellular automata. Physica D 69, 279–301 (1993).
Hanson, J. E. & Crutchfield, J. P. Computational mechanics of cellular automata: An example. Physica D 103, 169–189 (1997).
Shalizi, C. R., Shalizi, K. L. & Haslinger, R. Quantifying self-organization with optimal predictors. Phys. Rev. Lett. 93, 118701 (2004).
Crutchfield, J. P. & Feldman, D. P. Statistical complexity of simple one-dimensional spin systems. Phys. Rev. E 55, 239R–1243R (1997).
Feldman, D. P. & Crutchfield, J. P. Structural information in two-dimensional patterns: Entropy convergence and excess entropy. Phys. Rev. E 67, 051103 (2003).
Bonner, J. T. The Evolution of Complexity by Means of Natural Selection (Princeton Univ. Press, 1988).
Eigen, M. Natural selection: A phase transition? Biophys. Chem. 85, 101–123 (2000).
Adami, C. What is complexity? BioEssays 24, 1085–1094 (2002).
Frenken, K. Innovation, Evolution and Complexity Theory (Edward Elgar Publishing, 2005).
McShea, D. W. The evolution of complexity without natural selection—A possible large-scale trend of the fourth kind. Paleobiology 31, 146–156 (2005).
Krakauer, D. Darwinian demons, evolutionary complexity, and information maximization. Chaos 21, 037111 (2011).
Tononi, G., Edelman, G. M. & Sporns, O. Complexity and coherency: Integrating information in the brain. Trends Cogn. Sci. 2, 474–484 (1998).
Bialek, W., Nemenman, I. & Tishby, N. Predictability, complexity, and learning. Neural Comput. 13, 2409–2463 (2001).
Sporns, O., Chialvo, D. R., Kaiser, M. & Hilgetag, C. C. Organization, development, and function of complex brain networks. Trends Cogn. Sci. 8, 418–425 (2004).
Crutchfield, J. P. & Mitchell, M. The evolution of emergent computation. Proc. Natl Acad. Sci. USA 92, 10742–10746 (1995).
Lizier, J., Prokopenko, M. & Zomaya, A. Information modification and particle collisions in distributed computation. Chaos 20, 037109 (2010).
Flecker, B., Alford, W., Beggs, J. M., Williams, P. L. & Beer, R. D. Partial information decomposition as a spatiotemporal filter. Chaos 21, 037104 (2011).
Rissanen, J. Stochastic Complexity in Statistical Inquiry (World Scientific, 1989).
Balasubramanian, V. Statistical inference, Occam’s razor, and statistical mechanics on the space of probability distributions. Neural Comput. 9, 349–368 (1997).
Glymour, C. & Cooper, G. F. (eds) in Computation, Causation, and Discovery (AAAI Press, 1999).
Shalizi, C. R., Shalizi, K. L. & Crutchfield, J. P. Pattern Discovery in Time Series, Part I: Theory, Algorithm, Analysis, and Convergence Santa Fe Institute Working Paper 02-10-060 (2002).
MacKay, D. J. C. Information Theory, Inference and Learning Algorithms (Cambridge Univ. Press, 2003).
Still, S., Crutchfield, J. P. & Ellison, C. J. Optimal causal inference. Chaos 20, 037111 (2007).
Wheeler, J. A. in Entropy, Complexity, and the Physics of Information volume VIII (ed. Zurek, W.) (SFI Studies in the Sciences of Complexity, Addison-Wesley, 1990).
I thank the Santa Fe Institute and the Redwood Center for Theoretical Neuroscience, University of California Berkeley, for their hospitality during a sabbatical visit.
The author declares no competing financial interests.
About this article
Cite this article
Crutchfield, J. Between order and chaos. Nature Phys 8, 17–24 (2012). https://doi.org/10.1038/nphys2190
This article is cited by
Nature Astronomy (2022)
npj Systems Biology and Applications (2022)
Nonlinear Dynamics (2022)
Journal of Statistical Physics (2022)
Journal of Statistical Physics (2022)