Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Between order and chaos

Nature Physics volume 8, pages 17–24 (2012)Cite this article

A Corrigendum to this article was published on 03 June 2013

This article has been updated

Abstract

What is a pattern? How do we come to recognize patterns never seen before? Quantifying the notion of pattern and formalizing the process of pattern discovery go right to the heart of physical science. Over the past few decades physics’ view of nature’s lack of structure—its unpredictability—underwent a major renovation with the discovery of deterministic chaos, overthrowing two centuries of Laplace’s strict determinism in classical physics. Behind the veil of apparent randomness, though, many processes are highly ordered, following simple rules. Tools adapted from the theories of information and computation have brought physical science to the brink of automatically discovering hidden patterns and quantifying their structural complexity.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: ε-machines for four information sources.

© 1994 Elsevier

Figure 2: Structure versus randomness.
Figure 3: Complexity–entropy diagrams.

Similar content being viewed by others

Change history

  • 17 May 2013

    In the version of this Review Article originally published online there were several errors. In the section entitled 'Complicated yes, but is it complex?', seventh paragraph, the subscripts should appear as follows: past X:t; future Xt:; and blocks Xt:t'. All instances of x:l should appear as x0:l. In the section entitled 'Applications', in the third and seventh paragraphs, x: should appear as x0:. In the caption of Fig. 1d, the distributions should be listed as Pr(A, B, C, D), and panel d should have been attributed to ref. 45. These errors have now been corrected in the PDF and HTML versions of the Review Article.

References

  1. Press, W. H. Flicker noises in astronomy and elsewhere. Comment. Astrophys. 7, 103–119 (1978).

    ADS  Google Scholar 

  2. van der Pol, B. & van der Mark, J. Frequency demultiplication. Nature 120, 363–364 (1927).

    ADS  Google Scholar 

  3. Goroff, D. (ed.) in H. Poincaré New Methods of Celestial Mechanics, 1: Periodic And Asymptotic Solutions (American Institute of Physics, 1991).

  4. Goroff, D. (ed.) H. Poincaré New Methods Of Celestial Mechanics, 2: Approximations by Series (American Institute of Physics, 1993).

  5. Goroff, D. (ed.) in H. Poincaré New Methods Of Celestial Mechanics, 3: Integral Invariants and Asymptotic Properties of Certain Solutions (American Institute of Physics, 1993).

  6. Crutchfield, J. P., Packard, N. H., Farmer, J. D. & Shaw, R. S. Chaos. Sci. Am. 255, 46–57 (1986).

    Google Scholar 

  7. Binney, J. J., Dowrick, N. J., Fisher, A. J. & Newman, M. E. J. The Theory of Critical Phenomena (Oxford Univ. Press, 1992).

    MATH  Google Scholar 

  8. Cross, M. C. & Hohenberg, P. C. Pattern formation outside of equilibrium. Rev. Mod. Phys. 65, 851–1112 (1993).

    ADS  MATH  Google Scholar 

  9. Manneville, P. Dissipative Structures and Weak Turbulence (Academic, 1990).

    MATH  Google Scholar 

  10. Shannon, C. E. A mathematical theory of communication. Bell Syst. Tech. J. 27 379–423; 623–656 (1948).

  11. Cover, T. M. & Thomas, J. A. Elements of Information Theory 2nd edn (Wiley–Interscience, 2006).

    MATH  Google Scholar 

  12. Kolmogorov, A. N. Entropy per unit time as a metric invariant of automorphisms. Dokl. Akad. Nauk. SSSR 124, 754–755 (1959).

    MathSciNet  MATH  Google Scholar 

  13. Sinai, Ja G. On the notion of entropy of a dynamical system. Dokl. Akad. Nauk. SSSR 124, 768–771 (1959).

    MathSciNet  MATH  Google Scholar 

  14. Anderson, P. W. More is different. Science 177, 393–396 (1972).

    ADS  Google Scholar 

  15. Turing, A. M. On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. Math. Soc. 2 42, 230–265 (1936).

    MathSciNet  MATH  Google Scholar 

  16. Solomonoff, R. J. A formal theory of inductive inference: Part I. Inform. Control 7, 1–24 (1964).

    MathSciNet  MATH  Google Scholar 

  17. Solomonoff, R. J. A formal theory of inductive inference: Part II. Inform. Control 7, 224–254 (1964).

    MathSciNet  MATH  Google Scholar 

  18. Minsky, M. L. in Problems in the Biological Sciences Vol. XIV (ed. Bellman, R. E.) (Proceedings of Symposia in Applied Mathematics, American Mathematical Society, 1962).

    Google Scholar 

  19. Chaitin, G. On the length of programs for computing finite binary sequences. J. ACM 13, 145–159 (1966).

    MathSciNet  MATH  Google Scholar 

  20. Kolmogorov, A. N. Three approaches to the concept of the amount of information. Probab. Inform. Trans. 1, 1–7 (1965).

    Google Scholar 

  21. Martin-Löf, P. The definition of random sequences. Inform. Control 9, 602–619 (1966).

    MathSciNet  MATH  Google Scholar 

  22. Brudno, A. A. Entropy and the complexity of the trajectories of a dynamical system. Trans. Moscow Math. Soc. 44, 127–151 (1983).

    MATH  Google Scholar 

  23. Zvonkin, A. K. & Levin, L. A. The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russ. Math. Survey 25, 83–124 (1970).

    ADS  MATH  Google Scholar 

  24. Chaitin, G. Algorithmic Information Theory (Cambridge Univ. Press, 1987).

    MATH  Google Scholar 

  25. Li, M. & Vitanyi, P. M. B. An Introduction to Kolmogorov Complexity and its Applications (Springer, 1993).

    MATH  Google Scholar 

  26. Rissanen, J. Universal coding, information, prediction, and estimation. IEEE Trans. Inform. Theory IT-30, 629–636 (1984).

    MathSciNet  MATH  Google Scholar 

  27. Rissanen, J. Complexity of strings in the class of Markov sources. IEEE Trans. Inform. Theory IT-32, 526–532 (1986).

    MathSciNet  MATH  Google Scholar 

  28. Blum, L., Shub, M. & Smale, S. On a theory of computation over the real numbers: NP-completeness, Recursive Functions and Universal Machines. Bull. Am. Math. Soc. 21, 1–46 (1989).

    MathSciNet  MATH  Google Scholar 

  29. Moore, C. Recursion theory on the reals and continuous-time computation. Theor. Comput. Sci. 162, 23–44 (1996).

    MathSciNet  MATH  Google Scholar 

  30. Shannon, C. E. Communication theory of secrecy systems. Bell Syst. Tech. J. 28, 656–715 (1949).

    MathSciNet  MATH  Google Scholar 

  31. Ruelle, D. & Takens, F. On the nature of turbulence. Comm. Math. Phys. 20, 167–192 (1974).

    ADS  MathSciNet  MATH  Google Scholar 

  32. Packard, N. H., Crutchfield, J. P., Farmer, J. D. & Shaw, R. S. Geometry from a time series. Phys. Rev. Lett. 45, 712–716 (1980).

    ADS  Google Scholar 

  33. Takens, F. in Symposium on Dynamical Systems and Turbulence, Vol. 898 (eds Rand, D. A. & Young, L. S.) 366–381 (Springer, 1981).

    Google Scholar 

  34. Brandstater, A. et al. Low-dimensional chaos in a hydrodynamic system. Phys. Rev. Lett. 51, 1442–1445 (1983).

    ADS  MathSciNet  Google Scholar 

  35. Crutchfield, J. P. & McNamara, B. S. Equations of motion from a data series. Complex Syst. 1, 417–452 (1987).

    MathSciNet  MATH  Google Scholar 

  36. Crutchfield, J. P. & Young, K. Inferring statistical complexity. Phys. Rev. Lett. 63, 105–108 (1989).

    ADS  MathSciNet  Google Scholar 

  37. Crutchfield, J. P. & Shalizi, C. R. Thermodynamic depth of causal states: Objective complexity via minimal representations. Phys. Rev. E 59, 275–283 (1999).

    ADS  MathSciNet  Google Scholar 

  38. Shalizi, C. R. & Crutchfield, J. P. Computational mechanics: Pattern and prediction, structure and simplicity. J. Stat. Phys. 104, 817–879 (2001).

    MathSciNet  MATH  Google Scholar 

  39. Young, K. The Grammar and Statistical Mechanics of Complex Physical Systems. PhD thesis, Univ. California (1991).

  40. Koppel, M. Complexity, depth, and sophistication. Complexity 1, 1087–1091 (1987).

    MathSciNet  MATH  Google Scholar 

  41. Koppel, M. & Atlan, H. An almost machine-independent theory of program-length complexity, sophistication, and induction. Information Sciences 56, 23–33 (1991).

    MathSciNet  MATH  Google Scholar 

  42. Crutchfield, J. P. & Young, K. in Entropy, Complexity, and the Physics of Information Vol. VIII (ed. Zurek, W.) 223–269 (SFI Studies in the Sciences of Complexity, Addison-Wesley, 1990).

    Google Scholar 

  43. William of Ockham Philosophical Writings: A Selection, Translated, with an Introduction (ed. Philotheus Boehner, O. F. M.) (Bobbs-Merrill, 1964).

  44. Farmer, J. D. Information dimension and the probabilistic structure of chaos. Z. Naturf. 37a, 1304–1325 (1982).

    ADS  MathSciNet  Google Scholar 

  45. Crutchfield, J. P. The calculi of emergence: Computation, dynamics, and induction. Physica D 75, 11–54 (1994).

    ADS  MATH  Google Scholar 

  46. Crutchfield, J. P. in Complexity: Metaphors, Models, and Reality Vol. XIX (eds Cowan, G., Pines, D. & Melzner, D.) 479–497 (Santa Fe Institute Studies in the Sciences of Complexity, Addison-Wesley, 1994).

    Google Scholar 

  47. Crutchfield, J. P. & Feldman, D. P. Regularities unseen, randomness observed: Levels of entropy convergence. Chaos 13, 25–54 (2003).

    ADS  MathSciNet  MATH  Google Scholar 

  48. Mahoney, J. R., Ellison, C. J., James, R. G. & Crutchfield, J. P. How hidden are hidden processes? A primer on crypticity and entropy convergence. Chaos 21, 037112 (2011).

    ADS  MathSciNet  MATH  Google Scholar 

  49. Ellison, C. J., Mahoney, J. R., James, R. G., Crutchfield, J. P. & Reichardt, J. Information symmetries in irreversible processes. Chaos 21, 037107 (2011).

    ADS  MathSciNet  MATH  Google Scholar 

  50. Crutchfield, J. P. in Nonlinear Modeling and Forecasting Vol. XII (eds Casdagli, M. & Eubank, S.) 317–359 (Santa Fe Institute Studies in the Sciences of Complexity, Addison-Wesley, 1992).

    Google Scholar 

  51. Crutchfield, J. P., Ellison, C. J. & Mahoney, J. R. Time’s barbed arrow: Irreversibility, crypticity, and stored information. Phys. Rev. Lett. 103, 094101 (2009).

    ADS  Google Scholar 

  52. Ellison, C. J., Mahoney, J. R. & Crutchfield, J. P. Prediction, retrodiction, and the amount of information stored in the present. J. Stat. Phys. 136, 1005–1034 (2009).

    ADS  MathSciNet  MATH  Google Scholar 

  53. Ruelle, D. Do turbulent crystals exist? Physica A 113, 619–623 (1982).

    ADS  MathSciNet  Google Scholar 

  54. Varn, D. P., Canright, G. S. & Crutchfield, J. P. Discovering planar disorder in close-packed structures from X-ray diffraction: Beyond the fault model. Phys. Rev. B 66, 174110 (2002).

    ADS  Google Scholar 

  55. Varn, D. P. & Crutchfield, J. P. From finite to infinite range order via annealing: The causal architecture of deformation faulting in annealed close-packed crystals. Phys. Lett. A 234, 299–307 (2004).

    ADS  MATH  Google Scholar 

  56. Varn, D. P., Canright, G. S. & Crutchfield, J. P. Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part I: ε-machine Spectral Reconstruction Theory Santa Fe Institute Working Paper 03-03-021 (2002).

  57. Varn, D. P., Canright, G. S. & Crutchfield, J. P. Inferring pattern and disorder in close-packed structures via ε-machine reconstruction theory: Structure and intrinsic computation in Zinc Sulphide. Acta Cryst. B 63, 169–182 (2002).

    Google Scholar 

  58. Welberry, T. R. Diffuse x-ray scattering and models of disorder. Rep. Prog. Phys. 48, 1543–1593 (1985).

    ADS  Google Scholar 

  59. Guinier, A. X-Ray Diffraction in Crystals, Imperfect Crystals and Amorphous Bodies (W. H. Freeman, 1963).

    Google Scholar 

  60. Sebastian, M. T. & Krishna, P. Random, Non-Random and Periodic Faulting in Crystals (Gordon and Breach Science Publishers, 1994).

    Google Scholar 

  61. Feldman, D. P., McTague, C. S. & Crutchfield, J. P. The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing. Chaos 18, 043106 (2008).

    ADS  MathSciNet  Google Scholar 

  62. Mitchell, M., Hraber, P. & Crutchfield, J. P. Revisiting the edge of chaos: Evolving cellular automata to perform computations. Complex Syst. 7, 89–130 (1993).

    MATH  Google Scholar 

  63. Johnson, B. D., Crutchfield, J. P., Ellison, C. J. & McTague, C. S. Enumerating Finitary Processes Santa Fe Institute Working Paper 10-11-027 (2010).

  64. Lind, D. & Marcus, B. An Introduction to Symbolic Dynamics and Coding (Cambridge Univ. Press, 1995).

    MATH  Google Scholar 

  65. Hopcroft, J. E. & Ullman, J. D. Introduction to Automata Theory, Languages, and Computation (Addison-Wesley, 1979).

    MATH  Google Scholar 

  66. Upper, D. R. Theory and Algorithms for Hidden Markov Models and Generalized Hidden Markov Models. Ph.D. thesis, Univ. California (1997).

  67. Kelly, D., Dillingham, M., Hudson, A. & Wiesner, K. Inferring hidden Markov models from noisy time sequences: A method to alleviate degeneracy in molecular dynamics. Preprint at http://arxiv.org/abs/1011.2969 (2010).

  68. Ryabov, V. & Nerukh, D. Computational mechanics of molecular systems: Quantifying high-dimensional dynamics by distribution of Poincaré recurrence times. Chaos 21, 037113 (2011).

    ADS  MathSciNet  MATH  Google Scholar 

  69. Li, C-B., Yang, H. & Komatsuzaki, T. Multiscale complex network of protein conformational fluctuations in single-molecule time series. Proc. Natl Acad. Sci. USA 105, 536–541 (2008).

    ADS  Google Scholar 

  70. Crutchfield, J. P. & Wiesner, K. Intrinsic quantum computation. Phys. Lett. A 372, 375–380 (2006).

    ADS  MathSciNet  MATH  Google Scholar 

  71. Goncalves, W. M., Pinto, R. D., Sartorelli, J. C. & de Oliveira, M. J. Inferring statistical complexity in the dripping faucet experiment. Physica A 257, 385–389 (1998).

    ADS  Google Scholar 

  72. Clarke, R. W., Freeman, M. P. & Watkins, N. W. The application of computational mechanics to the analysis of geomagnetic data. Phys. Rev. E 67, 160–203 (2003).

    Google Scholar 

  73. Crutchfield, J. P. & Hanson, J. E. Turbulent pattern bases for cellular automata. Physica D 69, 279–301 (1993).

    ADS  MathSciNet  MATH  Google Scholar 

  74. Hanson, J. E. & Crutchfield, J. P. Computational mechanics of cellular automata: An example. Physica D 103, 169–189 (1997).

    ADS  MathSciNet  MATH  Google Scholar 

  75. Shalizi, C. R., Shalizi, K. L. & Haslinger, R. Quantifying self-organization with optimal predictors. Phys. Rev. Lett. 93, 118701 (2004).

    ADS  Google Scholar 

  76. Crutchfield, J. P. & Feldman, D. P. Statistical complexity of simple one-dimensional spin systems. Phys. Rev. E 55, 239R–1243R (1997).

    ADS  Google Scholar 

  77. Feldman, D. P. & Crutchfield, J. P. Structural information in two-dimensional patterns: Entropy convergence and excess entropy. Phys. Rev. E 67, 051103 (2003).

    ADS  MathSciNet  Google Scholar 

  78. Bonner, J. T. The Evolution of Complexity by Means of Natural Selection (Princeton Univ. Press, 1988).

    Google Scholar 

  79. Eigen, M. Natural selection: A phase transition? Biophys. Chem. 85, 101–123 (2000).

    Google Scholar 

  80. Adami, C. What is complexity? BioEssays 24, 1085–1094 (2002).

    Google Scholar 

  81. Frenken, K. Innovation, Evolution and Complexity Theory (Edward Elgar Publishing, 2005).

    Google Scholar 

  82. McShea, D. W. The evolution of complexity without natural selection—A possible large-scale trend of the fourth kind. Paleobiology 31, 146–156 (2005).

    Google Scholar 

  83. Krakauer, D. Darwinian demons, evolutionary complexity, and information maximization. Chaos 21, 037111 (2011).

    ADS  MathSciNet  Google Scholar 

  84. Tononi, G., Edelman, G. M. & Sporns, O. Complexity and coherency: Integrating information in the brain. Trends Cogn. Sci. 2, 474–484 (1998).

    Google Scholar 

  85. Bialek, W., Nemenman, I. & Tishby, N. Predictability, complexity, and learning. Neural Comput. 13, 2409–2463 (2001).

    MATH  Google Scholar 

  86. Sporns, O., Chialvo, D. R., Kaiser, M. & Hilgetag, C. C. Organization, development, and function of complex brain networks. Trends Cogn. Sci. 8, 418–425 (2004).

    Google Scholar 

  87. Crutchfield, J. P. & Mitchell, M. The evolution of emergent computation. Proc. Natl Acad. Sci. USA 92, 10742–10746 (1995).

    ADS  MATH  Google Scholar 

  88. Lizier, J., Prokopenko, M. & Zomaya, A. Information modification and particle collisions in distributed computation. Chaos 20, 037109 (2010).

    ADS  MathSciNet  MATH  Google Scholar 

  89. Flecker, B., Alford, W., Beggs, J. M., Williams, P. L. & Beer, R. D. Partial information decomposition as a spatiotemporal filter. Chaos 21, 037104 (2011).

    ADS  MathSciNet  MATH  Google Scholar 

  90. Rissanen, J. Stochastic Complexity in Statistical Inquiry (World Scientific, 1989).

    MATH  Google Scholar 

  91. Balasubramanian, V. Statistical inference, Occam’s razor, and statistical mechanics on the space of probability distributions. Neural Comput. 9, 349–368 (1997).

    MATH  Google Scholar 

  92. Glymour, C. & Cooper, G. F. (eds) in Computation, Causation, and Discovery (AAAI Press, 1999).

  93. Shalizi, C. R., Shalizi, K. L. & Crutchfield, J. P. Pattern Discovery in Time Series, Part I: Theory, Algorithm, Analysis, and Convergence Santa Fe Institute Working Paper 02-10-060 (2002).

  94. MacKay, D. J. C. Information Theory, Inference and Learning Algorithms (Cambridge Univ. Press, 2003).

    MATH  Google Scholar 

  95. Still, S., Crutchfield, J. P. & Ellison, C. J. Optimal causal inference. Chaos 20, 037111 (2007).

    ADS  MATH  Google Scholar 

  96. Wheeler, J. A. in Entropy, Complexity, and the Physics of Information volume VIII (ed. Zurek, W.) (SFI Studies in the Sciences of Complexity, Addison-Wesley, 1990).

    Google Scholar 

Download references

Acknowledgements

I thank the Santa Fe Institute and the Redwood Center for Theoretical Neuroscience, University of California Berkeley, for their hospitality during a sabbatical visit.

Author information

Authors and Affiliations

  1. Complexity Sciences Center and Physics Department, University of California at Davis, One Shields Avenue, Davis, California 95616, USA

    James P. Crutchfield

Authors
  1. James P. Crutchfield
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James P. Crutchfield.

Ethics declarations

Competing interests

The author declares no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Crutchfield, J. Between order and chaos. Nature Phys 8, 17–24 (2012). https://doi.org/10.1038/nphys2190

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nphys2190

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing