Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Mutual information, neural networks and the renormalization group

Nature Physics volume 14, pages 578–582 (2018)Cite this article

Subjects

Abstract

Physical systems differing in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains ‘slow’ degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine-learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, which performs this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine-learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The RSMI algorithm.
Fig. 2: The weights of the RSMI network trained on the Ising model.
Fig. 3: The dimer model.
Fig. 4: The weights of the RSMI network trained on dimer model data.
Fig. 5: RG flow for the 2D Ising model.

Similar content being viewed by others

References

  1. LeCun, Y., Bengio, Y. & Hinton, G. E. Deep learning. Nature521, 436–444 (2015).

    Article  ADS  Google Scholar 

  2. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature529, 584–589 (2016).

    Article  Google Scholar 

  3. Hershey, J. R., Rennie, S. J., Olsen, P. A. & Kristjansson, T. T. Super-human multi-talker speech recognition: A graphical modeling approach. Comput. Speech Lang.24, 45–66 (2010).

    Article  Google Scholar 

  4. Carrasquilla, J. & Melko, R. G. Machine learning phases of matter. Nat. Phys.13, 431–434 (2017).

    Article  Google Scholar 

  5. Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B94, 165134 (2016).

    Article  ADS  Google Scholar 

  6. van Nieuwenburg, E. P. L., Liu, Y.-H. & Huber, S. D. Learning phase transitions by confusion. Nat. Phys.13, 435–439 (2017).

    Article  Google Scholar 

  7. Wang, L. Discovering phase transitions with unsupervised learning. Phys. Rev. B94, 195105 (2016).

    Article  ADS  Google Scholar 

  8. Ohtsuki, T. & Ohtsuki, T. Deep learning the quantum phase transitions in random electron systems: applications to three dimensions. J. Phys. Soc. Jpn86, 044708 (2017).

    Article  ADS  Google Scholar 

  9. Ronhovde, P.et al Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering. Eur. Phys. J. E 34, 105 (2011).

    Article  Google Scholar 

  10. Ronhovde, P.et al Detection of hidden structures for arbitrary scales in complex physical systems. Sci. Rep. 2, 329 (2012).

    Article  Google Scholar 

  11. Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science313, 504–507 (2006).

    Article  ADS  MathSciNet  Google Scholar 

  12. Lin, H. W. & Tegmark, M. Why does deep and cheap learning work so well? J. Stat. Phys.168, 1223–1247 (2017).

    Article  ADS  MathSciNet  Google Scholar 

  13. Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science355, 602–606 (2017).

    Article  ADS  MathSciNet  Google Scholar 

  14. Deng, D.-L., Li, X. & Sarma, S. D. Machine learning topological states. Phys. Rev. B96, 195145 (2017).

    Article  ADS  Google Scholar 

  15. Wilson, K. G. The renormalization group: Critical phenomena and the Kondo problem. Rev. Mod. Phys.47, 773–840 (1975).

    Article  ADS  MathSciNet  Google Scholar 

  16. Politzer, H. D. Reliable perturbative results for strong interactions? Phys. Rev. Lett.30, 1346–1349 (1973).

    Article  ADS  Google Scholar 

  17. Berezinskii, V. L. Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group I. Classical systems. Sov. J. Exp. Theor. Phys.32, 493 (1971).

    ADS  MathSciNet  Google Scholar 

  18. Kosterlitz, J. M. & Thouless, D. Ordering, metastability and phase transitions in two-dimensional systems. J. Phys. C6, 1181 (1973).

    Article  ADS  Google Scholar 

  19. Kadanoff, L. P. Scaling laws for Ising models near T(c). Physics2, 263–272 (1966).

    Article  MathSciNet  Google Scholar 

  20. Wetterich, C. Exact evolution equation for the effective potential. Phys. Lett. B301, 90–94 (1993).

    Article  ADS  Google Scholar 

  21. White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett.69, 2863–2866 (1992).

    Article  ADS  Google Scholar 

  22. Ma, S.-k, Dasgupta, C. & Hu, C.-k Random antiferromagnetic chain. Phys. Rev. Lett.43, 1434–1437 (1979).

    Article  ADS  Google Scholar 

  23. Corboz, P. & Mila, F. Tensor network study of the Shastry–Sutherland model in zero magnetic field. Phys. Rev. B87, 115144 (2013).

    Article  ADS  Google Scholar 

  24. Capponi, S., Chandra, V. R., Auerbach, A. & Weinstein, M. p6 chiral resonating valence bonds in the kagome antiferromagnet. Phys. Rev. B87, 161118 (2013).

    Article  ADS  Google Scholar 

  25. Auerbach, A. Interacting Electrons and Quantum Magnetism (Springer, New York, NY, 1994).

  26. Gaite, J. & O’Connor, D. Field theory entropy, the h theorem, and the renormalization group. Phys. Rev. D54, 5163–5173 (1996).

    Article  ADS  MathSciNet  Google Scholar 

  27. Preskill, J. Quantum information and physics: some future directions. J. Mod. Opt.47, 127–137 (2000).

    Article  ADS  MathSciNet  Google Scholar 

  28. Apenko, S. M. Information theory and renormalization group flows. Phys. A391, 62–77 (2012).

    Article  MathSciNet  Google Scholar 

  29. Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science342, 604–607 (2013).

    Article  ADS  Google Scholar 

  30. Beny, C. & Osborne, T. J. The renormalization group via statistical inference. New. J. Phys.17, 083005 (2015).

    Article  ADS  MathSciNet  Google Scholar 

  31. Stephan, J.-M., Inglis, S., Fendley, P. & Melko, R. G. Geometric mutual information at classical critical points. Phys. Rev. Lett.112, 127204 (2014).

    Article  ADS  Google Scholar 

  32. Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. In Proc. 37th Allerton Conf. on Communication, Control and Computation (eds ​Hajek, B. & Sreenivas, R. S.) 49, 368–377 (University of Illinois, 2001).

  33. Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput.14, 1771–1800 (2002).

    Article  Google Scholar 

  34. Ludwig, A. W. W. & Cardy, J. L. Perturbative evaluation of the conformal anomaly at new critical points with applications to random systems. Nucl. Phys. B285, 687–718 (1987).

    Article  ADS  MathSciNet  Google Scholar 

  35. Fisher, M. E. & Stephenson, J. Statistical mechanics of dimers on a plane lattice. II. Dimer correlations and monomers. Phys. Rev.132, 1411–1431 (1963).

    Article  ADS  MathSciNet  Google Scholar 

  36. Fradkin, E. Field Theories of Condensed Matter Physics (Cambridge Univ. Press, Cambridge, 2013).

  37. Mehta, P. & Schwab, D. J. An exact mapping between the variational renormalization group and deep learning. Preprint at abs/1410.3831 (2014).

  38. McCoy, B. M. & Wu, T. T. The Two-Dimensional Ising Model (Harvard Univ. Press, Cambridge, MA, 1973).

  39. Schoenholz, S. S., Cubuk, E. D., Sussman, D. M., Kaxiras, E. & Liu, A. J. A structural approach to relaxation in glassy liquids. Nat. Phys.12, 469–471 (2016).

    Article  Google Scholar 

  40. Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science349, 255–260 (2015).

    Article  ADS  MathSciNet  Google Scholar 

  41. Slonim, N. & Tishby, N. Document clustering using word clusters via the information bottleneck method. In Proc. 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, SIGIR ’00 208–215 (ACM, 2000).

Download references

Acknowledgements

We thank S. Huber and P. Fendley for discussions. M.K.-J. gratefully acknowledges the support of the Swiss National Science Foundation. Z.R. was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 657111.

Author information

Authors and Affiliations

  1. Institute for Theoretical Physics, ETH Zurich, Zurich, Switzerland

    Maciej Koch-Janusz

  2. Racah Institute of Physics, Hebrew University of Jerusalem, Jerusalem, Israel

    Zohar Ringel

Authors
  1. Maciej Koch-Janusz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zohar Ringel
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.K.-J. and Z.R. contributed equally to this work.

Corresponding author

Correspondence to Maciej Koch-Janusz.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Methods

Methods, Supplementary Figures 1–6, Supplementary References 1–12

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koch-Janusz, M., Ringel, Z. Mutual information, neural networks and the renormalization group. Nature Phys 14, 578–582 (2018). https://doi.org/10.1038/s41567-018-0081-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41567-018-0081-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics