Article | Published:

Mutual information, neural networks and the renormalization group

Nature Physicsvolume 14pages578582 (2018) | Download Citation

Abstract

Physical systems differing in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains ‘slow’ degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine-learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, which performs this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine-learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publishers note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    LeCun, Y., Bengio, Y. & Hinton, G. E. Deep learning. Nature 521, 436–444 (2015).

  2. 2.

    Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 584–589 (2016).

  3. 3.

    Hershey, J. R., Rennie, S. J., Olsen, P. A. & Kristjansson, T. T. Super-human multi-talker speech recognition: A graphical modeling approach. Comput. Speech Lang. 24, 45–66 (2010).

  4. 4.

    Carrasquilla, J. & Melko, R. G. Machine learning phases of matter. Nat. Phys. 13, 431–434 (2017).

  5. 5.

    Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B 94, 165134 (2016).

  6. 6.

    van Nieuwenburg, E. P. L., Liu, Y.-H. & Huber, S. D. Learning phase transitions by confusion. Nat. Phys. 13, 435–439 (2017).

  7. 7.

    Wang, L. Discovering phase transitions with unsupervised learning. Phys. Rev. B 94, 195105 (2016).

  8. 8.

    Ohtsuki, T. & Ohtsuki, T. Deep learning the quantum phase transitions in random electron systems: applications to three dimensions. J. Phys. Soc. Jpn 86, 044708 (2017).

  9. 9.

    Ronhovde, P.et al Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering. Eur. Phys. J. E 34, 105 (2011).

  10. 10.

    Ronhovde, P.et al Detection of hidden structures for arbitrary scales in complex physical systems. Sci. Rep. 2, 329 (2012).

  11. 11.

    Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006).

  12. 12.

    Lin, H. W. & Tegmark, M. Why does deep and cheap learning work so well? J. Stat. Phys. 168, 1223–1247 (2017).

  13. 13.

    Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science 355, 602–606 (2017).

  14. 14.

    Deng, D.-L., Li, X. & Sarma, S. D. Machine learning topological states. Phys. Rev. B 96, 195145 (2017).

  15. 15.

    Wilson, K. G. The renormalization group: Critical phenomena and the Kondo problem. Rev. Mod. Phys. 47, 773–840 (1975).

  16. 16.

    Politzer, H. D. Reliable perturbative results for strong interactions? Phys. Rev. Lett. 30, 1346–1349 (1973).

  17. 17.

    Berezinskii, V. L. Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group I. Classical systems. Sov. J. Exp. Theor. Phys. 32, 493 (1971).

  18. 18.

    Kosterlitz, J. M. & Thouless, D. Ordering, metastability and phase transitions in two-dimensional systems. J. Phys. C 6, 1181 (1973).

  19. 19.

    Kadanoff, L. P. Scaling laws for Ising models near T(c). Physics 2, 263–272 (1966).

  20. 20.

    Wetterich, C. Exact evolution equation for the effective potential. Phys. Lett. B 301, 90–94 (1993).

  21. 21.

    White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett. 69, 2863–2866 (1992).

  22. 22.

    Ma, S.-k, Dasgupta, C. & Hu, C.-k Random antiferromagnetic chain. Phys. Rev. Lett. 43, 1434–1437 (1979).

  23. 23.

    Corboz, P. & Mila, F. Tensor network study of the Shastry–Sutherland model in zero magnetic field. Phys. Rev. B 87, 115144 (2013).

  24. 24.

    Capponi, S., Chandra, V. R., Auerbach, A. & Weinstein, M. p6 chiral resonating valence bonds in the kagome antiferromagnet. Phys. Rev. B 87, 161118 (2013).

  25. 25.

    Auerbach, A. Interacting Electrons and Quantum Magnetism (Springer, New York, NY, 1994).

  26. 26.

    Gaite, J. & O’Connor, D. Field theory entropy, the h theorem, and the renormalization group. Phys. Rev. D 54, 5163–5173 (1996).

  27. 27.

    Preskill, J. Quantum information and physics: some future directions. J. Mod. Opt. 47, 127–137 (2000).

  28. 28.

    Apenko, S. M. Information theory and renormalization group flows. Phys. A 391, 62–77 (2012).

  29. 29.

    Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science 342, 604–607 (2013).

  30. 30.

    Beny, C. & Osborne, T. J. The renormalization group via statistical inference. New. J. Phys. 17, 083005 (2015).

  31. 31.

    Stephan, J.-M., Inglis, S., Fendley, P. & Melko, R. G. Geometric mutual information at classical critical points. Phys. Rev. Lett. 112, 127204 (2014).

  32. 32.

    Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. In Proc. 37th Allerton Conf. on Communication, Control and Computation (eds ​Hajek, B. & Sreenivas, R. S.) 49, 368–377 (University of Illinois, 2001).

  33. 33.

    Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771–1800 (2002).

  34. 34.

    Ludwig, A. W. W. & Cardy, J. L. Perturbative evaluation of the conformal anomaly at new critical points with applications to random systems. Nucl. Phys. B 285, 687–718 (1987).

  35. 35.

    Fisher, M. E. & Stephenson, J. Statistical mechanics of dimers on a plane lattice. II. Dimer correlations and monomers. Phys. Rev. 132, 1411–1431 (1963).

  36. 36.

    Fradkin, E. Field Theories of Condensed Matter Physics (Cambridge Univ. Press, Cambridge, 2013).

  37. 37.

    Mehta, P. & Schwab, D. J. An exact mapping between the variational renormalization group and deep learning. Preprint at abs/1410.3831 (2014).

  38. 38.

    McCoy, B. M. & Wu, T. T. The Two-Dimensional Ising Model (Harvard Univ. Press, Cambridge, MA, 1973).

  39. 39.

    Schoenholz, S. S., Cubuk, E. D., Sussman, D. M., Kaxiras, E. & Liu, A. J. A structural approach to relaxation in glassy liquids. Nat. Phys. 12, 469–471 (2016).

  40. 40.

    Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science 349, 255–260 (2015).

  41. 41.

    Slonim, N. & Tishby, N. Document clustering using word clusters via the information bottleneck method. In Proc. 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, SIGIR ’00 208–215 (ACM, 2000).

Download references

Acknowledgements

We thank S. Huber and P. Fendley for discussions. M.K.-J. gratefully acknowledges the support of the Swiss National Science Foundation. Z.R. was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 657111.

Author information

Affiliations

  1. Institute for Theoretical Physics, ETH Zurich, Zurich, Switzerland

    • Maciej Koch-Janusz
  2. Racah Institute of Physics, Hebrew University of Jerusalem, Jerusalem, Israel

    • Zohar Ringel

Authors

  1. Search for Maciej Koch-Janusz in:

  2. Search for Zohar Ringel in:

Contributions

M.K.-J. and Z.R. contributed equally to this work.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Maciej Koch-Janusz.

Supplementary information

  1. Methods

    Methods, Supplementary Figures 1–6, Supplementary References 1–12

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41567-018-0081-4