Mutual information, neural networks and the renormalization group

Abstract

Physical systems differing in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains ‘slow’ degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine-learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, which performs this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine-learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from\$8.99

All prices are NET prices.

References

1. 1.

LeCun, Y., Bengio, Y. & Hinton, G. E. Deep learning. Nature521, 436–444 (2015).

2. 2.

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature529, 584–589 (2016).

3. 3.

Hershey, J. R., Rennie, S. J., Olsen, P. A. & Kristjansson, T. T. Super-human multi-talker speech recognition: A graphical modeling approach. Comput. Speech Lang.24, 45–66 (2010).

4. 4.

Carrasquilla, J. & Melko, R. G. Machine learning phases of matter. Nat. Phys.13, 431–434 (2017).

5. 5.

Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B94, 165134 (2016).

6. 6.

van Nieuwenburg, E. P. L., Liu, Y.-H. & Huber, S. D. Learning phase transitions by confusion. Nat. Phys.13, 435–439 (2017).

7. 7.

Wang, L. Discovering phase transitions with unsupervised learning. Phys. Rev. B94, 195105 (2016).

8. 8.

Ohtsuki, T. & Ohtsuki, T. Deep learning the quantum phase transitions in random electron systems: applications to three dimensions. J. Phys. Soc. Jpn86, 044708 (2017).

9. 9.

Ronhovde, P.et al Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering. Eur. Phys. J. E 34, 105 (2011).

10. 10.

Ronhovde, P.et al Detection of hidden structures for arbitrary scales in complex physical systems. Sci. Rep. 2, 329 (2012).

11. 11.

Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science313, 504–507 (2006).

12. 12.

Lin, H. W. & Tegmark, M. Why does deep and cheap learning work so well? J. Stat. Phys.168, 1223–1247 (2017).

13. 13.

Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science355, 602–606 (2017).

14. 14.

Deng, D.-L., Li, X. & Sarma, S. D. Machine learning topological states. Phys. Rev. B96, 195145 (2017).

15. 15.

Wilson, K. G. The renormalization group: Critical phenomena and the Kondo problem. Rev. Mod. Phys.47, 773–840 (1975).

16. 16.

Politzer, H. D. Reliable perturbative results for strong interactions? Phys. Rev. Lett.30, 1346–1349 (1973).

17. 17.

Berezinskii, V. L. Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group I. Classical systems. Sov. J. Exp. Theor. Phys.32, 493 (1971).

18. 18.

Kosterlitz, J. M. & Thouless, D. Ordering, metastability and phase transitions in two-dimensional systems. J. Phys. C6, 1181 (1973).

19. 19.

Kadanoff, L. P. Scaling laws for Ising models near T(c). Physics2, 263–272 (1966).

20. 20.

Wetterich, C. Exact evolution equation for the effective potential. Phys. Lett. B301, 90–94 (1993).

21. 21.

White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett.69, 2863–2866 (1992).

22. 22.

Ma, S.-k, Dasgupta, C. & Hu, C.-k Random antiferromagnetic chain. Phys. Rev. Lett.43, 1434–1437 (1979).

23. 23.

Corboz, P. & Mila, F. Tensor network study of the Shastry–Sutherland model in zero magnetic field. Phys. Rev. B87, 115144 (2013).

24. 24.

Capponi, S., Chandra, V. R., Auerbach, A. & Weinstein, M. p6 chiral resonating valence bonds in the kagome antiferromagnet. Phys. Rev. B87, 161118 (2013).

25. 25.

Auerbach, A. Interacting Electrons and Quantum Magnetism (Springer, New York, NY, 1994).

26. 26.

Gaite, J. & O’Connor, D. Field theory entropy, the h theorem, and the renormalization group. Phys. Rev. D54, 5163–5173 (1996).

27. 27.

Preskill, J. Quantum information and physics: some future directions. J. Mod. Opt.47, 127–137 (2000).

28. 28.

Apenko, S. M. Information theory and renormalization group flows. Phys. A391, 62–77 (2012).

29. 29.

Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science342, 604–607 (2013).

30. 30.

Beny, C. & Osborne, T. J. The renormalization group via statistical inference. New. J. Phys.17, 083005 (2015).

31. 31.

Stephan, J.-M., Inglis, S., Fendley, P. & Melko, R. G. Geometric mutual information at classical critical points. Phys. Rev. Lett.112, 127204 (2014).

32. 32.

Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. In Proc. 37th Allerton Conf. on Communication, Control and Computation (eds ​Hajek, B. & Sreenivas, R. S.) 49, 368–377 (University of Illinois, 2001).

33. 33.

Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput.14, 1771–1800 (2002).

34. 34.

Ludwig, A. W. W. & Cardy, J. L. Perturbative evaluation of the conformal anomaly at new critical points with applications to random systems. Nucl. Phys. B285, 687–718 (1987).

35. 35.

Fisher, M. E. & Stephenson, J. Statistical mechanics of dimers on a plane lattice. II. Dimer correlations and monomers. Phys. Rev.132, 1411–1431 (1963).

36. 36.

Fradkin, E. Field Theories of Condensed Matter Physics (Cambridge Univ. Press, Cambridge, 2013).

37. 37.

Mehta, P. & Schwab, D. J. An exact mapping between the variational renormalization group and deep learning. Preprint at abs/1410.3831 (2014).

38. 38.

McCoy, B. M. & Wu, T. T. The Two-Dimensional Ising Model (Harvard Univ. Press, Cambridge, MA, 1973).

39. 39.

Schoenholz, S. S., Cubuk, E. D., Sussman, D. M., Kaxiras, E. & Liu, A. J. A structural approach to relaxation in glassy liquids. Nat. Phys.12, 469–471 (2016).

40. 40.

Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science349, 255–260 (2015).

41. 41.

Slonim, N. & Tishby, N. Document clustering using word clusters via the information bottleneck method. In Proc. 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, SIGIR ’00 208–215 (ACM, 2000).

Download references

Acknowledgements

We thank S. Huber and P. Fendley for discussions. M.K.-J. gratefully acknowledges the support of the Swiss National Science Foundation. Z.R. was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 657111.

Author information

Authors

Contributions

M.K.-J. and Z.R. contributed equally to this work.

Corresponding author

Correspondence to Maciej Koch-Janusz.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publishers note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Methods

Methods, Supplementary Figures 1–6, Supplementary References 1–12

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Koch-Janusz, M., Ringel, Z. Mutual information, neural networks and the renormalization group. Nature Phys 14, 578–582 (2018). https://doi.org/10.1038/s41567-018-0081-4

Download citation

• Received:

• Accepted:

• Published:

• Issue Date:

Further reading

• Entropic Dynamics in Neural Networks, the Renormalization Group and the Hamilton-Jacobi-Bellman Equation

• Nestor Caticha

Entropy (2020)

• Learning the Ising model with generative neural networks

• Francesco D'Angelo
•  & Lucas Böttcher

Physical Review Research (2020)

• Precise measurement of quantum observables with neural-network estimators

• Giacomo Torlai
• , Guglielmo Mazzola
• , Giuseppe Carleo
•  & Antonio Mezzacapo

Physical Review Research (2020)

• Asymptotically unbiased estimation of physical observables with neural samplers

• Kim A. Nicoli
• , Shinichi Nakajima
• , Nils Strodthoff
• , Wojciech Samek
• , Klaus-Robert Müller
•  & Pan Kessel

Physical Review E (2020)

• Machine learning for quantum matter

• Juan Carrasquilla

Advances in Physics: X (2020)