Mutual information, neural networks and the renormalization group

Abstract

Physical systems differing in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains ‘slow’ degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine-learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, which performs this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine-learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: The RSMI algorithm.
Fig. 2: The weights of the RSMI network trained on the Ising model.
Fig. 3: The dimer model.
Fig. 4: The weights of the RSMI network trained on dimer model data.
Fig. 5: RG flow for the 2D Ising model.

References

  1. 1.

    LeCun, Y., Bengio, Y. & Hinton, G. E. Deep learning. Nature521, 436–444 (2015).

    ADS  Article  Google Scholar 

  2. 2.

    Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature529, 584–589 (2016).

    Article  Google Scholar 

  3. 3.

    Hershey, J. R., Rennie, S. J., Olsen, P. A. & Kristjansson, T. T. Super-human multi-talker speech recognition: A graphical modeling approach. Comput. Speech Lang.24, 45–66 (2010).

    Article  Google Scholar 

  4. 4.

    Carrasquilla, J. & Melko, R. G. Machine learning phases of matter. Nat. Phys.13, 431–434 (2017).

    Article  Google Scholar 

  5. 5.

    Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B94, 165134 (2016).

    ADS  Article  Google Scholar 

  6. 6.

    van Nieuwenburg, E. P. L., Liu, Y.-H. & Huber, S. D. Learning phase transitions by confusion. Nat. Phys.13, 435–439 (2017).

    Article  Google Scholar 

  7. 7.

    Wang, L. Discovering phase transitions with unsupervised learning. Phys. Rev. B94, 195105 (2016).

    ADS  Article  Google Scholar 

  8. 8.

    Ohtsuki, T. & Ohtsuki, T. Deep learning the quantum phase transitions in random electron systems: applications to three dimensions. J. Phys. Soc. Jpn86, 044708 (2017).

    ADS  Article  Google Scholar 

  9. 9.

    Ronhovde, P.et al Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering. Eur. Phys. J. E 34, 105 (2011).

    Article  Google Scholar 

  10. 10.

    Ronhovde, P.et al Detection of hidden structures for arbitrary scales in complex physical systems. Sci. Rep. 2, 329 (2012).

    Article  Google Scholar 

  11. 11.

    Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science313, 504–507 (2006).

    ADS  MathSciNet  Article  Google Scholar 

  12. 12.

    Lin, H. W. & Tegmark, M. Why does deep and cheap learning work so well? J. Stat. Phys.168, 1223–1247 (2017).

    ADS  MathSciNet  Article  Google Scholar 

  13. 13.

    Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science355, 602–606 (2017).

    ADS  MathSciNet  Article  Google Scholar 

  14. 14.

    Deng, D.-L., Li, X. & Sarma, S. D. Machine learning topological states. Phys. Rev. B96, 195145 (2017).

    ADS  Article  Google Scholar 

  15. 15.

    Wilson, K. G. The renormalization group: Critical phenomena and the Kondo problem. Rev. Mod. Phys.47, 773–840 (1975).

    ADS  MathSciNet  Article  Google Scholar 

  16. 16.

    Politzer, H. D. Reliable perturbative results for strong interactions? Phys. Rev. Lett.30, 1346–1349 (1973).

    ADS  Article  Google Scholar 

  17. 17.

    Berezinskii, V. L. Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group I. Classical systems. Sov. J. Exp. Theor. Phys.32, 493 (1971).

    ADS  MathSciNet  Google Scholar 

  18. 18.

    Kosterlitz, J. M. & Thouless, D. Ordering, metastability and phase transitions in two-dimensional systems. J. Phys. C6, 1181 (1973).

    ADS  Article  Google Scholar 

  19. 19.

    Kadanoff, L. P. Scaling laws for Ising models near T(c). Physics2, 263–272 (1966).

    MathSciNet  Article  Google Scholar 

  20. 20.

    Wetterich, C. Exact evolution equation for the effective potential. Phys. Lett. B301, 90–94 (1993).

    ADS  Article  Google Scholar 

  21. 21.

    White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett.69, 2863–2866 (1992).

    ADS  Article  Google Scholar 

  22. 22.

    Ma, S.-k, Dasgupta, C. & Hu, C.-k Random antiferromagnetic chain. Phys. Rev. Lett.43, 1434–1437 (1979).

    ADS  Article  Google Scholar 

  23. 23.

    Corboz, P. & Mila, F. Tensor network study of the Shastry–Sutherland model in zero magnetic field. Phys. Rev. B87, 115144 (2013).

    ADS  Article  Google Scholar 

  24. 24.

    Capponi, S., Chandra, V. R., Auerbach, A. & Weinstein, M. p6 chiral resonating valence bonds in the kagome antiferromagnet. Phys. Rev. B87, 161118 (2013).

    ADS  Article  Google Scholar 

  25. 25.

    Auerbach, A. Interacting Electrons and Quantum Magnetism (Springer, New York, NY, 1994).

  26. 26.

    Gaite, J. & O’Connor, D. Field theory entropy, the h theorem, and the renormalization group. Phys. Rev. D54, 5163–5173 (1996).

    ADS  MathSciNet  Article  Google Scholar 

  27. 27.

    Preskill, J. Quantum information and physics: some future directions. J. Mod. Opt.47, 127–137 (2000).

    ADS  MathSciNet  Article  Google Scholar 

  28. 28.

    Apenko, S. M. Information theory and renormalization group flows. Phys. A391, 62–77 (2012).

    MathSciNet  Article  Google Scholar 

  29. 29.

    Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science342, 604–607 (2013).

    ADS  Article  Google Scholar 

  30. 30.

    Beny, C. & Osborne, T. J. The renormalization group via statistical inference. New. J. Phys.17, 083005 (2015).

    ADS  MathSciNet  Article  Google Scholar 

  31. 31.

    Stephan, J.-M., Inglis, S., Fendley, P. & Melko, R. G. Geometric mutual information at classical critical points. Phys. Rev. Lett.112, 127204 (2014).

    ADS  Article  Google Scholar 

  32. 32.

    Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. In Proc. 37th Allerton Conf. on Communication, Control and Computation (eds ​Hajek, B. & Sreenivas, R. S.) 49, 368–377 (University of Illinois, 2001).

  33. 33.

    Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput.14, 1771–1800 (2002).

    Article  Google Scholar 

  34. 34.

    Ludwig, A. W. W. & Cardy, J. L. Perturbative evaluation of the conformal anomaly at new critical points with applications to random systems. Nucl. Phys. B285, 687–718 (1987).

    ADS  MathSciNet  Article  Google Scholar 

  35. 35.

    Fisher, M. E. & Stephenson, J. Statistical mechanics of dimers on a plane lattice. II. Dimer correlations and monomers. Phys. Rev.132, 1411–1431 (1963).

    ADS  MathSciNet  Article  Google Scholar 

  36. 36.

    Fradkin, E. Field Theories of Condensed Matter Physics (Cambridge Univ. Press, Cambridge, 2013).

  37. 37.

    Mehta, P. & Schwab, D. J. An exact mapping between the variational renormalization group and deep learning. Preprint at abs/1410.3831 (2014).

  38. 38.

    McCoy, B. M. & Wu, T. T. The Two-Dimensional Ising Model (Harvard Univ. Press, Cambridge, MA, 1973).

  39. 39.

    Schoenholz, S. S., Cubuk, E. D., Sussman, D. M., Kaxiras, E. & Liu, A. J. A structural approach to relaxation in glassy liquids. Nat. Phys.12, 469–471 (2016).

    Article  Google Scholar 

  40. 40.

    Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science349, 255–260 (2015).

    ADS  MathSciNet  Article  Google Scholar 

  41. 41.

    Slonim, N. & Tishby, N. Document clustering using word clusters via the information bottleneck method. In Proc. 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, SIGIR ’00 208–215 (ACM, 2000).

Download references

Acknowledgements

We thank S. Huber and P. Fendley for discussions. M.K.-J. gratefully acknowledges the support of the Swiss National Science Foundation. Z.R. was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 657111.

Author information

Affiliations

Authors

Contributions

M.K.-J. and Z.R. contributed equally to this work.

Corresponding author

Correspondence to Maciej Koch-Janusz.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publishers note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Methods

Methods, Supplementary Figures 1–6, Supplementary References 1–12

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Koch-Janusz, M., Ringel, Z. Mutual information, neural networks and the renormalization group. Nature Phys 14, 578–582 (2018). https://doi.org/10.1038/s41567-018-0081-4

Download citation

Further reading