Physical systems differing in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains ‘slow’ degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine-learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, which performs this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine-learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.
Subscribe to Journal
Get full journal access for 1 year
only $15.58 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
LeCun, Y., Bengio, Y. & Hinton, G. E. Deep learning. Nature521, 436–444 (2015).
Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature529, 584–589 (2016).
Hershey, J. R., Rennie, S. J., Olsen, P. A. & Kristjansson, T. T. Super-human multi-talker speech recognition: A graphical modeling approach. Comput. Speech Lang.24, 45–66 (2010).
Carrasquilla, J. & Melko, R. G. Machine learning phases of matter. Nat. Phys.13, 431–434 (2017).
Torlai, G. & Melko, R. G. Learning thermodynamics with Boltzmann machines. Phys. Rev. B94, 165134 (2016).
van Nieuwenburg, E. P. L., Liu, Y.-H. & Huber, S. D. Learning phase transitions by confusion. Nat. Phys.13, 435–439 (2017).
Wang, L. Discovering phase transitions with unsupervised learning. Phys. Rev. B94, 195105 (2016).
Ohtsuki, T. & Ohtsuki, T. Deep learning the quantum phase transitions in random electron systems: applications to three dimensions. J. Phys. Soc. Jpn86, 044708 (2017).
Ronhovde, P.et al Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering. Eur. Phys. J. E 34, 105 (2011).
Ronhovde, P.et al Detection of hidden structures for arbitrary scales in complex physical systems. Sci. Rep. 2, 329 (2012).
Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science313, 504–507 (2006).
Lin, H. W. & Tegmark, M. Why does deep and cheap learning work so well? J. Stat. Phys.168, 1223–1247 (2017).
Carleo, G. & Troyer, M. Solving the quantum many-body problem with artificial neural networks. Science355, 602–606 (2017).
Deng, D.-L., Li, X. & Sarma, S. D. Machine learning topological states. Phys. Rev. B96, 195145 (2017).
Wilson, K. G. The renormalization group: Critical phenomena and the Kondo problem. Rev. Mod. Phys.47, 773–840 (1975).
Politzer, H. D. Reliable perturbative results for strong interactions? Phys. Rev. Lett.30, 1346–1349 (1973).
Berezinskii, V. L. Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group I. Classical systems. Sov. J. Exp. Theor. Phys.32, 493 (1971).
Kosterlitz, J. M. & Thouless, D. Ordering, metastability and phase transitions in two-dimensional systems. J. Phys. C6, 1181 (1973).
Kadanoff, L. P. Scaling laws for Ising models near T(c). Physics2, 263–272 (1966).
Wetterich, C. Exact evolution equation for the effective potential. Phys. Lett. B301, 90–94 (1993).
White, S. R. Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett.69, 2863–2866 (1992).
Ma, S.-k, Dasgupta, C. & Hu, C.-k Random antiferromagnetic chain. Phys. Rev. Lett.43, 1434–1437 (1979).
Corboz, P. & Mila, F. Tensor network study of the Shastry–Sutherland model in zero magnetic field. Phys. Rev. B87, 115144 (2013).
Capponi, S., Chandra, V. R., Auerbach, A. & Weinstein, M. p6 chiral resonating valence bonds in the kagome antiferromagnet. Phys. Rev. B87, 161118 (2013).
Auerbach, A. Interacting Electrons and Quantum Magnetism (Springer, New York, NY, 1994).
Gaite, J. & O’Connor, D. Field theory entropy, the h theorem, and the renormalization group. Phys. Rev. D54, 5163–5173 (1996).
Preskill, J. Quantum information and physics: some future directions. J. Mod. Opt.47, 127–137 (2000).
Apenko, S. M. Information theory and renormalization group flows. Phys. A391, 62–77 (2012).
Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science342, 604–607 (2013).
Beny, C. & Osborne, T. J. The renormalization group via statistical inference. New. J. Phys.17, 083005 (2015).
Stephan, J.-M., Inglis, S., Fendley, P. & Melko, R. G. Geometric mutual information at classical critical points. Phys. Rev. Lett.112, 127204 (2014).
Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. In Proc. 37th Allerton Conf. on Communication, Control and Computation (eds Hajek, B. & Sreenivas, R. S.) 49, 368–377 (University of Illinois, 2001).
Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput.14, 1771–1800 (2002).
Ludwig, A. W. W. & Cardy, J. L. Perturbative evaluation of the conformal anomaly at new critical points with applications to random systems. Nucl. Phys. B285, 687–718 (1987).
Fisher, M. E. & Stephenson, J. Statistical mechanics of dimers on a plane lattice. II. Dimer correlations and monomers. Phys. Rev.132, 1411–1431 (1963).
Fradkin, E. Field Theories of Condensed Matter Physics (Cambridge Univ. Press, Cambridge, 2013).
Mehta, P. & Schwab, D. J. An exact mapping between the variational renormalization group and deep learning. Preprint at abs/1410.3831 (2014).
McCoy, B. M. & Wu, T. T. The Two-Dimensional Ising Model (Harvard Univ. Press, Cambridge, MA, 1973).
Schoenholz, S. S., Cubuk, E. D., Sussman, D. M., Kaxiras, E. & Liu, A. J. A structural approach to relaxation in glassy liquids. Nat. Phys.12, 469–471 (2016).
Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science349, 255–260 (2015).
Slonim, N. & Tishby, N. Document clustering using word clusters via the information bottleneck method. In Proc. 23rd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, SIGIR ’00 208–215 (ACM, 2000).
We thank S. Huber and P. Fendley for discussions. M.K.-J. gratefully acknowledges the support of the Swiss National Science Foundation. Z.R. was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no. 657111.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Koch-Janusz, M., Ringel, Z. Mutual information, neural networks and the renormalization group. Nature Phys 14, 578–582 (2018). https://doi.org/10.1038/s41567-018-0081-4
Entropic Dynamics in Neural Networks, the Renormalization Group and the Hamilton-Jacobi-Bellman Equation
Physical Review Research (2020)
Physical Review Research (2020)
Physical Review E (2020)
Advances in Physics: X (2020)