Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The low-rank hypothesis of complex systems

Nature Physics volume 20, pages 294–302 (2024)Cite this article

Subjects

Abstract

Complex systems are high-dimensional nonlinear dynamical systems with heterogeneous interactions among their constituents. To make interpretable predictions about their large-scale behaviour, it is typically assumed that these dynamics can be reduced to a few equations involving a low-rank matrix describing the network of interactions. Our Article sheds light on this low-rank hypothesis and questions its validity. Using fundamental theorems on singular-value decomposition, we probe the hypothesis for various random graphs, either by making explicit their low-rank formulation or by demonstrating the exponential decrease of their singular values. We verify the hypothesis for real networks by revealing the rapid decrease of their singular values, which has major consequences for their effective ranks. We then evaluate the impact of the low-rank hypothesis for general dynamical systems on networks through an optimal dimension reduction. This allows us to prove that recurrent neural networks can be exactly reduced, and we can connect the rapidly decreasing singular values of real networks to the dimension reduction error of the nonlinear dynamics they support. Finally, we prove that higher-order interactions naturally emerge from the dimension reduction, thus providing insights into the origin of higher-order interactions in complex systems.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Experimental verification of the low-rank hypothesis for real networks.
Fig. 2: Three indicators of the low-rank hypothesis for random graphs.
Fig. 3: The low-rank hypothesis of complex systems and the emergence of higher-order interactions.
Fig. 4: Dimension reduction errors for nonlinear dynamics on real complex networks related to their singular values and effective ranks.

Similar content being viewed by others

Data availability

All the details about the real networks data used in the paper, mostly from the network repository Netzschleuder, are given in Supplementary Information section IV. The data to generate Figs. 1, 2 and 4 are available on Zenodo (https://doi.org/10.5281/zenodo.8342130).

Code availability

The Python code used to generate the results of the paper is available on Zenodo (https://doi.org/10.5281/zenodo.8342130). The code for the optimal shrinkage of singular values is a Python implementation of the Matlab codes optimal_singval_threshold (ref. 67) and optimal_singval_shrink (ref. 24), which is partly based on the repository optht by B. Erichson.

References

  1. Bellman, R. Dynamic Programming (Princeton Univ. Press, 1957).

  2. Ganguli, S. & Sompolinsky, H. Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. Annu. Rev. Neurosci. 35, 485 (2012).

    Article  CAS  PubMed  Google Scholar 

  3. Abbott, L. F. et al. The mind of a mouse. Cell 182, 1372 (2020).

    Article  CAS  PubMed  Google Scholar 

  4. Anderson, P. W. More is different. Science 177, 393 (1972).

    Article  CAS  PubMed  ADS  Google Scholar 

  5. Strogatz, S. et al. Fifty years of ‘More is different’. Nat. Rev. Phys. 4, 508 (2022).

    Article  Google Scholar 

  6. May, R. M. Simple mathematical models with very complicated dynamics. Nature 261, 459 (1976).

    Article  CAS  PubMed  ADS  Google Scholar 

  7. von Neumann, J. The general and logical theory of automata. In John von Neumann Collected Work, Vol. V (ed. Taub, A. H.) 288–328 (Pergamon, 1963).

  8. Wolfram, S. Cellular automata as models of complexity. Nature 311, 419 (1984).

    Article  ADS  Google Scholar 

  9. Parisi, G. Statistical physics and biology. Phys. World 6, 42 (1993).

    Article  CAS  Google Scholar 

  10. Stein, D. L. & Newman, C. M. Spin Glasses and Complexity (Princeton Univ. Press, 2013).

  11. Funahashi, K. I. & Nakamura, Y. Approximation of dynamical systems by continuous time recurrent neural networks. Neural Netw. 6, 801 (1993).

    Article  Google Scholar 

  12. Fortunato, S. & Newman, M. E. J. 20 years of network community detection. Nat. Phys. 18, 848 (2022).

    Article  CAS  Google Scholar 

  13. Bianconi, G. Higher-Order Networks (Cambridge Univ. Press, 2021).

  14. Battiston, F. et al. The physics of higher-order interactions in complex systems. Nat. Phys. 17, 1093 (2021).

    Article  CAS  Google Scholar 

  15. Wilf, H. S. The eigenvalues of a graph and its chromatic number. J. Lond. Math. Soc. 1, 330 (1967).

    Article  MathSciNet  Google Scholar 

  16. Donath, W. E. & Hoffman, A. J. Lower bounds for the partitioning of graphs. IBM J. Res. Dev. 17, 420 (1973).

    Article  MathSciNet  Google Scholar 

  17. Bonacich, P. Factoring and weighting approaches to status scores and clique identification. J. Math. Sociol. 2, 113 (1972).

    Article  Google Scholar 

  18. Restrepo, J. G., Ott, E. & Hunt, B. R. Onset of synchronization in large networks of coupled oscillators. Phys. Rev. E 71, 036151 (2005).

    Article  MathSciNet  ADS  Google Scholar 

  19. Horn, R. A. & Johnson, C. R. Matrix Analysis (Cambridge Univ. Press, 2013).

  20. Weyl, H. Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung). Math. Ann. 71, 441 (1912).

    Article  MathSciNet  Google Scholar 

  21. Fan, K. Maximum properties and inequalities for the eigenvalues of completely continuous operators. Proc. Natl Acad. Sci. USA 37, 760 (1951).

    Article  MathSciNet  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  22. Cai, J.-F., Candès, E. J. & Shen, Z. A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 46, 1956 (2010).

    Article  MathSciNet  Google Scholar 

  23. Kutz, J. N., Brunton, S. L. & Brunton, B. W. Dynamic Mode Decomposition (SIAM, 2016).

  24. Gavish, M. & Donoho, D. L. Optimal shrinkage of singular values. IEEE Trans. Inf. Theory 63, 2137 (2017).

    Article  MathSciNet  Google Scholar 

  25. Kalman, R. E. On the general theory of control systems. In Proc. 1st International IFAC Congress on Automatic and Remote Control (eds. Coales, J. F., Ragazzini, D. J. & Fuller, A. T.) 491–502 (Elsevier, 1960).

  26. Kalman, R. E. Contributions to the theory of time-optimal control. Bol. Soc. Mat. Mex. 5, 102 (1960).

    Google Scholar 

  27. Yan, G. et al. Network control principles predict neuron function in the Caenorhabditis elegans connectome. Nature 550, 519 (2017).

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  28. Marčenko, V. A. & Pastur, L. A. Distribution of eigenvalues for some sets of random matrices. Math. USSR-Sbornik 1, 457 (1967).

    Article  Google Scholar 

  29. Féral, D. & Péché, S. The largest eigenvalue of rank one deformation of large Wigner matrices. Commun. Math. Phys. 272, 185 (2007).

    Article  MathSciNet  ADS  Google Scholar 

  30. Capitaine, M., Donati-Martin, C. & Féral, D. The largest eigenvalues of finite rank deformation of large Wigner matrices: convergence and nonuniversality of the fluctuation. Ann. Probab. 37, 1 (2009).

    Article  MathSciNet  Google Scholar 

  31. Benaych-Georges, F. & Nadakuditi, R. R. The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math. 227, 494 (2011).

    Article  MathSciNet  Google Scholar 

  32. Benaych-Georges, F. & Nadakuditi, R. R. The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivar. Anal. 111, 120 (2012).

    Article  MathSciNet  Google Scholar 

  33. Pizzo, A., Renfrew, D. & Soshnikov, A. On finite rank deformations of Wigner matrices. Ann. I. H. Poincaré PR 49, 64–94 (2013).

  34. Baik, J., Ben Arous, G. & Péché, S. Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33, 1643 (2005).

    Article  MathSciNet  Google Scholar 

  35. Valdano, E. & Arenas, A. Exact rank reduction of network models. Phys. Rev. X 9, 031050 (2019).

    CAS  Google Scholar 

  36. Beiran, M., Dubreuil, A., Valente, A., Mastrogiuseppe, F. & Ostojic, S. Shaping dynamics with multiple populations in low-rank recurrent networks. Neural Comput. 33, 1572 (2021).

    Article  MathSciNet  PubMed  Google Scholar 

  37. Gao, P. & Ganguli, S. On simplicity and complexity in the brave new world of large-scale neuroscience. Curr. Opin. Neurobiol. 32, 148 (2015).

    Article  CAS  PubMed  Google Scholar 

  38. Beckermann, B. & Townsend, A. On the singular values of matrices with displacement structure. SIAM J. Matrix Anal. Appl. 38, 1227 (2017).

    Article  MathSciNet  Google Scholar 

  39. Udell, M. & Townsend, A. Why are big data matrices approximately low rank? SIAM J. Math. Data Sci. 1, 144 (2019).

    Article  MathSciNet  Google Scholar 

  40. Gao, J., Barzel, B. & Barabási, A.-L. Universal resilience patterns in complex networks. Nature 530, 307 (2016).

    Article  CAS  PubMed  ADS  Google Scholar 

  41. Tu, C., Grilli, J., Schuessler, F. & Suweis, S. Collapse of resilience patterns in generalized Lotka–Volterra dynamics and beyond. Phys. Rev. E 95, 062307 (2017).

    Article  MathSciNet  PubMed  ADS  Google Scholar 

  42. Jiang, J. et al. Predicting tipping points in mutualistic networks through dimension reduction. Proc. Natl Acad. Sci. USA 115, E639 (2018).

    Article  MathSciNet  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  43. Laurence, E., Doyon, N., Dubé, L. J. & Desrosiers, P. Spectral dimension reduction of complex dynamical networks. Phys. Rev. X 9, 011042 (2019).

    CAS  Google Scholar 

  44. Vegué, M., Thibeault, V., Desrosiers, P. & Allard, A. Dimension reduction of dynamics on modular and heterogeneous directed networks. PNAS Nexus 2, pgad150 (2023).

  45. Kundu, P., Kori, H. & Masuda, N. Accuracy of a one-dimensional reduction of dynamical systems on networks. Phys. Rev. E 105, 024305 (2022).

    Article  MathSciNet  CAS  PubMed  ADS  Google Scholar 

  46. Thibeault, V., St-Onge, G., Dubé, L. J. & Desrosiers, P. Threefold way to the dimension reduction of dynamics on networks: an application to synchronization. Phys. Rev. Res. 2, 043215 (2020).

    Article  CAS  Google Scholar 

  47. Kuehn, C. & Bick, C. A universal route to explosive phenomena. Sci. Adv. 7, 1 (2021).

    Article  Google Scholar 

  48. St-Onge, G., Thibeault, V., Allard, A., Dubé, L. J. & Hébert-Dufresne, L. Social confinement and mesoscopic localization of epidemics on networks. Phys. Rev. Lett. 126, 098301 (2021).

    Article  MathSciNet  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  49. Battiston, F. et al. Networks beyond pairwise interactions: structure and dynamics. Phys. Rep. 874, 1 (2020).

    Article  MathSciNet  ADS  Google Scholar 

  50. Matheny, M. H. et al. Exotic states in a simple network of nanoelectromechanical oscillators. Science 363, 1057 (2019).

    Article  MathSciNet  Google Scholar 

  51. Nijholt, E., Ocampo-Espindola, J. L., Eroglu, D., Kiss, I. Z. & Pereira, T. Emergent hypernetworks in weakly coupled oscillators. Nat. Commun. 13, 4849 (2022).

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  52. Gallo, G., Longo, G., Pallottino, S. & Nguyen, S. Directed hypergraphs and applications. Discret. Appl. Math. 42, 177 (1993).

    Article  MathSciNet  Google Scholar 

  53. Palla, G., Derényi, I., Farkas, I. & Vicsek, T. Uncovering the overlapping community structure of complex networks in nature and society. Nature 435, 814 (2005).

    Article  CAS  PubMed  ADS  Google Scholar 

  54. Yu, S. et al. Higher-order interactions characterized in cortical activity. J. Neurosci. 31, 17514 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Mayfield, M. M. & Stouffer, D. B. Higher-order interactions capture unexplained complexity in diverse communities. Nat. Ecol. Evol. 1, 1 (2017).

    Article  Google Scholar 

  56. Ferraz de Arruda, G., Tizzani, M. & Moreno, Y. Phase transitions and stability of dynamical processes on hypergraphs. Commun. Phys. 4, 24 (2021).

    Article  Google Scholar 

  57. Qi, L. & Luo, Z. Tensor Analysis (SIAM, 2017).

  58. Watanabe, S. & Strogatz, S. H. Constants of motion for superconducting Josephson arrays. Phys. D 74, 197 (1994).

    Article  Google Scholar 

  59. Brunton, S. L., Budišić, M., Kaiser, E. & Kutz, J. N. Modern Koopman theory for dynamical systems. SIAM Rev. 64, 229 (2022).

    Article  MathSciNet  Google Scholar 

  60. Valente, A., Pillow, J. W. & Ostojic, S. Extracting computational mechanisms from neural data using low-rank RNNs. In Advances in Neural Information Processing Systems (eds. Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K. & Oh, A.) 24072–24086 (Curran Associates, Inc., 2022).

  61. Holland, J. H. Hidden Order: How Adaptation Builds Complexity (Addison-Wesley, 1995).

  62. Montanari, A. N., Duan, C., Aguirre, L. A. & Motter, A. E. Functional observability and target state estimation in large-scale networks. Proc. Natl Acad. Sci. USA 119, e2113750119 (2022).

    Article  MathSciNet  CAS  PubMed  Google Scholar 

  63. Sanhedrai, H. et al. Reviving a failed network through microscopic interventions. Nat. Phys. 18, 338 (2022).

    Article  CAS  Google Scholar 

  64. Desrosiers, P. & Roy-Pomerleau, X. One for all. Nat. Phys. 18, 238 (2022).

    Article  CAS  Google Scholar 

  65. Martin, C. H. & Mahoney, M. W. Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning. J. Mach. Learn. Res. 22, 1 (2021).

    MathSciNet  Google Scholar 

  66. Gower, J. Properties of Euclidean and non-Euclidean distance matrices. Linear Algebra Appl. 67, 81 (1985).

    Article  MathSciNet  Google Scholar 

  67. Gavish, M. & Donoho, D. L. The optimal hard threshold for singular values is \(4/\sqrt{3}\). IEEE Trans. Inf. Theory 60, 5040 (2014).

    Article  Google Scholar 

  68. Donoho, D., Gavish, M. & Johnstone, I. Optimal shrinkage of eigenvalues in the spiked covariance model. Ann. Statis. 46, 1742 (2018).

    Article  MathSciNet  Google Scholar 

  69. Scheffer, L. K. et al. A connectome and analysis of the adult Drosophila central brain. eLife 9, 1 (2020).

    Article  Google Scholar 

  70. Malinowski, E. R. Theory of error in factor analysis. Anal. Chem. 49, 606 (1977).

    Article  CAS  Google Scholar 

  71. Sánchez, E. & Kowalski, B. R. Generalized rank annihilation factor analysis. Anal. Chem. 58, 496 (1986).

    Article  Google Scholar 

  72. Abdi, H. & Williams, L. J. Principal component analysis. WIREs Comput. Stat. 2, 433 (2010).

    Article  Google Scholar 

  73. Almagro, P., Boguñá, M. & Ángeles Serrano, M. Detecting the ultra low dimensionality of real networks. Nat. Commun. 13, 6096 (2022).

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  74. Lynn, C. W. & Bassett, D. S. Compressibility of complex networks. Proc. Natl Acad. Sci. USA 118, e2023473118 (2021).

    Article  MathSciNet  CAS  PubMed  PubMed Central  Google Scholar 

  75. Perry, P. O. Cross-Validation for Unsupervised Learning. PhD thesis, Stanford Univ. (2009).

  76. Städter, P., Schälte, Y., Schmiester, L., Hasenauer, J. & Stapor, P. L. Benchmarking of numerical integration methods for ODE models of biological systems. Sci. Rep. 11, 2696 (2021).

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  77. Sompolinsky, H., Crisanti, A. & Sommers, H.-J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259 (1988).

    Article  MathSciNet  CAS  PubMed  ADS  Google Scholar 

Download references

Acknowledgements

We are grateful to G. Eilerstein for sharing the code to extract the weight matrices from the repository NWS, G. Jékely for sharing the neuronal and desmosomal connectomes of Platynereis dumerilii, C. Murphy for useful discussions on artificial neural networks, G. St-Onge for his comments on the preprint and X. Roy-Pomerleau for helping us to explore the microbial dynamics numerically. We thank É. Boran for his fundamental contribution to linear algebra. This work was supported by the Fonds de recherche du Québec—Nature et technologies (V.T. and P.D.), the Natural Sciences and Engineering Research Council of Canada (V.T., A.A. and P.D.) and the Sentinelle Nord programme of Université Laval, funded by the Canada First Research Excellence Fund (V.T., A.A. and P.D.).

Author information

Authors and Affiliations

  1. Département de physique, de génie physique et d’optique, Université Laval, Québec, Québec, Canada

    Vincent Thibeault, Antoine Allard & Patrick Desrosiers

  2. Centre interdisciplinaire en modélisation mathématique de l’Université Laval, Québec, Québec, Canada

    Vincent Thibeault, Antoine Allard & Patrick Desrosiers

  3. Centre de recherche CERVO, Québec, Québec, Canada

    Patrick Desrosiers

Authors
  1. Vincent Thibeault
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antoine Allard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Desrosiers
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the formulation of the study, the interpretation of the results and the editing of the paper. V.T. and P.D. obtained the mathematical results and conceived the conceptual basis of the project. V.T. led the writing of the manuscript, wrote the supplementary information with P.D., designed the figures, wrote the code and performed the numerical experiments to generate the results. V.T., A.A. and P.D. contributed to the code and analysed the data to generate Fig. 1.

Corresponding authors

Correspondence to Vincent Thibeault or Patrick Desrosiers.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Physics thanks Jianxi Gao and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Table 1 Vector fields of typical nonlinear dynamics on a network and their least-square optimal reduced dynamics
Full size table

Supplementary information

Supplementary Information

Supplementary mathematical details to support the results of the paper, Fig. 1–12, discussion and details of the real network data.

Supplementary Table 1

The table lists all the real networks that were used in the paper. All the names have a clickable link to the source of the data, along with their number of vertices N, their rank and their effective ranks.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thibeault, V., Allard, A. & Desrosiers, P. The low-rank hypothesis of complex systems. Nat. Phys. 20, 294–302 (2024). https://doi.org/10.1038/s41567-023-02303-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41567-023-02303-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics