Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Unifying pairwise interactions in complex dynamics

Nature Computational Science volume 3, pages 883–893 (2023)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Scientists have developed hundreds of techniques to measure the interactions between pairs of processes in complex systems, but these computational methods—from contemporaneous correlation coefficients to causal inference methods—define and formulate interactions differently, using distinct quantitative theories that remain largely disconnected. Here we introduce a large assembled library of 237 statistics of pairwise interactions, and assess their behavior on 1,053 multivariate time series from a wide range of real-world and model-generated systems. Our analysis highlights commonalities between disparate mathematical formulations of interactions, providing a unified picture of a rich interdisciplinary literature. Using three real-world case studies, we then show that simultaneously leveraging diverse methods can uncover those most suitable for addressing a given problem, facilitating interpretable understanding of the quantitative formulation of pairwise dependencies that drive successful performance. Our results and accompanying software enable comprehensive analysis of time-series interactions by drawing on decades of diverse methodological contributions.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The behavior of a scientific library of 237 SPIs was evaluated using a collection of 1,053 MTS.
Fig. 2: Statistics for measuring pairwise interactions between time series can be organized into 14 modules on the basis of their behavior on over 1,000 MTS, providing an intuitive, data-driven organization of interdisciplinary scientific literature.
Fig. 3: A comprehensive library of SPIs can be used to accurately classify and understand differences in human movement and neural activity datasets.

Similar content being viewed by others

Data availability

The full database of 1,053 diverse real-world and simulated MTS analyzed here is available on the Zenodo repository at: https://doi.org/10.5281/zenodo.7118947 (ref. 58). This resource could be used to test scientific methods on a diverse sample of MTS. Time-series data used in the three case studies are from open sources, as described in the ‘Classification case studies’ section in the Methods. Source data are provided with this paper.

Code availability

Accompanying this paper is an extendable Python-based package called pyspi57, which includes implementations of all 237 SPIs. This software allows users to compare the behavior of an interdisciplinary literature on methods for quantifying interactions between pairs of time series. Furthermore, code and instructions for reproducing the results and figures presented in this work can be found on the Zenodo repository at: https://doi.org/10.5281/zenodo.8027702 (ref. 67).

References

  1. Bassett, D. S. & Sporns, O. Network neuroscience. Nat. Neurosci. 20, 353–364 (2017).

    Article  Google Scholar 

  2. Buckner, R. L. et al. Molecular, structural, and functional characterization of Alzheimer’s disease: evidence for a relationship between default activity, amyloid, and memory. J. Neurosci. 25, 7709 (2005).

    Article  Google Scholar 

  3. Sugihara, G. et al. Detecting causality in complex ecosystems. Science 338, 496–500 (2012).

    Article  MATH  Google Scholar 

  4. Runge, J. et al. Inferring causation from time series in Earth system sciences. Nat. Commun. 10, 2553 (2019).

    Article  Google Scholar 

  5. Engle, R. F. & Granger, C. W. Co-integration and error correction: representation, estimation, and testing. Econometrica 55, 251–276 (1987).

    Article  MathSciNet  MATH  Google Scholar 

  6. Reinsel, G. C. Elements of Multivariate Time Series Analysis (Springer, 2003).

  7. Stephens, Z. D. PLoS Biol. 13, e1002195 (2015).

    Article  Google Scholar 

  8. van den Heuvel, M. P. & Hulshoff Pol, H. E. Exploring the brain network: a review on resting-state fMRI functional connectivity. Eur. Neuropsychopharmacol. 20, 519 (2010).

    Article  Google Scholar 

  9. Cliff, O. M., Novelli, L., Fulcher, B. D., Shine, J. M. & Lizier, J. T. Assessing the significance of directed and multivariate measures of linear dependence between time series. Phys. Rev. Res. 3, 013145 (2021).

    Article  Google Scholar 

  10. Sakoe, H. & Chiba, S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Signal Process 26, 43 (1978).

    Article  MATH  Google Scholar 

  11. Granger, C. W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424 (1969).

    Article  MATH  Google Scholar 

  12. Fulcher, B. D., Little, M. A. & Jones, N. S. Highly comparative time-series analysis: the empirical structure of time series and their methods. J. R. Soc. Interface 10, 20130048 (2013).

    Article  Google Scholar 

  13. Peach, R. L. et al. HCGA: Highly comparative graph analysis for network phenotyping. Patterns 2, 100227 (2021).

    Article  Google Scholar 

  14. Massey, J. Causality, feedback and directed information. In International Symposium on Information Theory and its Applications (ISITA-90) 303–305 (IEEE, 1990).

  15. Tononi, G. An information integration theory of consciousness. BMC Neurosci. 5, 42 (2004).

    Article  Google Scholar 

  16. Oizumi, M., Amari, S.-i, Yanagawa, T., Fujii, N. & Tsuchiya, N. Measuring integrated information from the decoding perspective. PLoS Comput. Biol. 12, e1004654 (2016).

    Article  Google Scholar 

  17. Wiener, N. in Modern Mathematics for Engineers (ed. Beckenback, E.) (McGraw-Hill, 1956).

  18. Pesaran, M. H. & Shin, Y. An autoregressive distributed lag modelling approach to cointegration analysis. In Econometrics and Economic Theory in the 20th Century: The Ragnar Frisch Centennial Symposium (ed. Strøm, S.) (Cambridge Univ. Press, 1999).

  19. Nolte, G. et al. Robustly estimating the flow direction of information in complex physical systems. Phys. Rev. Lett. 100, 234101 (2008).

    Article  Google Scholar 

  20. Schreiber, T. Measuring information transfer. Phys. Rev. Lett. 85, 461 (2000).

    Article  Google Scholar 

  21. Bastos, A. M. & Schoffelen, J.-M. A tutorial review of functional connectivity analysis methods and their interpretational pitfalls. Front. Syst. Neurosci. https://doi.org/10.3389/fnsys.2015.00175 (2016).

  22. Oizumi, M., Tsuchiya, N. & Amari, S.-I. Unified framework for information integration based on information geometry. Proc. Natl Acad. Sci. USA 113, 14817 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  23. Ay, N. Information geometry on complexity and stochastic interaction. Entropy 17, 2432 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  24. Geweke, J. Measurement of linear dependence and feedback between multiple time series. J. Am. Stat Assoc. 77, 304 (1982).

    Article  MathSciNet  MATH  Google Scholar 

  25. Barnett, L., Barrett, A. B. & Seth, A. K. Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103, 238701 (2009).

    Article  Google Scholar 

  26. Cliff, O. M., Prokopenko, M. & Fitch, R. Minimising the Kullback–Leibler divergence for model selection in distributed nonlinear systems. Entropy 20, 51 (2018).

    Article  MathSciNet  Google Scholar 

  27. Lachaux, J.-P., Rodriguez, E., Martinerie, J. & Varela, F. J. Measuring phase synchrony in brain signals. Hum Brain Mapp 8, 194–208 (1999).

    Article  Google Scholar 

  28. MacKay, D. J. in Information Theory, Inference and Learning Algorithms (Cambridge Univ. Press, 2003).

  29. Shen, C., Priebe, C. E. & Vogelstein, J. T. From distance correlation to multiscale graph correlation. J. Am. Stat Assoc. 115, 280 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  30. Gretton, A. et al. A kernel statistical test of independence. In Advances in Neural Information Processing Systems (eds Roweis, S. et al.) Vol. 20 (Curran Associates, 2008).

  31. Sejdinovic, D., Sriperumbudur, B., Gretton, A. & Fukumizu, K. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Annals Stat. 41, 2263–2291 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  32. Heller, R., Heller, Y. & Gorfine, M. A consistent multivariate test of association based on ranks of distances. Biometrika 100, 503–510 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  33. Ruiz, A. P., Flynn, M., Large, J., Middlehurst, M. & Bagnall, A. The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 35, 401–449 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  34. Bagnall, A. et al. The UEA multivariate time series classification archive. Preprint at https://arxiv.org/abs/1811.00075 (2018).

  35. Birbaumer, N. et al. A spelling device for the paralysed. Nature 398, 297–298 (1999).

    Article  Google Scholar 

  36. Byrge, L. & Kennedy, D. P. Accurate prediction of individual subject identity and task, but not autism diagnosis, from functional connectomes. Hum. Brain Mapp. 41, 2249 (2020).

    Article  Google Scholar 

  37. Liu, Y. & Aviyente, S. Quantification of effective connectivity in the brain using a measure of directed information. Comput. Math Methods Med. 2012, 635103 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  38. Mehta, K. & Kliewer, J. Directional and causal information flow in EEG for assessing perceived audio quality. IEEE Trans. Mol. Biol. Multiscale Commun. 3, 150–165 (2017).

    Google Scholar 

  39. Wang, G. & Takigawa, M. Directed coherence as a measure of interhemispheric correlation of EEG. Int J. Psychophysiol. 13, 119–128 (1992).

    Article  Google Scholar 

  40. Schelter, B. et al. Testing for directed influences among neural signals using partial directed coherence. J. Neurosci. Methods 152, 210–219 (2006).

    Article  Google Scholar 

  41. Smith, S. M. et al. Network modelling methods for FMRI. NeuroImage 54, 875–891 (2011).

    Article  Google Scholar 

  42. Evans, T. & Lambiotte, R. Line graphs, link partitions, and overlapping communities. Phys. Rev. E 80, 016105 (2009).

    Article  Google Scholar 

  43. Ahn, Y.-Y., Bagrow, J. P. & Lehmann, S. Link communities reveal multiscale complexity in networks. Nature 466, 761–764 (2010).

    Article  Google Scholar 

  44. Fulcher, B. D. Feature-based time-series analysis. In Feature Engineering for Machine Learning and Data Analytics 87–116 (CRC, 2018).

  45. Fulcher, B. D. & Jones, N. S. hctsa: a computational framework for automated time-series phenotyping using massive feature extraction. Cell Syst. 5, 527–531.e3 (2017).

    Google Scholar 

  46. Lubba, C. H. et al. catch22: CAnonical Time-series CHaracteristics. Data Min. Knowl. Discov. 33, 1821–1852 (2019).

    Article  Google Scholar 

  47. Phinyomark, A. et al. Navigating features: A topologically informed chart of electromyographic features space. J. R. Soc. 14, 20170734 (2017).

    Google Scholar 

  48. Nosek, B. A., Ebersole, C. R., DeHaven, A. C. & Mellor, D. T. The preregistration revolution. Proc. Natl Acad. Sci. USA 115, 2600 (2018).

    Article  Google Scholar 

  49. Leung, A. et al. Towards blinded classification of loss of consciousness: distinguishing wakefulness from general anesthesia and sleep in flies using a massive library of univariate time series analyses. Preprint at https://psyarxiv.com/rmsv8/ (2023).

  50. Peixoto, T. P. Network reconstruction and community detection from dynamics. Phys. Rev. Lett. 123, 128301 (2019).

    Article  Google Scholar 

  51. Banerjee, A., Chandra, S. & Ott, E. Network inference from short, noisy, low time-resolution, partial measurements: application to C. elegans neuronal calcium dynamics. Proc Natl Acad. Sci. USA 120, e2216030120 (2023).

    Article  MathSciNet  Google Scholar 

  52. Hoffmann, T., Peel, L., Lambiotte, R. & Jones, N. S. Community detection in networks without observing edges. Sci Adv 6, eaav1478 (2020).

    Article  Google Scholar 

  53. McCabe, S. et al. Netrd: A library for network reconstruction and graph distances. J. Open Source Softw. 6, 2990 (2021).

    Article  Google Scholar 

  54. Wollstadt, P. et al. IDTxl: The Information Dynamics Toolkit xl: a Python package for the efficient analysis of multivariate information dynamics in networks. J. Open Source Softw. 4, 1081 (2019).

    Article  Google Scholar 

  55. Peel, L., Peixoto, T. P. & De Domenico, M. Statistical inference links data and theory in network science. Nat. Commun. 13, 6794 (2022).

    Article  Google Scholar 

  56. Battiston, F. et al. The physics of higher-order interactions in complex systems. Nat. Phys. 17, 1093–1098 (2021).

    Article  Google Scholar 

  57. Cliff, O. M. DynamicsAndNeuralSystems/pyspi (Zenodo, 2023); https://doi.org/10.5281/zenodo.8223340

  58. Cliff, O. M. Library of Multivariate Time Series (Zenodo, 2022); https://doi.org/10.5281/zenodo.7118947

  59. Kendall, M. G. A new measure of rank correlation. Biometrika 30, 81 (1938).

    Article  MATH  Google Scholar 

  60. Dhamala, M., Rangarajan, G. & Ding, M. Estimating Granger causality from Fourier and wavelet transforms of time series data. Phys. Rev. Lett. 100, 018701 (2008).

    Article  Google Scholar 

  61. Kaneko, K. & Tsuda, I. Complex systems: Chaos and Beyond (Springer, 2011).

  62. Strogatz, S. H. From Kuramoto to Crawford: exploring the onset of synchronization in populations of coupled oscillators. Physica D 143, 1–20 (2000).

    Article  MathSciNet  MATH  Google Scholar 

  63. Betzel, R., Kennedy, D. & Byrge, L. Resting-State and Movie-Watching Data (Figshare, 2020); https://doi.org/10.6084/m9.figshare.12971162.v2

  64. Yeo, B. T. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 1125–1165 (2011).

  65. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825 (2011).

    MathSciNet  MATH  Google Scholar 

  66. Bonferroni, C. Teoria Statistica Delle Classi e Calcolo Delle Probabilita Vol. 8, 3 (Seeber, 1936).

  67. Bryant, A. G. DynamicsAndNeuralSystems/pyspi_paper_classification: Documentation ft. readme and code comments (Zenodo, 2023); https://doi.org/10.5281/zenodo.8027702

Download references

Acknowledgements

O.M.C., N.T. and B.D.F. were supported by NHMRC Ideas Grant 1183280. N.T. was supported by by Japan Society for the Promotion of Science, Grant-in-Aid for Transformative Research Areas (20H05710, 23H04830, 23H04829). A.G.B. was supported by an Australian Government Research Training Program Scholarship, an American Australian Association Graduate Education Fund Scholarship, and the University of Sydney Physics Foundation. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the paper. High-performance computing facilities provided by the School of Physics, the University of Sydney contributed to our results.

Author information

Authors and Affiliations

  1. School of Physics, The University of Sydney, Camperdown, New South Wales, Australia

    Oliver M. Cliff, Annie G. Bryant & Ben D. Fulcher

  2. Centre for Complex Systems, The University of Sydney, Camperdown, New South Wales, Australia

    Oliver M. Cliff, Annie G. Bryant, Joseph T. Lizier & Ben D. Fulcher

  3. School of Computer Science, The University of Sydney, Camperdown, New South Wales, Australia

    Joseph T. Lizier

  4. Turner Institute for Brain and Mental Health & School of Psychological Sciences, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Melbourne, Victoria, Australia

    Naotsugu Tsuchiya

  5. Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Suita-shi, Japan

    Naotsugu Tsuchiya

  6. Advanced Telecommunications Research Computational Neuroscience Laboratories, Seika-cho, Japan

    Naotsugu Tsuchiya

Authors
  1. Oliver M. Cliff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annie G. Bryant
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joseph T. Lizier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naotsugu Tsuchiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ben D. Fulcher
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

N.T. and B.D.F. conceived of the project. O.M.C. developed the software and large data repositories and performed the main analysis. A.G.B. performed the classification case study analyses. B.D.F. supervised the project with input from J.T.L. and N.T. O.M.C. and B.D.F. wrote the paper, with input from all other co-authors.

Corresponding author

Correspondence to Ben D. Fulcher.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks the anonymous reviewers for their contribution to the peer review of this work. Primary Handling Editor: Jie Pan, in collaboration with the Nature Computational Science team. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary text (41 pages) containing Supplementary Notes 1 (full descriptions of our library of 237 SPIs), 2 (full descriptions of the 1,053 MTS included in our data library), and 3 (supplementary methods and results), and Supplementary Figs. 1, 2 and 3.

Reporting Summary

Peer Review File

Supplementary Data 1

Information on the 237 SPIs measured for the empirical similarity index construction and classification analysis, including directionality, literature category and module. Note that these SPIs refer to the pyspi v.0.2.0 implementation applied to the empirical similarity index computations.

Supplementary Data 2

SPIs dropped due to numerical issues and/or constant values by classification problem.

Supplementary Data 3

Cross-validated accuracy per SPI across the three classification case studies (smartwatch activity, EEG state and fMRI film). Note that these SPIs refer to the pyspi v.0.4.0 implementation applied to the classification case studies.

Source data

Source Data Fig. 3

Statistical source data for histograms (Fig. 3b,e,h) and violin plots (Fig. 3c,f,i). Note that these data files can be produced by Generate_Figure3_Visuals.Rmd in the accompanying GitHub repository: https://github.com/DynamicsAndNeuralSystems/pyspi_paper_code

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cliff, O.M., Bryant, A.G., Lizier, J.T. et al. Unifying pairwise interactions in complex dynamics. Nat Comput Sci 3, 883–893 (2023). https://doi.org/10.1038/s43588-023-00519-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-023-00519-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics