Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Weak pairwise correlations imply strongly correlated network states in a neural population

Nature volume 440, pages 1007–1012 (2006)Cite this article

Abstract

Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher-order interactions among large groups of elements have an important role. Here we show, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons. We find that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions. These maximum entropy models are equivalent to Ising models, and predict that larger networks are completely dominated by correlation effects. This suggests that the neural code has associative or error-correcting properties, and we provide preliminary evidence for such behaviour. As a first test for the generality of these ideas, we show that similar results are obtained from networks of cultured cortical neurons.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Weak pairwise cross-correlations and the failure of the independent approximation.
Figure 2: A maximum entropy model including all pairwise interactions gives an excellent approximation of the full network correlation structure.
Figure 3: Pairwise interactions and individual cell biases, as in equation (1).
Figure 4: Interactions and local fields in networks of different size.
Figure 5: Extrapolation to larger networks.

Similar content being viewed by others

References

  1. Hopfield, J. J. & Tank, D. W. Computing with neural circuits: a model. Science 233, 625–633 (1986)

    ADS  CAS  PubMed  Google Scholar 

  2. Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)

    ADS  CAS  PubMed  Google Scholar 

  3. Hartwell, L. H., Hopfield, J. J., Leibler, S. & Murray, A. W. From molecular to modular cell biology. Nature 402 (Suppl. C), 47–52 (1999)

    Google Scholar 

  4. Barabási, A.-L. & Oltvai, Z. N. Network biology: Understanding the cell's functional organization. Nature Rev. Genet. 5, 101–113 (2004)

    PubMed  Google Scholar 

  5. Perkel, D. H. & Bullock, T. H. Neural coding. Neurosci. Res. Prog. Sum. 3, 221–348 (1968)

    Google Scholar 

  6. Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994)

    ADS  CAS  PubMed  Google Scholar 

  7. Meister, M., Lagnado, L. & Baylor, D. A. Concerted signaling by retinal ganglion cells. Science 270, 1207–1210 (1995)

    ADS  CAS  PubMed  Google Scholar 

  8. Riehle, A., Grun, S., Diesmann, M. & Aertsen, A. Spike synchronization and rate modulation differentially involved in motor cortical function. Science 278, 1950–1953 (1997)

    ADS  CAS  PubMed  Google Scholar 

  9. Dan, Y., Alonso, J. M., Usrey, W. M. & Reid, R. C. Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neurosci. 1, 501–507 (1998)

    CAS  PubMed  Google Scholar 

  10. Hatsopoulos, N., Ojakangas, C., Paninski, L. & Donoghue, J. Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. Natl Acad. Sci. USA 95, 15706–15711 (1998)

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  11. Abbott, L. F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999)

    CAS  PubMed  Google Scholar 

  12. Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001)

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Shamir, M. & Sompolinsky, H. Nonlinear population codes. Neural Comput. 16, 1105–1136 (2004)

    PubMed  MATH  Google Scholar 

  14. Eisen, M. B., Spellman, P. T., Brown, P. O. & Botstein, D. Cluster analysis and display of genome-wide expression patterns. Proc. Natl Acad. Sci. USA 95, 14863–14868 (1998)

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  15. Alter, O., Brown, P. O. & Botstein, D. Singular value decomposition for genome-wide expression data processing and modeling. Proc. Natl Acad. Sci. USA 97, 10101–10106 (2000)

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  16. Holter, N. S., Maritan, A., Cieplak, M., Federoff, N. V. & Banavar, J. R. Dynamic modeling of gene expression data. Proc. Natl Acad. Sci. USA 98, 1693–1698 (2001)

    ADS  CAS  PubMed  PubMed Central  Google Scholar 

  17. Meister, M., Pine, J. & Baylor, D. A. Multi-neuronal signals from the retina: acquisition and analysis. J. Neurosci. Methods 51, 95–106 (1994)

    CAS  PubMed  Google Scholar 

  18. Segev, R., Goodhouse, J., Puchalla, J. L. & Berry, M. J. II . Recoding spikes from a large fraction of the ganglion cells in a retinal patch. Nature Neurosci. 7, 1155–1162 (2004)

    Google Scholar 

  19. Puchalla, J. L., Schneidman, E., Harris, R. A. & Berry, M. J. II . Redundancy in the population code of the retina. Neuron 46, 492–504 (2005)

    Google Scholar 

  20. Frechette, E. S. et al. Fidelity of the ensemble code for visual motion in primate retina. J. Neurophysiol. 94, 119–135 (2005)

    CAS  PubMed  Google Scholar 

  21. Rieke, F., Warland, D., de Ruyter van Steveninck, R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, 1997)

    MATH  Google Scholar 

  22. Martignon, L. et al. Neural coding: higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Comput. 12, 2621–2653 (2000)

    CAS  PubMed  Google Scholar 

  23. Grun, S., Diesmann, M. & Aertsen, A. Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Comput. 14, 43–80 (2002)

    PubMed  MATH  Google Scholar 

  24. Schnitzer, M. J. & Meister, M. Multineuronal firing patterns in the signal from eye to brain. Neuron 37, 499–511 (2003)

    CAS  PubMed  Google Scholar 

  25. Brillouin, L. Science and Information Theory (Academic, New York, 1962)

    MATH  Google Scholar 

  26. Jaynes, E. T. Information theory and statistical mechanics. Phys. Rev. 106, 62–79 (1957)

    ADS  MathSciNet  MATH  Google Scholar 

  27. Schneidman, E., Still, S., Berry, M. J. II & Bialek, W. Network information and connected correlations. Phys. Rev. Lett. 91, 238701 (2003)

    ADS  PubMed  Google Scholar 

  28. Landau, L. D. & Lifshitz, E. M. Statistical Physics 3rd edn (Pergamon, Oxford, 1980)

    MATH  Google Scholar 

  29. Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982)

    ADS  MathSciNet  CAS  PubMed  PubMed Central  MATH  Google Scholar 

  30. Amit, D. J. Modeling Brain Function: The World of Attractor Neural Networks (Cambridge Univ. Press, Cambridge, UK, 1989)

    MATH  Google Scholar 

  31. Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley & Sons, New York, 1991)

    MATH  Google Scholar 

  32. Eytan, D., Brenner, N. & Marom, S. Selective adaptation in networks of cortical neurons. J. Neurosci. 23, 9349–9356 (2003)

    CAS  PubMed  PubMed Central  Google Scholar 

  33. Perkel, D. H., Gerstein, G. L. & Moore, G. P. Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J. 7, 419–440 (1967)

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Mezard, M., Parisi, G. & Virasoro, M. A. Spin Glass Theory and Beyond (World Scientific, Singapore, 1987)

    MATH  Google Scholar 

  35. Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273, 1868–1871 (1996)

    ADS  CAS  PubMed  Google Scholar 

  36. Bi, G. & Poo, M. M. Synaptic modification by correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139–166 (2001)

    CAS  PubMed  Google Scholar 

  37. Barlow, H. Conditions for versatile learning, Helmholtz's unconscious inference, and the task of perception. Vision Res. 30, 1561–1571 (1990)

    CAS  PubMed  Google Scholar 

  38. Smirnakis, S., Berry, M. J. II, Warland, D. K., Bialek, W. & Meister, M. Adaptation of retinal processing to image contrast and spatial scale. Nature 386, 69–73 (1997)

    ADS  CAS  PubMed  Google Scholar 

  39. Hosoya, T., Baccus, S. A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 71–77 (2005)

    ADS  CAS  PubMed  Google Scholar 

  40. NIPS 2003 Workshop. Estimation of entropy and information of undersampled probability distributions. http://nips.cc/Conferences/2003/Workshops/#EstimationofEntropy (2003).

  41. Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R. & Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998)

    ADS  CAS  Google Scholar 

  42. Darroch, J. N. & Ratcliff, D. Generalized iterative scaling for log–linear models. Ann. Math. Stat. 43, 1470–1480 (1972)

    MathSciNet  MATH  Google Scholar 

  43. Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory 37, 145–151 (1991)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We thank G. Stephens and G. Tkačik for discussions, N. Tkachuk for help with the experiments, S. Marom for sharing his lab's cultured cortical networks data with us, and E. J. Chichilnisky for sharing his lab's primate retina results with us. This work was supported in part by the NIH and by the E. Matilda Zeigler Foundation.

Author information

Authors and Affiliations

  1. Joseph Henry Laboratories of Physics, New Jersey, 08544, Princeton, USA

    Elad Schneidman & William Bialek

  2. Department of Molecular Biology, New Jersey, 08544, Princeton, USA

    Elad Schneidman, Michael J. Berry II & Ronen Segev

  3. Lewis-Sigler Institute for Integrative Genomics, Princeton University, New Jersey, 08544, Princeton, USA

    Elad Schneidman & William Bialek

Authors
  1. Elad Schneidman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J. Berry II
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ronen Segev
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William Bialek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elad Schneidman.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Figure 1

Effect of bin size discretization on the second order maximum entropy model. The success of the second order maximum entropy model is not sensitive to the temporal discretization bin size. (PDF 77 kb)

Supplementary Figure 2

The success of the second order maximum entropy model is not sensitive to the size of the network. The success of the second order maximum entropy model is not sensitive to the size of the sub-network modeled. (PDF 72 kb)

Supplementary Figure 3

Average interaction field vs. average local field in maximum entropy models of different size sub-networks. The average interaction field versus the average local field, in the second order maximum entropy models. For larger sub-networks the interaction field dominates the local field. (DOC 40 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schneidman, E., Berry, M., Segev, R. et al. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006). https://doi.org/10.1038/nature04701

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature04701

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing