Weak pairwise correlations imply strongly correlated network states in a neural population

Abstract

Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher-order interactions among large groups of elements have an important role. Here we show, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons. We find that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions. These maximum entropy models are equivalent to Ising models, and predict that larger networks are completely dominated by correlation effects. This suggests that the neural code has associative or error-correcting properties, and we provide preliminary evidence for such behaviour. As a first test for the generality of these ideas, we show that similar results are obtained from networks of cultured cortical neurons.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Weak pairwise cross-correlations and the failure of the independent approximation.
Figure 2: A maximum entropy model including all pairwise interactions gives an excellent approximation of the full network correlation structure.
Figure 3: Pairwise interactions and individual cell biases, as in equation (1).
Figure 4: Interactions and local fields in networks of different size.
Figure 5: Extrapolation to larger networks.

References

  1. 1

    Hopfield, J. J. & Tank, D. W. Computing with neural circuits: a model. Science 233, 625–633 (1986)

  2. 2

    Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)

  3. 3

    Hartwell, L. H., Hopfield, J. J., Leibler, S. & Murray, A. W. From molecular to modular cell biology. Nature 402 (Suppl. C), 47–52 (1999)

  4. 4

    Barabási, A.-L. & Oltvai, Z. N. Network biology: Understanding the cell's functional organization. Nature Rev. Genet. 5, 101–113 (2004)

  5. 5

    Perkel, D. H. & Bullock, T. H. Neural coding. Neurosci. Res. Prog. Sum. 3, 221–348 (1968)

  6. 6

    Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994)

  7. 7

    Meister, M., Lagnado, L. & Baylor, D. A. Concerted signaling by retinal ganglion cells. Science 270, 1207–1210 (1995)

  8. 8

    Riehle, A., Grun, S., Diesmann, M. & Aertsen, A. Spike synchronization and rate modulation differentially involved in motor cortical function. Science 278, 1950–1953 (1997)

  9. 9

    Dan, Y., Alonso, J. M., Usrey, W. M. & Reid, R. C. Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neurosci. 1, 501–507 (1998)

  10. 10

    Hatsopoulos, N., Ojakangas, C., Paninski, L. & Donoghue, J. Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. Natl Acad. Sci. USA 95, 15706–15711 (1998)

  11. 11

    Abbott, L. F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999)

  12. 12

    Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001)

  13. 13

    Shamir, M. & Sompolinsky, H. Nonlinear population codes. Neural Comput. 16, 1105–1136 (2004)

  14. 14

    Eisen, M. B., Spellman, P. T., Brown, P. O. & Botstein, D. Cluster analysis and display of genome-wide expression patterns. Proc. Natl Acad. Sci. USA 95, 14863–14868 (1998)

  15. 15

    Alter, O., Brown, P. O. & Botstein, D. Singular value decomposition for genome-wide expression data processing and modeling. Proc. Natl Acad. Sci. USA 97, 10101–10106 (2000)

  16. 16

    Holter, N. S., Maritan, A., Cieplak, M., Federoff, N. V. & Banavar, J. R. Dynamic modeling of gene expression data. Proc. Natl Acad. Sci. USA 98, 1693–1698 (2001)

  17. 17

    Meister, M., Pine, J. & Baylor, D. A. Multi-neuronal signals from the retina: acquisition and analysis. J. Neurosci. Methods 51, 95–106 (1994)

  18. 18

    Segev, R., Goodhouse, J., Puchalla, J. L. & Berry, M. J. II . Recoding spikes from a large fraction of the ganglion cells in a retinal patch. Nature Neurosci. 7, 1155–1162 (2004)

  19. 19

    Puchalla, J. L., Schneidman, E., Harris, R. A. & Berry, M. J. II . Redundancy in the population code of the retina. Neuron 46, 492–504 (2005)

  20. 20

    Frechette, E. S. et al. Fidelity of the ensemble code for visual motion in primate retina. J. Neurophysiol. 94, 119–135 (2005)

  21. 21

    Rieke, F., Warland, D., de Ruyter van Steveninck, R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, 1997)

  22. 22

    Martignon, L. et al. Neural coding: higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Comput. 12, 2621–2653 (2000)

  23. 23

    Grun, S., Diesmann, M. & Aertsen, A. Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Comput. 14, 43–80 (2002)

  24. 24

    Schnitzer, M. J. & Meister, M. Multineuronal firing patterns in the signal from eye to brain. Neuron 37, 499–511 (2003)

  25. 25

    Brillouin, L. Science and Information Theory (Academic, New York, 1962)

  26. 26

    Jaynes, E. T. Information theory and statistical mechanics. Phys. Rev. 106, 62–79 (1957)

  27. 27

    Schneidman, E., Still, S., Berry, M. J. II & Bialek, W. Network information and connected correlations. Phys. Rev. Lett. 91, 238701 (2003)

  28. 28

    Landau, L. D. & Lifshitz, E. M. Statistical Physics 3rd edn (Pergamon, Oxford, 1980)

  29. 29

    Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982)

  30. 30

    Amit, D. J. Modeling Brain Function: The World of Attractor Neural Networks (Cambridge Univ. Press, Cambridge, UK, 1989)

  31. 31

    Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley & Sons, New York, 1991)

  32. 32

    Eytan, D., Brenner, N. & Marom, S. Selective adaptation in networks of cortical neurons. J. Neurosci. 23, 9349–9356 (2003)

  33. 33

    Perkel, D. H., Gerstein, G. L. & Moore, G. P. Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J. 7, 419–440 (1967)

  34. 34

    Mezard, M., Parisi, G. & Virasoro, M. A. Spin Glass Theory and Beyond (World Scientific, Singapore, 1987)

  35. 35

    Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273, 1868–1871 (1996)

  36. 36

    Bi, G. & Poo, M. M. Synaptic modification by correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139–166 (2001)

  37. 37

    Barlow, H. Conditions for versatile learning, Helmholtz's unconscious inference, and the task of perception. Vision Res. 30, 1561–1571 (1990)

  38. 38

    Smirnakis, S., Berry, M. J. II, Warland, D. K., Bialek, W. & Meister, M. Adaptation of retinal processing to image contrast and spatial scale. Nature 386, 69–73 (1997)

  39. 39

    Hosoya, T., Baccus, S. A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 71–77 (2005)

  40. 40

    NIPS 2003 Workshop. Estimation of entropy and information of undersampled probability distributions. http://nips.cc/Conferences/2003/Workshops/#EstimationofEntropy (2003).

  41. 41

    Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R. & Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998)

  42. 42

    Darroch, J. N. & Ratcliff, D. Generalized iterative scaling for log–linear models. Ann. Math. Stat. 43, 1470–1480 (1972)

  43. 43

    Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory 37, 145–151 (1991)

Download references

Acknowledgements

We thank G. Stephens and G. Tkačik for discussions, N. Tkachuk for help with the experiments, S. Marom for sharing his lab's cultured cortical networks data with us, and E. J. Chichilnisky for sharing his lab's primate retina results with us. This work was supported in part by the NIH and by the E. Matilda Zeigler Foundation.

Author information

Correspondence to Elad Schneidman.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Figure 1

Effect of bin size discretization on the second order maximum entropy model. The success of the second order maximum entropy model is not sensitive to the temporal discretization bin size. (PDF 77 kb)

Supplementary Figure 2

The success of the second order maximum entropy model is not sensitive to the size of the network. The success of the second order maximum entropy model is not sensitive to the size of the sub-network modeled. (PDF 72 kb)

Supplementary Figure 3

Average interaction field vs. average local field in maximum entropy models of different size sub-networks. The average interaction field versus the average local field, in the second order maximum entropy models. For larger sub-networks the interaction field dominates the local field. (DOC 40 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Schneidman, E., Berry, M., Segev, R. et al. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006). https://doi.org/10.1038/nature04701

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.