Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher-order interactions among large groups of elements have an important role. Here we show, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons. We find that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions. These maximum entropy models are equivalent to Ising models, and predict that larger networks are completely dominated by correlation effects. This suggests that the neural code has associative or error-correcting properties, and we provide preliminary evidence for such behaviour. As a first test for the generality of these ideas, we show that similar results are obtained from networks of cultured cortical neurons.
Subscribe to Journal
Get full journal access for 1 year
only $3.83 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Hopfield, J. J. & Tank, D. W. Computing with neural circuits: a model. Science 233, 625–633 (1986)
Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)
Hartwell, L. H., Hopfield, J. J., Leibler, S. & Murray, A. W. From molecular to modular cell biology. Nature 402 (Suppl. C), 47–52 (1999)
Barabási, A.-L. & Oltvai, Z. N. Network biology: Understanding the cell's functional organization. Nature Rev. Genet. 5, 101–113 (2004)
Perkel, D. H. & Bullock, T. H. Neural coding. Neurosci. Res. Prog. Sum. 3, 221–348 (1968)
Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994)
Meister, M., Lagnado, L. & Baylor, D. A. Concerted signaling by retinal ganglion cells. Science 270, 1207–1210 (1995)
Riehle, A., Grun, S., Diesmann, M. & Aertsen, A. Spike synchronization and rate modulation differentially involved in motor cortical function. Science 278, 1950–1953 (1997)
Dan, Y., Alonso, J. M., Usrey, W. M. & Reid, R. C. Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neurosci. 1, 501–507 (1998)
Hatsopoulos, N., Ojakangas, C., Paninski, L. & Donoghue, J. Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. Natl Acad. Sci. USA 95, 15706–15711 (1998)
Abbott, L. F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999)
Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001)
Shamir, M. & Sompolinsky, H. Nonlinear population codes. Neural Comput. 16, 1105–1136 (2004)
Eisen, M. B., Spellman, P. T., Brown, P. O. & Botstein, D. Cluster analysis and display of genome-wide expression patterns. Proc. Natl Acad. Sci. USA 95, 14863–14868 (1998)
Alter, O., Brown, P. O. & Botstein, D. Singular value decomposition for genome-wide expression data processing and modeling. Proc. Natl Acad. Sci. USA 97, 10101–10106 (2000)
Holter, N. S., Maritan, A., Cieplak, M., Federoff, N. V. & Banavar, J. R. Dynamic modeling of gene expression data. Proc. Natl Acad. Sci. USA 98, 1693–1698 (2001)
Meister, M., Pine, J. & Baylor, D. A. Multi-neuronal signals from the retina: acquisition and analysis. J. Neurosci. Methods 51, 95–106 (1994)
Segev, R., Goodhouse, J., Puchalla, J. L. & Berry, M. J. II . Recoding spikes from a large fraction of the ganglion cells in a retinal patch. Nature Neurosci. 7, 1155–1162 (2004)
Puchalla, J. L., Schneidman, E., Harris, R. A. & Berry, M. J. II . Redundancy in the population code of the retina. Neuron 46, 492–504 (2005)
Frechette, E. S. et al. Fidelity of the ensemble code for visual motion in primate retina. J. Neurophysiol. 94, 119–135 (2005)
Rieke, F., Warland, D., de Ruyter van Steveninck, R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, 1997)
Martignon, L. et al. Neural coding: higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Comput. 12, 2621–2653 (2000)
Grun, S., Diesmann, M. & Aertsen, A. Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Comput. 14, 43–80 (2002)
Schnitzer, M. J. & Meister, M. Multineuronal firing patterns in the signal from eye to brain. Neuron 37, 499–511 (2003)
Brillouin, L. Science and Information Theory (Academic, New York, 1962)
Jaynes, E. T. Information theory and statistical mechanics. Phys. Rev. 106, 62–79 (1957)
Schneidman, E., Still, S., Berry, M. J. II & Bialek, W. Network information and connected correlations. Phys. Rev. Lett. 91, 238701 (2003)
Landau, L. D. & Lifshitz, E. M. Statistical Physics 3rd edn (Pergamon, Oxford, 1980)
Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982)
Amit, D. J. Modeling Brain Function: The World of Attractor Neural Networks (Cambridge Univ. Press, Cambridge, UK, 1989)
Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley & Sons, New York, 1991)
Eytan, D., Brenner, N. & Marom, S. Selective adaptation in networks of cortical neurons. J. Neurosci. 23, 9349–9356 (2003)
Perkel, D. H., Gerstein, G. L. & Moore, G. P. Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J. 7, 419–440 (1967)
Mezard, M., Parisi, G. & Virasoro, M. A. Spin Glass Theory and Beyond (World Scientific, Singapore, 1987)
Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273, 1868–1871 (1996)
Bi, G. & Poo, M. M. Synaptic modification by correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139–166 (2001)
Barlow, H. Conditions for versatile learning, Helmholtz's unconscious inference, and the task of perception. Vision Res. 30, 1561–1571 (1990)
Smirnakis, S., Berry, M. J. II, Warland, D. K., Bialek, W. & Meister, M. Adaptation of retinal processing to image contrast and spatial scale. Nature 386, 69–73 (1997)
Hosoya, T., Baccus, S. A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 71–77 (2005)
NIPS 2003 Workshop. Estimation of entropy and information of undersampled probability distributions. http://nips.cc/Conferences/2003/Workshops/#EstimationofEntropy (2003).
Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R. & Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998)
Darroch, J. N. & Ratcliff, D. Generalized iterative scaling for log–linear models. Ann. Math. Stat. 43, 1470–1480 (1972)
Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory 37, 145–151 (1991)
We thank G. Stephens and G. Tkačik for discussions, N. Tkachuk for help with the experiments, S. Marom for sharing his lab's cultured cortical networks data with us, and E. J. Chichilnisky for sharing his lab's primate retina results with us. This work was supported in part by the NIH and by the E. Matilda Zeigler Foundation.
Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.
Effect of bin size discretization on the second order maximum entropy model. The success of the second order maximum entropy model is not sensitive to the temporal discretization bin size. (PDF 77 kb)
The success of the second order maximum entropy model is not sensitive to the size of the network. The success of the second order maximum entropy model is not sensitive to the size of the sub-network modeled. (PDF 72 kb)
Average interaction field vs. average local field in maximum entropy models of different size sub-networks. The average interaction field versus the average local field, in the second order maximum entropy models. For larger sub-networks the interaction field dominates the local field. (DOC 40 kb)
About this article
Cite this article
Schneidman, E., Berry, M., Segev, R. et al. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006). https://doi.org/10.1038/nature04701
Physical Review E (2020)
Scientific Reports (2020)
European Journal of Physics (2020)
From Synaptic Interactions to Collective Dynamics in Random Neuronal Networks Models: Critical Role of Eigenvectors and Transient Behavior
Neural Computation (2020)
Information Sciences (2020)