Computational strategies used by the brain strongly depend on the amount of information that can be stored in population activity, which in turn strongly depends on the pattern of noise correlations. In vivo, noise correlations tend to be positive and proportional to the similarity in tuning properties. Such correlations are thought to limit information, which has led to the suggestion that decorrelation increases information. In contrast, we found, analytically and numerically, that decorrelation does not imply an increase in information. Instead, the only information-limiting correlations are what we refer to as differential correlations: correlations proportional to the product of the derivatives of the tuning curves. Unfortunately, differential correlations are likely to be very small and buried under correlations that do not limit information, making them particularly difficult to detect. We found, however, that the effect of differential correlations on information can be detected with relatively simple decoders.
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Tolhurst, D.J., Movshon, J.A. & Dean, A.D. The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 23, 775–785 (1983).
Averbeck, B.B., Latham, P.E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).
Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implication for psychophysical performance. Nature 370, 140–143 (1994).
Maynard, E.M. et al. Neuronal interactions improve cortical population coding of movement direction. J. Neurosci. 19, 8083–8093 (1999).
Averbeck, B.B. & Lee, D. Neural noise and movement-related codes in the macaque supplementary motor area. J. Neurosci. 23, 7630–7641 (2003).
Gu, Y. et al. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron 71, 750–761 (2011).
Adibi, M., McDonald, J.S., Clifford, C.W. & Arabzadeh, E. Adaptation improves neural coding efficiency despite increasing correlations in variability. J. Neurosci. 33, 2108–2120 (2013).
Abbott, L.F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).
Renart, A. et al. The asynchronous state in cortical circuits. Science 327, 587–590 (2010).
Cohen, M.R. & Maunsell, J.H. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594–1600 (2009).
Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).
Ecker, A.S. et al. Decorrelated neuronal firing in cortical microcircuits. Science 327, 584–587 (2010).
Sompolinsky, H., Yoon, H., Kang, K. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 64, 051904 (2001).
Yoon, H. & Sompolinsky, H. The effect of correlations on the Fisher information of population codes. in Advances in Neural Information Processing Systems (eds. Kearns, M.S., Solla, S. & Cohn, D.A.) 167–173 (MIT Press, Cambridge, Massachusetts, 1999).
Shamir, M. & Sompolinsky, H. Implications of neuronal diversity on population coding. Neural Comput. 18, 1951–1986 (2006).
Ecker, A.S., Berens, P., Tolias, A.S. & Bethge, M. The effect of noise correlations in populations of diversely tuned neurons. J. Neurosci. 31, 14272–14283 (2011).
Papoulis, A. Probability, Random Variables and Stochastic Process (McGraw-Hill, New York, 1991).
Paradiso, M.A. A theory of the use of visual orientation information which exploits the columnar structure of striate cortex. Biol. Cybern. 58, 35–49 (1988).
Graf, A.B., Kohn, A., Jazayeri, M. & Movshon, J.A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–245 (2011).
Berens, P. et al. A fast and simple population code for orientation in primate V1. J. Neurosci. 32, 10618–10626 (2012).
Maimon, G. & Assad, J.A. Beyond Poisson: increased spike-time regularity across primate parietal cortex. Neuron 62, 426–440 (2009).
Gershon, E.D., Wiener, M.C., Latham, P.E. & Richmond, B.J. Coding strategies in monkey V1 and inferior temporal cortices. J. Neurophysiol. 79, 1135–1144 (1998).
Qi, X.L. & Constantinidis, C. Variability of prefrontal neuronal discharges before and after training in a working memory task. PLoS ONE 7, e41053 (2012).
Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).
Churchland, M.M. & Shenoy, K.V. Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. J. Neurophysiol. 97, 4235–4257 (2007).
Berens, P., Ecker, A.S., Gerwinn, S., Tolias, A.S. & Bethge, M. Reassessing optimal neural population codes with neurometric functions. Proc. Natl. Acad. Sci. USA 108, 4423–4428 (2011).
Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E. & Pouget, A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 74, 30–39 (2012).
Shadlen, M.N. & Newsome, W.T. The variable discharge of cortical neurons: Implications for connectivity, computation and information coding. J. Neurosci. 18, 3870–3896 (1998).
Romo, R., Hernandez, A., Zainos, A. & Salinas, E. Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 38, 649–657 (2003).
Goris, R.L., Movshon, J.A. & Simoncelli, E.P. Partitioning neuronal variability. Nat. Neurosci. 17, 858–865 (2014).
Ecker, A.S. et al. State dependence of noise correlations in macaque primary visual cortex. Neuron 82, 235–248 (2014).
Kohn, A. & Smith, M.A. Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J. Neurosci. 25, 3661–3673 (2005).
Wimmer, K., Nykamp, D.Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).
Huang, X. & Lisberger, S.G. Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. J. Neurophysiol. 101, 3012–3030 (2009).
Ponce-Alvarez, A., Thiele, A., Albright, T.D., Stoner, G.R. & Deco, G. Stimulus-dependent variability and noise correlations in cortical MT neurons. Proc. Natl. Acad. Sci. USA 110, 13162–13167 (2013).
Newsome, W.T., Britten, K.H. & Movshon, J.A. Neuronal correlates of a perceptual decision. Nature 341, 52–54 (1989).
Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, T.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).
Series, P., Latham, P. & Pouget, A. Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations. Nat. Neurosci. 7, 1129–1135 (2004).
Barlow, H. Redundancy reduction revisited. Network 12, 241–253 (2001).
Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear Fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).
R.M.-B. was supported by the Ramón y Cajal Spanish Award RYC-2010-05952 and by the Marie Curie FP7-PEOPLE-2010-IRG grant PIRG08-GA-2010-276795. X.P. was supported in part by US National Institutes of Health grant T32DC009974. P.L. was supported by the Gatsby Charitable Foundation. A.P. was supported by a grant from the Swiss National Science Foundation (#31003A_143707), and a grant from the Human Frontier Science Program.
The authors declare no competing financial interests.
Integrated supplementary information
Output information for a non-leaky integrate and fire network as a function of network size, N for several values of the observation time, T :100 ms (pink), 500 ms (purple), 2 seconds (red), and 10 seconds (green). The black solid line is the information in the input, Eq. (25).Parameters correspond to those of the network with high correlations (Table 1 with JIE = 5.4 and JII = 8.9); this is the same network we used to make Fig. 2d of the main text. The open green circles here correspond to the ones in Fig. 2d of the main text; slight differences between the open green circles in this figure and in Fig. 2d are because we used a different seed in the random number generator in the two sets of simulations; those differences are within error bars. Small observation times reduce information, but the effect diminishes as the number of neurons increases.
All figures lie in firing rate space, here shown as two (out of N) dimensional. Red indicates the noise; in panels a, c and d, it denotes the noise covariance ellipse. We are assuming that one of the eigenvalues of the noise covariance ellipse is O(N) and the rest are O(1).
a. When the angle between f′(s) and the long (i.e, O(N)) direction of the noise covariance ellipse is O(1/N1/2), the correlations are information limiting.
b. Realistic information limiting noise tracks f(s). The thin black curve is the covariance ellipse from panel a.
c. When there are information limiting correlations, a suboptimal linear decoder, w, predicts that information saturates with N even if the angle between w and the long direction of the noise covariance ellipse is O(1).
d. When there are no information limiting correlations, so that information is O(N), the same linear decoder still predicts that information saturates with N. To observe the O(N) scaling, w would have to be nearly perpendicular to the long direction of the covariance ellipse.
The black lines on the right show the upper bound on information given by 1/ò (see Eq. (64)); the dashed blue lines are the true information. Different colors indicate different number of samples (M) used to estimate information: from top to bottom, M is 400, 800, 1600, 2400, 3200, and 4000. On the left, differential correlations are not present and information grows with the number of neurons. On the right, differential correlations are present and information saturates at the black line.
First row: The ridge parameter, λ, is fixed at 0.1.
Second row: The ridge parameter is chosen so that the effective number of degrees of freedom of the resulting covariance matrix is 90\% of the maximum number of degrees of freedom possible (see Eq. (107)); typically λ = 0.25.
Third row: The inverse of the variance of a decoder regularized using ridge regression and optimized by minimizing cross-validation error.
About this article
Cite this article
Moreno-Bote, R., Beck, J., Kanitscheider, I. et al. Information-limiting correlations. Nat Neurosci 17, 1410–1417 (2014). https://doi.org/10.1038/nn.3807
The Journal of Mathematical Neuroscience (2021)
Nature Neuroscience (2021)
Nature Neuroscience (2021)
Nature Reviews Neuroscience (2021)
Scaling of sensory information in large neural populations shows signatures of information-limiting correlations
Nature Communications (2021)