Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Information-limiting correlations

Subjects

Abstract

Computational strategies used by the brain strongly depend on the amount of information that can be stored in population activity, which in turn strongly depends on the pattern of noise correlations. In vivo, noise correlations tend to be positive and proportional to the similarity in tuning properties. Such correlations are thought to limit information, which has led to the suggestion that decorrelation increases information. In contrast, we found, analytically and numerically, that decorrelation does not imply an increase in information. Instead, the only information-limiting correlations are what we refer to as differential correlations: correlations proportional to the product of the derivatives of the tuning curves. Unfortunately, differential correlations are likely to be very small and buried under correlations that do not limit information, making them particularly difficult to detect. We found, however, that the effect of differential correlations on information can be detected with relatively simple decoders.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: The effect of correlations on a population code with translation invariant tuning curves.
Figure 2: Decorrelation does not necessarily increase information.
Figure 3: Differential correlations induced by a shifting hill.
Figure 4: Shared connections do not necessarily induce information limiting correlations.
Figure 5: Differential correlations.
Figure 6: Small differential correlations can have a large effect on information.
Figure 7: Estimating information from neuronal populations with differential correlations (Supplementary Modeling).

References

  1. 1

    Tolhurst, D.J., Movshon, J.A. & Dean, A.D. The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 23, 775–785 (1983).

    CAS  Article  Google Scholar 

  2. 2

    Averbeck, B.B., Latham, P.E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).

    CAS  Article  Google Scholar 

  3. 3

    Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implication for psychophysical performance. Nature 370, 140–143 (1994).

    CAS  Article  Google Scholar 

  4. 4

    Maynard, E.M. et al. Neuronal interactions improve cortical population coding of movement direction. J. Neurosci. 19, 8083–8093 (1999).

    CAS  Article  Google Scholar 

  5. 5

    Averbeck, B.B. & Lee, D. Neural noise and movement-related codes in the macaque supplementary motor area. J. Neurosci. 23, 7630–7641 (2003).

    CAS  Article  Google Scholar 

  6. 6

    Gu, Y. et al. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron 71, 750–761 (2011).

    CAS  Article  Google Scholar 

  7. 7

    Adibi, M., McDonald, J.S., Clifford, C.W. & Arabzadeh, E. Adaptation improves neural coding efficiency despite increasing correlations in variability. J. Neurosci. 33, 2108–2120 (2013).

    CAS  Article  Google Scholar 

  8. 8

    Abbott, L.F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).

    CAS  Article  Google Scholar 

  9. 9

    Renart, A. et al. The asynchronous state in cortical circuits. Science 327, 587–590 (2010).

    CAS  Article  Google Scholar 

  10. 10

    Cohen, M.R. & Maunsell, J.H. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594–1600 (2009).

    CAS  Article  Google Scholar 

  11. 11

    Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).

    CAS  Article  Google Scholar 

  12. 12

    Ecker, A.S. et al. Decorrelated neuronal firing in cortical microcircuits. Science 327, 584–587 (2010).

    CAS  Article  Google Scholar 

  13. 13

    Sompolinsky, H., Yoon, H., Kang, K. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 64, 051904 (2001).

    CAS  Article  Google Scholar 

  14. 14

    Yoon, H. & Sompolinsky, H. The effect of correlations on the Fisher information of population codes. in Advances in Neural Information Processing Systems (eds. Kearns, M.S., Solla, S. & Cohn, D.A.) 167–173 (MIT Press, Cambridge, Massachusetts, 1999).

  15. 15

    Shamir, M. & Sompolinsky, H. Implications of neuronal diversity on population coding. Neural Comput. 18, 1951–1986 (2006).

    Article  Google Scholar 

  16. 16

    Ecker, A.S., Berens, P., Tolias, A.S. & Bethge, M. The effect of noise correlations in populations of diversely tuned neurons. J. Neurosci. 31, 14272–14283 (2011).

    CAS  Article  Google Scholar 

  17. 17

    Papoulis, A. Probability, Random Variables and Stochastic Process (McGraw-Hill, New York, 1991).

  18. 18

    Paradiso, M.A. A theory of the use of visual orientation information which exploits the columnar structure of striate cortex. Biol. Cybern. 58, 35–49 (1988).

    CAS  Article  Google Scholar 

  19. 19

    Graf, A.B., Kohn, A., Jazayeri, M. & Movshon, J.A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–245 (2011).

    CAS  Article  Google Scholar 

  20. 20

    Berens, P. et al. A fast and simple population code for orientation in primate V1. J. Neurosci. 32, 10618–10626 (2012).

    CAS  Article  Google Scholar 

  21. 21

    Maimon, G. & Assad, J.A. Beyond Poisson: increased spike-time regularity across primate parietal cortex. Neuron 62, 426–440 (2009).

    CAS  Article  Google Scholar 

  22. 22

    Gershon, E.D., Wiener, M.C., Latham, P.E. & Richmond, B.J. Coding strategies in monkey V1 and inferior temporal cortices. J. Neurophysiol. 79, 1135–1144 (1998).

    CAS  Article  Google Scholar 

  23. 23

    Qi, X.L. & Constantinidis, C. Variability of prefrontal neuronal discharges before and after training in a working memory task. PLoS ONE 7, e41053 (2012).

    CAS  Article  Google Scholar 

  24. 24

    Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).

    CAS  Article  Google Scholar 

  25. 25

    Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).

    Article  Google Scholar 

  26. 26

    Churchland, M.M. & Shenoy, K.V. Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. J. Neurophysiol. 97, 4235–4257 (2007).

    Article  Google Scholar 

  27. 27

    Berens, P., Ecker, A.S., Gerwinn, S., Tolias, A.S. & Bethge, M. Reassessing optimal neural population codes with neurometric functions. Proc. Natl. Acad. Sci. USA 108, 4423–4428 (2011).

    CAS  Article  Google Scholar 

  28. 28

    Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E. & Pouget, A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 74, 30–39 (2012).

    CAS  Article  Google Scholar 

  29. 29

    Shadlen, M.N. & Newsome, W.T. The variable discharge of cortical neurons: Implications for connectivity, computation and information coding. J. Neurosci. 18, 3870–3896 (1998).

    CAS  Article  Google Scholar 

  30. 30

    Romo, R., Hernandez, A., Zainos, A. & Salinas, E. Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 38, 649–657 (2003).

    CAS  Article  Google Scholar 

  31. 31

    Goris, R.L., Movshon, J.A. & Simoncelli, E.P. Partitioning neuronal variability. Nat. Neurosci. 17, 858–865 (2014).

    CAS  Article  Google Scholar 

  32. 32

    Ecker, A.S. et al. State dependence of noise correlations in macaque primary visual cortex. Neuron 82, 235–248 (2014).

    CAS  Article  Google Scholar 

  33. 33

    Kohn, A. & Smith, M.A. Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J. Neurosci. 25, 3661–3673 (2005).

    CAS  Article  Google Scholar 

  34. 34

    Wimmer, K., Nykamp, D.Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

    CAS  Article  Google Scholar 

  35. 35

    Huang, X. & Lisberger, S.G. Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. J. Neurophysiol. 101, 3012–3030 (2009).

    Article  Google Scholar 

  36. 36

    Ponce-Alvarez, A., Thiele, A., Albright, T.D., Stoner, G.R. & Deco, G. Stimulus-dependent variability and noise correlations in cortical MT neurons. Proc. Natl. Acad. Sci. USA 110, 13162–13167 (2013).

    CAS  Article  Google Scholar 

  37. 37

    Newsome, W.T., Britten, K.H. & Movshon, J.A. Neuronal correlates of a perceptual decision. Nature 341, 52–54 (1989).

    CAS  Article  Google Scholar 

  38. 38

    Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, T.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).

    CAS  Article  Google Scholar 

  39. 39

    Series, P., Latham, P. & Pouget, A. Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations. Nat. Neurosci. 7, 1129–1135 (2004).

    CAS  Article  Google Scholar 

  40. 40

    Barlow, H. Redundancy reduction revisited. Network 12, 241–253 (2001).

    CAS  Article  Google Scholar 

  41. 41

    Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear Fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).

    Article  Google Scholar 

Download references

Acknowledgements

R.M.-B. was supported by the Ramón y Cajal Spanish Award RYC-2010-05952 and by the Marie Curie FP7-PEOPLE-2010-IRG grant PIRG08-GA-2010-276795. X.P. was supported in part by US National Institutes of Health grant T32DC009974. P.L. was supported by the Gatsby Charitable Foundation. A.P. was supported by a grant from the Swiss National Science Foundation (#31003A_143707), and a grant from the Human Frontier Science Program.

Author information

Affiliations

Authors

Contributions

R.M.-B., J.B., P.L. and A.P. conceived the project. R.M.-B., J.B., I.K., X.P., P.L. and A.P. developed the theory and wrote the manuscript.

Corresponding author

Correspondence to Alexandre Pouget.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Effect of observation time.

Output information for a non-leaky integrate and fire network as a function of network size, N for several values of the observation time, T :100 ms (pink), 500 ms (purple), 2 seconds (red), and 10 seconds (green). The black solid line is the information in the input, Eq. (25).Parameters correspond to those of the network with high correlations (Table 1 with JIE = 5.4 and JII = 8.9); this is the same network we used to make Fig. 2d of the main text. The open green circles here correspond to the ones in Fig. 2d of the main text; slight differences between the open green circles in this figure and in Fig. 2d are because we used a different seed in the random number generator in the two sets of simulations; those differences are within error bars. Small observation times reduce information, but the effect diminishes as the number of neurons increases.

Supplementary Figure 2 Visualizing information limiting correlations.

All figures lie in firing rate space, here shown as two (out of N) dimensional. Red indicates the noise; in panels a, c and d, it denotes the noise covariance ellipse. We are assuming that one of the eigenvalues of the noise covariance ellipse is O(N) and the rest are O(1).

a. When the angle between f′(s) and the long (i.e, O(N)) direction of the noise covariance ellipse is O(1/N1/2), the correlations are information limiting.

b. Realistic information limiting noise tracks f(s). The thin black curve is the covariance ellipse from panel a.

c. When there are information limiting correlations, a suboptimal linear decoder, w, predicts that information saturates with N even if the angle between w and the long direction of the noise covariance ellipse is O(1).

d. When there are no information limiting correlations, so that information is O(N), the same linear decoder still predicts that information saturates with N. To observe the O(N) scaling, w would have to be nearly perpendicular to the long direction of the covariance ellipse.

Supplementary Figure 3 Empirically estimated information using ridge regularization.

The black lines on the right show the upper bound on information given by 1/ò (see Eq. (64)); the dashed blue lines are the true information. Different colors indicate different number of samples (M) used to estimate information: from top to bottom, M is 400, 800, 1600, 2400, 3200, and 4000. On the left, differential correlations are not present and information grows with the number of neurons. On the right, differential correlations are present and information saturates at the black line.

First row: The ridge parameter, λ, is fixed at 0.1.

Second row: The ridge parameter is chosen so that the effective number of degrees of freedom of the resulting covariance matrix is 90\% of the maximum number of degrees of freedom possible (see Eq. (107)); typically λ = 0.25.

Third row: The inverse of the variance of a decoder regularized using ridge regression and optimized by minimizing cross-validation error.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–3 and Supplementary Modeling (PDF 2065 kb)

Supplementary Methods Checklist (PDF 346 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moreno-Bote, R., Beck, J., Kanitscheider, I. et al. Information-limiting correlations. Nat Neurosci 17, 1410–1417 (2014). https://doi.org/10.1038/nn.3807

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing