Optimal decoding of correlated neural population responses in the primate visual cortex


Even the simplest environmental stimuli elicit responses in large populations of neurons in early sensory cortical areas. How these distributed responses are read out by subsequent processing stages to mediate behavior remains unknown. Here we used voltage-sensitive dye imaging to measure directly population responses in the primary visual cortex (V1) of monkeys performing a demanding visual detection task. We then evaluated the ability of different decoding rules to detect the target from the measured neural responses. We found that small visual targets elicit widespread responses in V1, and that response variability at distant sites is highly correlated. These correlations render most previously proposed decoding rules inefficient relative to one that uses spatially antagonistic center-surround summation. This optimal decoder consistently outperformed the monkey in the detection task, demonstrating the sensitivity of our techniques. Overall, our results suggest an unexpected role for inhibitory mechanisms in efficient decoding of neural population responses.

NOTE: In the supplementary information initially published online to accompany this article,the “ ¢ ” symbols in Supplementary Figure 6 and Supplementary Methods were incorrectly placed in the equations. The correct symbol should be “ ' ”. The error has been corrected online.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Figure 1: Behavioral task and recording chamber.
Figure 2: Neural population responses in V1 to a Gabor target (see Methods) measured with VSD imaging in one experiment.
Figure 3: Optimal two-site and multiple-site pooling.
Figure 4: Methods for measuring neural and behavioral detection sensitivity.
Figure 5: Comparing detection sensitivity of candidate pooling rules.
Figure 6: Effect of spatial binning on accuracy of the three pooling rules that rely on the response in a single site.
Figure 7: Effects of timing parameters on the accuracy of the seven pooling rules.

Change history

  • 08 November 2006

    Files replaced


  1. 1.

    In the supplementary information initially published online to accompany this article,the “ ¢ ” symbols in Supplementary Figure 6 and Supplementary Methods were incorrectly placed in the equations. The correct symbol should be “ ' ”. The error has been corrected online.


  1. 1

    Woolsey, C.N. in The Biology of Mental Health and Disease, 193–206 (Hoeber, New York, 1952).

  2. 2

    Hubel, D.H. & Weisel, T.N. Ferrier lecture. Functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1–59 (1977).

  3. 3

    Knudsen, E.I., Lac, S. & Esterly, S.D. Computational maps in the brain. Annu. Rev. Neurosci. 10, 41–65 (1987).

  4. 4

    de Valois, R.L. & de Valois, K.K. Spatial Vision (Oxford Univ. Press, New York, 1988).

  5. 5

    Georgopoulos, A.P., Kalaska, J.F., Caminiti, R. & Massey, J.T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).

  6. 6

    Mountcastle, V.B. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J. Neurophysiol. 20, 408–434 (1957).

  7. 7

    Hubel, D.H. & Weisel, T.N. Shape and arrangement of columns in cat's striate cortex. J. Physiol. (Lond.) 165, 559–568 (1963).

  8. 8

    Grinvald, A., Lieke, E.E., Frostig, R.D. & Hildesheim, R. Cortical point-spread function and long-range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual-cortex. J. Neurosci. 14, 2545–2568 (1994).

  9. 9

    McLlwain, J.T. Point images in the visual system: new interest in an old idea. Trends Neurosci. 9, 354–358 (1986).

  10. 10

    Georgopoulos, A.P., Schwartz, A.B. & Kettner, R.E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).

  11. 11

    Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, J.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).

  12. 12

    Parker, A.J. & Newsome, W.T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

  13. 13

    Purushothaman, G. & Bradley, D.C. Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8, 99–106 (2005).

  14. 14

    Johnson, K.O. Sensory discrimination – neural processes preceding discrimination decision. J. Neurophysiol. 43, 1793–1815 (1980).

  15. 15

    Paradiso, M.A. A theory for the use of visual orientation information which exploits the columnar structure of striate cortex. Biol. Cybern. 58, 35–49 (1988).

  16. 16

    Barlow, H.B. in The Cognitive Neurosciences (ed. Gazzaniga, M.S.) 401–414 (MIT Press, Cambridge, Massachusetts, 1995).

  17. 17

    Geisler, W.S. & Albrecht, D.G. Visual cortex neurons in monkeys and cats: Detection, discrimination, and identification. Vis. Neurosci. 14, 897–919 (1997).

  18. 18

    Pouget, A., Dayan, P. & Zemel, R.S. Inference and computation with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003).

  19. 19

    Sparks, D.L., Lee, C. & Rohrer, W.H. Population coding of the direction, amplitude, and velocity of saccadic eye-movements by neurons in the superior colliculus. Cold Spring Harb. Symp. Quant. Biol. 55, 805–811 (1990).

  20. 20

    Grinvald, A. & Hildesheim, R. VSDI: a new era in functional imaging of cortical dynamics. Nat. Rev. Neurosci. 5, 874–885 (2004).

  21. 21

    Seidemann, E., Arieli, A., Grinvald, A. & Slovin, H. Dynamics of depolarization and hyperpolarization in the frontal cortex and saccade goal. Science 295, 862–865 (2002).

  22. 22

    Slovin, H., Arieli, A., Hildesheim, R. & Grinvald, A. Long-term voltage-sensitive dye imaging reveals cortical dynamics in behaving monkeys. J. Neurophysiol. 88, 3421–3438 (2002).

  23. 23

    Grinvald, A., Lieke, E., Frostig, R.D., Gilbert, C.D. & Wiesel, T.N. Functional architecture of cortex revealed by optical imaging of intrinsic signals. Nature 324, 361–364 (1986).

  24. 24

    Blasdel, G.G. & Salama, G. Voltage-sensitive dyes reveal a modular organization in monkey striate cortex. Nature 321, 579–585 (1986).

  25. 25

    Bosking, W.H., Zhang, Y., Schofield, B. & Fitzpatrick, D. Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosci. 17, 2112–2127 (1997).

  26. 26

    Snippe, H.P. & Koenderink, J.J. Information in channel-coded systems – correlated receivers. Biol. Cybern. 67, 183–190 (1992).

  27. 27

    Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994).

  28. 28

    Abbott, L.F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).

  29. 29

    Sompolinsky, H., Yoon, H., Kang, K.J. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E 64, 051904–51915 (2001).

  30. 30

    Blasdel, G. & Campbell, D. Functional retinotopy of monkey visual cortex. J. Neurosci. 21, 8286–8301 (2001).

  31. 31

    Green, D.M. & Swets, J.A. Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

  32. 32

    Tolhurst, D.J., Movshon, J.A. & Dean, A.F. The statistical reliability of signals in single neurons in cat and monkey visual-cortex. Vision Res. 23, 775–785 (1983).

  33. 33

    Gawne, T.J. & Richmond, B.J. How independent are the messages carried by adjacent inferior temporal cortical-neurons. J. Neurosci. 13, 2758–2771 (1993).

  34. 34

    Lee, D., Port, N.L., Kruse, W. & Georgopoulos, A.P. Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. J. Neurosci. 18, 1161–1170 (1998).

  35. 35

    Romo, R., Hernandez, A., Zainos, A. & Salinas, E. Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 38, 649–657 (2003).

  36. 36

    Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science 273, 1868–1871 (1996).

  37. 37

    Grinvald, A. et al. in Modern Techniques in Neuroscience Research (eds. Windhorst, U. & Johansson, H.) 893–969 (Springer, New York, 1999).

  38. 38

    Duda, R.O., Hart, P.E. & Stork, D.G. Pattern Classification (Wiley, New York, 2001).

  39. 39

    Quick, R.F. A vector magnitude model of contrast detection. Kybernetik 16, 65–67 (1974).

  40. 40

    Efron, B. & Tibshirani, R.J. An Introduction to the Bootstrap (Chapman & Hall, London, 1993).

  41. 41

    Simoncelli, E.P. & Olshausen, B.A. Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001).

  42. 42

    Lund, J.S. Anatomical organization of macaque monkey striate visual cortex. Annu. Rev. Neurosci. 11, 253–288 (1988).

  43. 43

    Albright, T.D., Desimone, R. & Gross, C.G. Columnar organization of directionally selective cells in visual area mt of the macaque. J. Neurophysiol. 51, 16–31 (1984).

  44. 44

    Arieli, A., Grinvald, A. & Slovin, H. Dural substitute for long-term imaging of cortical activity in behaving monkeys and its clinical implications. J. Neurosci. Methods 114, 119–133 (2002).

  45. 45

    Shoham, D. et al. Imaging cortical dynamics at high spatial and temporal resolution with novel blue voltage-sensitive dyes. Neuron 24, 791–802 (1999).

Download references


We thank W. Bosking, D. Heeger, E. Kaplan, W. Newsome, R. Romo, M. Shadlen and H. Shouval for comments on earlier versions of this manuscript, C. Palmer, C. Michelson and Z. Yang for assistance with experiments and for discussions, and T. Cakic, C. Creeger, M. Hawthorne and M. Wu for technical support. This work was supported by the National Eye Institute, US National Institutes of Health and a Sloan Fellowship (to E.S.). W.S.G. was supported by the National Eye Institute, US National Institutes of Health.

Author information

The research was conceived by E.S.; the data were collected by E.S. and Y.C.; all authors were involved in the data analysis, modeling and writing.

Note: Supplementary information is available on the Nature Neuroscience website.

Correspondence to Eyal Seidemann.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Effect of contrast on response spread. (PDF 200 kb)

Supplementary Fig. 2

Effect of eye position on response spread. (PDF 218 kb)

Supplementary Fig. 3

Variability in V1 population responses, as measured by VSD imaging, can be described as an additive (stimulus independent) Gaussian noise with widespread spatial correlations. (PDF 252 kb)

Supplementary Fig. 4

Effect of pool size on correlations in population responses. (PDF 281 kb)

Supplementary Fig. 5

Comparison between spatial correlations measured from V1 and spatial correlations measured during control experiment with light emitting diode (LED) (see text). (PDF 239 kb)

Supplementary Fig. 6

Optimal two-point pooling in the example experiment. (PDF 175 kb)

Supplementary Methods (PDF 111 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chen, Y., Geisler, W. & Seidemann, E. Optimal decoding of correlated neural population responses in the primate visual cortex. Nat Neurosci 9, 1412–1420 (2006). https://doi.org/10.1038/nn1792

Download citation

Further reading