Sensory information is encoded by populations of neurons. The responses of individual neurons are inherently noisy, so the brain must interpret this information as reliably as possible. In most situations, the optimal strategy for decoding the population signal is to compute the likelihoods of the stimuli that are consistent with an observed neural response. But it has not been clear how the brain can directly compute likelihoods. Here we present a simple and biologically plausible model that can realize the likelihood function by computing a weighted sum of sensory neuron responses. The model provides the basis for an optimal decoding of sensory information. It explains a variety of psychophysical observations on detection, discrimination and identification, and it also directly predicts the relative contributions that different sensory neurons make to perceptual judgments.
This is a preview of subscription content, access via your institution
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
*NOTE: In the supplementary information initially published online to accompany this article, equation 5 and the last two equations on page 4 contained typographical errors. The errors have been corrected online.
Pouget, A., Dayan, P. & Zemel, R.S. Inference and computation with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003).
Georgopoulos, A.P., Kalaska, J.F., Caminiti, R. & Massey, J.T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).
Deneve, S., Latham, P.E. & Pouget, A. Reading population codes: a neural implementation of ideal observers. Nat. Neurosci. 2, 740–745 (1999).
Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).
Hillis, J.M., Watt, S.J., Landy, M.S. & Banks, M.S. Slant from texture and disparity cues: optimal cue combination. J. Vis. 4, 967–992 (2004).
Weiss, Y., Simoncelli, E.P. & Adelson, E.H. Motion illusions as optimal percepts. Nat. Neurosci. 5, 598–604 (2002).
Seung, H.S. & Sompolinsky, H. Simple models for reading neuronal population codes. Proc. Natl. Acad. Sci. USA 90, 10749–10753 (1993).
Softky, W.R. & Koch, C. The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350 (1993).
Shadlen, M.N. & Newsome, W.T. The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J. Neurosci. 18, 3870–3896 (1998).
Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994).
Bair, W., Zohary, E. & Newsome, W.T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001).
Kohn, A. & Smith, M.A. Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J. Neurosci. 25, 3661–3673 (2005).
Shamir, M. & Sompolinsky, H. Nonlinear population codes. Neural Comput. 16, 1105–1136 (2004).
Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745–4765 (1992).
Maunsell, J.H. & van Essen, D.C. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J. Neurosci. 3, 2563–2586 (1983).
Salzman, C.D., Murasugi, C.M., Britten, K.H. & Newsome, W.T. Microstimulation in visual area MT: effects on direction discrimination performance. J. Neurosci. 12, 2331–2355 (1992).
Salzman, C.D. & Newsome, W.T. Neural mechanisms for forming a perceptual decision. Science 264, 231–237 (1994).
Mardia, K.V. Statistics of Directional Data (Academic Press, London, New York, 1972).
Swindale, N.V. Orientation tuning curves: empirical description and estimation of parameters. Biol. Cybern. 78, 45–56 (1998).
Britten, K.H. & Newsome, W.T. Tuning bandwidths for near-threshold stimuli in area MT. J. Neurophysiol. 80, 762–770 (1998).
Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci. 10, 1157–1169 (1993).
Adelson, E.H. & Bergen, J.R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2, 284–299 (1985).
Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, J.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).
Regan, D. & Beverley, K.I. Postadaptation orientation discrimination. J. Opt. Soc. Am. A 2, 147–155 (1985).
Hol, K. & Treue, S. Different populations of neurons contribute to the detection and discrimination of visual motion. Vision Res. 41, 685–689 (2001).
Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87–100 (1996).
Purushothaman, G. & Bradley, D.C. Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8, 99–106 (2005).
Salinas, E. & Abbott, L.F. Vector reconstruction from firing rates. J. Comput. Neurosci. 1, 89–107 (1994).
Gold, J.I. & Shadlen, M.N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5, 10–16 (2001).
Salzman, C.D., Britten, K.H. & Newsome, W.T. Cortical microstimulation influences perceptual judgments of motion direction. Nature 346, 174–177 (1990).
Groh, J.M., Born, R.T. & Newsome, W.T. How is a sensory map read out? Effects of microstimulation in visual area MT on saccades and smooth pursuit eye movements. J. Neurosci. 17, 4312–4330 (1997).
Nichols, M.J. & Newsome, W.T. Middle temporal visual area microstimulation influences veridical judgments of motion direction. J. Neurosci. 22, 9530–9540 (2002).
Sekuler, R. & Ball, K. Mental set alters visibility of moving targets. Science 32, 60–62 (1977).
Smith, P.L. & Ratcliff, R. Psychology and neurobiology of simple decisions. Trends Neurosci. 27, 161–168 (2004).
Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).
Wang, X.J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).
Koulakov, A.A., Raghavachari, S., Kepecs, A. & Lisman, J.E. Model for a robust neural integrator. Nat. Neurosci. 5, 775–782 (2002).
Movshon, J.A., Adelson, E.H., Gizzi, M.S. & Newsome, W.T. The analysis of moving visual patterns. in Experimental Brain Research Supplementum II: Pattern Recognition Mechanisms (eds. Chagas, C., Gattass, R. & Gross, C.) 117–151 (Springer, New York, 1986).
Simoncelli, E.P. & Heeger, D.J. A model of neuronal responses in visual area MT. Vision Res. 38, 743–761 (1998).
Mazurek, M.E., Roitman, J.D., Ditterich, J. & Shadlen, M.N. A role for neural integrators in perceptual decision making. Cereb. Cortex 13, 1257–1269 (2003).
Snyder, L.H., Batista, A.P. & Andersen, R.A. Coding of intention in the posterior parietal cortex. Nature 386, 167–170 (1997).
Shadlen, M.N. & Newsome, W.T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936 (2001).
This work was supported by a research grant from the US National Institutes of Health (EY2017). We thank P. Latham, B. Lau and E.P. Simoncelli for helpful advice and discussion.
The authors declare no competing financial interests.
About this article
Cite this article
Jazayeri, M., Movshon, J. Optimal representation of sensory information by neural populations. Nat Neurosci 9, 690–696 (2006). https://doi.org/10.1038/nn1691
This article is cited by
Nature Neuroscience (2023)
Principal neuron diversity in the murine lateral superior olive supports multiple sound localization strategies and segregation of information in higher processing centers
Communications Biology (2023)
Scientific Reports (2023)
Nature Communications (2023)
Nature Communications (2023)