Article | Published:

Dopamine neurons share common response function for reward prediction error

Nature Neuroscience volume 19, pages 479486 (2016) | Download Citation

Subjects

Abstract

Dopamine neurons are thought to signal reward prediction error, or the difference between actual and predicted reward. How dopamine neurons jointly encode this information, however, remains unclear. One possibility is that different neurons specialize in different aspects of prediction error; another is that each neuron calculates prediction error in the same way. We recorded from optogenetically identified dopamine neurons in the lateral ventral tegmental area (VTA) while mice performed classical conditioning tasks. Our tasks allowed us to determine the full prediction error functions of dopamine neurons and compare them to each other. We found marked homogeneity among individual dopamine neurons: their responses to both unexpected and expected rewards followed the same function, just scaled up or down. As a result, we were able to describe both individual and population responses using just two parameters. Such uniformity ensures robust information coding, allowing each dopamine neuron to contribute fully to the prediction error signal.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).

  2. 2.

    & The mysterious motivational functions of mesolimbic dopamine. Neuron 76, 470–485 (2012).

  3. 3.

    , & Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815–834 (2010).

  4. 4.

    et al. Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. J. Neurosci. 29, 444–453 (2009).

  5. 5.

    et al. The discovery of central monoamine neurons gave volume transmission to the wired brain. Prog. Neurobiol. 90, 82–100 (2010).

  6. 6.

    & Intracellular and extracellular electrophysiology of nigral dopaminergic neurons. 1. Identification and characterization. Neuroscience 10, 301–315 (1983).

  7. 7.

    , & Electrical synapses between dopaminergic neurons of the substantia nigra pars compacta. J. Neurosci. 25, 291–298 (2005).

  8. 8.

    et al. Synchronization of midbrain dopaminergic neurons is enhanced by rewarding events. Neuron 62, 695–704 (2009).

  9. 9.

    , , , & Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133–143 (2004).

  10. 10.

    , & Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning. PLoS One 7, e29766 (2012).

  11. 11.

    Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).

  12. 12.

    Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238 (2013).

  13. 13.

    Dissecting the diversity of midbrain dopamine neurons. Trends Neurosci. 36, 336–342 (2013).

  14. 14.

    et al. Temporal-spatial changes in Sonic Hedgehog expression and signaling reveal different potentials of ventral mesencephalic progenitors to populate distinct ventral midbrain nuclei. Neural Dev. 6, 29 (2011).

  15. 15.

    , , & The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J. Physiol. (Lond.) 577, 907–924 (2006).

  16. 16.

    , , & I(h) channels contribute to the different functional properties of identified dopaminergic subpopulations in the midbrain. J. Neurosci. 22, 1290–1302 (2002).

  17. 17.

    et al. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491, 212–217 (2012).

  18. 18.

    , , , & Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858–873 (2012).

  19. 19.

    et al. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57, 760–773 (2008).

  20. 20.

    , & Diversity and homogeneity in responses of midbrain dopamine neurons. J. Neurosci. 33, 4693–4709 (2013).

  21. 21.

    & Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

  22. 22.

    & Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

  23. 23.

    , , , & Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88 (2012).

  24. 24.

    , , & Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

  25. 25.

    , , & BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319, 1264–1267 (2008).

  26. 26.

    , & Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).

  27. 27.

    , & A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

  28. 28.

    et al. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).

  29. 29.

    , & Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).

  30. 30.

    , & Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).

  31. 31.

    & Measuring and interpreting neuronal correlations. Nat. Neurosci. 14, 811–819 (2011).

  32. 32.

    Twenty-five lessons from computational neuromodulation. Neuron 76, 240–256 (2012).

  33. 33.

    & The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).

  34. 34.

    , & Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13, 1189–1215 (1984).

  35. 35.

    An electron microscopic morphometric comparison of tyrosine hydroxylase immunoreactive innervation in the neostriatum and the nucleus accumbens core and shell. Brain Res. 575, 341–346 (1992).

  36. 36.

    Homeostatic signaling: the positive side of negative feedback. Curr. Opin. Neurobiol. 17, 318–324 (2007).

  37. 37.

    Homeostatic control of neural activity: from phenomenology to molecular design. Annu. Rev. Neurosci. 29, 307–323 (2006).

  38. 38.

    , & Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J. Neurosci. 33, 4710–4725 (2013).

  39. 39.

    Towards a general theory of neural computation based on prediction by single neurons. PLoS One 3, e3298 (2008).

  40. 40.

    et al. One-dimensional dynamics of attention and decision making in LIP. Neuron 58, 15–25 (2008).

  41. 41.

    et al. Biased associative representations in parietal cortex. Neuron 77, 180–191 (2013).

  42. 42.

    & Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79, 1011–1024 (2013).

  43. 43.

    et al. Activation of dopamine neurons is critical for aversive conditioning and prevention of generalized anxiety. Nat. Neurosci. 14, 620–626 (2011).

  44. 44.

    , , & Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 30, 220–227 (2007).

  45. 45.

    et al. Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. eLife 4, e10032 (2015).

  46. 46.

    et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635–647 (2015).

  47. 47.

    et al. Circuit architecture of VTA dopamine neurons revealed by systematic input-output mapping. Cell 162, 622–634 (2015).

  48. 48.

    , , , & Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum. Mol. Pharmacol. 76, 396–404 (2009).

  49. 49.

    , & Dopaminergic neurons inhibit striatal output through non-canonical release of GABA. Nature 490, 262–266 (2012).

  50. 50.

    , , , & Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate. J. Neurosci. 30, 8229–8233 (2010).

  51. 51.

    et al. Characterization of a mouse strain expressing Cre recombinase from the 3′ untranslated region of the dopamine transporter locus. Genesis 44, 383–390 (2006).

  52. 52.

    , , , & Millisecond-timescale, genetically targeted optical control of neural activity. Nat. Neurosci. 8, 1263–1268 (2005).

  53. 53.

    , , & A FLEX switch targets Channelrhodopsin-2 to multiple cell types for imaging and long-range circuit mapping. J. Neurosci. 28, 7025–7030 (2008).

  54. 54.

    & Speed and accuracy of olfactory discrimination in the rat. Nat. Neurosci. 6, 1224–1229 (2003).

  55. 55.

    & Habenula lesions reveal that multiple mechanisms underlie dopamine prediction errors. Neuron 87, 1304–1316 (2015).

  56. 56.

    & Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J. Neurophysiol. 91, 2259–2272 (2004).

  57. 57.

    , , & PINP: a new method of tagging neuronal populations for identification during in vivo electrophysiological recording. PLoS One 4, e6099 (2009).

  58. 58.

    et al. Distinct behavioural and network correlates of two interneuron types in prefrontal cortex. Nature 498, 363–366 (2013).

  59. 59.

    , & Divisive normalization in olfactory population codes. Neuron 66, 287–299 (2010).

Download references

Acknowledgements

We thank J. Fitzgerald for assistance with analysis, J. Assad, R. Born, J. Maunsell, R. Wilson and members of the Uchida laboratory for discussions, C. Dulac for sharing resources, and K. Deisseroth (Stanford University) for the AAV-FLEX-ChR2 construct. This work was supported by a Sackler Fellowship in Psychobiology (N.E.) and US National Institutes of Health grants T32GM007753 (N.E.), F30MH100729 (N.E.), 2T32MH020017-16 (M.B.), 5T32MH020017-17 (M.B.), R01MH095953 (N.U.) and R01MH101207 (N.U.).

Author information

Affiliations

  1. Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, Cambridge, Massachusetts, USA.

    • Neir Eshel
    • , Ju Tian
    • , Michael Bukwich
    •  & Naoshige Uchida
  2. MD-PhD Program, Harvard Medical School, Boston, Massachusetts, USA.

    • Neir Eshel

Authors

  1. Search for Neir Eshel in:

  2. Search for Ju Tian in:

  3. Search for Michael Bukwich in:

  4. Search for Naoshige Uchida in:

Contributions

N.E., J.T. and N.U. designed the experiments. N.E. and M.B. collected data for the variable-reward task. J.T. collected data for the variable-expectation task. N.E. analyzed data and wrote the manuscript, with comments from J.T. and N.U.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Naoshige Uchida.

Integrated supplementary information

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–6 and Supplementary Note

  2. 2.

    Supplementary Methods Checklist

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.4239

Further reading