Dopamine restores reward prediction errors in old age

  • A Corrigendum to this article was published on 21 November 2014

Abstract

Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Two-armed bandit task design and performance in young and older adults.
Figure 2: Reinforcement learning model and behavior.
Figure 3: Reward prediction in the nucleus accumbens in 32 older adults.
Figure 4: Nigro-striatal tract connectivity strength and functional prediction errors.

Change history

  • 22 May 2013

    In the version of this article initially published, an affiliation for author Quentin Huys read Translational Neuroimaging Unit. The correct name is Translational Neuromodeling Unit. The error has been corrected in the HTML and PDF versions of the article.

References

  1. 1

    Eppinger, B., Hämmerer, D. & Li, S.-C. Neuromodulation of reward-based learning and decision making in human aging. Ann. NY Acad. Sci. 1235, 1–17 (2011).

    PubMed  Article  PubMed Central  Google Scholar 

  2. 2

    Mell, T. et al. Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563 (2005).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  3. 3

    Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  4. 4

    Salamone, J.D., Correa, M., Mingote, S.M. & Weber, S.M. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol. 5, 34–41 (2005).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  5. 5

    Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  6. 6

    O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  7. 7

    O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  8. 8

    Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  9. 9

    Knutson, B. & Gibbs, S. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl.) 191, 813–822 (2007).

    CAS  Article  Google Scholar 

  10. 10

    Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045 (2006).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  11. 11

    Rutledge, R.B. et al. Dopaminergic drugs modulate learning rates and perseveration in parkinson's patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114 (2009).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  12. 12

    Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C. & Farde, L. The correlative triad among aging, dopamine and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807 (2006).

    Article  CAS  Google Scholar 

  13. 13

    Düzel, E., Bunzeck, N., Guitart-Masip, M. & Duzel, S. Novelty-related motivation of anticipation and exploration by dopamine (NOMAD): implications for healthy aging. Neurosci. Biobehav. Rev. 34, 660–669 (2010).

    PubMed  Article  CAS  PubMed Central  Google Scholar 

  14. 14

    Fearnley, J.M. & Lees, A.J. Ageing and Parkinson's disease: substantia nigra regional selectivity. Brain 114, 2283–2301 (1991).

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15

    Vaillancourt, D.E., Spraker, M.B., Prodoehl, J., Zhou, X.J. & Little, D.M. Effects of aging on the ventral and dorsal substantia nigra using diffusion tensor imaging. Neurobiol. Aging 33, 35–42 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  16. 16

    Samanez-Larkin, G.R., Kuhnen, C.M., Yoo, D.J. & Knutson, B. Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 30, 1426–1434 (2010).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  17. 17

    Schott, B.H. et al. Ageing and early-stage Parkinson's disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130, 2412–2424 (2007).

    Article  Google Scholar 

  18. 18

    Cox, K.M., Aizenstein, H.J. & Fiez, J.A. Striatal outcome processing in healthy aging. Cogn. Affect. Behav. Neurosci. 8, 304–317 (2008).

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19

    Behrens, T.E.J., Hunt, L.T., Woolrich, M.W. & Rushworth, M.F.S. Associative learning of social value. Nature 456, 245–249 (2008).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  20. 20

    Cools, R. & D'Esposito, M. Inverted U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 69, e113–e125 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  21. 21

    Li, J. & Daw, N.D. Signals in human striatum are appropriate for policy update rather than value prediction. J. Neurosci. 31, 5504–5511 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  22. 22

    Eppinger, B., Kray, J., Mock, B. & Mecklinger, A. Better or worse than expected? Aging, learning and the ERN. Neuropsychologia 46, 521–539 (2008).

    PubMed  Article  PubMed Central  Google Scholar 

  23. 23

    Samanez-Larkin, G.R., Wagner, A.D. & Knutson, B. Expected value information improves financial risk taking across the adult life span. Soc. Cogn. Affect. Neurosci. 6, 207–217 (2011).

    PubMed  Article  PubMed Central  Google Scholar 

  24. 24

    Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154–166 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  25. 25

    Dickinson, A. & Balleine, B. The Role of Learning in the Operation of Motivational Systems (John Wiley & Sons, New York, 2002).

  26. 26

    Dickinson, A., Smith, J. & Mirenowicz, J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483 (2000).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  27. 27

    Worthy, D.A., Gorlick, M.A., Pacheco, J.L., Schnyer, D.M. & Maddox, W.T. With age comes wisdom. Psychol. Sci. 22, 1375–1380 (2011).

    PubMed  PubMed Central  Article  Google Scholar 

  28. 28

    de Wit, S. et al. Reliance on habits at the expense of goal-directed control following dopamine precursor depletion. Psychopharmacology (Berl.) 219, 621–631 (2012).

    CAS  Article  Google Scholar 

  29. 29

    Wunderlich, K., Smittenaar, P. & Dolan, R.J. Dopamine enhances model-based over model-free choice behavior. Neuron 75, 418–424 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  30. 30

    Gläscher, J., Daw, N., Dayan, P. & O'Doherty, J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).

    PubMed  PubMed Central  Google Scholar 

  31. 31

    Jocham, G., Klein, T.A. & Ullsperger, M. Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J. Neurosci. 31, 1606–1613 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  32. 32

    Shohamy, D. & Adcock, R.A. Dopamine and adaptive memory. Trends Cogn. Sci. 14, 464–472 (2010).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  33. 33

    Nicola, S.M., Surmeier, D.J. & Malenka, R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185–215 (2000).

    CAS  Article  Google Scholar 

  34. 34

    Daw, N.D. & Gershman, S.J. Seymour, B., Dayan, P. & Dolan, R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  35. 35

    Simon, D.A. & Daw, N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526–5539 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  36. 36

    Guitart-Masip, M. et al. Action controls dopaminergic enhancement of reward representations. Proc. Natl. Acad. Sci. USA 109, 7511–7516 (2012).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  37. 37

    Choi, W.Y., Balsam, P.D. & Horvitz, J.C. Extended habit training reduces dopamine mediation of appetitive response expression. J. Neurosci. 25, 6729–6733 (2005).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  38. 38

    Klein-Flügge, M.C., Hunt, L.T., Bach, D.R., Dolan, R.J. & Behrens, T.E.J. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 72, 654–664 (2011).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  39. 39

    Coxon, J.P., Van Impe, A., Wenderoth, N. & Swinnen, S.P. Aging and inhibitory control of action: cortico-subthalamic connection strength predicts stopping performance. J. Neurosci. 32, 8401–8412 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  40. 40

    Forstmann, B.U. et al. Cortico-subthalamic white matter tract strength predicts interindividual efficacy in stopping a motor response. Neuroimage 60, 370–375 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  41. 41

    Le Bihan, D. & Johansen-Berg, H. Diffusion MRI at 25: exploring brain tissue structure and function. Neuroimage 61, 324–341 (2012).

    PubMed  Article  PubMed Central  Google Scholar 

  42. 42

    Draganski, B. et al. Regional specificity of MRI contrast parameter changes in normal ageing revealed by voxel-based quantification (VBQ). Neuroimage 55, 1423–1434 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  43. 43

    Li, S.-C. & Sikström, S. Integrative neurocomputational perspectives on cognitive aging, neuromodulation, and representation. Neurosci. Biobehav. Rev. 26, 795–808 (2002).

    PubMed  Article  PubMed Central  Google Scholar 

  44. 44

    Schönberg, T., Daw, N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–12867 (2007).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  45. 45

    Huys, Q.J.M. et al. Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  46. 46

    Lutti, A., Hutton, C., Finsterbusch, J., Helms, G. & Weiskopf, N. Optimization and validation of methods for mapping of the radiofrequency transmit field at 3T. Magn. Reson. Med. 64, 229–238 (2010).

    PubMed  PubMed Central  Article  Google Scholar 

  47. 47

    Reese, T.G., Heid, O., Weisskoff, R.M. & Wedeen, V.J. Reduction of eddy-current-induced distortion in diffusion MRI using a twice-refocused spin echo. Magn. Reson. Med. 49, 177–182 (2003).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  48. 48

    Nagy, Z., Weiskopf, N., Alexander, D.C. & Deichmann, R. A method for improving the performance of gradient systems for diffusion-weighted MRI. Magn. Reson. Med. 58, 763–768 (2007).

    PubMed  PubMed Central  Article  Google Scholar 

  49. 49

    Andersson, J.L.R., Skare, S. & Ashburner, J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20, 870–888 (2003).

    PubMed  Article  PubMed Central  Google Scholar 

  50. 50

    Ashburner, J. A fast diffeomorphic image registration algorithm. Neuroimage 38, 95–113 (2007).

    PubMed  PubMed Central  Article  Google Scholar 

  51. 51

    Knutson, B. & Cooper, J.C. Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18, 411–417 (2005).

    PubMed  Article  PubMed Central  Google Scholar 

  52. 52

    Brett, M., Anton, J.-L., Valabregue, R. & Poline, J.-B. Region of interest analysis using an SPM toolbox. Neuroimage 16, 1140–1141 (2002).

    Google Scholar 

  53. 53

    Forstmann, B.U. et al. The speed-accuracy tradeoff in the elderly brain: a structural model-based approach. J. Neurosci. 31, 17242–17249 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  54. 54

    Schwarzkopf, D.S., de Haas, B. & Rees, G. Better ways to improve standards in brain-behavior correlation analysis. Front. Hum. Neurosci. 6, 200 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

Download references

Acknowledgements

We thank J. Medhora and L. Sasse for their assistance with data collection, and H. Barron and M. Klein-Flügge for their assistance with time course analyses. R.C. is supported by a Wellcome Trust Research Training Fellowship (WT088286MA). R.J.D. is supported by the Wellcome Trust (grant number 078865/Z/05/Z). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z).

Author information

Affiliations

Authors

Contributions

R.C. and M.G.-M. conducted the experiment, analyzed the data and prepared the manuscript. C.L., P.D., Q.H. and E.D. contributed to data analysis and manuscript preparation. R.J.D. contributed to data analysis and manuscript preparation and supervised the project.

Corresponding author

Correspondence to Rumana Chowdhury.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 and Supplementary Tables 1–6 (PDF 4051 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chowdhury, R., Guitart-Masip, M., Lambert, C. et al. Dopamine restores reward prediction errors in old age. Nat Neurosci 16, 648–653 (2013). https://doi.org/10.1038/nn.3364

Download citation

Further reading