Abstract
Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Change history
22 May 2013
In the version of this article initially published, an affiliation for author Quentin Huys read Translational Neuroimaging Unit. The correct name is Translational Neuromodeling Unit. The error has been corrected in the HTML and PDF versions of the article.
References
Eppinger, B., Hämmerer, D. & Li, S.-C. Neuromodulation of reward-based learning and decision making in human aging. Ann. NY Acad. Sci. 1235, 1–17 (2011).
Mell, T. et al. Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563 (2005).
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Salamone, J.D., Correa, M., Mingote, S.M. & Weber, S.M. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol. 5, 34–41 (2005).
Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).
O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).
O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).
Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
Knutson, B. & Gibbs, S. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl.) 191, 813–822 (2007).
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045 (2006).
Rutledge, R.B. et al. Dopaminergic drugs modulate learning rates and perseveration in parkinson's patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114 (2009).
Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C. & Farde, L. The correlative triad among aging, dopamine and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807 (2006).
Düzel, E., Bunzeck, N., Guitart-Masip, M. & Duzel, S. Novelty-related motivation of anticipation and exploration by dopamine (NOMAD): implications for healthy aging. Neurosci. Biobehav. Rev. 34, 660–669 (2010).
Fearnley, J.M. & Lees, A.J. Ageing and Parkinson's disease: substantia nigra regional selectivity. Brain 114, 2283–2301 (1991).
Vaillancourt, D.E., Spraker, M.B., Prodoehl, J., Zhou, X.J. & Little, D.M. Effects of aging on the ventral and dorsal substantia nigra using diffusion tensor imaging. Neurobiol. Aging 33, 35–42 (2012).
Samanez-Larkin, G.R., Kuhnen, C.M., Yoo, D.J. & Knutson, B. Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 30, 1426–1434 (2010).
Schott, B.H. et al. Ageing and early-stage Parkinson's disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130, 2412–2424 (2007).
Cox, K.M., Aizenstein, H.J. & Fiez, J.A. Striatal outcome processing in healthy aging. Cogn. Affect. Behav. Neurosci. 8, 304–317 (2008).
Behrens, T.E.J., Hunt, L.T., Woolrich, M.W. & Rushworth, M.F.S. Associative learning of social value. Nature 456, 245–249 (2008).
Cools, R. & D'Esposito, M. Inverted U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 69, e113–e125 (2011).
Li, J. & Daw, N.D. Signals in human striatum are appropriate for policy update rather than value prediction. J. Neurosci. 31, 5504–5511 (2011).
Eppinger, B., Kray, J., Mock, B. & Mecklinger, A. Better or worse than expected? Aging, learning and the ERN. Neuropsychologia 46, 521–539 (2008).
Samanez-Larkin, G.R., Wagner, A.D. & Knutson, B. Expected value information improves financial risk taking across the adult life span. Soc. Cogn. Affect. Neurosci. 6, 207–217 (2011).
Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154–166 (2012).
Dickinson, A. & Balleine, B. The Role of Learning in the Operation of Motivational Systems (John Wiley & Sons, New York, 2002).
Dickinson, A., Smith, J. & Mirenowicz, J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483 (2000).
Worthy, D.A., Gorlick, M.A., Pacheco, J.L., Schnyer, D.M. & Maddox, W.T. With age comes wisdom. Psychol. Sci. 22, 1375–1380 (2011).
de Wit, S. et al. Reliance on habits at the expense of goal-directed control following dopamine precursor depletion. Psychopharmacology (Berl.) 219, 621–631 (2012).
Wunderlich, K., Smittenaar, P. & Dolan, R.J. Dopamine enhances model-based over model-free choice behavior. Neuron 75, 418–424 (2012).
Gläscher, J., Daw, N., Dayan, P. & O'Doherty, J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).
Jocham, G., Klein, T.A. & Ullsperger, M. Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J. Neurosci. 31, 1606–1613 (2011).
Shohamy, D. & Adcock, R.A. Dopamine and adaptive memory. Trends Cogn. Sci. 14, 464–472 (2010).
Nicola, S.M., Surmeier, D.J. & Malenka, R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185–215 (2000).
Daw, N.D. & Gershman, S.J. Seymour, B., Dayan, P. & Dolan, R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).
Simon, D.A. & Daw, N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526–5539 (2011).
Guitart-Masip, M. et al. Action controls dopaminergic enhancement of reward representations. Proc. Natl. Acad. Sci. USA 109, 7511–7516 (2012).
Choi, W.Y., Balsam, P.D. & Horvitz, J.C. Extended habit training reduces dopamine mediation of appetitive response expression. J. Neurosci. 25, 6729–6733 (2005).
Klein-Flügge, M.C., Hunt, L.T., Bach, D.R., Dolan, R.J. & Behrens, T.E.J. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 72, 654–664 (2011).
Coxon, J.P., Van Impe, A., Wenderoth, N. & Swinnen, S.P. Aging and inhibitory control of action: cortico-subthalamic connection strength predicts stopping performance. J. Neurosci. 32, 8401–8412 (2012).
Forstmann, B.U. et al. Cortico-subthalamic white matter tract strength predicts interindividual efficacy in stopping a motor response. Neuroimage 60, 370–375 (2012).
Le Bihan, D. & Johansen-Berg, H. Diffusion MRI at 25: exploring brain tissue structure and function. Neuroimage 61, 324–341 (2012).
Draganski, B. et al. Regional specificity of MRI contrast parameter changes in normal ageing revealed by voxel-based quantification (VBQ). Neuroimage 55, 1423–1434 (2011).
Li, S.-C. & Sikström, S. Integrative neurocomputational perspectives on cognitive aging, neuromodulation, and representation. Neurosci. Biobehav. Rev. 26, 795–808 (2002).
Schönberg, T., Daw, N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–12867 (2007).
Huys, Q.J.M. et al. Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).
Lutti, A., Hutton, C., Finsterbusch, J., Helms, G. & Weiskopf, N. Optimization and validation of methods for mapping of the radiofrequency transmit field at 3T. Magn. Reson. Med. 64, 229–238 (2010).
Reese, T.G., Heid, O., Weisskoff, R.M. & Wedeen, V.J. Reduction of eddy-current-induced distortion in diffusion MRI using a twice-refocused spin echo. Magn. Reson. Med. 49, 177–182 (2003).
Nagy, Z., Weiskopf, N., Alexander, D.C. & Deichmann, R. A method for improving the performance of gradient systems for diffusion-weighted MRI. Magn. Reson. Med. 58, 763–768 (2007).
Andersson, J.L.R., Skare, S. & Ashburner, J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20, 870–888 (2003).
Ashburner, J. A fast diffeomorphic image registration algorithm. Neuroimage 38, 95–113 (2007).
Knutson, B. & Cooper, J.C. Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18, 411–417 (2005).
Brett, M., Anton, J.-L., Valabregue, R. & Poline, J.-B. Region of interest analysis using an SPM toolbox. Neuroimage 16, 1140–1141 (2002).
Forstmann, B.U. et al. The speed-accuracy tradeoff in the elderly brain: a structural model-based approach. J. Neurosci. 31, 17242–17249 (2011).
Schwarzkopf, D.S., de Haas, B. & Rees, G. Better ways to improve standards in brain-behavior correlation analysis. Front. Hum. Neurosci. 6, 200 (2012).
Acknowledgements
We thank J. Medhora and L. Sasse for their assistance with data collection, and H. Barron and M. Klein-Flügge for their assistance with time course analyses. R.C. is supported by a Wellcome Trust Research Training Fellowship (WT088286MA). R.J.D. is supported by the Wellcome Trust (grant number 078865/Z/05/Z). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z).
Author information
Authors and Affiliations
Contributions
R.C. and M.G.-M. conducted the experiment, analyzed the data and prepared the manuscript. C.L., P.D., Q.H. and E.D. contributed to data analysis and manuscript preparation. R.J.D. contributed to data analysis and manuscript preparation and supervised the project.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–7 and Supplementary Tables 1–6 (PDF 4051 kb)
Rights and permissions
About this article
Cite this article
Chowdhury, R., Guitart-Masip, M., Lambert, C. et al. Dopamine restores reward prediction errors in old age. Nat Neurosci 16, 648–653 (2013). https://doi.org/10.1038/nn.3364
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3364
This article is cited by
-
Dopamine release in human associative striatum during reversal learning
Nature Communications (2024)
-
Cortical processes of multisensory plausibility modulation of vibrotactile perception in virtual environments in middled-aged and older adults
Scientific Reports (2024)
-
L-DOPA and oxytocin influence the neural correlates of performance monitoring for self and others
Psychopharmacology (2024)
-
Dopamine regulates decision thresholds in human reinforcement learning in males
Nature Communications (2023)
-
How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?
npj Aging (2023)