Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.
Subscribe to Journal
Get full journal access for 1 year
only $18.75 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Eppinger, B., Hämmerer, D. & Li, S.-C. Neuromodulation of reward-based learning and decision making in human aging. Ann. NY Acad. Sci. 1235, 1–17 (2011).
Mell, T. et al. Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563 (2005).
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Salamone, J.D., Correa, M., Mingote, S.M. & Weber, S.M. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol. 5, 34–41 (2005).
Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).
O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).
O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).
Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
Knutson, B. & Gibbs, S. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl.) 191, 813–822 (2007).
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045 (2006).
Rutledge, R.B. et al. Dopaminergic drugs modulate learning rates and perseveration in parkinson's patients in a dynamic foraging task. J. Neurosci. 29, 15104–15114 (2009).
Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C. & Farde, L. The correlative triad among aging, dopamine and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807 (2006).
Düzel, E., Bunzeck, N., Guitart-Masip, M. & Duzel, S. Novelty-related motivation of anticipation and exploration by dopamine (NOMAD): implications for healthy aging. Neurosci. Biobehav. Rev. 34, 660–669 (2010).
Fearnley, J.M. & Lees, A.J. Ageing and Parkinson's disease: substantia nigra regional selectivity. Brain 114, 2283–2301 (1991).
Vaillancourt, D.E., Spraker, M.B., Prodoehl, J., Zhou, X.J. & Little, D.M. Effects of aging on the ventral and dorsal substantia nigra using diffusion tensor imaging. Neurobiol. Aging 33, 35–42 (2012).
Samanez-Larkin, G.R., Kuhnen, C.M., Yoo, D.J. & Knutson, B. Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 30, 1426–1434 (2010).
Schott, B.H. et al. Ageing and early-stage Parkinson's disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130, 2412–2424 (2007).
Cox, K.M., Aizenstein, H.J. & Fiez, J.A. Striatal outcome processing in healthy aging. Cogn. Affect. Behav. Neurosci. 8, 304–317 (2008).
Behrens, T.E.J., Hunt, L.T., Woolrich, M.W. & Rushworth, M.F.S. Associative learning of social value. Nature 456, 245–249 (2008).
Cools, R. & D'Esposito, M. Inverted U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 69, e113–e125 (2011).
Li, J. & Daw, N.D. Signals in human striatum are appropriate for policy update rather than value prediction. J. Neurosci. 31, 5504–5511 (2011).
Eppinger, B., Kray, J., Mock, B. & Mecklinger, A. Better or worse than expected? Aging, learning and the ERN. Neuropsychologia 46, 521–539 (2008).
Samanez-Larkin, G.R., Wagner, A.D. & Knutson, B. Expected value information improves financial risk taking across the adult life span. Soc. Cogn. Affect. Neurosci. 6, 207–217 (2011).
Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154–166 (2012).
Dickinson, A. & Balleine, B. The Role of Learning in the Operation of Motivational Systems (John Wiley & Sons, New York, 2002).
Dickinson, A., Smith, J. & Mirenowicz, J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483 (2000).
Worthy, D.A., Gorlick, M.A., Pacheco, J.L., Schnyer, D.M. & Maddox, W.T. With age comes wisdom. Psychol. Sci. 22, 1375–1380 (2011).
de Wit, S. et al. Reliance on habits at the expense of goal-directed control following dopamine precursor depletion. Psychopharmacology (Berl.) 219, 621–631 (2012).
Wunderlich, K., Smittenaar, P. & Dolan, R.J. Dopamine enhances model-based over model-free choice behavior. Neuron 75, 418–424 (2012).
Gläscher, J., Daw, N., Dayan, P. & O'Doherty, J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).
Jocham, G., Klein, T.A. & Ullsperger, M. Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J. Neurosci. 31, 1606–1613 (2011).
Shohamy, D. & Adcock, R.A. Dopamine and adaptive memory. Trends Cogn. Sci. 14, 464–472 (2010).
Nicola, S.M., Surmeier, D.J. & Malenka, R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185–215 (2000).
Daw, N.D. & Gershman, S.J. Seymour, B., Dayan, P. & Dolan, R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).
Simon, D.A. & Daw, N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526–5539 (2011).
Guitart-Masip, M. et al. Action controls dopaminergic enhancement of reward representations. Proc. Natl. Acad. Sci. USA 109, 7511–7516 (2012).
Choi, W.Y., Balsam, P.D. & Horvitz, J.C. Extended habit training reduces dopamine mediation of appetitive response expression. J. Neurosci. 25, 6729–6733 (2005).
Klein-Flügge, M.C., Hunt, L.T., Bach, D.R., Dolan, R.J. & Behrens, T.E.J. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 72, 654–664 (2011).
Coxon, J.P., Van Impe, A., Wenderoth, N. & Swinnen, S.P. Aging and inhibitory control of action: cortico-subthalamic connection strength predicts stopping performance. J. Neurosci. 32, 8401–8412 (2012).
Forstmann, B.U. et al. Cortico-subthalamic white matter tract strength predicts interindividual efficacy in stopping a motor response. Neuroimage 60, 370–375 (2012).
Le Bihan, D. & Johansen-Berg, H. Diffusion MRI at 25: exploring brain tissue structure and function. Neuroimage 61, 324–341 (2012).
Draganski, B. et al. Regional specificity of MRI contrast parameter changes in normal ageing revealed by voxel-based quantification (VBQ). Neuroimage 55, 1423–1434 (2011).
Li, S.-C. & Sikström, S. Integrative neurocomputational perspectives on cognitive aging, neuromodulation, and representation. Neurosci. Biobehav. Rev. 26, 795–808 (2002).
Schönberg, T., Daw, N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–12867 (2007).
Huys, Q.J.M. et al. Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).
Lutti, A., Hutton, C., Finsterbusch, J., Helms, G. & Weiskopf, N. Optimization and validation of methods for mapping of the radiofrequency transmit field at 3T. Magn. Reson. Med. 64, 229–238 (2010).
Reese, T.G., Heid, O., Weisskoff, R.M. & Wedeen, V.J. Reduction of eddy-current-induced distortion in diffusion MRI using a twice-refocused spin echo. Magn. Reson. Med. 49, 177–182 (2003).
Nagy, Z., Weiskopf, N., Alexander, D.C. & Deichmann, R. A method for improving the performance of gradient systems for diffusion-weighted MRI. Magn. Reson. Med. 58, 763–768 (2007).
Andersson, J.L.R., Skare, S. & Ashburner, J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. Neuroimage 20, 870–888 (2003).
Ashburner, J. A fast diffeomorphic image registration algorithm. Neuroimage 38, 95–113 (2007).
Knutson, B. & Cooper, J.C. Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18, 411–417 (2005).
Brett, M., Anton, J.-L., Valabregue, R. & Poline, J.-B. Region of interest analysis using an SPM toolbox. Neuroimage 16, 1140–1141 (2002).
Forstmann, B.U. et al. The speed-accuracy tradeoff in the elderly brain: a structural model-based approach. J. Neurosci. 31, 17242–17249 (2011).
Schwarzkopf, D.S., de Haas, B. & Rees, G. Better ways to improve standards in brain-behavior correlation analysis. Front. Hum. Neurosci. 6, 200 (2012).
We thank J. Medhora and L. Sasse for their assistance with data collection, and H. Barron and M. Klein-Flügge for their assistance with time course analyses. R.C. is supported by a Wellcome Trust Research Training Fellowship (WT088286MA). R.J.D. is supported by the Wellcome Trust (grant number 078865/Z/05/Z). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z).
The authors declare no competing financial interests.
About this article
Cite this article
Chowdhury, R., Guitart-Masip, M., Lambert, C. et al. Dopamine restores reward prediction errors in old age. Nat Neurosci 16, 648–653 (2013). https://doi.org/10.1038/nn.3364
WIREs Cognitive Science (2020)
Neurobiology of Aging (2020)
The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study
Cerebral Cortex (2020)
Impaired social learning predicts reduced real-life motivation in individuals with depression: A computational fMRI study
Journal of Affective Disorders (2020)