Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The dorsomedial striatum encodes net expected return, critical for energizing performance vigor

Nature Neuroscience volume 16, pages 639–647 (2013)Cite this article

Subjects

Abstract

Decision making requires an actor to not only steer behavior toward specific goals but also determine the optimal vigor of performance. Current research and models have largely focused on the former problem of how actions are directed while overlooking the latter problem of how they are energized. Here we designed a self-paced decision-making paradigm, which showed that rats' performance vigor globally fluctuates with the net value of their options, suggesting that they maintain long-term estimates of the value of their current state. Lesions of the dorsomedial striatum (DMS) and, to a lesser degree, in the ventral striatum impaired such state-dependent modulation of vigor, rendering vigor to depend more exclusively on the outcomes of immediately preceding trials. The lesions, however, spared choice biases. Neuronal recordings showed that the DMS is enriched in net value–coding neurons. In sum, the DMS encodes one's net expected return, which drives the general motivation to perform.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Design of a self-paced decision-making task: dissociating the effects of relative and net value on choice and performance vigor.
Figure 2: Net value of the rat's current block guides performance vigor.
Figure 3: Performance vigor depends on the integration of outcomes of multiple previous trials.
Figure 4: Lesions of DMS and ventral striatum (VS) have little effect on action selection.
Figure 5: Lesions in DMS impaired net value-dependent modulation of vigor.
Figure 6: Value-coding neurons in the rat striatum.
Figure 7: DMS, rather than ventral striatum (VS), predominantly encodes net value.
Figure 8: The outcome of immediately preceding trial does not significantly affect the activity of net value-coding neurons.

Similar content being viewed by others

References

  1. Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009).

    Article  CAS  Google Scholar 

  2. Lee, D., Seo, H. & Jung, M.W. Neural basis of reinforcement learning and decision making. Annu. Rev. Neurosci. 35, 287–308 (2012).

    Article  CAS  Google Scholar 

  3. Niv, Y., Joel, D. & Dayan, P. A normative perspective on motivation. Trends Cogn. Sci. 10, 375–381 (2006).

    Article  Google Scholar 

  4. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507–520 (2007).

    Article  CAS  Google Scholar 

  5. Guitart-Masip, M., Beierholm, U.R., Dolan, R., Duzel, E. & Dayan, P. Vigor in the face of fluctuating rates of reward: an experimental examination. J. Cogn. Neurosci. 23, 3933–3938 (2011).

    Article  Google Scholar 

  6. Hebb, D.O. Drives and the C.N.S. (conceptual nervous system). Psychol. Rev. 62, 243–254 (1955).

    Article  CAS  Google Scholar 

  7. Hull, C.L. Principles of Behavior: An Introduction to Behavior Theory (D. Appleton-Century Company, Inc., Oxford, England, 1943).

  8. Ferster, C.B. & Skinner, B.F. Schedules of Reinforcement (D. Appleton-Century Company, Inc., Oxford, England, 1957).

  9. Nakamura, K. & Hikosaka, O. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26, 5360–5369 (2006).

    Article  CAS  Google Scholar 

  10. Ding, L. & Gold, J.I. Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron 75, 865–874 (2012).

    Article  CAS  Google Scholar 

  11. Roesch, M.R. & Olson, C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).

    Article  CAS  Google Scholar 

  12. Gurney, K., Prescott, T.J. & Redgrave, P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84, 401–410 (2001).

    Article  CAS  Google Scholar 

  13. Hikosaka, O., Nakamura, K. & Nakahara, H. Basal ganglia orient eyes to reward. J. Neurophysiol. 95, 567–584 (2006).

    Article  Google Scholar 

  14. Turner, R.S. & Desmurget, M. Basal ganglia contributions to motor control: a vigorous tutor. Curr. Opin. Neurobiol. 20, 704–716 (2010).

    Article  CAS  Google Scholar 

  15. Lau, B. & Glimcher, P.W. Value representations in the primate striatum during matching behavior. Neuron 58, 451–463 (2008).

    Article  CAS  Google Scholar 

  16. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

    Article  CAS  Google Scholar 

  17. Cai, X., Kim, S. & Lee, D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69, 170–182 (2011).

    Article  CAS  Google Scholar 

  18. Kelley, A.E. Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci. Biobehav. Rev. 27, 765–776 (2004).

    Article  Google Scholar 

  19. Mogenson, G.J., Jones, D.L. & Yim, C.Y. From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69–97 (1980).

    Article  CAS  Google Scholar 

  20. Palmiter, R.D. Dopamine signaling in the dorsal striatum is essential for motivated behaviors: lessons from dopamine-deficient mice. Ann. NY Acad. Sci. 1129, 35–46 (2008).

    Article  CAS  Google Scholar 

  21. Amalric, M. & Koob, G.F. Depletion of dopamine in the caudate nucleus but not in nucleus accumbens impairs reaction-time performance in rats. J. Neurosci. 7, 2129–2134 (1987).

    Article  CAS  Google Scholar 

  22. Hauber, W. & Schmidt, W.J. Differential effects of lesions of the dorsomedial and dorsolateral caudate-putamen on reaction time performance in rats. Behav. Brain Res. 60, 211–215 (1994).

    Article  CAS  Google Scholar 

  23. Uchida, N. & Mainen, Z.F. Speed and accuracy of olfactory discrimination in the rat. Nat. Neurosci. 6, 1224–1229 (2003).

    Article  CAS  Google Scholar 

  24. Rorie, A.E., Gao, J., McClelland, J.L. & Newsome, W.T. Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey. PLoS ONE 5, e9308 (2010).

    Article  Google Scholar 

  25. Skinner, B.F. The Behavior of Organisms: An Experimental Analysis (D. Appleton-Century Company, Inc., Oxford, England, 1938).

  26. Tai, L.H., Lee, A.M., Benavidez, N., Bonci, A. & Wilbrecht, L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat. Neurosci. 15, 1281–1289 (2012).

    Article  CAS  Google Scholar 

  27. Mogenson, G.J., Takigawa, M., Robertson, A. & Wu, M. Self-stimulation of the nucleus accumbens and ventral tegmental area of Tsai attenuated by microinjections of spiroperidol into the nucleus accumbens. Brain Res. 171, 247–259 (1979).

    Article  CAS  Google Scholar 

  28. Imperato, A. & Di Chiara, G. Preferential stimulation of dopamine release in the nucleus accumbens of freely moving rats by ethanol. J. Pharmacol. Exp. Ther. 239, 219–228 (1986).

    CAS  PubMed  Google Scholar 

  29. Salamone, J.D., Correa, M., Farrar, A.M., Nunes, E.J. & Pardo, M. Dopamine, behavioral economics, and effort. Front. Behav. Neurosci. 3, 13 (2009).

    Article  Google Scholar 

  30. Newman, R. & Winans, S.S. An experimental study of the ventral striatum of the golden hamster. I. Neuronal connections of the nucleus accumbens. J. Comp. Neurol. 191, 167–192 (1980).

    Article  CAS  Google Scholar 

  31. Delgado, M.R., Stenger, V.A. & Fiez, J.A. Motivation-dependent responses in the human caudate nucleus. Cereb. Cortex 14, 1022–1030 (2004).

    Article  CAS  Google Scholar 

  32. Volkow, N.D. et al. “Nonhedonic” food motivation in humans involves dopamine in the dorsal striatum and methylphenidate amplifies this effect. Synapse 44, 175–180 (2002).

    Article  CAS  Google Scholar 

  33. Brown, V.J. & Robbins, T.W. Elementary processes of response selection mediated by distinct regions of the striatum. J. Neurosci. 9, 3760–3765 (1989).

    Article  CAS  Google Scholar 

  34. Bailey, K.R. & Mair, R.G. The role of striatum in initiation and execution of learned action sequences in rats. J. Neurosci. 26, 1016–1025 (2006).

    Article  CAS  Google Scholar 

  35. Thorn, C.A., Atallah, H., Howe, M. & Graybiel, A.M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781–795 (2010).

    Article  CAS  Google Scholar 

  36. Kimchi, E.Y. & Laubach, M. Dynamic encoding of action selection by the medial striatum. J. Neurosci. 29, 3148–3159 (2009).

    Article  CAS  Google Scholar 

  37. Balleine, B.W. & O'Doherty, J.P. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48–69 (2010).

    Article  Google Scholar 

  38. Yin, H.H., Knowlton, B.J. & Balleine, B.W. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur. J. Neurosci. 22, 505–512 (2005).

    Article  Google Scholar 

  39. Yin, H.H., Ostlund, S.B., Knowlton, B.J. & Balleine, B.W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523 (2005).

    Article  Google Scholar 

  40. Cury, K.M. & Uchida, N. Robust odor coding via inhalation-coupled transient activity in the mammalian olfactory bulb. Neuron 68, 570–585 (2010).

    Article  CAS  Google Scholar 

  41. Jongen-Relo, A.L. & Feldon, J. Specific neuronal protein: a new tool for histological evaluation of excitotoxic lesions. Physiol. Behav. 76, 449–456 (2002).

    Article  CAS  Google Scholar 

  42. Draper, N.R. & Smith, H. Applied Regression Analysis (John Wiley & Sons, Inc., New York, 1966).

  43. Ito, M. & Doya, K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J. Neurosci. 29, 9861–9874 (2009).

    Article  CAS  Google Scholar 

  44. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank J. Assad, K. Blum, O. Hikosaka, D. Lee, M. Livingstone, J. Maunsell and M. Meister for their comments on the manuscript; R. Born and the members of the Uchida laboratory for discussions; and S. Kim for support in behavioral experiments. This work was supported by a fellowship from National Science Foundation (A.Y.W.), Precursory Research for Embryonic Science and Technology (K.M.), a Smith Family New Investigator Award, the Alfred Sloan Foundation, the Milton Fund and the Startup Fund from Harvard University (N.U.).

Author information

Author notes

  1. Alice Y Wang

    Present address: Present address: Department of Neurobiology, Harvard Medical School, Boston, Massachusetts, USA.,

Authors and Affiliations

  1. Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, Massachusetts, USA

    Alice Y Wang, Keiji Miura & Naoshige Uchida

  2. Precursory Research for Embryonic Science and Technology (PRESTO), Japan Science and Technology Agency, Saitama, Japan

    Keiji Miura

  3. Graduate School of Information Sciences, Tohoku University, Sendai, Japan

    Keiji Miura

Authors
  1. Alice Y Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keiji Miura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naoshige Uchida
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.Y.W. wrote the manuscript, performed the experiments, performed the analyses and was involved in the experimental design. K.M. performed the simulations. N.U. was involved in preparing the manuscript and experimental design.

Corresponding author

Correspondence to Naoshige Uchida.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–10 (PDF 4193 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, A., Miura, K. & Uchida, N. The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nat Neurosci 16, 639–647 (2013). https://doi.org/10.1038/nn.3377

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3377

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing