Review Article | Published:

Striatal circuits for reward learning and decision-making

Abstract

The striatum is essential for learning which actions lead to reward and for implementing those actions. Decades of experimental and theoretical work have led to several influential theories and hypotheses about how the striatal circuit mediates these functions. However, owing to technical limitations, testing these hypotheses rigorously has been difficult. In this Review, we briefly describe some of the classic ideas of striatal function. We then review recent studies in rodents that take advantage of optical and genetic methods to test these classic ideas by recording and manipulating identified cell types within the circuit. This new body of work has provided experimental support of some longstanding ideas about the striatal circuit and has uncovered critical aspects of the classic view that are incorrect or incomplete.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Alexander, G. E., DeLong, M. R. & Strick, P. L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381 (1986).

  2. 2.

    Alexander, G. E. & Crutcher, M. D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271 (1990).

  3. 3.

    Redgrave, P. et al. Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nat. Rev. Neurosci. 11, 760–772 (2010).

  4. 4.

    Gerfen, C. R. & Bolam, J. P. in Handbook of Basal Ganglia Structure and Function 2nd edn Vol. 24 (eds Steiner, H. & Tseng, K. Y.) 3–32 (Elsevier, 2016).

  5. 5.

    Loopuijt, L. D. & van der Kooy, D. Organization of the striatum: collateralization of its efferent axons. Brain Res. 348, 86–99 (1985).

  6. 6.

    Gerfen, C. R. & Scott Young, W. Distribution of striatonigral and striatopallidal peptidergic neurons in both patch and matrix compartments: an in situ hybridization histochemistry and fluorescent retrograde tracing study. Brain Res. 460, 161–167 (1988).

  7. 7.

    Kawaguchi, Y., Wilson, C. J. & Emson, P. C. Projection subtypes of rat neostriatal matrix cells revealed by intracellular injection of biocytin. J. Neurosci. 10, 3421–3438 (1990).

  8. 8.

    Gerfen, C. R. et al. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250, 1429–1432 (1990).

  9. 9.

    Wu, Y., Richard, S. & Parent, A. The organization of the striatal output system: a single-cell juxtacellular labeling study in the rat. Neurosci. Res. 38, 49–62 (2000).

  10. 10.

    Albin, R. L., Young, A. B. & Penney, J. B. The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366–375 (1989).

  11. 11.

    Chevalier, G. & Deniau, J. M. Disinhibition as a basic process in the expression of striatal functions. Trends Neurosci. 13, 277–280 (1990).

  12. 12.

    DeLong, M. R. Primate models of movement disorders of basal ganglia origin. Trends Neurosci. 13, 281–285 (1990).

  13. 13.

    Mink, J. W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996).

  14. 14.

    Lanciego, J. L., Luquin, N. & Obeso, J. A. Functional neuroanatomy of the basal ganglia. Cold Spring Harb. Perspect. Med. 2, a009621 (2012).

  15. 15.

    Nelson, A. B. & Kreitzer, A. C. Reassessing models of basal ganglia function and dysfunction. Annu. Rev. Neurosci. 37, 117–135 (2014).

  16. 16.

    Bolam, J. P., Wainer, B. H. & Smith, A. D. Characterization of cholinergic neurons in the rat neostriatum. A combination of choline acetyltransferase immunocytochemistry, Golgi-impregnation and electron microscopy. Neuroscience 12, 711–718 (1984).

  17. 17.

    Burke, D. A., Rotstein, H. G. & Alvarez, V. A. Striatal local circuitry: a new framework for lateral inhibition. Neuron 96, 267–284 (2017).

  18. 18.

    Tepper, J. M. & Koós, T. in Handbook of Basal Ganglia Structure and Function 2nd edn Vol. 24 (eds Steiner, H. & Tseng, K. Y.) 157–178 (Elsevier, 2016).

  19. 19.

    Berke, J. D. Functional properties of striatal fast-spiking interneurons. Front. Syst. Neurosci. 5, 45 (2011).

  20. 20.

    Beckstead, R. M., Domesick, V. B. & Nauta, W. J. Efferent connections of the substantia nigra and ventral tegmental area in the rat. Brain Res. 175, 191–217 (1979).

  21. 21.

    Swanson, L. W. The projections of the ventral tegmental area and adjacent regions: a combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 9, 321–353 (1982).

  22. 22.

    Lammel, S. et al. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57, 760–773 (2008).

  23. 23.

    Lerner, T. N. et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635–647 (2015).

  24. 24.

    Beier, K. T. et al. Circuit architecture of VTA dopamine neurons revealed by systematic input-output mapping. Cell 162, 622–634 (2015).

  25. 25.

    Saunders, B. T., Richard, J. M., Margolis, E. B. & Janak, P. H. Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nat. Neurosci. 21, 1072–1083 (2018). This study shows how VTA DA activation increases the value of an associated conditioned stimulus whereas SNc DA activation increases conditioned responding to the conditioned stimulus without increasing its value.

  26. 26.

    Poulin, J.-F. et al. Mapping projections of molecularly defined dopamine neuron subtypes using intersectional genetic approaches. Nat. Neurosci. 21, 1260–1271 (2018).

  27. 27.

    Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).

  28. 28.

    Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997). This seminal paper connects DA activity with reinforcement learning models.

  29. 29.

    Mirenowicz, J. & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449–451 (1996).

  30. 30.

    Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

  31. 31.

    Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).

  32. 32.

    Fiorillo, C. D., Tobler, P. N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

  33. 33.

    Roesch, M. R., Calu, D. J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615–1624 (2007).

  34. 34.

    Day, J. J., Roitman, M. F., Wightman, R. M. & Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020–1028 (2007).

  35. 35.

    Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815–834 (2010).

  36. 36.

    Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B. & Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88 (2012). This study employs phototagging to confirm that VTA DA neurons represent RPE whereas VTA GABA neurons represent expected reward.

  37. 37.

    Eshel, N. et al. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).

  38. 38.

    Reynolds, J. N. J., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001).

  39. 39.

    Reynolds, J. N. J. & Wickens, J. R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15, 507–521 (2002).

  40. 40.

    Shen, W., Flajolet, M., Greengard, P. & Surmeier, D. J. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848–851 (2008).

  41. 41.

    Gerfen, C. R. & Surmeier, D. J. Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci. 34, 441–466 (2011).

  42. 42.

    Bamford, N. S., Wightman, R. M. & Sulzer, D. Dopamine’s effects on corticostriatal synapses during reward-based behaviors. Neuron 97, 494–510 (2018). This recent review discusses mechanisms by which DA affects corticostriatal synapses and MSN activity during reward-seeking behaviours.

  43. 43.

    Olds, J. Self-stimulation of the brain; its use to study local effects of hunger, sex, and drugs. Science 127, 315–324 (1958).

  44. 44.

    Corbett, D. & Wise, R. A. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable electrode mapping study. Brain Res. 185, 1–15 (1980).

  45. 45.

    Fouriezos, G. & Wise, R. A. Pimozide-induced extinction of intracranial self-stimulation: response patterns rule out motor or performance deficits. Brain Res. 103, 377–380 (1976).

  46. 46.

    Wise, R. A. Forebrain substrates of reward and motivation. J. Comp. Neurol. 493, 115–121 (2005).

  47. 47.

    Steinberg, E. E. et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973 (2013).

  48. 48.

    Sharpe, M. J. et al. Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat. Neurosci. 20, 735–742 (2017).

  49. 49.

    Tsai, H.-C. et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080–1084 (2009).

  50. 50.

    Ilango, A. et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817–822 (2014).

  51. 51.

    Witten, I. B. et al. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 72, 721–733 (2011).

  52. 52.

    Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. Nat. Neurosci. 19, 117–126 (2016).

  53. 53.

    Adamantidis, A. R. et al. Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J. Neurosci. 31, 10829–10835 (2011).

  54. 54.

    Chang, C. Y. et al. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat. Neurosci. 19, 111–116 (2016).

  55. 55.

    Parker, N. F. et al. Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nat. Neurosci. 19, 845–854 (2016).

  56. 56.

    O’Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

  57. 57.

    Balleine, B. W., Delgado, M. R. & Hikosaka, O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161–8165 (2007).

  58. 58.

    Engelhard, B. et al. Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons. Nature https://doi.org/10.1038/s41586-019-1261-9 (2019).Cellular resolution imaging of VTA DA neurons reveals widespread reward representations multiplexed with specialized representations of task variables.

  59. 59.

    Howe, M. W. & Dombeck, D. A. Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature 535, 505–510 (2016). Axonal imaging of DA terminals in the dorsal striatum reveals that distinct axons signal locomotion and reward.

  60. 60.

    Menegas, W., Akiti, K., Amo, R., Uchida, N. & Watabe-Uchida, M. Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli. Nat. Neurosci. 275, 1593 (2018). This paper shows that DA in the TS supports learning to avoid threatening stimuli whereas DA in the NAc supports learning to pursue rewarding stimuli.

  61. 61.

    Menegas, W., Babayan, B. M., Uchida, N. & Watabe-Uchida, M. Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. eLife 6, e21886 (2017).

  62. 62.

    Lee, R. S., Mattar, M. G., Parker, N. F., Witten, I. B. & Daw, N. D. Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons. eLife 8, e42992 (2019).

  63. 63.

    da Silva, J. A., Tecuapetla, F., Paixão, V. & Costa, R. M. Dopamine neuron activity before action initiation gates and invigorates future movements. Nature 554, 244–248 (2018).

  64. 64.

    Barter, J. W. et al. Beyond reward prediction errors: the role of dopamine in movement kinematics. Front. Integr. Neurosci. 9, 39 (2015).

  65. 65.

    Joshua, M., Adler, A., Mitelman, R., Vaadia, E. & Bergman, H. Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J. Neurosci. 28, 11673–11684 (2008).

  66. 66.

    Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

  67. 67.

    Brischoux, F., Chakraborty, S., Brierley, D. I. & Ungless, M. A. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl Acad. Sci. USA 106, 4894–4899 (2009).

  68. 68.

    Gangarossa, G. et al. Spatial distribution of D1R- and D2R-expressing medium-sized spiny neurons differs along the rostro-caudal axis of the mouse dorsal striatum. Front. Neural Circuits 7, 124 (2013).

  69. 69.

    Hikosaka, O., Takikawa, Y. & Kawagoe, R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80, 953–978 (2000).

  70. 70.

    Calabresi, P., Picconi, B., Tozzi, A., Ghiglieri, V. & Di Filippo, M. Direct and indirect pathways of basal ganglia: a critical reappraisal. Nat. Neurosci. 17, 1022–1030 (2014).

  71. 71.

    Oldenburg, I. A. & Sabatini, B. L. Antagonistic but not symmetric regulation of primary motor cortex by basal ganglia direct and indirect pathways. Neuron 86, 1174–1181 (2015).

  72. 72.

    Roseberry, T. K. et al. Cell-type-specific control of brainstem locomotor circuits by basal ganglia. Cell 164, 526–537 (2016).

  73. 73.

    Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002).

  74. 74.

    Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

  75. 75.

    Lau, B. & Glimcher, P. W. Value representations in the primate striatum during matching behavior. Neuron 58, 451–463 (2008).

  76. 76.

    Ding, L. & Gold, J. I. Caudate encodes multiple computations for perceptual decisions. J. Neurosci. 30, 15747–15759 (2010).

  77. 77.

    Gong, S. et al. Targeting Cre recombinase to specific neuron populations with bacterial artificial chromosome constructs. J. Neurosci. 27, 9817–9823 (2007).

  78. 78.

    Gerfen, C. R., Paletzki, R. & Heintz, N. GENSAT BAC cre-recombinase driver lines to study the functional organization of cerebral cortical and basal ganglia circuits. Neuron 80, 1368–1383 (2013).

  79. 79.

    Cui, G. et al. Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238–242 (2013). This study is the first to show that D1R and D2R MSNs are co-activated during movement and inactive during immobility, contrary to some theories of striatal function.

  80. 80.

    Barbera, G. et al. Spatially compact neural clusters in the dorsal striatum encode locomotion relevant information. Neuron 92, 202–213 (2016).

  81. 81.

    Klaus, A. et al. The spatiotemporal organization of the striatum encodes action space. Neuron 96, 949 (2017).

  82. 82.

    Markowitz, J. E. et al. The striatum organizes 3D behavior via moment-to-moment action selection. Cell 174, 44–58 (2018). This study uses machine learning algorithms to characterize spontaneous behaviour into discrete, subsecond components and describes D1R and D2R MSN responses to the identity and sequence of these behavioural components.

  83. 83.

    Parker, J. G. et al. Diametric neural ensemble dynamics in parkinsonian and dyskinetic states. Nature 557, 177–182 (2018). This study thoroughly probes the effects of DA depletion and subsequent administration of dopaminergic agonists and antagonists on the activity of D1R-expressing and D2R-expressing MSNs in vivo.

  84. 84.

    Meng, C. et al. Spectrally resolved fiber photometry for multi-component analysis of brain circuits. Neuron 98, 707–717 (2018).

  85. 85.

    London, T. D. et al. Coordinated ramping of dorsal striatal pathways preceding food approach and consumption. J. Neurosci. 38, 3547–3558 (2018).

  86. 86.

    Isomura, Y. et al. Reward-modulated motor information in identified striatum neurons. J. Neurosci. 33, 10209–10220 (2013).

  87. 87.

    Jin, X., Tecuapetla, F. & Costa, R. M. Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences. Nat. Neurosci. 17, 423–430 (2014).

  88. 88.

    Sippy, T., Lapray, D., Crochet, S. & Petersen, C. C. H. Cell-type-specific sensorimotor processing in striatal projection neurons during goal-directed behavior. Neuron 88, 298–305 (2015).

  89. 89.

    Geddes, C. E., Li, H. & Jin, X. Optogenetic editing reveals the hierarchical organization of learned action sequences. Cell 174, 32–43 (2018).

  90. 90.

    Nonomura, S. et al. Monitoring and updating of action selection for goal-directed behavior through the striatal direct and indirect pathways. Neuron 99, 1302–1314 (2018).

  91. 91.

    Donahue, C. H., Liu, M. & Kreitzer, A. Distinct value encoding in striatal direct and indirect pathways during adaptive learning. Preprint at bioRxiv https://doi.org/10.1101/277855 (2018).

  92. 92.

    Tecuapetla, F., Matias, S., Dugue, G. P., Mainen, Z. F. & Costa, R. M. Balanced activity in basal ganglia projection pathways is critical for contraversive movements. Nat. Commun. 5, 4315 (2014).

  93. 93.

    Cazorla, M. et al. Dopamine D2 receptors regulate the anatomical and functional balance of basal ganglia circuitry. Neuron 81, 153–164 (2014).

  94. 94.

    Collins, A. G. E. & Frank, M. J. Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol. Rev. 121, 337–366 (2014).

  95. 95.

    Bariselli, S., Fobbs, W. C., Creed, M. C. & Kravitz, A. V. A competitive model for striatal action selection. Brain Res. 1713, 70–79 (2018).

  96. 96.

    Frank, M. J., Seeberger, L. C. & O’reilly, R. C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943 (2004).

  97. 97.

    Peak, J., Hart, G. & Balleine, B. W. From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning. Eur. J. Neurosci. 49, 658–671 (2019).

  98. 98.

    Yartsev, M. M., Hanks, T. D., Yoon, A. M. & Brody, C. D. Causal contribution and dynamical encoding in the striatum during evidence accumulation. eLife 7, e34929 (2018).

  99. 99.

    Shin, J. H., Kim, D. & Jung, M. W. Differential coding of reward and movement information in the dorsomedial striatal direct and indirect pathways. Nat. Commun. 9, 404 (2018).

  100. 100.

    Zalocusky, K. A. et al. Nucleus accumbens D2R cells signal prior outcomes and control risky decision-making. Nature 531, 642–646 (2016).

  101. 101.

    Lau, B. & Glimcher, P. W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005).

  102. 102.

    Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A. & Wilbrecht, L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat. Neurosci. 15, 1281–1289 (2012).

  103. 103.

    Kravitz, A. V., Tye, L. D. & Kreitzer, A. C. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 15, 816–818 (2012). This paper shows how D1R and D2R MSN activity in DMS is sufficient to positively and negatively reinforce intracranial self-stimulation, respectively, but this learning does not depend on DA transmission.

  104. 104.

    Yttri, E. A. & Dudman, J. T. Opponent and bidirectional control of movement velocity in the basal ganglia. Nature 533, 402–406 (2016). This study shows that D1R and D2R MSN activity in DMS is sufficient to positively and negatively reinforce movement velocity, respectively, and this learning depends on DA transmission.

  105. 105.

    Lobo, M. K. et al. Cell type-specific loss of BDNF signaling mimics optogenetic control of cocaine reward. Science 330, 385–390 (2010). D1R and D2R MSN activity in NAc enhances or suppresses, respectively, the establishment of a cocaine CPP.

  106. 106.

    Wang, L., Rangarajan, K. V., Gerfen, C. R. & Krauzlis, R. J. Activation of striatal neurons causes a perceptual decision bias during visual change detection in mice. Neuron 98, 669 (2018).

  107. 107.

    Tecuapetla, F., Jin, X., Lima, S. Q. & Costa, R. M. Complementary contributions of striatal projection pathways to action initiation and execution. Cell 166, 703–715 (2016).

  108. 108.

    Vicente, A. M., Galvão-Ferreira, P., Tecuapetla, F. & Costa, R. M. Direct and indirect dorsolateral striatum pathways reinforce different action strategies. Curr. Biol. 26, R267–269 (2016).

  109. 109.

    Cole, S. L., Robinson, M. J. F. & Berridge, K. C. Optogenetic self-stimulation in the nucleus accumbens: D1 reward versus D2 ambivalence. PLOS ONE 13, e0207694 (2018).

  110. 110.

    Soares-Cunha, C. et al. Activation of D2 dopamine receptor-expressing neurons in the nucleus accumbens increases motivation. Nat. Commun. 7, 11829 (2016).

  111. 111.

    Carvalho Poyraz, F. et al. Decreasing striatopallidal pathway function enhances motivation by energizing the initiation of goal-directed action. J. Neurosci. 36, 5988–6001 (2016).

  112. 112.

    Gallo, E. F. et al. Accumbens dopamine D2 receptors increase motivation by decreasing inhibitory transmission to the ventral pallidum. Nat. Commun. 9, 1086 (2018).

  113. 113.

    Apicella, P. The role of the intrinsic cholinergic system of the striatum: what have we learned from TAN recordings in behaving animals? Neuroscience 360, 81–94 (2017).

  114. 114.

    Hasselmo, M. E. The role of acetylcholine in learning and memory. Curr. Opin. Neurobiol. 16, 710–715 (2006).

  115. 115.

    Macintosh, F. C. The distribution of acetylcholine in the peripheral and the central nervous system. J. Physiol. 99, 436–442 (1941).

  116. 116.

    Hebb, C. O. & Silver, A. Gradient of cholinesterase activity and of choline acetylase activity in nerve fibres: gradient of choline acetylase activity. Nature 189, 123–125 (1961).

  117. 117.

    Lim, S. A. O., Kang, U. J. & McGehee, D. S. Striatal cholinergic interneuron regulation and circuit effects. Front. Synaptic Neurosci. 6, 22 (2014).

  118. 118.

    Wilson, C. J., Chang, H. T. & Kitai, S. T. Firing patterns and synaptic potentials of identified giant aspiny interneurons in the rat neostriatum. J. Neurosci. 10, 508–519 (1990).

  119. 119.

    Inokawa, H., Yamada, H., Matsumoto, N., Muranishi, M. & Kimura, M. Juxtacellular labeling of tonically active neurons and phasically active neurons in the rat striatum. Neuroscience 168, 395–404 (2010).

  120. 120.

    Schulz, J. M., Oswald, M. J. & Reynolds, J. N. J. Visual-induced excitation leads to firing pauses in striatal cholinergic interneurons. J. Neurosci. 31, 11133–11143 (2011).

  121. 121.

    Kimura, M., Rajkowski, J. & Evarts, E. Tonically discharging putamen neurons exhibit set-dependent responses. Proc. Natl Acad. Sci. USA 81, 4998–5001 (1984).

  122. 122.

    Aosaki, T. et al. Responses of tonically active neurons in the primate’s striatum undergo systematic changes during behavioral sensorimotor conditioning. J. Neurosci. 14, 3969–3984 (1994).

  123. 123.

    Graybiel, A. M., Aosaki, T., Flaherty, A. W. & Kimura, M. The basal ganglia and adaptive motor control. Science 265, 1826–1831 (1994).

  124. 124.

    Aosaki, T., Graybiel, A. M. & Kimura, M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265, 412–415 (1994).

  125. 125.

    Ravel, S., Legallet, E. & Apicella, P. Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli. Exp. Brain Res. 128, 531–534 (1999).

  126. 126.

    Goldberg, J. A. & Reynolds, J. N. J. Spontaneous firing and evoked pauses in the tonically active cholinergic interneurons of the striatum. Neuroscience 198, 27–43 (2011).

  127. 127.

    Witten, I. B. et al. Cholinergic interneurons control local circuit activity and cocaine conditioning. Science 330, 1677–1681 (2010).

  128. 128.

    Lee, J., Finkelstein, J., Choi, J. Y. & Witten, I. B. Linking cholinergic interneurons, synaptic plasticity, and behavior during the extinction of a cocaine-context association. Neuron 90, 1071–1085 (2016). This study shows that CINs regulate glutamatergic synaptic plasticity in the NAc during cocaine context extinction in a manner that can explain the associated behavioural changes.

  129. 129.

    Bradfield, L. A., Bertran-Gonzalez, J., Chieng, B. & Balleine, B. W. The thalamostriatal pathway and cholinergic control of goal-directed action: interlacing new with existing learning in the striatum. Neuron 79, 153–166 (2013).

  130. 130.

    Aoki, S., Liu, A. W., Zucca, A., Zucca, S. & Wickens, J. R. Role of striatal cholinergic interneurons in set-shifting in the rat. J. Neurosci. 35, 9424–9431 (2015).

  131. 131.

    Okada, K. et al. Enhanced flexibility of place discrimination learning by targeting striatal cholinergic interneurons. Nat. Commun. 5, 3778 (2014).

  132. 132.

    Matamales, M. et al. Aging-related dysfunction of striatal cholinergic interneurons produces conflict in action selection. Neuron 90, 362–373 (2016).

  133. 133.

    Collins, A. L. et al. Nucleus accumbens cholinergic interneurons oppose cue-motivated behavior. Biol. Psychiatry https://doi.org/10.1016/j.biopsych.2019.02.014 (2019).

  134. 134.

    English, D. F. et al. GABAergic circuits mediate the reinforcement-related signals of striatal cholinergic interneurons. Nat. Neurosci. 15, 123–130 (2011).

  135. 135.

    Nelson, A. B. et al. Striatal cholinergic interneurons drive GABA release from dopamine terminals. Neuron 82, 63–70 (2014).

  136. 136.

    Tritsch, N. X., Oh, W.-J., Gu, C. & Sabatini, B. L. Midbrain dopamine neurons sustain inhibitory transmission using plasma membrane uptake of GABA, not synthesis. eLife 3, e01936 (2014).

  137. 137.

    Cachope, R. et al. Selective activation of cholinergic interneurons enhances accumbal phasic dopamine release: setting the tone for reward processing. Cell Rep. 2, 33–41 (2012).

  138. 138.

    Threlfell, S. et al. Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75, 58–64 (2012).

  139. 139.

    Selemon, L. D. & Goldman-Rakic, P. S. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J. Neurosci. 5, 776–794 (1985).

  140. 140.

    Groenewegen, H. J., Berendse, H. W., Wolters, J. G. & Lohman, A. H. The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization. Prog. Brain Res. 85, 95–116 (1990).

  141. 141.

    Flaherty, A. W. & Graybiel, A. M. Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-part representations. J. Neurophysiol. 66, 1249–1263 (1991).

  142. 142.

    Berendse, H. W., Galis-de Graaf, Y. & Groenewegen, H. J. Topographical organization and relationship with ventral striatal compartments of prefrontal corticostriatal projections in the rat. J. Comp. Neurol. 316, 314–347 (1992).

  143. 143.

    Pan, W. X., Mao, T. & Dudman, J. T. Inputs to the dorsal striatum of the mouse reflect the parallel circuit architecture of the forebrain. Front. Neuroanat. 4, 147 (2010).

  144. 144.

    Wall, N. R., De La Parra, M., Callaway, E. M. & Kreitzer, A. C. Differential innervation of direct- and indirect-pathway striatal projection neurons. Neuron 79, 347–360 (2013).

  145. 145.

    Guo, Q. et al. Whole-brain mapping of inputs to projection neurons and cholinergic interneurons in the dorsal striatum. PLOS ONE 10, e0123381 (2015).

  146. 146.

    Heilbronner, S. R., Rodriguez-Romaguera, J., Quirk, G. J., Groenewegen, H. J. & Haber, S. N. Circuit-based corticostriatal homologies between rat and primate. Biol. Psychiatry 80, 509–521 (2016).

  147. 147.

    Hunnicutt, B. J. et al. A comprehensive excitatory input map of the striatum reveals novel functional organization. eLife 5, e19103 (2016).

  148. 148.

    Hintiryan, H. et al. The mouse cortico-striatal projectome. Nat. Neurosci. 19, 1100–1114 (2016). This paper and that of Hunnicutt et al. (2016) provide detailed cortical and thalamic input maps to the striatum and use clustering methods on the anatomical distribution of these inputs to identify striatal subdomains.

  149. 149.

    Stuber, G. D. et al. Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking. Nature 475, 377–380 (2011).

  150. 150.

    Britt, J. P. et al. Synaptic and behavioral profile of multiple glutamatergic inputs to the nucleus accumbens. Neuron 76, 790–803 (2012).

  151. 151.

    Koralek, A. C., Jin, X., Long, J. D. 2nd, Costa, R. M. & Carmena, J. M. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335 (2012). This study shows that corticostriatal plasticity is required for learning neuroprosthetic control of motor cortex neurons, irrespective of movement, and that the activity of striatal neurons is modulated by this type of goal-directed learning.

  152. 152.

    Znamenskiy, P. & Zador, A. M. Corticostriatal neurons in auditory cortex drive decisions during auditory discrimination. Nature 497, 482–485 (2013).

  153. 153.

    MacAskill, A. F., Cassel, J. M. & Carter, A. G. Cocaine exposure reorganizes cell type- and input-specific connectivity in the nucleus accumbens. Nat. Neurosci. 17, 1198–1207 (2014).

  154. 154.

    Pascoli, V. et al. Contrasting forms of cocaine-evoked plasticity control components of relapse. Nature 509, 459–464 (2014).

  155. 155.

    Friedman, A. et al. A corticostriatal path targeting striosomes controls decision-making under conflict. Cell 161, 1320–1333 (2015).

  156. 156.

    Rothwell, P. E. et al. Input- and output-specific regulation of serial order performance by corticostriatal circuits. Neuron 88, 345–356 (2015).

  157. 157.

    Namburi, P. et al. A circuit mechanism for differentiating positive and negative associations. Nature 520, 675–678 (2015).

  158. 158.

    Xiong, Q., Znamenskiy, P. & Zador, A. M. Selective corticostriatal plasticity during acquisition of an auditory discrimination task. Nature 521, 348–351 (2015).

  159. 159.

    Christoffel, D. J. et al. Excitatory transmission at thalamo-striatal synapses mediates susceptibility to social stress. Nat. Neurosci. 18, 962–964 (2015).

  160. 160.

    Zhu, Y., Wienecke, C. F. R., Nachtrab, G. & Chen, X. A thalamic input to the nucleus accumbens mediates opiate dependence. Nature 530, 219–222 (2016).

  161. 161.

    Okuyama, T., Kitamura, T., Roy, D. S., Itohara, S. & Tonegawa, S. Ventral CA1 neurons store social memory. Science 353, 1536–1541 (2016).

  162. 162.

    Yoo, J. H. et al. Ventral tegmental area glutamate neurons co-release GABA and promote positive reinforcement. Nat. Commun. 7, 13697 (2016).

  163. 163.

    Beyeler, A. et al. Divergent routing of positive and negative information from the amygdala during memory retrieval. Neuron 90, 348–361 (2016).

  164. 164.

    Otis, J. M. et al. Prefrontal cortex output circuits guide reward seeking through divergent cue encoding. Nature 543, 103–107 (2017).

  165. 165.

    Murugan, M. et al. Combined social and spatial coding in a descending projection from the prefrontal cortex. Cell 171, 1663–1677 (2017).

  166. 166.

    Kupferschmidt, D. A., Juczewski, K., Cui, G., Johnson, K. A. & Lovinger, D. M. Parallel, but dissociable, processing in discrete corticostriatal inputs encodes skill learning. Neuron 96, 476–489 (2017).

  167. 167.

    Friedman, A. et al. Chronic stress alters striosome-circuit dynamics, leading to aberrant decision-making. Cell 171, 1191–1205 (2017).

  168. 168.

    Kim, C. K. et al. Molecular and circuit-dynamical identification of top-down neural mechanisms for restraint of reward seeking. Cell 170, 1013–1027 (2017).

  169. 169.

    Amadei, E. A. et al. Dynamic corticostriatal activity biases social bonding in monogamous female prairie voles. Nature 546, 297–301 (2017). This paper demonstrates that pair bonding in prairie voles modulates the projection from the mPFC to the NAc and that stimulation of this projection increases preference for a social target.

  170. 170.

    Sweis, B. M., Larson, E. B., Redish, A. D. & Thomas, M. J. Altering gain of the infralimbic-to-accumbens shell circuit alters economically dissociable decision-making algorithms. Proc. Natl Acad. Sci. USA 115, E6347–E6355 (2018).

  171. 171.

    Cui, Q., Li, Q., Geng, H., Chen, L. & Ip, N. Y. Dopamine receptors mediate strategy abandoning via modulation of a specific prelimbic cortex–nucleus accumbens pathway in mice. Proc. Natl Acad. Sci. USA 115, E4890–E4899 (2018).

  172. 172.

    Díaz-Hernández, E. et al. The thalamostriatal projections contribute to the initiation and execution of a sequence of movements. Neuron 100, 739–752 (2018).

  173. 173.

    LeGates, T. A. et al. Reward behaviour is regulated by the strength of hippocampus-nucleus accumbens synapses. Nature 564, 258–262 (2018).

  174. 174.

    Hart, G., Bradfield, L. A., Fok, S. Y., Chieng, B. & Balleine, B. W. The bilateral prefronto-striatal pathway is necessary for learning new goal-directed actions. Curr. Biol. 28, 2218–2229 (2018).

  175. 175.

    Trouche, S. et al. A hippocampus-accumbens tripartite neuronal motif guides appetitive memory in space. Cell 176, 1393–1406 (2019).

  176. 176.

    Chen, L., Wang, X., Ge, S. & Xiong, Q. Medial geniculate body and primary auditory cortex differentially contribute to striatal sound representations. Nat. Commun. 10, 418 (2019).

  177. 177.

    Yamamoto, S., Monosov, I. E., Yasuda, M. & Hikosaka, O. What and where information in the caudate tail guides saccades to visual objects. J. Neurosci. 32, 11005–11016 (2012).

  178. 178.

    Yamamoto, S., Kim, H. F. & Hikosaka, O. Reward value-contingent changes of visual responses in the primate caudate tail associated with a visuomotor skill. J. Neurosci. 33, 11227–11238 (2013).

  179. 179.

    Kim, H. F. & Hikosaka, O. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79, 1001–1010 (2013).

  180. 180.

    Kim, H. F., Amita, H. & Hikosaka, O. Indirect pathway of caudal basal ganglia for rejection of valueless visual objects. Neuron 94, 920–930 (2017).

  181. 181.

    Surmeier, D. J., Ding, J., Day, M., Wang, Z. & Shen, W. D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 30, 228–235 (2007).

  182. 182.

    Pawlak, V. & Kerr, J. N. D. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J. Neurosci. 28, 2435–2446 (2008).

  183. 183.

    Kravitz, A. V. et al. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626 (2010).

  184. 184.

    Bartholomew, R. A. et al. Striatonigral control of movement velocity in mice. Eur. J. Neurosci. 43, 1097–1110 (2016).

  185. 185.

    Brown, M. T. C. et al. Ventral tegmental area GABA projections pause accumbal cholinergic interneurons to enhance associative learning. Nature 492, 452–456 (2012).

  186. 186.

    Kawai, R. et al. Motor cortex is required for learning but not for executing a motor skill. Neuron 86, 800–812 (2015).

  187. 187.

    Graybiel, A. M. Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359–387 (2008).

  188. 188.

    Liljeholm, M. & O’Doherty, J. P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 16, 467–475 (2012).

  189. 189.

    Gruber, A. J. & McDonald, R. J. Context, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior. Front. Behav. Neurosci. 6, 50 (2012).

  190. 190.

    Balleine, B. W. & O’Doherty, J. P. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48–69 (2010). This paper reviews human and rodent studies investigating striatal involvement in goal-directed and habitual behaviour.

  191. 191.

    Yin, H. H. & Knowlton, B. J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464–476 (2006).

  192. 192.

    Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z. & Graybiel, A. M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161 (2005).

  193. 193.

    Yin, H. H. et al. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341 (2009).

  194. 194.

    Thorn, C. A., Atallah, H., Howe, M. & Graybiel, A. M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781–795 (2010).

  195. 195.

    Dolan, R. J. & Dayan, P. Goals and habits in the brain. Neuron 80, 312–325 (2013).

  196. 196.

    O’Hare, J. K. et al. Pathway-specific striatal substrates for habitual behavior. Neuron 89, 472–479 (2016). This paper shows correlations between performance of habitual behaviour and the strengthening of cortically evoked activity in D1R and D2R MSNs in the DLS as well as changes in the relative timing of activation of the two pathways.

  197. 197.

    Yin, H. H., Ostlund, S. B., Knowlton, B. J. & Balleine, B. W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523 (2005).

  198. 198.

    Setlow, B., Schoenbaum, G. & Gallagher, M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636 (2003).

  199. 199.

    Roitman, M. F., Wheeler, R. A. & Carelli, R. M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597 (2005).

  200. 200.

    Groenewegen, H. J., Wright, C. I., Beijer, A. V. & Voorn, P. Convergence and segregation of ventral striatal inputs and outputs. Ann. NY Acad. Sci. 877, 49–63 (1999).

  201. 201.

    Watabe-Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A. & Uchida, N. Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858–873 (2012).

  202. 202.

    Mannella, F., Gurney, K. & Baldassarre, G. The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 7, 135 (2013).

  203. 203.

    Menegas, W. et al. Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. eLife 4, e10032 (2015).

  204. 204.

    Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M. & Graybiel, A. M. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575–579 (2013).

  205. 205.

    Freeze, B. S., Kravitz, A. V., Hammack, N., Berke, J. D. & Kreitzer, A. C. Control of basal ganglia output by direct and indirect pathway projection neurons. J. Neurosci. 33, 18531–18539 (2013).

  206. 206.

    Kim, K. M. et al. Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLOS ONE 7, e33612 (2012).

Download references

Acknowledgements

The authors thank L. Pinto for comments on this manuscript and W. Fleming for providing a figure schematic. This work was funded by New York Stem Cell Foundation (NYSCF), Pew, McKnight, NARSAD (US National Alliance for Research on Schizophrenia and Depression) and Sloan Foundation grants to I.B.W.; US National Institutes of Health (NIH) grants U19 NS104648-01, DP2 DA035149-01 and 5R01MH106689-02 (to I.B.W.) and F32 MH112320-02 (to J.C.); and Army Research Office grant W911NF-17-1-0554. I.B.W. is an NYSCF–Robertson Investigator.

Reviewer information

Nature Reviews Neuroscience thanks D. Sulzer and the other, anonymous reviewers for their contribution to the peer review of this work.

Author information

Both authors researched data for the article, made substantial contributions to the discussion of content, wrote the manuscript and reviewed or edited the manuscript before submission.

Competing interests

The authors declare no competing interests.

Correspondence to Ilana B. Witten.

Glossary

Basal ganglia

An evolutionarily conserved group of interconnected subcortical nuclei that are involved in motor, cognitive and limbic processes.

Reinforcement learning

A learning process in which performance of a behaviour is modified by positive or negative feedback.

Medial forebrain bundle

A white-matter tract that contains dopaminergic axons travelling from the ventral tegmental area and substantia nigra pars compacta to the striatum.

Stimulus–outcome associations

Associations between sensory stimuli and the outcomes they predict, which induce conditioned behaviours, although experience of the outcome is independent of that behaviour.

Stimulus–response associations

Associations that result in the performance of actions in response to sensory stimuli, regardless of the value of the outcomes of the actions.

Action–outcome associations

Associations between actions (or responses) and the outcomes of those actions, the performance of which depends on the value of the outcomes.

Probabilistic reversal learning task

A behavioural task in which participants learn associations between actions and reward probabilities that are then reversed, requiring updating of learned associations.

Conditioned place preference

(CPP). An assay for measuring context-reward associations that evaluates how much time animals spend in a spatial location associated with a particular stimulus.

Devaluation test

A measurement of performance of an action with a learned outcome that becomes devalued (for example, with satiety) to assess whether a behaviour is more goal-directed or habitual.

Cost–benefit comparison

A comparison between actions that are associated with both a benefit (such as reward) and a cost (such as punishment).

Rights and permissions

Reprints and Permissions

About this article

Fig. 1: Heterogeneity of midbrain dopamine neurons.
Fig. 2: Direct and indirect pathway regulation of behaviour.
Fig. 3: Cholinergic interneurons modulate synaptic plasticity and cocaine context extinction learning.
Fig. 4: Glutamatergic inputs to the striatum.