Separate neural substrates for skill learning and performance in the ventral and dorsal striatum

Abstract

It is widely accepted that the striatum of the basal ganglia is a primary substrate for the learning and performance of skills. We provide evidence that two regions of the rat striatum, ventral and dorsal, play distinct roles in instrumental conditioning (skill learning), with the ventral striatum being critical for learning and the dorsal striatum being important for performance but, notably, not for learning. This implies an actor (dorsal) versus director (ventral) division of labor, which is a new variant of the widely discussed actor-critic architecture. Our results also imply that the successful performance of a skill can ultimately result in its establishment as a habit outside the basal ganglia.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Effects of muscimol and AP-5 in the ventral striatum on the instrumental task.
Figure 2: Response latencies (seconds) after injections in the ventral and dorsal striatum.
Figure 3: Effects of muscimol and AP-5 in the dorsal striatum on the instrumental task.
Figure 4: The effect of muscimol in dorsal striatum on subsequent test performance.
Figure 5: The effect of pavlovian pretraining on instrumental performance.
Figure 6: The effect of pavlovian pretraining on a pavlovian test.

References

  1. 1

    Packard, M.G., Hirsh, R. & White, N.M. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9, 1465–1472 (1989).

  2. 2

    Kantak, K.M., Green-Jordan, K., Valencia, E., Kremin, T. & Eichenbaum, H.B. Cognitive task performance after lidocaine-induced inactivation of different sites within the basolateral amygdala and dorsal striatum. Behav. Neurosci. 115, 589–601 (2001).

  3. 3

    Packard, M.G. & McGaugh, J.L. Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems. Behav. Neurosci. 106, 439–446 (1992).

  4. 4

    Nakamura, K. & Hikosaka, O. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26, 5360–5369 (2006).

  5. 5

    Williams, Z.M. & Eskandar, E.N. Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat. Neurosci. 9, 562–568 (2006).

  6. 6

    Featherstone, R.E. & McDonald, R.J. Dorsal striatum and stimulus-response learning: lesions of the dorsolateral, but not dorsomedial, striatum impair acquisition of a simple discrimination task. Behav. Brain Res. 150, 15–23 (2004).

  7. 7

    Poldrack, R.A., Prabhakaran, V., Seger, C.A. & Gabrieli, J.D. Striatal activation during memory acquisition of cognitive skill. Neuropsychology 13, 564–574 (1999).

  8. 8

    Knowlton, B.J., Mangels, J.A. & Squire, L.R. A neostriatal habit learning system in humans. Science 273, 1399–1402 (1996).

  9. 9

    Hernandez, P.J., Sadeghian, K. & Kelley, A.E. Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens. Nat. Neurosci. 5, 1327–1331 (2002).

  10. 10

    O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

  11. 11

    Alexander, G.E., Crutcher, M.D. & DeLong, M.R. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, “prefrontal” and “limbic” functions. Prog. Brain Res. 85, 119–146 (1990).

  12. 12

    Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

  13. 13

    Haber, S.N., Kim, K., Mailly, P. & Calzavara, R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368–8376 (2006).

  14. 14

    Seger, C.A. & Cincotta, C.M. The role of the caudate nucleus in human classification learning. J. Neurosci. 25, 2941–2951 (2005).

  15. 15

    Corbit, L.H., Muir, J.L. & Balleine, B.W. The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between core and shell. J. Neurosci. 21, 3251–3260 (2001).

  16. 16

    Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction. (MIT Press, Cambridge, Massachusetts, USA, 1998).

  17. 17

    Houk, J.C. & Wise, S.P. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95–110 (1995).

  18. 18

    Corbit, L.H. & Balleine, B.W. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J. Neurosci. 25, 962–970 (2005).

  19. 19

    Holland, P.C. Relations between pavlovian-instrumental transfer and reinforcer devaluation. J. Exp. Psychol. Anim. Behav. Process. 30, 104–117 (2004).

  20. 20

    Hernandez, P.J., Schiltz, C.A. & Kelley, A.E. Dynamic shifts in corticostriatal expression patterns of the immediate early genes Homer 1a and Zif268 during early and late phases of instrumental training. Learn. Mem. 13, 599–608 (2006).

  21. 21

    Han, J.S., McMahan, R.W., Holland, P. & Gallagher, M. The role of an amygdalo-nigrostriatal pathway in associative learning. J. Neurosci. 17, 3913–3919 (1997).

  22. 22

    Andrzejewski, M.E., Sadeghian, K. & Kelley, A.E. Central amygdalar and dorsal striatal NMDA receptor involvement in instrumental learning and spontaneous behavior. Behav. Neurosci. 118, 715–729 (2004).

  23. 23

    Haruno, M. & Kawato, M. Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J. Neurophysiol. 95, 948–959 (2005).

  24. 24

    Rodriguez, P.F., Aron, A.R. & Poldrack, R.A. Ventral-striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Hum. Brain Mapp. 27, 306–313 (2006).

  25. 25

    Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behavior in humans. Nature 442, 1042–1045 (2006).

  26. 26

    Corlett, P.R. et al. Prediction error during retrospective revaluation of causal associations in humans: fMRI evidence in favor of associative model of learning. Neuron 44, 877–888 (2004).

  27. 27

    O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

  28. 28

    Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

  29. 29

    Miyachi, S., Hikosaka, O. & Lu, X. Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp. Brain Res. 146, 122–126 (2002).

  30. 30

    Reep, R.L., Cheatwood, J.L. & Corwin, J.V. The associative striatum: organization of cortical projections to the dorsocentral striatum in rats. J. Comp. Neurol. 467, 271–292 (2003).

  31. 31

    Cheatwood, J.L., Corwin, J.V. & Reep, R.L. Overlap and interdigitation of cortical and thalamic afferents to dorsocentral striatum in the rat. Brain Res. 1036, 90–100 (2005).

  32. 32

    Atallah, H.E., Frank, M.J. & O'Reilly, R.C. Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems. Neurobiol. Learn. Mem. 82, 253–267 (2004).

  33. 33

    Frank, M.J. Dynamic dopamine modulation of the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. J. Cogn. Neurosci. 17, 51–72 (2005).

  34. 34

    Pasupathy, A. & Miller, E.K. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433, 873–876 (2005).

  35. 35

    Dickinson, A., Balleine, B.W., Watt, A., Gonzales, F. & Boakes, R.A. Overtraining and the motivational control of instrumental action. Anim. Learn. Behav. 22, 197–206 (1995).

  36. 36

    Dickinson, A. & Balleine, B. Motivational control of goal directed action. Anim. Learn. Behav. 22, 1–18 (1994).

  37. 37

    Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).

  38. 38

    Yin, H.H., Ostlund, S.B., Knowlton, B.J. & Balleine, B.W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523 (2005).

  39. 39

    Yin, H.H., Knowlton, B.J. & Balleine, B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).

  40. 40

    Balleine, B.W. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol. Behav. 86, 717–730 (2005).

  41. 41

    O'Reilly, R.C. & Frank, M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283–328 (2006).

  42. 42

    Schultz, W. Getting formal with dopamine. Neuron 36, 241–263 (2002).

  43. 43

    Frank, M.J. & Claus, E.D. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol. Rev. 113, 300–326 (2006).

  44. 44

    Joel, D. & Weiner, I. The organization of the basal ganglia-thalamocortical circuits: open interconnected rather than closed segregated. Neuroscience 63, 363–379 (1994).

  45. 45

    Joel, D. & Weiner, I. Striatal contention scheduling and the split circuit scheme of basal ganglia-thalamocortical circuitry: from anatomy to behavior. In Brain Dynamics and the Striatal Complex (eds. Miller, R. & Wickens, J.R.) 209–236 (Harwood Academic Publishers, Amsterdam, 2000).

  46. 46

    Joel, D. & Weiner, I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451–474 (2000).

  47. 47

    Paxinos, G. & Watson, C. The Rat Brain in Stereotaxic Coordinates (Academic, San Diego, 1997).

Download references

Acknowledgements

We thank M.J. Frank, T.E. Hazy and the members of the Computational Cognitive Neuroscience laboratory for comments on this manuscript. This work was supported by Defense Advanced Research Projects Agency–Office of Naval Research N00014-05-1-0880 and US National Institutes of Health MH069597-01.

Author information

Correspondence to Hisham E Atallah or Randall C O'Reilly.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Injection sites in ventral striatum for the acquisition groups. (PDF 362 kb)

Supplementary Fig. 2

Injection sites in ventral striatum for the test groups. (PDF 368 kb)

Supplementary Fig. 3

Injection sites in dorsal striatum for the acquisition groups. (PDF 360 kb)

Supplementary Fig. 4

Injection sites in dorsal striatum for the test groups. (PDF 581 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Atallah, H., Lopez-Paniagua, D., Rudy, J. et al. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10, 126–131 (2007). https://doi.org/10.1038/nn1817

Download citation

Further reading