A selective role for dopamine in stimulus–reward learning

Abstract

Individuals make choices and prioritize goals using complex processes that assign value to rewards and associated stimuli. During Pavlovian learning, previously neutral stimuli that predict rewards can acquire motivational properties, becoming attractive and desirable incentive stimuli. However, whether a cue acts solely as a predictor of reward, or also serves as an incentive stimulus, differs between individuals. Thus, individuals vary in the degree to which cues bias choice and potentially promote maladaptive behaviour. Here we use rats that differ in the incentive motivational properties they attribute to food cues to probe the role of the neurotransmitter dopamine in stimulus–reward learning. We show that intact dopamine transmission is not required for all forms of learning in which reward cues become effective predictors. Rather, dopamine acts selectively in a form of stimulus–reward learning in which incentive salience is assigned to reward cues. In individuals with a propensity for this form of learning, reward cues come to powerfully motivate and control behaviour. This work provides insight into the neurobiology of a form of stimulus–reward learning that confers increased susceptibility to disorders of impulse control.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Development of sign-tracking versus goal-tracking CRs in bHR and bLR rats.
Figure 2: Phasic dopamine signalling in response to CS and US presentation during the acquisition of Pavlovian conditional approach behaviour in bHR and bLR rats.
Figure 3: Conditional responses and phasic dopamine signalling in response to CS and US presentation in outbred rats.
Figure 4: Dopamine is necessary for learning CS–US associations that lead to sign-tracking, but not goal-tracking.

References

  1. 1

    Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006)

    Article  Google Scholar 

  2. 2

    Wise, R. A. Dopamine, learning and motivation. Nature Rev. Neurosci. 5, 483–494 (2004)

    CAS  Article  Google Scholar 

  3. 3

    Day, J. J., Roitman, M. F., Wightman, R. M. & Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nature Neurosci. 10, 1020–1028 (2007)

    CAS  Article  Google Scholar 

  4. 4

    Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)

    CAS  Article  Google Scholar 

  5. 5

    Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996)

    CAS  Article  Google Scholar 

  6. 6

    Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001)

    ADS  CAS  Article  Google Scholar 

  7. 7

    Balleine, B. W., Daw, N. D. & O’Doherty, J. P. in Neuroeconomics: Decision Making and the Brain (eds Glimcher, P. W., Camerer, C. F., Fehr, E. & Poldrack, R. A.) 367–389 (Academic Press, 2008)

    Google Scholar 

  8. 8

    Berridge, K. C. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology 191, 391–431 (2007)

    CAS  Article  Google Scholar 

  9. 9

    Berridge, K. C. & Robinson, T. E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998)

    CAS  Article  Google Scholar 

  10. 10

    Berridge, K. C., Robinson, T. E. & Aldridge, J. W. Dissecting components of reward: 'liking', 'wanting', and learning. Curr. Opin. Pharmacol. 9, 65–73 (2009)

    CAS  Article  Google Scholar 

  11. 11

    Panksepp, J. Affective consciousness: core emotional feelings in animals and humans. Conscious. Cogn. 14, 30–80 (2005)

    Article  Google Scholar 

  12. 12

    Hearst, E. & Jenkins, H. Sign-Tracking: The Stimulus-Reinforcer Relation and Directed Action (Monograph of the Psychonomic Society, 1974)

    Google Scholar 

  13. 13

    Boakes, R. in Operant-Pavlovian Interactions (eds Davis, H. & Hurwitz, H. M. B.) 67–97 (Erlbaum, 1977)

    Google Scholar 

  14. 14

    Robinson, T. E. & Flagel, S. B. Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biol. Psychiat. 65, 869–873 (2009)

    Article  Google Scholar 

  15. 15

    Stead, J. D. et al. Selective breeding for divergence in novelty-seeking traits: heritability and enrichment in spontaneous anxiety-related behaviors. Behav. Genet. 36, 697–712 (2006)

    Article  Google Scholar 

  16. 16

    Flagel, S. B. et al. An animal model of genetic vulnerability to behavioral disinhibition and responsiveness to reward-related cues: implications for addiction. Neuropsychopharmacology 35, 388–400 (2010)

    Article  Google Scholar 

  17. 17

    Berridge, K. C. in Psychology of Learning and Motivation (ed. Medin, D. L.) 223–278 (Academic Press, 2001)

    Google Scholar 

  18. 18

    Cardinal, R. N., Parkinson, J. A., Hall, J. & Everitt, B. J. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci. Biobehav. Rev. 26, 321–352 (2002)

    Article  Google Scholar 

  19. 19

    Kelley, A. E. Functional specificity of ventral striatal compartments in appetitive behaviors. Ann. NY Acad. Sci. 877, 71–90 (1999)

    ADS  CAS  Article  Google Scholar 

  20. 20

    Dalley, J. W. et al. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc. Natl Acad. Sci. USA 102, 6189–6194 (2005)

    ADS  CAS  Article  Google Scholar 

  21. 21

    Di Ciano, P., Cardinal, R. N., Cowell, R. A., Little, S. J. & Everitt, B. J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J. Neurosci. 21, 9471–9477 (2001)

    CAS  Article  Google Scholar 

  22. 22

    Parkinson, J. A. et al. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav. Brain Res. 137, 149–163 (2002)

    CAS  Article  Google Scholar 

  23. 23

    Parkinson, J. A., Olmstead, M. C., Burns, L. H., Robbins, T. W. & Everitt, B. J. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J. Neurosci. 19, 2401–2411 (1999)

    CAS  Article  Google Scholar 

  24. 24

    Clark, J. J. et al. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nature Methods 7, 126–129 (2010)

    CAS  Article  Google Scholar 

  25. 25

    Robinson, T. E. & Berridge, K. C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Brain Res. Rev. 18, 247–291 (1993)

    CAS  Article  Google Scholar 

  26. 26

    Tindell, A. J., Smith, K. S., Berridge, K. C. & Aldridge, J. W. Dynamic computation of incentive salience: "wanting" what was never "liked". J. Neurosci. 29, 12220–12228 (2009)

    CAS  Article  Google Scholar 

  27. 27

    Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S. & Aldridge, J. W. A neural computational model of incentive salience. PLOS Comput. Biol. 5, e1000437 (2009)

    ADS  MathSciNet  Article  Google Scholar 

  28. 28

    Beckmann, J. S., Marusich, J. A., Gipson, C. D. & Bardo, M. T. Novelty seeking, incentive salience and acquisition of cocaine self-administration in the rat. Behav. Brain Res. 216, 159–165 (2011)

    CAS  Article  Google Scholar 

  29. 29

    Wyvell, C. L. & Berridge, K. C. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J. Neurosci. 20, 8122–8130 (2000)

    CAS  Article  Google Scholar 

  30. 30

    Wyvell, C. L. & Berridge, K. C. Incentive sensitization by previous amphetamine exposure: increased cue-triggered "wanting" for sucrose reward. J. Neurosci. 21, 7831–7840 (2001)

    CAS  Article  Google Scholar 

  31. 31

    Tomie, A., Aguado, A. S., Pohorecky, L. A. & Benjamin, D. Ethanol induces impulsive-like responding in a delay-of-reward operant choice procedure: impulsivity predicts autoshaping. Psychopharmacology 139, 376–382 (1998)

    CAS  Article  Google Scholar 

  32. 32

    Kuo, W. J., Sjostrom, T., Chen, Y. P., Wang, Y. H. & Huang, C. Y. Intuition and deliberation: two systems for strategizing in the brain. Science 324, 519–522 (2009)

    ADS  CAS  Article  Google Scholar 

  33. 33

    Clinton, S. M, et al. Individual differences in novelty-seeking and emotional reactivity correlate with variation in maternal behavior. Horm. Behav. 51, 655–664 (2007)

    Article  Google Scholar 

  34. 34

    Verbeke, G. &. Molenberghs, G. Linear Mixed Models for Longitudinal Data (Springer, 2000)

    Google Scholar 

  35. 35

    Heien, M. L. Johnson, M. A. & Wightman, R. M. Resolving neurotransmitters detected by fast-scan cyclic voltammetry. Anal. Chem. 76, 5697–5704 (2004)

    CAS  Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Institutes of Health grants: R01-MH079292 (to P.E.M.P.), R01-DA027858 (to P.E.M.P.), T32-DA07278 (to J.J.C.), F32-DA24540 (to J.J.C.), R37-DA04294 ( to T.E.R.), and 5P01-DA021633-02 (to T.E.R. and H.A.). The selective breeding colony was supported by a grant from the Office of Naval Research to H.A. (N00014-02-1-0879). We thank K. Berridge and J. Morrow for comments on earlier versions of the manuscript, and S. Ng-Evans for technical support.

Author information

Affiliations

Authors

Contributions

S.B.F, J.J.C., T.E.R., P.E.M.P. and H.A. designed the experiments and wrote the manuscript. S.B.F., J.J.C., L.M., A.C., I.W. and C.A.A. conducted the experiments, S.M.C. oversaw the selective breeding colony, and S.B.F. and J.J.C. analysed the data.

Corresponding authors

Correspondence to Paul E. M. Phillips or Huda Akil.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Information

The file contains Supplementary Results, additional references, Supplementary Figures 1-6 with legends and legends for Supplementary Movies 1-2. (PDF 986 kb)

Supplementary Movie 1

The movie shows a single trial of CS-US pairing for a bHR rat during the 6thPavlovian conditioning session (see Supplementary Information file for full legend). This movie file was replaced on 11 January 2011. (MOV 3660 kb)

Supplementary Movie 2

The movie shows a single trial of CS-US pairing for a bLR rat during the 6thPavlovian conditioning session (see Supplementary Information file for full legend). This movie file was replaced on 11 January 2011. (MOV 3544 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Flagel, S., Clark, J., Robinson, T. et al. A selective role for dopamine in stimulus–reward learning. Nature 469, 53–57 (2011). https://doi.org/10.1038/nature09588

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.