Article | Published:

Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties

Nature Neurosciencevolume 21pages10721083 (2018) | Download Citation

Subjects

Abstract

Environmental cues, through Pavlovian learning, become conditioned stimuli that guide animals toward the acquisition of rewards (for example, food) that are necessary for survival. We tested the fundamental role of midbrain dopamine neurons in conferring predictive and motivational properties to cues, independent of external rewards. We found that brief phasic optogenetic excitation of dopamine neurons, when presented in temporal association with discrete sensory cues, was sufficient to instantiate those cues as conditioned stimuli that subsequently both evoked dopamine neuron activity on their own and elicited cue-locked conditioned behavior. Notably, we identified highly parcellated functions for dopamine neuron subpopulations projecting to different regions of striatum, revealing dissociable dopamine systems for the generation of incentive value and conditioned movement invigoration. Our results indicate that dopamine neurons orchestrate Pavlovian conditioning via functionally heterogeneous, circuit-specific motivational signals to create, gate, and shape cue-controlled behaviors.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Robinson, T. E. & Berridge, K. C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Brain Res. Rev. 18, 247–291 (1993).

  2. 2.

    Dauer, W. & Przedborski, S. Parkinson’s disease: mechanisms and models. Neuron 39, 889–909 (2003).

  3. 3.

    Keiflin, R. & Janak, P. H. Dopamine prediction errors in reward learning and addiction: from theory to neural circuitry. Neuron 88, 247–263 (2015).

  4. 4.

    Steinberg, E. E. et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973 (2013).

  5. 5.

    Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. Nat. Neurosci. 19, 117–126 (2016).

  6. 6.

    Chang, C. Y. et al. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat. Neurosci. 19, 111–116 (2016).

  7. 7.

    Tsai, H.-C. et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080–1084 (2009).

  8. 8.

    Morales, M. & Margolis, E. B. Ventral tegmental area: cellular heterogeneity, connectivity and behaviour. Nat. Rev. Neurosci. 18, 73–85 (2017).

  9. 9.

    Witten, I. B. et al. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 72, 721–733 (2011).

  10. 10.

    Howe, M. W. & Dombeck, D. A. Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature 535, 505–510 (2016).

  11. 11.

    Dodson, P. D. et al. Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc. Natl. Acad. Sci. USA 113, E2180–E2188 (2016).

  12. 12.

    da Silva, J. A., Tecuapetla, F., Paixão, V. & Costa, R. M. Dopamine neuron activity before action initiation gates and invigorates future movements. Nature 554, 244–248 (2018).

  13. 13.

    Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B. & Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88 (2012).

  14. 14.

    Day, J. J., Roitman, M. F., Wightman, R. M. & Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020–1028 (2007).

  15. 15.

    Flagel, S. B. et al. A selective role for dopamine in stimulus-reward learning. Nature 469, 53–57 (2011).

  16. 16.

    Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).

  17. 17.

    Cui, G. et al. Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238–242 (2013).

  18. 18.

    Eshel, N. et al. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).

  19. 19.

    Stuber, G. D. et al. Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science 321, 1690–1692 (2008).

  20. 20.

    Berridge, K. C. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacol. (Berl.) 191, 391–431 (2007).

  21. 21.

    Haber, S. N., Fudge, J. L. & McFarland, N. R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

  22. 22.

    Belin, D. & Everitt, B. J. Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57, 432–441 (2008).

  23. 23.

    Willuhn, I., Burgeno, L. M., Everitt, B. J. & Phillips, P. E. M. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc. Natl. Acad. Sci. USA 109, 20703–20708 (2012).

  24. 24.

    Ilango, A. et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817–822 (2014).

  25. 25.

    Saunders, B. T. & Robinson, T. E. The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses. Eur. J. Neurosci. 36, 2521–2532 (2012).

  26. 26.

    Taylor, J. R., & Robbins, T. W.. Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacol. (Berl.) 84, 405–412 (1984).

  27. 27.

    Kelley, A. E. & Delfs, J. M. Dopamine and conditioned reinforcement. Psychopharmacol. (Berl.) 103, 187–196 (1991).

  28. 28.

    Di Ciano, P., Cardinal, R. N., Cowell, R. A., Little, S. J. & Everitt, B. J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J. Neurosci. 21, 9471–9477 (2001).

  29. 29.

    Swanson, L. W. The projections of the ventral tegmental area and adjacent regions: a combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 9, 321–353 (1982).

  30. 30.

    Eshel, N., Tian, J., Bukwich, M. & Uchida, N. Dopamine neurons share common response function for reward prediction error. Nat. Neurosci. 19, 479–486 (2016).

  31. 31.

    Wassum, K. M., Ostlund, S. B., Loewinger, G. C. & Maidment, N. T. Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer. Biol. Psychiatry 73, 747–755 (2013).

  32. 32.

    McClure, S. M., Daw, N. D. & Montague, P. R. A computational substrate for incentive salience. Trends Neurosci. 26, 423–428 (2003).

  33. 33.

    Sharpe, M. J. et al. Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat. Neurosci. 20, 735–742 (2017).

  34. 34.

    Dayan, P. & Berridge, K. C. Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation. Cogn. Affect. Behav. Neurosci. 14, 473–492 (2014).

  35. 35.

    Pascoli, V., Terrier, J., Hiver, A. & Lüscher, C. Sufficiency of mesolimbic dopamine neuron stimulation for the progression to addiction. Neuron 88, 1054–1066 (2015).

  36. 36.

    Burgess, C. P. et al. High-yield methods for accurate two-alternative visual psychophysics in head-fixed mice. Cell Rep. 20, 2513–2524 (2017).

  37. 37.

    Mazzoni, P., Hristova, A. & Krakauer, J. W. Why don’t we move faster? Parkinson’s disease, movement vigor, and implicit motivation. J. Neurosci. 27, 7105–7116 (2007).

  38. 38.

    Fraser, K. M. & Janak, P. H. Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking. Eur. J. Neurosci. 46, 2047–2055 (2017).

  39. 39.

    Corbit, L. H. & Janak, P. H. Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J. Neurosci. 27, 13977–13981 (2007).

  40. 40.

    Lerner, T. N. et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635–647 (2015).

  41. 41.

    Parker, N. F. et al. Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nat. Neurosci. 19, 845–854 (2016).

  42. 42.

    Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

  43. 43.

    Klockgether, T., Borutta, M., Rapp, H., Spieker, S. & Dichgans, J. A defect of kinesthesia in Parkinson’s disease. Mov. Disord. 10, 460–465 (1995).

  44. 44.

    Azulay, J. P. et al. Visual control of locomotion in Parkinson’s disease. Brain 122, 111–120 (1999).

  45. 45.

    Saddoris, M. P., Cacciapaglia, F., Wightman, R. M. & Carelli, R. M. Differential dopamine release dynamics in the nucleus accumbens core and shell reveal complementary signals for error prediction and incentive motivation. J. Neurosci. 35, 11572–11582 (2015).

  46. 46.

    Bassareo, V. & Di Chiara, G. Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments. Neuroscience 89, 637–641 (1999).

  47. 47.

    Richard, J. M. & Berridge, K. C. Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D(1) alone for appetitive eating but D(1) and D(2) together for fear. J. Neurosci. 31, 12866–12879 (2011).

  48. 48.

    Millan, E. Z., Kim, H. A. & Janak, P. H. Optogenetic activation of amygdala projections to nucleus accumbens can arrest conditioned and unconditioned alcohol consummatory behavior. Neuroscience 360, 106–117 (2017).

  49. 49.

    Margolis, E. B., Mitchell, J. M., Ishikawa, J., Hjelmstad, G. O. & Fields, H. L. Midbrain dopamine neurons: projection target determines action potential duration and dopamine D(2) receptor inhibition. J. Neurosci. 28, 8908–8913 (2008).

  50. 50.

    Yang, H. et al. Nucleus accumbens subnuclei regulate motivated behavior via direct inhibition and disinhibition of VTA dopamine subpopulations. Neuron 97, 434–449.e4 (2018).

  51. 51.

    Rothermel, M., Brunert, D., Zabawa, C., Díaz-Quesada, M. & Wachowiak, M. Transgene expression in target-defined neuron populations mediated by retrograde infection with adeno-associated viral vectors. J. Neurosci. 33, 15195–15206 (2013).

  52. 52.

    Steinberg, E. E. et al. Positive reinforcement mediated by midbrain dopamine neurons requires D1 and D2 receptor activation in the nucleus accumbens. PLoS One 9, e94771 (2014).

  53. 53.

    Margolis, E. B., Hjelmstad, G. O., Fujita, W. & Fields, H. L. Direct bidirectional μ-opioid control of midbrain dopamine neurons. J. Neurosci. 34, 14707–14716 (2014).

  54. 54.

    Klapoetke, N. C. et al. Independent optical excitation of distinct neural populations. Nat. Methods 11, 338–346 (2014).

  55. 55.

    Kim, C. K. et al. Simultaneous fast measurement of circuit dynamics at multiple sites across the mammalian brain. Nat. Methods 13, 325–328 (2016).

Download references

Acknowledgements

We thank R. Keiflin and all members of the Janak laboratory for discussion and comments on the manuscript; A. Haimbaugh, D. Acs, H. Pribut, K. Lineback, N. Pettas, B. Persaud, and L. Kinny for assistance with histology and behavioral video scoring; P. Fong for conducting surgical procedures for ex vivo physiology studies; K. Deisseroth (Stanford) for the ChR2 construct; E. Boyden (MIT) for the ChrimsonR construct; and the Janelia Research Campus GENIE Project and Stanford Gene Vector and Virus Core for the GCaMP6f construct. This work was supported by National Institutes of Health grants DA036996 (B.T.S.), DA042895 (B.T.S.), AA022290 (J.M.R.), AA025384 (J.M.R.), DA030529 (E.B.M.), and DA035943 (P.H.J.), as well as grants from the Brain and Behavior Research Foundation (B.T.S. and J.M.R.).

Author information

Author notes

    • Benjamin T. Saunders

    Present address: Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA

Affiliations

  1. Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA

    • Benjamin T. Saunders
    • , Jocelyn M. Richard
    •  & Patricia H. Janak
  2. Kavli Neuroscience Discovery Institute, Johns Hopkins School of Medicine, Baltimore, MD, USA

    • Benjamin T. Saunders
    • , Jocelyn M. Richard
    •  & Patricia H. Janak
  3. Department of Neurology, Wheeler Center for the Neurobiology of Addiction, Alcoholism and Addiction Research Group, University of California San Francisco, San Francisco, CA, USA

    • Elyssa B. Margolis
  4. Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD, USA

    • Patricia H. Janak

Authors

  1. Search for Benjamin T. Saunders in:

  2. Search for Jocelyn M. Richard in:

  3. Search for Elyssa B. Margolis in:

  4. Search for Patricia H. Janak in:

Contributions

B.T.S. and P.H.J. designed the experiments. B.T.S. collected and analyzed data from ChR2 experiments. B.T.S. and J.M.R. collected and analyzed the photometry data. E.B.M. collected and analyzed the ex vivo physiology data. B.T.S. and P.H.J. wrote the manuscript with input from all the authors.

Competing interests

The authors declare no competing interests.

Corresponding authors

Correspondence to Benjamin T. Saunders or Patricia H. Janak.

Integrated supplementary information

  1. Supplementary Figure 1 Highly specific targeting of ChR2-eYFP to TH+ neurons in TH-Cre rats.

    (a) Injection of a cre-dependent ChR2-eYFP AAV vector resulted in targeting of ChR2-eYFP to TH+ neurons in the (b) SNc and (c) VTA. (d) Targeting specificity was high (96.9%; 1027 TH+/1060 eYFP+ neurons counted) across medial/lateral and anterior/posterior sections of the midbrain.

  2. Supplementary Figure 2 Optic fiber placements.

    Coronal plates showing the location of optic fiber tips relative to Bregma for TH-cre+ and cre- control rats in the (a) ChR2 experiments, (b) fiber photometry experiments, (c) ChrimsonR intracranial self-stimulation, and (d) Projection-specific ChR2 experiments.

  3. Supplementary Figure 3 Acquisition of Pavlovian conditioned responses.

    (a) VTA and SNc cre+ paired rats learned conditioned responses during the cue period at the same rate (2-way repeated measures ANOVA, no interaction, F(3,42) = 0.691, p = 0.563). (b) Neither VTA nor SNc cre+ unpaired rats developed conditioned responses during the laser period. (c) Learning curves for individual rats in all groups during the cue period. (d) Learning curves for individual rats in all groups during the laser period. (e) The average cumulative probability of CR occurrence for VTA and SNc paired rats within 7-sec cue periods across training. VTA and SNc rats acquire CRs rapidly, and CRs emerged earlier in the cue period as training progressed. Data expressed as mean ± SEM.

  4. Supplementary Figure 4 One-second laser optogenetic Pavlovian conditioning.

    (a) TH-cre+ rats (n = 5; 3 SNc targeted, 2 VTA targeted) underwent optogenetic Pavlovian conditioning where they received 1-sec long laser presentations, delivered during the final 1 sec of each 7-sec cue. (b) Across training, these rats developed conditioned behavior (i.e., locomotion) in response to cue presentations, relative to unpaired controls, that did not differ in likelihood when compared with 5-sec laser paired conditioning groups (2-way repeated measures (RM) ANOVA, session X group interaction, F(2,32) = 66.44, p < 0.0001; post hoc comparisons with paired and unpaired groups). (c) The onset latency of conditioned behavior during cue presentations decreased across training for 1 and 5-sec paired stimulation groups (main effect of session F(1,19) = 60.96, p < 0.0001; no effect of stimulation group, F(1,19) = .0446, p = .835). Data expressed as mean ± SEM. ***p < 0.0001.

  5. Supplementary Figure 5 Fiber photometry validation and analysis.

    (a) AAVs containing cre-driven GCaMP6f and ChrimsonR were co-injected into the midbrain in TH-cre rats. (b and c) This led high GCaMP6/ChrimsonR co-expression and specificity to TH+ neurons within the recording area below optic fiber placements. (d) Delivery of 20 or 100 5-ms 590-nm laser light pulses resulted in rapid increases in GCaMP6f fluorescence (depicted as ΔF/F, the change in fluorescence during the stimulation period over baseline, n = 5 rats) that showed stable peak levels, and rapid offset. (e) Intracranial self-stimulation was used to assess the effectiveness of ChrimsonR activ ation to support behavior. (f) ChrimsonR activation via 590-nm laser delivery to dopamine neurons in the midbrain supported robust self-stimulation behavior (n = 7), measured as nose pokes, that rapidly extinguished when a 473-nm laser was substituted (2-way repeated measures ANOVA, session X response type interaction, F(2,12) = 37.27, p < 0.0001; post hoc comparisons with inactive responses). (g) Optogenetic activation of dopamine neurons was compared to activation produced by sucrose (n = 5). Delivery of 100 5-ms 590-nm laser pulses produced an overall pattern and duration of increased fluorescence that was similar to that produced by consumption of 0.1 ml of a 10% sucrose solution. (h) Example whole session trace of the demodulated 465-nm LED signal. (i) Example whole session trace of the demodulated 405-nm LED signal. (j) Trace shown in (i) after applying a least-squares fit. (k) Normalized 465 signal (ΔF/F) = (465-nm signal – fitted 405-nm signal)/(fitted 405-nm signal). Laser-evoked fluorescence is denoted by the orange bars. Data expressed as mean ± SEM. ***p < 0.0001.

  6. Supplementary Figure 6 Acquisition of conditioned approach for individual rats.

    VTA cre+ paired rats developed (a) cue approach conditioned behavior, relative to cre+ unpaired and cre- control groups. (b) No SNc cre+ paired rats developed cue approach.

  7. Supplementary Figure 7 Proximity to cue location at cue onset and stimulation hemisphere are not related to behavioral results.

    (a) The location of each rat in the experimental chamber was recorded at the onset of each cue presentation. The average location at cue onset across training was determined by assigning a value of “1” to a trial if the rat was located on the side of the chamber containing the cue light, or a value of “2” to a trial if the rat was located in the back half of the chamber at cue onset. (b) The average location at cue onset for cre+ paired VTA and SNC rats did not differ, nor did it change across training (2-way RM ANOVA, no effect of group, F(1,14) = 3.178, p = 0.0963; no interaction, F(3,42) = 0.706, p = 0.554). (c-f) The probability of locomotion, approach, and rotation during cue presentations did not differ across rats depending on the hemisphere in which they received optogenetic stimulation. Data expressed as mean ± SEM.

  8. Supplementary Figure 8 Movement tracking examples.

    (a) Behavioral videos were recorded during optogenetic Pavlovian conditioning. (b-e) Videos from the final day of conditioning were analyzed using Ethovision software to localize the frame-by-frame position of each rat’s head within the experimental chambers. Depicted are screenshots of an example rat from each experimental group, superimposed with the behavior tracking data for three cue-laser trials for that subject. Rats in the paired groups exhibited movement during the cue period (cue directed for the VTA rats, rotational for the SNc rats), while unpaired rats were immobile.

  9. Supplementary Figure 9 Similar light-evoked responses in ventral and dorsal striatal projecting dopamine neurons.

    (a) Among quiescent neurons, both NAc projectors and DS projectors showed high fidelity up to 50-Hz stimulation. Fidelity was observed in response to both 1-ms and 5-ms pulse trains. Each marker represents a cell, but some cells were tested with more than one frequency. (b) Summary of the locations of the recorded neurons in the horizontal slice. DS projectors were localized in the substantia nigra pars compacta (SNc) and NAc projectors were located in the VTA. (c) Example recording in a ChR2 expressing VTA neuron that was also firing spontaneously during recording. Although the LED light stimulation did increase the firing rate of the cell (lower panel), the increase in firing was not due to AP firing time-locked to the light pulses (upper panel). (d) Example spontaneous firing in the same cell, without light stimulation. (e) Summary of the impact of light pulses on spontaneously firing, ChR2 expressing neurons. While fidelity was moderate, stimulation did increase the firing rate in these cells.

  10. Supplementary Figure 10 Retrograde targeting of dopamine neurons reveals projection-specific expression patterns in the midbrain.

    (a) Transfection in striatum of TH-cre rats with a retrogradely-transported DIO-ChR2-eYFP construct resulted in robust expression of ChR2-eYFP in TH+ cells in the midbrain, but the expression pattern varied according to striatal target. (b) Summary of expression patterns for NAc shell (n = 5 rats, 6-10 slices per rat), NAc core (n = 4), and dorsal striatum (n = 5) projecting dopamine neurons. Shell projections were concentrated in the ventromedial VTA, while core projections were concentrated in the laterodorsal VTA. Projections to the dorsal striatum were localized throughout the SNc.

Supplementary information

  1. Supplementary Text and Figures

    Supplementary Figures 1–10

  2. Reporting Summary

  3. Supplementary Video 1

    Cue approach in a VTA Cre+ paired rat.

  4. Supplementary Video 2

    Cue presentation for a Cre+ unpaired rat.

  5. Supplementary Video 3

    Laser delivery for a Cre+ unpaired rat.

  6. Supplementary Video 4

    Cue presentation for a Cre paired rat.

  7. Supplementary Video 5

    Cue-evoked rotation in an SNc Cre+ paired rat.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41593-018-0191-4