Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Temporally restricted dopaminergic control of reward-conditioned movements

Nature Neuroscience volume 23, pages 209–216 (2020)Cite this article

Subjects

Abstract

Midbrain dopamine (DA) neurons encode both reward- and movement-related events and are implicated in disorders of reward processing as well as movement. Consequently, disentangling the contribution of DA neurons in reinforcing versus generating movements is challenging and has led to lasting controversy. In this study, we dissociated these functions by parametrically varying the timing of optogenetic manipulations in a Pavlovian conditioning task and examining the influence on anticipatory licking before reward delivery. Inhibiting both ventral tegmental area and substantia nigra pars compacta DA neurons in the post-reward period had a significantly greater behavioral effect than inhibition in the pre-reward period of the task. Furthermore, the contribution of DA neurons to behavior decreased linearly as a function of elapsed time after reward. Together, the results indicate a temporally restricted role of DA neurons primarily related to reinforcing stimulus–reward associations and suggest that directly generating movements is a comparatively less important function.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Pre- and post-reward DA signals differentially control conditioned movements.
Fig. 2: Similar optogenetic reduction of DA neuron activity in the pre- and post-reward periods.
Fig. 3: Fiber photometry measurements of VTA DA neuron activity.
Fig. 4: Prolonged DA neuron inhibition does not compound behavioral effects.
Fig. 5: Post-reward DA signals control temporally specific cue–reward associations.
Fig. 6: Post-reward DA signals are sufficient to maintain conditioned responding during extinction.
Fig. 7: Temporal dissection of the post-reward DA signal.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request. Source data for Figs. 1–7 and Extended Data Figs. 1–10 are presented with the paper.

Code availability

Custom MATLAB code for analysis of behavior and neural activity is available from the corresponding author upon reasonable request.

References

  1. Fanselow, M. S. & Wassum, K. M. The origins and organization of vertebrate Pavlovian conditioning. Cold Spring Harb. Perspect. Biol. 8, a021717 (2016).

    PubMed Central  Google Scholar 

  2. Schultz, W. Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Saunders, B. T., Richard, J. M., Margolis, E. B. & Janak, P. H. Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nat. Neurosci. 21, 1072–1083 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  4. Everitt, B. J. & Robbins, T. W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489 (2005).

    CAS  PubMed  Google Scholar 

  5. Leventhal, D. K. et al. Dissociable effects of dopamine on learning and performance within sensorimotor striatum. Basal Ganglia 4, 43–54 (2014).

    PubMed  Google Scholar 

  6. Berke, J. D. What does dopamine mean? Nat. Neurosci. 21, 787–793 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  7. Wickens, J. Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model. J. Neural Transm. Gen. Sect. 80, 9–31 (1990).

    CAS  PubMed  Google Scholar 

  8. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    CAS  PubMed  Google Scholar 

  9. Glimcher, P. W. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl Acad. Sci. USA 108, 15647–15654 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Steinberg, E. E. et al. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Chang, C. Y. et al. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat Neurosci 19, 111–116 (2016).

    CAS  PubMed  Google Scholar 

  12. Day, J. J., Roitman, M. F., Wightman, R. M. & Carelli, R. M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020–1028 (2007).

    CAS  PubMed  Google Scholar 

  13. Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. Nat. Neurosci. 19, 117–126 (2016).

    CAS  PubMed  Google Scholar 

  14. Wise, R. A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).

    CAS  PubMed  Google Scholar 

  15. Berridge, K. C. & Robinson, T. E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998).

    CAS  PubMed  Google Scholar 

  16. Dodson, P. D. et al. Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc. Natl Acad. Sci. USA 113, E2180–E2188 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Howe, M. W. & Dombeck, D. A. Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature 535, 505–510 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. da Silva, J. A., Tecuapetla, F., Paixao, V. & Costa, R. M. Dopamine neuron activity before action initiation gates and invigorates future movements. Nature 554, 244–248 (2018).

    PubMed  Google Scholar 

  19. Barter, J. W. et al. Beyond reward prediction errors: the role of dopamine in movement kinematics. Front. Integr. Neurosci. 9, 39 (2015).

    PubMed  PubMed Central  Google Scholar 

  20. Coddington, L. T. & Dudman, J. T. The timing of action determines reward prediction signals in identified midbrain dopamine neurons. Nat. Neurosci. 21, 1563–1573 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Syed, E. C. et al. Action initiation shapes mesolimbic dopamine encoding of future rewards. Nat. Neurosci. 19, 34–36 (2016).

    CAS  PubMed  Google Scholar 

  22. Engelhard, B. et al. Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons. Nature 570, 509–513 (2019).

  23. Zhu, Y. et al. Dynamic salience processing in paraventricular thalamus gates associative learning. Science 362, 423–429 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B. & Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  25. van Zessen, R., Phillips, J. L., Budygin, E. A. & Stuber, G. D. Activation of VTA GABA neurons disrupts reward consumption. Neuron 73, 1184–1194 (2012).

    PubMed  PubMed Central  Google Scholar 

  26. Xiao, C. et al. Cholinergic mesopontine signals govern locomotion and reward through dissociable midbrain pathways. Neuron 90, 333–347 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  27. Kim, H. F., Ghazizadeh, A. & Hikosaka, O. Dopamine neurons encoding long-term memory of object value for habitual behavior. Cell 163, 1165–1175 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Parker, N. F. et al. Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nat. Neurosci. 19, 845–854 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Komiyama, T. et al. Learning-related fine-scale specificity imaged in motor cortex circuits of behaving mice. Nature 464, 1182–1186 (2010).

    CAS  PubMed  Google Scholar 

  30. Gunaydin, L. A. et al. Natural neural projection dynamics underlying social behavior. Cell 157, 1535–1551 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Markowitz, J. E. et al. The striatum organizes 3D behavior via moment-to-moment action selection. Cell 174, 44–58 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Chang, C. Y., Gardner, M. P. H., Conroy, J. C., Whitaker, L. R. & Schoenbaum, G. Brief, but not prolonged, pauses in the firing of midbrain dopamine neurons are sufficient to produce a conditioned inhibitor. J. Neurosci. 38, 8822–8830 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  33. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).

    CAS  PubMed  Google Scholar 

  34. Sharpe, M. J. et al. Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat. Neurosci. 20, 735–742 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Fischbach, S. & Janak, P. H. Decreases in cued reward seeking after reward-paired inhibition of mesolimbic dopamine. Neuroscience 412, 259–269 (2019).

    CAS  PubMed  Google Scholar 

  36. Albin, R. L., Young, A. B. & Penney, J. B. The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366–375 (1989).

    CAS  PubMed  Google Scholar 

  37. Howard, C. D., Li, H., Geddes, C. E. & Jin, X. Dynamic nigrostriatal dopamine biases action selection. Neuron 93, 1436–1450 (2017). e1438.

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Ilango, A. et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817–822 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Wise, R. A. Roles for nigrostriatal—not just mesocorticolimbic—dopamine in reward and addiction. Trends Neurosci. 32, 517–524 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Tian, J. et al. Distributed and mixed information in monosynaptic inputs to dopamine neurons. Neuron 91, 1374–1389 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Liljeholm, M. & O’Doherty, J. P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 16, 467–475 (2012).

    PubMed  PubMed Central  Google Scholar 

  42. de Jong, J. W. et al. A neural circuit mechanism for encoding aversive stimuli in the mesolimbic dopamine system. Neuron 101, 133–151 (2019).

    PubMed  Google Scholar 

  43. Lerner, T. N. et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635–647 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  44. Lammel, S., Ion, D. I., Roeper, J. & Malenka, R. C. Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855–862 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Menegas, W., Akiti, K., Amo, R., Uchida, N. & Watabe-Uchida, M. Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli. Nat. Neurosci. 21, 1421–1430 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815–834 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Soares, S., Atallah, B. V. & Paton, J. J. Midbrain dopamine neurons control judgment of time. Science 354, 1273–1277 (2016).

    CAS  PubMed  Google Scholar 

  48. Maes, E. J., Sharpe, M. J., Gardner, M. P. H., Chang, C. Y., Schoenbaum, G. & Iordanova, M. D. Causal evidence supporting the proposal that dopamine transients function as a temporal difference prediction error. Preprint at bioRxiv https://doi.org/10.1101/520965 (2019).

  49. Reynolds, J. N., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001).

    CAS  PubMed  Google Scholar 

  50. Yagishita, S. et al. A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science 345, 1616–1620 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  51. Backman, C. M. et al. Characterization of a mouse strain expressing Cre recombinase from the 3′ untranslated region of the dopamine transporter locus. Genesis 44, 383–390 (2006).

    CAS  PubMed  Google Scholar 

  52. Gradinaru, V. et al. Molecular and cellular approaches for diversifying and extending optogenetics. Cell 141, 154–165 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  53. Klapoetke, N. C. et al. Independent optical excitation of distinct neural populations. Nat. Methods 11, 338–346 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Chen, T. W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  55. Yang, L., Lee, K., Villagracia, J. & Masmanidis, S. C. Open source silicon microprobes for high throughput neural recording. J. Neural Eng. https://doi.org/10.1088/1741-2552/ab581a (2019).

    PubMed  PubMed Central  Google Scholar 

  56. Pachitariu, M., Steinmetz, N., Kadir, S., Carandini, M. & Harris, K. D. Kilosort: realtime spike-sorting for extracellular electrophysiology with hundreds of channels. Preprint at bioRxiv https://doi.org/10.1101/061481 (2016).

Download references

Acknowledgements

We thank C.D. Fiorillo for valuable discussions, T.J. Davidson for technical assistance with photometry and the investigators who shared resources, including viruses for optogenetics and calcium imaging, as well as DAT-Cre mice. S.C.M. was supported by National Institutes of Health grants NS100050, NS096994, DA042739 and DA005010 and National Science Foundation NeuroNex Award 1707408.

Author information

Author notes

  1. Leslie D. Claar

    Present address: Allen Institute for Brain Science, Seattle, WA, USA

  2. Konstantin I. Bakhurin

    Present address: Department of Psychology & Neuroscience, Duke University, Durham, NC, USA

  3. These authors contributed equally: Kwang Lee, Leslie D. Claar.

Authors and Affiliations

  1. Department of Neurobiology, University of California, Los Angeles, Los Angeles, CA, USA

    Kwang Lee, Ayaka Hachisuka, Jacquelyn Nguyen & Sotiris C. Masmanidis

  2. Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA

    Leslie D. Claar

  3. Neuroscience Interdepartmental Program, University of California, Los Angeles, Los Angeles, CA, USA

    Konstantin I. Bakhurin & Jeremy M. Trott

  4. Medical Scientist Training Program, University of California, Los Angeles, Los Angeles, CA, USA

    Jay L. Gill

  5. California Nanosystems Institute, University of California, Los Angeles, Los Angeles, CA, USA

    Sotiris C. Masmanidis

Authors
  1. Kwang Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leslie D. Claar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ayaka Hachisuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konstantin I. Bakhurin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacquelyn Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jeremy M. Trott
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jay L. Gill
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sotiris C. Masmanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

K.L. and L.D.C. designed and carried out experiments, analyzed data and wrote the manuscript. K.I.B., A.H., J.N., J.M.T. and J.L.G. carried out experiments and analyzed data. S.C.M. conceived the project, designed experiments, analyzed data and wrote the manuscript.

Corresponding author

Correspondence to Sotiris C. Masmanidis.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Neuroscience thanks Naoshige Uchida and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Optogenetic inhibition of spontaneous activity in the VTA, in the absence of behavior.

(a) (Top) Illustration of recording with a 64 electrode silicon microprobe during optogenetic inhibition of VTA DA neurons. (Bottom) Silicon microprobe with attached optical fiber under bright field and laser illumination. Scale bars: 0.2 mm. (b) Spike raster (top) and mean firing rate versus time (bottom) of two VTA neurons in response to optogenetic inhibition. Orange bar indicates the duration of the laser stimulus. Insets show the spike waveform of each unit in blue (scale bar: 1 ms). Note the rebound excitation effect by the left unit. (c) Firing rate modulation index distribution of 142 VTA neurons from 4 mice in response to optogenetic inhibition. The mean value was significantly less than zero (two-sided paired t-test, t141 = –5.6, P < 0.0001). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 2 Effect of post-reward VTA DA neuron inhibition on anticipatory licking.

(a) (Left) Trial structure of Pavlovian conditioning task with post-reward VTA DA inhibition. (Right) Mean lick rate versus time for NpHR3.0 (n = 18) and YFP (n = 14) expressing animals undergoing post-reward inhibition. Black and green lines represent block 1 (trials 1 – 40, laser off) and block 2 (trials 41 – 80, laser on), respectively. (b) (Left) During post-reward inhibition, anticipatory lick number was significantly reduced in the laser block compared to controls (n = 18 eNpHR3.0+ and 14 YFP+ mice, two-way RM ANOVA, group effect: F1,30 = 2.4, P = 0.13, trial block effect: F2,60 = 24.8, P < 0.0001). Post-hoc Sidak’s test: **P = 0.003. (Right) Anticipatory lick onset time was not significantly altered in the laser block relative to controls (two-way RM ANOVA, group effect: F1,30 = 0.0001, P = 0.99, trial block effect: F2,60 = 9.2, P = 0.0003). (c) Mean number of anticipatory licks per animal (n = 18 eNpHR3.0+ and 14 YFP+ mice) as a function of trial number, for post-reward laser stimulation. Data are normalized to the mean lick count in the first trial block corresponding to laser off. Data are aligned to the start of the third trial block. Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 3 Reward size reduction resembles post-reward DA neuron inhibition.

(a) (Top) Schematic illustration the test using reduced reward size. The reward was reduced from 5 µL in block 1 to 2 or 3 µL in block 2, then reinstated to 5 µL in block 3. (Middle and bottom) Mean lick rate versus time for a reduced reward of 2 and 3 µL (n = 7 mice). Black and green lines represent trial blocks 1 and 2, respectively. (b) Anticipatory lick probability in each of the three trial blocks for the two reduced reward conditions (n = 7 mice, two-way RM ANOVA, reward size effect: F1,6 = 34.2, P = 0.001, trial block effect: F2,12 = 27, P < 0.0001. Post-hoc Sidak’s test: ****P < 0.0001. (c) Mean normalized number of anticipatory licks per animal as a function of trial number for the two reduced reward size conditions. Left plot shows data aligned to the start of block 2, and right plot shows data aligned to the start of block 3. (d) Fractional change in anticipatory licking probability as a function of reward size in block 2 (n = 7 mice, one-way RM ANOVA, F = 37.6, P < 0.0001). Post-hoc Tukey’s test: 2 vs 3 µL P = 0.002, 2 vs 5 µL P = 0.0004, 3 vs 5 µL P = 0.09. Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 4 Comparison of pre and post-reward VTA DA neuron inhibition on behavior.

(a) (Left) Trial structure of Pavlovian conditioning task with pre-reward VTA DA inhibition. (Right) Mean lick rate versus time for NpHR3.0 (n = 18) and YFP (n = 14) expressing animals undergoing pre-reward inhibition. Black and green lines represent trial blocks 1 and 2, respectively. (b) (Left) Anticipatory lick number in each of the three trial blocks during pre-reward inhibition (n = 18 eNpHR3.0+ and 14 YFP+ mice, two-way RM ANOVA, group effect: F1,30 = 0.005, P = 0.94, trial block effect: F2,60 = 23.3, P < 0.0001). (Right) Anticipatory lick onset time in each of the three trial blocks during pre-reward inhibition (two-way RM ANOVA, group effect: F1,30 = 0.1, P = 0.73, trial block effect: F2,60 = 30.1, P < 0.0001). (c) Mean normalized number of anticipatory licks per animal as a function of trial number during pre-reward VTA DA inhibition (n = 7 mice). Left plot shows data aligned to the start of block 2, and right plot shows data aligned to the start of block 3. (d) The fractional change in anticipatory lick number was significantly more reduced by post-reward VTA DA inhibition (n = 18 eNpHR3.0+ mice, two-sided paired t-test, t17 = 5.6, ****P < 0.0001). (e) Control experiments with YFP expression. (Left) Fractional change in anticipatory lick probability caused by pre- and post-reward inhibition (n = 14 YFP+ mice, two-sided paired t-test, t13 = 0.9, P = 0.37). (Right) Fractional change in anticipatory lick number caused by pre- and post-reward inhibition (two-sided paired t-test, t13 = 1.9, P = 0.074). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 5 Effect of SNc DA neuron inhibition with random laser trial schedule.

(a) (Top) Trial structure of a Pavlovian reward conditioning task, in which pre-reward laser stimulation was given to SNc DA neurons on random trials (50 %) rather than in a continuous block of trials as with other experiments in this study. (Bottom left) The probability of generating anticipatory licks was significantly reduced on trials with laser compared to laser off trials (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 4.7, **P = 0.002). (Bottom right) The mean anticipatory lick onset time was increased on trials with laser (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 2.3, #P = 0.051). (b) Comparison of random trial versus continuous 40 trial block SNc DA neuron inhibition in the pre-reward period. (Left) Anticipatory lick probability (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 0.7, P = 0.52). (Right) Anticipatory lick number (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 0.5, P = 0.64). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 6 Comparison of pre and post-reward SNc DA neuron inhibition on behavior.

(a) (Top) Lick raster of a mouse during post-reward SNc DA inhibition. (Bottom) Lick raster of the same mouse during pre-reward SNc DA inhibition. Orange shaded area indicates timing of the laser stimulus given on trials 41 – 80. (b) Mean lick rate versus time on sessions with post-reward (top) and pre-reward (bottom) SNc DA neuron inhibition (n = 9 eNpHR3.0+ mice). Black and green lines represent trial blocks 1 and 2, respectively. (c) Mean normalized number of anticipatory licks per animal as a function of trial number during pre-reward (blue) and post-reward (red) SNc DA inhibition (n = 9 eNpHR3.0+ mice). Left plot shows data aligned to the start of block 2, and right plot shows data aligned to the start of block 3. (d) The fractional change in anticipatory lick number was significantly more reduced by post-reward SNc DA inhibition (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 2.6, *P = 0.03). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 7 Optogenetic inhibition of M2 excitatory neurons and behavioral effects.

(a) Spike raster (top) and mean spontaneous firing rate versus time (bottom) of an M2 neuron in response to optogenetic inhibition, in the absence of behavior. Orange bar indicates the duration of the laser stimulus. Inset shows the corresponding spike waveform in blue (scale bar: 1 ms). (b) Firing rate modulation index distribution of 232 M2 neurons from 3 mice in response to optogenetic inhibition. The mean value was significantly less than zero (two-sided paired t-test, t231 = –19.8, P < 0.0001). (c) Mean firing rate versus time of all M2 units during optogenetic inhibition (n = 232). (d) Histologically determined optical fiber tracks for the 9 eNpHR3.0+ mice targeting M2 for behavioral experiments in Fig. 1. Grid lines are spaced 1 mm apart. (e) Mean normalized number of anticipatory licks per animal as a function of trial number during pre-reward (blue) and post-reward (red) M2 inhibition (n = 9 eNpHR3.0+ mice). Left plot shows data aligned to the start of block 2, and right plot shows data aligned to the start of block 3. (f) Mean lick rate versus time on sessions with post-reward (top) and pre-reward (bottom) M2 neuron inhibition (n = 9 eNpHR3.0+ mice). Black and green lines represent trial blocks 1 and 2, respectively. (g) (Top) Trial structure of the Pavlovian reward conditioning task with pre-reward inhibition in M2. Orange bar indicates the timing of the laser. (Bottom) Inhibiting M2 excitatory neurons in the pre-reward period significantly reduced the anticipatory lick probability relative to controls (n = 9 eNpHR3.0+ and 9 YFP+ mice, two-way RM ANOVA, group effect: F1,16 = 1.6, P = 0.23, trial block effect: F2,32 = 4, P = 0.03). Post-hoc Sidak’s test: **P = 0.009. (h) The fractional change in anticipatory lick number was significantly more reduced by pre-reward M2 inhibition (n = 9 eNpHR3.0+ mice, two-sided paired t-test, t8 = 9.9, ****P < 0.0001). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 8 Electrophysiological recordings during optogenetic DA neuron inhibition in behaving mice.

(a) auROC time series plots of 140 cells recorded from 5 mice, after hierarchical clustering yielded three types of clusters. There were 85 type I cells (putative DA neurons), 36 Type II cells, and 19 Type III cells. (b) First three principal components of each cell’s auROC, color-coded by cluster type. (c) Mean firing rate versus time of one Type II cluster cell on laser-free trials (n = 28 trials). (d) Mean firing rate versus time of one Type III cluster cell on laser-free trials (n = 28 trials). (e) Mean baseline firing rate of cells in each type of cluster (n = 85 Type I, 36 Type II, 19 Type III, one-way ANOVA, F2,137 = 6.2, P = 0.003). Post-hoc Tukey’s test: Type I vs II P = 0.02, Type I vs III P = 0.02, Type II vs III P = 0.89. (f) (Left) The mean firing rate of Type II cells in the post-reward period was not significantly reduced by application of post-reward laser (n = 36 cells, two-sided paired t-test, t35 = 1.6, P = 0.12). (Right) The mean firing rate of Type II cells in the pre-reward period was not significantly reduced by application of pre-reward laser (n = 36 cells, two-sided paired t-test, t35 = 0.7, P = 0.52). (g) (Left) The mean firing rate of Type III cells in the post-reward period was not significantly reduced by application of post-reward laser (n = 19 cells, two-sided paired t-test, t18 = 0.4, P = 0.68). (Right) The mean firing rate of Type III cells in the pre-reward period was not significantly reduced by application of pre-reward laser (n = 19 cells, two-sided paired t-test, t18 = 1.4, P = 0.17). (h) Cumulative distribution of the fractional change in pre- and post-reward firing caused by the laser, for Type I cluster cells. Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 9 Behavioral effect of VTA DA neuron activation during reward extinction.

(a) (Top) Trial structure of a Pavlovian reward conditioning task with extinction, in which the physical reward (milk) reward was omitted and substituted for VTA DA neuron activation during the post-reward period (2 s continuous laser duration on trials 41 – 80). Reward was given on all other trials. (Bottom) Histologically determined optical fiber tracks for the 10 Chrimson+ mice targeting the VTA for behavioral experiments in Fig. 6. Grid lines are spaced 1 mm apart. (b) Mean lick rate versus time for YFP (n = 8, top) and Chrimson (n = 10, bottom) expressing animals undergoing reward extinction with 2 s continuous laser stimulation. Black and green lines represent trial blocks 1 and 2, respectively. (c) (Left) Activating VTA DA neurons during extinction maintains a higher number of anticipatory licks in the laser block compared to controls (n = 10 Chrimson+ and 8 YFP+ mice, two-way RM ANOVA, group effect: F1,16 = 0.1, P = 0.79, trial block effect: F2,32 = 18.3, P < 0.0001). Post-hoc Sidak’s test: *P = 0.036. (Right) Activating VTA DA neurons during extinction does not have a significant effect on anticipatory lick onset time compared to controls (two-way RM ANOVA, group effect: F1,16 = 0.4, P = 0.52, trial block effect: F2,32 = 6.5, P = 0.004). Data are expressed as mean ± SEM.

Source data

Extended Data Fig. 10 Similar effect of pulsed and continuous laser stimulation during reward extinction.

(a) (Top) Illustration of the pulsed laser stimulation (as opposed to 2 s continuous used in Fig. 6) protocol used to activate DA neurons during reward extinction. (Bottom) Histologically determined optical fiber tracks for the 4 Chrimson+ mice targeting the VTA. Grid lines are spaced 1 mm apart. (b) Anticipatory lick probability in each of the three trial blocks on extinction sessions with laser (blue) and without laser (black) (n = 4 Chrimson+ mice, two-way RM ANOVA, group effect: F1,3 = 30.2, P = 0.01, trial block effect: F2,6 = 40.9, P = 0.0003. Post-hoc Sidak’s test: ****P < 0.0001. (c) Mean lick rate versus time for Chrimson expressing animals undergoing reward extinction on extinction sessions without laser (top) and with pulsed laser stimulation (bottom) (n = 4 mice). Black and green lines represent trial blocks 1 and 2, respectively. (d) Fractional change in anticipatory lick probability during reward extinction experiments with 2 s continuous (n = 10 mice) and 20 Hz pulsed laser (n = 4 mice). There is no significant difference between these groups (two-sided unpaired t-test, t12 = 0.1, P = 0.91). Data are expressed as mean ± SEM.

Source data

Supplementary information

Supplementary Information

Supplementary Figs. 1–5.

Reporting Summary

Supplementary Table

Numerical source data for Supplementary Figs. 1–5.

Source data

Source Data Fig. 1

Numerical data for Fig. 1.

Source Data Fig. 2

Numerical data for Fig. 2.

Source Data Fig. 3

Numerical data for Fig. 3.

Source Data Fig. 4

Numerical data for Fig. 4.

Source Data Fig. 5

Numerical data for Fig. 5.

Source Data Fig. 6

Numerical data for Fig. 6.

Source Data Fig. 7

Numerical data for Fig. 7.

Source Data Extended Data Fig. 1

Numerical data for Extended Data Fig. 1.

Source Data Extended Data Fig. 2

Numerical data for Extended Data Fig. 2.

Source Data Extended Data Fig. 3

Numerical data for Extended Data Fig. 3.

Source Data Extended Data Fig. 4

Numerical data for Extended Data Fig. 4.

Source Data Extended Data Fig. 5

Numerical data for Extended Data Fig. 5.

Source Data Extended Data Fig. 6

Numerical data for Extended Data Fig. 6.

Source Data Extended Data Fig. 7

Numerical data for Extended Data Fig. 7.

Source Data Extended Data Fig. 8

Numerical data for Extended Data Fig. 8.

Source Data Extended Data Fig. 9

Numerical data for Extended Data Fig. 9.

Source Data Extended Data Fig. 10

Numerical data for Extended Data Fig. 10.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, K., Claar, L.D., Hachisuka, A. et al. Temporally restricted dopaminergic control of reward-conditioned movements. Nat Neurosci 23, 209–216 (2020). https://doi.org/10.1038/s41593-019-0567-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-019-0567-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing