A novel multidimensional reinforcement task in mice elucidates sex-specific behavioral strategies

Abstract

A large body of work has focused on understanding stimulus-driven behavior, sex differences in these processes, and the neural circuits underlying them. Many preclinical mouse models present rewarding or aversive stimuli in isolation, ignoring that ethologically, reward seeking requires the consideration of potential aversive outcomes. In addition, the context (or reinforcement schedule under) in which stimuli are encountered can engender different behavioral responses to the same stimulus. Thus, delineating neural control of behavior requires a dissociation between stimulus valence and stimulus-driven behavior. We developed the Multidimensional Cue Outcome Action Task (MCOAT) to dissociate motivated action from cue learning and valence in mice. First, mice acquire positive and negative reinforcement in the presence of discrete discriminative stimuli. Next, discriminative stimuli are presented concurrently allowing for parsing innate behavioral strategies based on reward seeking and avoidance. Lastly, responding in the face of punishment is assessed, thus examining  how positive and negative outcomes are relatively valued. First, we identified sex-specific behavioral strategies, showing that females prioritize avoidance of negative outcomes over seeking positive, while males have the opposite strategy. Next, we show that chemogenetically inhibiting D1 medium spiny neurons (MSNs) in the nucleus accumbens—a population that has been linked to reward-driven behavior—reduces positive and increases negative reinforcement learning rates. Thus, D1 MSNs modulate stimulus processing, rather than motivated responses or the reinforcement process itself. Together, the MCOAT has broad utility for understanding complex behaviors as well as the definition of the discrete information encoded within cellular populations.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Schematic of the Multidimensional Cue Outcome Action Task (MCOAT).
Fig. 2: Sex differences in reinforcement learning for positive and negative reinforcement.
Fig. 3: Females are biased toward shock avoidance.
Fig. 4: Extensive training on the MCOAT does not alter female bias toward avoiding aversive stimuli.
Fig. 5: Females are more sensitive to punishment.
Fig. 6: Chemogenetic inhibition of D1 MSNs in the NAc disrupts positive reinforcement and enhances negative reinforcement learning.

References

  1. 1.

    Altemus M, Sarvaiya N, Neill Epperson C. Sex differences in anxiety and depression clinical perspectives. Front Neuroendocrinol. 2014;35,320–30.

    Article  Google Scholar 

  2. 2.

    Becker JB, Hu M. Sex differences in drug abuse. Front Neuroendocrinol. 2008;29:36–47.

    CAS  Article  Google Scholar 

  3. 3.

    Calipari ES, Juarez B, Morel C, Walker DM, Cahill ME, Ribeiro E, et al. Dopaminergic dynamics underlying sex-specific cocaine reward. Nat Commun. 2017. https://doi.org/10.1038/ncomms13877.

  4. 4.

    Chowdhury, T. G., Wallin-Miller, K. G., Rear, A. A., Park, J., Diaz, V., Simon, N. W., & Moghaddam, B. Sex differences in reward-and punishment-guided actions. Cognitive, Affective, & Behavioral Neuroscience, 2019;19:1404–17.

  5. 5.

    Johnson AR, Thibeault KC, Lopez AJ, Peck EG, Sands LP, Sanders CM, et al. Cues play a critical role in estrous cycle-dependent enhancement of cocaine reinforcement. Neuropsychopharmacology. 2019. https://doi.org/10.1038/s41386-019-0320-0.

  6. 6.

    Becker JB, Chartoff E. Sex differences in neural mechanisms mediating reward and addiction. Neuropsychopharmacology. 2019;44:166–83.

    CAS  Article  Google Scholar 

  7. 7.

    Hu M, Crombag HS, Robinson TE, Becker JB. Biological Basis of Sex Differences in the Propensity to Self-administer Cocaine. Neuropsychopharmacology. 2004. 2004. https://doi.org/10.1038/sj.npp.1300301.

  8. 8.

    Dalla C, Shors TJ. Sex differences in learning processes of classical and operant conditioning. Physiol Behav. 2009. https://doi.org/10.1016/j.physbeh.2009.02.035.

  9. 9.

    Chowdhury, T. G., Wallin-Miller, K. G., Rear, A. A., Park, J., Diaz, V., Simon, N. W., & Moghaddam, B. Sex differences in reward-and punishment-guided actions. Cognitive, Affective, & Behavioral Neuroscience, 2019;19:1404–17.

    Article  Google Scholar 

  10. 10.

    Townsend EA, Negus SS, Caine SB, Thomsen M, Banks ML Sex differences in opioid reinforcement under a fentanyl vs. food choice procedure in rats. Neuropsychopharmacology. 2019. https://doi.org/10.1038/s41386-019-0356-1.

  11. 11.

    Lynch WJ, Carroll ME Sex differences in the acquisition of intravenously self-administered cocaine and heroin in rats. Psychopharmacology. 1999. https://doi.org/10.1007/s002130050979.

  12. 12.

    Lynch WJ, Roth ME, Carroll ME. Biological basis of sex differences in drug abuse: Preclinical and clinical studies. Psychopharmacology. 2002;164:121–37.

    CAS  Article  Google Scholar 

  13. 13.

    Orsini CA, Willis ML, Gilbert RJ, Bizon JL, Setlow B. Sex differences in a rat model of risky decision making. Behav Neurosci. 2016. https://doi.org/10.1037/bne0000111.

  14. 14.

    Ishii H, Onodera M, Ohara S, Tsutsui KI, Iijima T. Sex differences in risk preference and c-Fos expression in paraventricular thalamic nucleus of rats during gambling task. Front Behav Neurosci. 2018. https://doi.org/10.3389/fnbeh.2018.00068.

  15. 15.

    Van den Bos R, Jolles J, Van der Knaap L, Baars A, De Visser L. Male and female Wistar rats differ in decision-making performance in a rodent version of the Iowa Gambling Task. Behav Brain Res. 2012. https://doi.org/10.1016/j.bbr.2012.07.015.

  16. 16.

    Orsini CA, Setlow B. Sex differences in animal models of decision making. J Neurosci Res. 2017.

  17. 17.

    Jolles JW, Boogert NJ, van den Bos R. Sex differences in risk-taking and associative learning in rats. R Soc Open Sci. 2015. https://doi.org/10.1098/rsos.150485.

  18. 18.

    Calipari ES, Bagot RC, Purushothaman I, Davidson TJ, Yorgason JT, Peña CJ, et al. In vivo imaging identifies temporal signature of D1 and D2 medium spiny neurons in cocaine reward. Proc Natl Acad Sci USA. 2016. https://doi.org/10.1073/pnas.1521238113.

  19. 19.

    Branch MN. On the role of “memory” in the analysis of behavior. J Exp Anal Behav. 1977. https://doi.org/10.1901/jeab.1977.28-171.

  20. 20.

    Kangas BD, Berry MS, Branch MN. On the Development and Mechanics of Delayed Matching-to-Sample Performance. J Exp Anal Behav. 2011. https://doi.org/10.1901/jeab.2011.95-221.

  21. 21.

    Azrin NH. Punishment and recovery during fixed-ratio performance. J Exp Anal Behav. 1959;2:301.

    CAS  Article  Google Scholar 

  22. 22.

    Cahill L. Why sex matters for neuroscience. Nat Rev Neurosci. 2006;7:477–84.

    CAS  Article  Google Scholar 

  23. 23.

    Kiraly DD, Walker DM, Calipari ES, Labonte B, Issler O, Pena CJ, et al. Alterations of the host microbiome affect behavioral responses to cocaine. Sci Rep. 2016;6.

  24. 24.

    Siciliano CA. Capturing the complexity of sex differences requires multidimensional behavioral models. Neuropsychopharmacology. 2019;44:1997–8.

    Article  Google Scholar 

  25. 25.

    Zachry JE, Johnson AR, Calipari ES. Sex differences in value-based decision making underlie substance use disorders in females. Alcohol Alcohol. 2019;54:339–41.

    Article  Google Scholar 

  26. 26.

    Gu X, FitzGerald THB. Interoceptive inference: homeostasis and decision-making. Trends Cogn Sci. 2014. https://doi.org/10.1016/j.tics.2014.02.001.

  27. 27.

    Georgiou P, Zanos P, Bhat S, Tracy JK, Merchenthaler IJ, McCarthy MM, et al. Dopamine and stress system modulation of sex differences in decision making. Neuropsychopharmacology. 2018. https://doi.org/10.1038/npp.2017.161.

  28. 28.

    Brown KJ, Grunberg NE. Effects of environmental conditions on food consumption in female and male rats. Physiol Behav. 1996. https://doi.org/10.1016/0031-9384(96)00020-0.

  29. 29.

    Hong S, Flashner B, Chiu M, ver Hoeve E, Luz S, Bhatnagar SSocial isolation in adolescence alters behaviors in the forced swim and sucrose preference tests in female but not in male rats. Physiol Behav. 2012. https://doi.org/10.1016/j.physbeh.2011.08.036.

  30. 30.

    Middaugh LD, Kelley BM, Bandy ALE, McGroarty KK. Ethanol consumption by C57BL/6 mice: Influence of gender and procedural variables. Alcohol. 1999. https://doi.org/10.1016/S0741-8329(98)00055-X.

  31. 31.

    Jackson LR, Robinson TE, Becker JB Sex differences and hormonal influences on acquisition of cocaine self-administration in rats. Neuropsychopharmacology. 2006. https://doi.org/10.1038/sj.npp.1300778.

  32. 32.

    Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B. Balancing risk and reward: A rat model of risky decision making. Neuropsychopharmacology. 2009. https://doi.org/10.1038/npp.2009.48.

  33. 33.

    Lobo MK, Covington HE, Chaudhury D, Friedman AK, Sun HS, Damez-Werno D, et al. Cell type—specific loss of BDNF signaling mimics optogenetic control of cocaine reward. Science (80). 2010. https://doi.org/10.1126/science.1188472.

  34. 34.

    Soares-Cunha C, Coimbra B, Sousa N, Rodrigues AJ. Reappraising striatal D1- and D2-neurons in reward and aversion. Neurosci Biobehav Rev. 2016;68:370–86.

    CAS  Article  Google Scholar 

  35. 35.

    Harkin T, Snowe OJ, Mikulski BA, Waxman HA. US General Accounting Office: Most Drugs Withdrawn in Recent Years Had Greater Health Risks for Women. US Gen Account Off. 2001;01-286R.

  36. 36.

    Beery AK, Zucker I. Sex bias in neuroscience and biomedical research. Neurosci Biobehav Rev. 2011;35:565–72.

    Article  Google Scholar 

  37. 37.

    Lee SK. Sex as an important biological variable in biomedical research. BMB Rep. 2018;51:167–73.

    CAS  Article  Google Scholar 

Download references

Acknowledgements

This work was supported by NIH grants DA042111, DA048931 to ESC, DA045103 to CAS, GM07628 to JEZ, MH065215 and DA047777 to ARJ, MH064913 and DA050410 to KCT, and DA048436 to AJL as well as by funds from the VUMC Faculty Research Scholar Award to MGK, the Vanderbilt Academic Pathways Fellowship to LJB, Brain and Behavior Research Foundation to MGK, ESC, and CAS, the Whitehall Foundation to ESC, and the Edward Mallincrodt Jr. Foundation to ESC.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Erin S. Calipari.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kutlu, M.G., Zachry, J.E., Brady, L.J. et al. A novel multidimensional reinforcement task in mice elucidates sex-specific behavioral strategies. Neuropsychopharmacol. 45, 1463–1472 (2020). https://doi.org/10.1038/s41386-020-0692-1

Download citation

Further reading

Search