Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Altered learning under uncertainty in unmedicated mood and anxiety disorders

Nature Human Behaviour volume 3, pages 1116–1123 (2019)Cite this article

Subjects

Matters Arising to this article was published on 18 July 2022

Abstract

Anxiety is characterized by altered responses under uncertain conditions, but the precise mechanism by which uncertainty changes the behaviour of anxious individuals is unclear. Here we probe the computational basis of learning under uncertainty in healthy individuals and individuals suffering from a mix of mood and anxiety disorders. Participants were asked to choose between four competing slot machines with fluctuating reward and punishment outcomes during safety and stress. We predicted that anxious individuals under stress would learn faster about punishments and exhibit choices that were more affected by those punishments, thus formalizing our predictions as parameters in reinforcement learning accounts of behaviour. Overall, the data suggest that anxious individuals are quicker to update their behaviour in response to negative outcomes (increased punishment learning rates). When treating anxiety, it may therefore be more fruitful to encourage anxious individuals to integrate information over longer horizons when bad things happen, rather than try to blunt their responses to negative outcomes.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Task schematic.
Fig. 2: Group difference in parameters.
Fig. 3: Sensitivity plots.
Fig. 4: Continuous symptom analysis.

Similar content being viewed by others

Data availability

All data used in this analysis are available on OSF at https://doi.org/10.17605/OSF.IO/UB6J7 (ref. 16).

Code availability

Scripts for model fitting are available on OSF at https://doi.org/10.17605/OSF.IO/UB6J7 (ref. 16) and in the Supplementary Software. For the hBayesDM package, please see https://github.com/CCS-Lab/hBayesDM.

References

  1. GBD Compare Data Visualization (IHME, accessed 17 November 2016).

  2. LeDoux, J. E. & Pine, D. S. Using neuroscience to help understand fear and anxiety: a two-system framework. Am. J. Psychiat. 173, 1083–1093 (2016).

    Article  Google Scholar 

  3. Grupe, D. W. & Nitschke, J. B. Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nat. Rev. Neurosci. 14, 488–501 (2013).

    Article  CAS  Google Scholar 

  4. Birrell, J., Meares, K., Wilkinson, A. & Freeston, M. Toward a definition of intolerance of uncertainty: a review of factor analytical studies of the Intolerance of Uncertainty Scale. Clin. Psychol. Rev. 31, 1198–1208 (2011).

    Article  Google Scholar 

  5. Charpentier, C. J., Aylward, J., Roiser, J. P. & Robinson, O. J. Enhanced risk aversion, but not loss aversion, in unmedicated pathological anxiety. Biol. Psychiat. 81, 1014–1022 (2017).

    Article  Google Scholar 

  6. Grillon, C. Models and mechanisms of anxiety: evidence from startle studies. Psychopharmacology 199, 421–437 (2008).

    Article  CAS  Google Scholar 

  7. Robinson, O. J., Overstreet, C., Allen, P. S., Pine, D. S. & Grillon, C. Acute tryptophan depletion increases translational indices of anxiety but not fear: serotonergic modulation of the bed nucleus of the stria terminalis? Neuropsychopharmacology 37, 1963–1971 (2012).

    Article  CAS  Google Scholar 

  8. Robinson, O. J. et al. The dorsal medial prefrontal (anterior cingulate) cortex–amygdala aversive amplification circuit in unmedicated generalised and social anxiety disorders: an observational study. Lancet Psychiat. 1, 294–302 (2014).

    Article  Google Scholar 

  9. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  Google Scholar 

  10. Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P. & Dolan, R. Serotonin selectively modulates reward value in human decision-making. J. Neurosci. 32, 5833–5842 (2012).

    Article  CAS  Google Scholar 

  11. Sharp, P. B. & Eldar, E. Computational models of anxiety: nascent efforts and future directions. Curr. Dir. Psychol. Sci. 28, 170–176 (2019).

    Article  Google Scholar 

  12. Robinson, O. J., Vytal, K., Cornwell, B. R. & Grillon, C. The impact of anxiety upon cognition: perspectives from human threat of shock studies. Front. Human Neurosci. 7, 203 (2013).

    Google Scholar 

  13. Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P. & Robinson, O. J. Modeling avoidance in mood and anxiety disorders using reinforcement learning. Biol. Psychiat. 82, 532–539 (2017).

    Article  Google Scholar 

  14. Gagne, C., Dayan, P. & Bishop, S. J. When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD. Curr. Opin. Behav. Sci. 24, 89–95 (2018).

    Article  Google Scholar 

  15. Monroe, S. M. & Simons, A. D. Diathesis-stress theories in the context of life stress research: implications for the depressive disorders. Psychol. Bull. 110, 406–425 (1991).

    Article  CAS  Google Scholar 

  16. Robinson, O. J. Altered learning under uncertainty in unmedicated mood and anxiety disorders—EU storage. Preprint at OSF https://doi.org/10.17605/OSF.IO/UB6J7 (2018).

  17. Gelman, A. & Rubin, D. B. Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–472 (1992).

    Article  Google Scholar 

  18. Bach, D. R. Anxiety-like behavioural inhibition is normative under environmental threat-reward correlations. PLoS Comput. Biol. 11, e1004646 (2015).

    Article  Google Scholar 

  19. Robinson, O. J., Overstreet, C., Charney, D. S., Vytal, K. & Grillon, C. Stress increases aversive prediction-error signal in the ventral striatum. Proc. Natl Acad. Sci. USA 110, 4129–4133 (2013).

    Article  CAS  Google Scholar 

  20. Deacon, B. J. & Abramowitz, J. S. Cognitive and behavioral treatments for anxiety disorders: a review of meta‐analytic findings. J. Clin. Psychol. 60, 429–441 (2004).

    Article  Google Scholar 

  21. Wilson, A., Fern, A., Ray, S. & Tadepalli, P. Multi-task reinforcement learning: a hierarchical Bayesian approach. In Proc. 24th International Conference on Machine Learning 1015−1022 (ACM, 2007).

  22. Browning, M., Behrens, T. E., Jocham, G., O’Reilly, J. X. & Bishop, S. J. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat. Neurosci. 18, 590–596 (2015).

    Article  CAS  Google Scholar 

  23. Lissek, S., Pine, D. S. & Grillon, C. The strong situation: a potential impediment to studying the psychobiology and pharmacology of anxiety disorders. Biol. Psychol. 72, 265–270 (2006).

    Article  Google Scholar 

  24. Robinson, O. J., Cools, R., Carlisi, C. O., Sahakian, B. J. & Drevets, W. C. Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive disorder. Am. J. Psychiat. 169, 152–159 (2012).

    Article  Google Scholar 

  25. Maxwell, S. E., Kelley, K. & Rausch, J. R. Sample size planning for statistical power and accuracy in parameter estimation. Annu. Rev. Psychol. 59, 537–563 (2008).

    Article  Google Scholar 

  26. Sheehan, D. et al. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur. Psychiat. 12, 232–241 (1997).

    Article  Google Scholar 

  27. Cogent 2000 Team at the FIL and the ICN. Cogent http://www.vislab.ucl.ac.uk/cogent.php (2013).

  28. Carlisi, C. O. & Robinson, O. J.. The role of prefrontal–subcortical circuitry in negative bias in anxiety: translational, developmental and treatment perspectives. Brain Neurosci. Adv. https://doi.org/10.1177/2398212818774223 (2018).

  29. Mkrtchian, A., Roiser, J. P. & Robinson, O. J. Threat of shock and aversive inhibition: induced anxiety modulates Pavlovian-instrumental interactions. J. Exp. Psychol. Gen. 146, 1694–1704 (2017).

    Article  Google Scholar 

  30. JASP Team. JASP (Version 0.7. 5.5) Google Sch. 765, 766 (2016).

  31. Ahn, W.-Y., Haines, N. & Zhang, L. Revealing neurocomputational mechanisms of reinforcement learning and decision-making with the hBayesDM package. Comp. Psychiat. 1, 24–57 (2017).

    Article  Google Scholar 

  32. Stan Development Team. RStan: the R interface to Stan. R package version 2.17.3, http://mc-stan.org/ (2018).

  33. Ahn, W.-Y., Krawitz, A., Kim, W., Busemeyer, J. R. & Brown, J. W. A model-based fMRI analysis with hierarchical Bayesian parameter estimation. J. Neurosci. Psychol. Econ. 4, 95–110 (2011).

    Article  Google Scholar 

  34. Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154–166 (2012).

    Article  Google Scholar 

  35. Huys, Q. J., Pizzagalli, D. A., Bogdan, R. & Dayan, P. Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol. Mood Anxiety Disord. 3, 12 (2013).

    Article  Google Scholar 

  36. Niv, Y. et al. Reinforcement learning in multidimensional environments relies on attention mechanisms. J. Neurosci. 35, 8145–8157 (2015).

    Article  CAS  Google Scholar 

  37. Ahn, W. Y., Busemeyer, J. R., Wagenmakers, E. J. & Stout, J. C. Comparison of decision learning models using the generalization criterion method. Cogn. Sci. 32, 1376–1402 (2008).

    Article  Google Scholar 

  38. Ahn, W.-Y. et al. Decision-making in stimulant and opiate addicts in protracted abstinence: evidence from computational modeling with pure users. Front. Psychol. 5, 849 (2014).

    Article  Google Scholar 

  39. Kruschke, J. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan (Academic, 2014).

  40. Allen, M., et al. Raincloud plots: a multi-platform tool for robust data visualization [version 1; peer review: 2 approved]. Wellcome Open Res. 4, 63 (2019).

Download references

Acknowledgements

This research was funded by a Medical Research Foundation Equipment Competition grant (no. C0497; principal investigator O.J.R.) and a Medical Research Council Career Development Award to O.J.R. (no. MR/K024280/1). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Neuroscience and Mental Health group, Institute of Cognitive Neuroscience, University College London, London, UK

    Jessica Aylward, Vincent Valton, Rebecca L. Bond, Jonathan P. Roiser & Oliver J. Robinson

  2. Department of Psychology, Seoul National University, Seoul, Korea

    Woo-Young Ahn

  3. Gatsby Computational Neuroscience Unit, University College London, London, UK

    Peter Dayan

  4. Research Department of Clinical, Educational and Health Psychology, University College London, London, UK

    Oliver J. Robinson

Authors
  1. Jessica Aylward
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent Valton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Woo-Young Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rebecca L. Bond
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Dayan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jonathan P. Roiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Oliver J. Robinson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

O.J.R., J.A. and R.L.B. conceived and designed the study and acquired the data. O.J.R., J.A., V.V., J.P.R., P.D. and W.-Y.A. analysed and interpreted the data. W.-Y.A. and O.J.R. contributed to the creation of new software used in this work. All authors drafted the Article or substantively revised it and all authors approved the Article and are individually accountable for their own contributions.

Corresponding author

Correspondence to Oliver J. Robinson.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information: Primary Handling Editor: Mary Elizabeth Sutherland.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods 1, Supplementary Results 1 and 2, Supplementary Tables 1−3 and Supplementary References.

Reporting Summary

Supplementary Software

A PDF version of the script used to fit the models reported in this paper. For the downloadable and editable script, as well as a link to the required hBayesDM package for R, please see the link in the Code availability statement.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aylward, J., Valton, V., Ahn, WY. et al. Altered learning under uncertainty in unmedicated mood and anxiety disorders. Nat Hum Behav 3, 1116–1123 (2019). https://doi.org/10.1038/s41562-019-0628-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-019-0628-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing