Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Stress-sensitive inference of task controllability

Nature Human Behaviour volume 6, pages 812–822 (2022)Cite this article

Subjects

Abstract

Estimating the controllability of the environment enables agents to better predict upcoming events and decide when to engage controlled action selection. How does the human brain estimate controllability? Trial-by-trial analysis of choices, decision times and neural activity in an explore-and-predict task demonstrate that humans solve this problem by comparing the predictions of an ‘actor’ model with those of a reduced ‘spectator’ model of their environment. Neural blood oxygen level-dependent responses within striatal and medial prefrontal areas tracked the instantaneous difference in the prediction errors generated by these two statistical learning models. Blood oxygen level-dependent activity in the posterior cingulate, temporoparietal and prefrontal cortices covaried with changes in estimated controllability. Exposure to inescapable stressors biased controllability estimates downward and increased reliance on the spectator model in an anxiety-dependent fashion. Taken together, these findings provide a mechanistic account of controllability inference and its distortion by stress exposure.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Theoretical framework and experimental protocol.
Fig. 2: Behavioural performance.
Fig. 3: Computational modelling and decision times.
Fig. 4: Neuroimaging.
Fig. 5: Stress experiment.

Data availability

The experimental paradigm and the code used to generate the figures are available at the following address: https://github.com/romainligneul/NHBcontrollability. Second-level SPM images are available at https://neurovault.org/collections/8810/. Anonymized data can be accessed using ORCID identification at https://data.donders.ru.nl/collections/di/dccn/DSC_3017049.01_905.

Code availability

The scripts used to collect and analyse data are available upon publication at https://github.com/romainligneul/NHBcontrollability.

References

  1. Bastos, A. M. et al. Canonical microcircuits for predictive coding. Neuron 76, 695–711 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Gläscher, J., Daw, N., Dayan, P. & O’Doherty, J. P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Rao, R. P. N. & Ballard, D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).

    Article  CAS  PubMed  Google Scholar 

  4. Niv, Y. Learning task-state representations. Nat. Neurosci. 22, 1544–1553 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Kim, D., Park, G. Y., O′Doherty, J. P. & Lee, S. W. Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning. Nat. Commun. 10, 5738 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Hamid, A. A., Frank, M. J. & Moore, C. I. Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment. Cell 184, 2733–2749 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Moscarello, J. M. & Hartley, C. A. Agency and the calibration of motivated behavior. Trends Cogn. Sci. 21, 725–735 (2017).

    Article  PubMed  Google Scholar 

  8. Maier, S. F. & Seligman, M. E. P. Learned helplessness at fifty: insights from neuroscience. Psychol. Rev. 123, 349–367 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cheng, C., Cheung, S. F., Chio, J. H. & Chan, M.-P. S. Cultural meaning of perceived control: a meta-analysis of locus of control and psychological symptoms across 18 cultural regions. Psychol. Bull. 139, 152–188 (2013).

    Article  PubMed  Google Scholar 

  10. Hammack, S. E., Cooper, M. A. & Lezak, K. R. Overlapping neurobiology of learned helplessness and conditioned defeat: implications for PTSD and mood disorders. Neuropharmacology 62, 565–575 (2012).

    Article  CAS  PubMed  Google Scholar 

  11. Harrow, M., Hansford, B. G. & Astrachan-Fletcher, E. B. Locus of control: relation to schizophrenia, to recovery, and to depression and psychosis — A 15-year longitudinal study. Psychiatry Res. 168, 186–192 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Gillan, C. M. et al. Obsessive–compulsive disorder patients have a reduced sense of control on the illusion of control task. Front. Psychol. 5, 204 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  13. Diener, C., Kuehner, C., Brusniak, W., Struve, M. & Flor, H. Effects of stressor controllability on psychophysiological, cognitive and behavioural responses in patients with major depression and dysthymia. Psychol. Med. 39, 77–86 (2009).

    Article  CAS  PubMed  Google Scholar 

  14. Amat, J. et al. Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat. Neurosci. 8, 365–371 (2005).

    Article  CAS  PubMed  Google Scholar 

  15. Bland, S. T. et al. Stressor controllability modulates stress-induced dopamine and serotonin efflux and morphine-induced serotonin efflux in the medial prefrontal cortex. Neuropsychopharmacology 28, 1589–1596 (2003).

    Article  CAS  PubMed  Google Scholar 

  16. Challis, C. et al. Raphe GABAergic neurons mediate the acquisition of avoidance after social defeat. J. Neurosci. 33, 13978–13988 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Kerr, D. L., McLaren, D. G., Mathy, R. M. & Nitschke, J. B. Controllability modulates the anticipatory response in the human ventromedial prefrontal cortex. Front. Psychol. 3, 557 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Wood, K. H. et al. Controllability modulates the neural response to predictable but not unpredictable threat in humans. NeuroImage 119, 371–381 (2015).

    Article  PubMed  Google Scholar 

  19. Alvarez, R. P. et al. Increased anterior insula activity in anxious individuals is linked to diminished perceived control. Transl. Psychiatry 5, e591–e591 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Bräscher, A.-K., Becker, S., Hoeppli, M.-E. & Schweinhardt, P. Different brain circuitries mediating controllable and uncontrollable pain. J. Neurosci. 36, 5013–5025 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  21. Haggard, P. Sense of agency in the human brain. Nat. Rev. Neurosci. 18, 196–207 (2017).

    Article  CAS  PubMed  Google Scholar 

  22. Kühn, S., Brass, M. & Haggard, P. Feeling in control: neural correlates of experience of agency. Cortex 49, 1935–1942 (2013).

    Article  PubMed  Google Scholar 

  23. Spengler, S., von Cramon, D. Y. & Brass, M. Was it me or was it you? How the sense of agency originates from ideomotor learning revealed by fMRI. NeuroImage 46, 290–298 (2009).

    Article  PubMed  Google Scholar 

  24. Liljeholm, M., Wang, S., Zhang, J. & O’Doherty, J. P. Neural correlates of the divergence of instrumental probability distributions. J. Neurosci. 33, 12519–12527 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Vejmelka, M. & Paluš, M. Inferring the directionality of coupling with conditional mutual information. Phys. Rev. E 77, 026214 (2008).

    Article  Google Scholar 

  26. Barnett, L., Barrett, A. B. & Seth, A. K. Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103, 238701 (2009).

    Article  PubMed  Google Scholar 

  27. Pearl, J. Interpretation and identification of causal mediation. Psychol. Methods 19, 459–481 (2014).

    Article  PubMed  Google Scholar 

  28. Dorfman, H. M. & Gershman, S. J. Controllability governs the balance between Pavlovian and instrumental action selection. Nat. Commun. 10, 5826 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Ly, V., Wang, K. S., Bhanji, J. & Delgado, M. R. A reward-based framework of perceived control. Front. Neurosci. 13, 65 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Huys, Q. J. M. & Dayan, P. A. Bayesian formulation of behavioral control. Cognition 113, 314–328 (2009).

    Article  PubMed  Google Scholar 

  31. Lee, S. W., Shimojo, S. & O’Doherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687–699 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A. & Daw, N. D. Working-memory capacity protects model-based learning from stress. Proc. Natl Acad. Sci. USA 110, 20941–20946 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Shahar, N. et al. Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling. PLoS Comput. Biol. 15, e1006803 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Shahar, N. et al. Credit assignment to state-independent task representations and its relationship with model-based decision making. Proc. Natl Acad. Sci. USA 116, 15871–15876 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Voss, M., Chambon, V., Wenke, D., Kühn, S. & Haggard, P. In and out of control: brain mechanisms linking fluency of action selection to self-agency in patients with schizophrenia. Brain 140, 2226–2239 (2017).

    Article  PubMed  Google Scholar 

  36. Maier, S. F. & Seligman, M. E. Learned helplessness: theory and evidence. J. Exp. Psychol. Gen. 105, 3–46 (1976).

    Article  Google Scholar 

  37. Miller, K. J., Botvinick, M. M. & Brody, C. D. Dorsal hippocampus contributes to model-based planning. Nat. Neurosci. 20, 1269–1276 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Westbrook, A. et al. Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work. Science 367, 1362–1366 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Hahn, A. et al. Reconfiguration of functional brain networks and metabolic cost converge during task performance. eLife 9, e52443 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. O’Doherty, J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

    Article  PubMed  Google Scholar 

  42. Garrison, J., Erdeniz, B. & Done, J. Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies. Neurosci. Biobehav. Rev. 37, 1297–1310 (2013).

    Article  PubMed  Google Scholar 

  43. Grogan, J. P., Sandhu, T. R., Hu, M. T. & Manohar, S. G. Dopamine promotes instrumental motivation, but reduces reward-related vigour. eLife 9, e58321 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Weiss, A., Chambon, V., Lee, J. K., Drugowitsch, J. & Wyart, V. Interacting with volatile environments stabilizes hidden-state inference and its brain signatures. Nat. Commun. 12, 2228 (2019).

    Article  Google Scholar 

  45. O’Callaghan, C., Vaghi, M. M., Brummerloh, B., Cardinal, R. N. & Robbins, T. W. Impaired awareness of action-outcome contingency and causality during healthy ageing and following ventromedial prefrontal cortex lesions. Neuropsychologia 128, 282–289 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Kim, D., Park, G. J., O’Doherty, J.-D. & Lee, S. W. Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning. Nat. Commun. 10, 5738 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Boureau, Y.-L., Sokol-Hessner, P. & Daw, N. D. Deciding how to decide: self-control and meta-decision making. Trends Cogn. Sci. 19, 700–710 (2015).

    Article  PubMed  Google Scholar 

  48. Piray, P., Toni, I. & Cools, R. Human choice strategy varies with anatomical projections from ventromedial prefrontal cortex to medial striatum. J. Neurosci. 36, 2857–2867 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Wanke, N. & Schwabe, L. Subjective uncontrollability over aversive events reduces working memory performance and related large-scale network interactions. Cereb. Cortex 30, 3116–3129 (2019).

    Article  Google Scholar 

  50. Preuschoff, K., Quartz, S. R. & Bossaerts, P. Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 28, 2745–2752 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Behrens, T. E. J., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. S. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  PubMed  Google Scholar 

  52. Albares, M. et al. The dorsal medial frontal cortex mediates automatic motor inhibition in uncertain contexts: evidence from combined fMRI and EEG studies. Hum. Brain Mapp. 35, 5517–5531 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  53. van Belle, J., Vink, M., Durston, S. & Zandbelt, B. B. Common and unique neural networks for proactive and reactive response inhibition revealed by independent component analysis of functional MRI data. NeuroImage 103, 65–74 (2014).

    Article  PubMed  Google Scholar 

  54. Shenhav, A., Cohen, J. D. & Botvinick, M. M. Dorsal anterior cingulate cortex and the value of control. Nat. Neurosci. 19, 1286–1291 (2016).

    Article  CAS  PubMed  Google Scholar 

  55. Leibfried, F., Pascual-Diaz, S. & Grau-Moya, J. A unified Bellman optimality principle combining reward maximization and empowerment. Preprint at https://arxiv.org/abs/1907.12392 (2020).

  56. Mohamed, S. & Jimenez Rezende, D. Variational information maximisation for intrinsically motivated reinforcement learning. Adv. Neural Inf. Process. Syst. 28, 2125–2133 (2015).

    Google Scholar 

  57. Dickerson, S. S. & Kemeny, M. E. Acute stressors and cortisol responses: a theoretical integration and synthesis of laboratory research. Psychol. Bull. 130, 355–391 (2004).

    Article  PubMed  Google Scholar 

  58. Daunizeau, J., Adam, V. & Rigoux, L. VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data. PLoS Comput. Biol. 10, e1003441 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  59. Hebart, M. N., Görgen, K. & Haynes, J.-D. The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data. Front. Neuroinform. 8, 88 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank M. Frank for his constructive comments on the manuscript and computational models. We thank P. Gaalman for help with fMRI data acquisition. We thank K. Elefteriadou, F. Dianellou, K. Koelbel and J. Breen and SOLO labsupport Leiden University for their help and support with data acquisition for the stress experiment. This work was supported by grants from the Fyssen Foundation and the Behaviour and Brain Research Foundation awarded to R.L. (Young Investigator 2017) and a Vici award from the Netherlands Organisation for Scientific Research to R.C. (NWO 453-14-005). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Author notes

  1. These authors contributed equally: Verena Ly, Roshan Cools.

Authors and Affiliations

  1. Champalimaud Research, Champalimaud Centre for the Unknown, Lisbon, Portugal

    Romain Ligneul & Zachary F. Mainen

  2. Donders Institute for Brain, Cognition and Behaviour, Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, the Netherlands

    Romain Ligneul, Verena Ly & Roshan Cools

  3. Department of Clinical Psychology, Institute of Psychology Office, Leiden Institute for Brain and Cognition, Leiden University, Leiden, the Netherlands

    Verena Ly

  4. Department of Psychiatry, Radboud University Medical Centre, Nijmegen, the Netherlands

    Roshan Cools

Authors
  1. Romain Ligneul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zachary F. Mainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Verena Ly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roshan Cools
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: R.L., R.C. and V.L. Methodology: R.L. and V.L. Software and formal analysis: R.L. Investigation: R.L. and V.L. Resources: R.C. and Z.M. Data curation: R.L. and V.L. Writing original draft: R.L. Writing review and editing: R.L., Z.M., V.L. and R.C. Visualization: R.L. Funding acquisition: R.L., Z.M., V.L. and R.C.

Corresponding authors

Correspondence to Romain Ligneul or Roshan Cools.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Peer review

Peer review information

Nature Human Behaviour thanks Sang Wan Lee and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information

Supplementary notes, methods, Figs. 1–5 and Tables 1–6.

Reporting summary.

Peer review information.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ligneul, R., Mainen, Z.F., Ly, V. et al. Stress-sensitive inference of task controllability. Nat Hum Behav 6, 812–822 (2022). https://doi.org/10.1038/s41562-022-01306-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-022-01306-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing