Article | Published:

Minimizing threat via heuristic and optimal policies recruits hippocampus and medial prefrontal cortex

Abstract

Jointly minimizing multiple threats over extended time horizons enhances survival. Consequently, many tests of approach–avoidance conflicts incorporate multiple threats for probing corollaries of animal and human anxiety. To facilitate computations necessary for threat minimization, the human brain may concurrently harness multiple decision policies and associated neural controllers, but it is unclear which. We combine a task that mimics foraging under predation with behavioural modelling and functional neuroimaging. Human choices rely on immediate predator probability—a myopic heuristic policy—and on the optimal policy, which integrates all relevant variables. Predator probability relates positively and the associated choice uncertainty relates negatively to activations in the anterior hippocampus, amygdala and dorsolateral prefrontal cortex. The optimal policy is positively associated with dorsomedial prefrontal cortex activity. We thus provide a decision-theoretic outlook on the role of the human hippocampus, amygdala and prefrontal cortex in resolving approach–avoidance conflicts relevant for anxiety and integral for survival.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Data availability

The behavioural data that support the findings of this study are publicly available at github (https://github.com/dnhi-lab/minimizing_threat.git) and at figshare (https://doi.org/10.6084/m9.figshare.7929914.v1). The neuroimaging data that support the findings of this study are publicly available at neurovault (https://neurovault.org/collections/5046/).

Code availability

The code used for the analyses is available at github (https://github.com/dnhi-lab/minimizing_threat.git).

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Bach, D. R. & Dayan, P. Algorithms for survival: a comparative perspective on emotions. Nat. Rev. Neurosci. 18, 311–319 (2017).

  2. 2.

    Korn, C. W. & Bach, D. R. Heuristic and optimal policy computations in the human brain during sequential decision-making. Nat. Commun. 9, 325 (2018).

  3. 3.

    Korn, C. W. & Bach, D. R. Maintaining homeostasis by decision-making. PLoS Comput. Biol. 11, e1004301 (2015).

  4. 4.

    Huys, Q. J. M. et al. Interplay of approximate planning strategies. Proc. Natl Acad. Sci. USA 112, 3098–3103 (2015).

  5. 5.

    Huys, Q. J. M. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).

  6. 6.

    Keramati, M., Smittenaar, P., Dolan, R. J. & Dayan, P. Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum. Proc. Natl Acad. Sci. USA 113, 12868–12873 (2016).

  7. 7.

    Keramati, M., Dezfouli, A. & Piray, P. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput. Biol. 7, e1002055 (2011).

  8. 8.

    LeDoux, J. & Daw, N. D. Surviving threats: neural circuit and computational implications of a new taxonomy of defensive behaviour. Nat. Rev. Neurosci. 19, 269–282 (2018).

  9. 9.

    Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. Foraging for foundations in decision neuroscience: insights from ethology. Nat. Rev. Neurosci. 19, 419–427 (2018).

  10. 10.

    Mobbs, D. The ethological deconstruction of fear(s). Curr. Opin. Behav. Sci. 24, 32–37 (2018).

  11. 11.

    Griebel, G. & Holmes, A. 50 years of hurdles and hope in anxiolytic drug discovery. Nat. Rev. Drug Discov. 12, 667–687 (2013).

  12. 12.

    Haller, J., Aliczki, M. & Gyimesine Pelczer, K. Classical and novel approaches to the preclinical testing of anxiolytics: a critical evaluation. Neurosci. Biobehav. Rev. 37, 2318–2330 (2013).

  13. 13.

    Cryan, J. F. & Sweeney, F. F. The age of anxiety: role of animal models of anxiolytic action in drug discovery. Br. J. Pharm. 164, 1129–1161 (2011).

  14. 14.

    Kirlic, N., Young, J. & Aupperle, R. L. Animal to human translational paradigms relevant for approach avoidance conflict decision making. Behav. Res. Ther. 96, 14–29 (2017).

  15. 15.

    Calhoon, G. G. & Tye, K. M. Resolving the neural circuits of anxiety. Nat. Neurosci. 18, 1394–1404 (2015).

  16. 16.

    Gray, J. A. & McNaughton, N. The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septohippocampal System (Oxford Univ. Press, 2000).

  17. 17.

    Aupperle, R. L., Melrose, A. J., Francisco, A., Paulus, M. P. & Stein, M. B. Neural substrates of approach–avoidance conflict decision-making. Hum. Brain Mapp. 36, 449–462 (2015).

  18. 18.

    Bach, D. R. et al. Human hippocampus arbitrates approach–avoidance conflict. Curr. Biol. 24, 541–547 (2014).

  19. 19.

    Loh, E. et al. Parsing the role of the hippocampus in approach–avoidance conflict. Cereb. Cortex 27, 201–215 (2016).

  20. 20.

    O’Neil, E. B. et al. Examining the role of the human hippocampus in approach–avoidance decision making using a novel conflict paradigm and multivariate functional magnetic resonance imaging. J. Neurosci. 35, 15039–15049 (2015).

  21. 21.

    Ito, R. & Lee, A. C. H. The role of the hippocampus in approach–avoidance conflict decision-making: evidence from rodent and human studies. Behav. Brain Res. 313, 345–357 (2016).

  22. 22.

    Schlund, M. W. et al. The tipping point: value differences and parallel dorsal–ventral frontal circuits gating human approach–avoidance behavior. Neuroimage 136, 94–105 (2016).

  23. 23.

    Mobbs, D. et al. When fear is near: threat imminence elicits prefrontal-periaqueductal gray shifts in humans. Science 317, 1079–1083 (2007).

  24. 24.

    Qi, S. et al. How cognitive and reactive fear circuits optimize escape decisions in humans. Proc. Natl Acad. Sci. USA 115, 3186–3191 (2018).

  25. 25.

    Bach, D. R. Anxiety-like behavioural inhibition is normative under environmental threat-reward correlations. PLoS Comput. Biol. 11, 1–20 (2015).

  26. 26.

    Korn, C. W. et al. Amygdala lesions reduce anxiety-like behavior in a human benzodiazepine-sensitive approach–avoidance conflict test. Biol. Psychiatry 82, 522–531 (2017).

  27. 27.

    Khemka, S., Barnes, G., Dolan, R. J. & Bach, D. R. Dissecting the function of hippocampal oscillations in a human anxiety model. J. Neurosci. 37, 6869–6876 (2017).

  28. 28.

    McNaughton, N. & Corr, P. J. Survival circuits and risk assessment. Curr. Opin. Behav. Sci. 24, 14–20 (2018).

  29. 29.

    Blanchard, D. C. Risk assessment: at the interface of cognition and emotion. Curr. Opin. Behav. Sci. 24, 69–74 (2018).

  30. 30.

    Amemori, K. & Graybiel, A. M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).

  31. 31.

    Symmonds, M., Wright, N. D., Bach, D. R. & Dolan, R. J. Deconstructing risk: separable encoding of variance and skewness in the brain. Neuroimage 58, 1139–1149 (2011).

  32. 32.

    Mohr, P. N. C., Biele, G. & Heekeren, H. R. Neural processing of risk. J. Neurosci. 30, 6613–6619 (2010).

  33. 33.

    Bach, D. R. & Dolan, R. J. Knowing how much you don’t know: a neural organization of uncertainty estimates. Nat. Rev. Neurosci. 13, 572–586 (2012).

  34. 34.

    Bartra, O., McGuire, J. T. & Kable, J. W. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 76, 412–427 (2013).

  35. 35.

    Clithero, J. A. & Rangel, A. Informatic parcellation of the network involved in the computation of subjective value. Soc. Cogn. Affect. Neurosci. 9, 1289–1302 (2014).

  36. 36.

    Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E. & Behrens, T. E. Frontal cortex and reward-guided learning and decision-making. Neuron 70, 1054–1069 (2011).

  37. 37.

    Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of foraging. Science 336, 95–98 (2012).

  38. 38.

    Kolling, N., Wittmann, M. & Rushworth, M. F. S. Multiple neural mechanisms of decision making and their competition under changing risk pressure. Neuron 81, 1190–1202 (2014).

  39. 39.

    Hayden, B. Y., Pearson, J. M. & Platt, M. L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933–939 (2011).

  40. 40.

    Mata, R., Wilke, A. & Czienskowski, U. Foraging across the life span: is there a reduction in exploration with aging? Front. Neurosci. 7, 53 (2013).

  41. 41.

    Shenhav, A., Straccia, M. A., Cohen, J. D. & Botvinick, M. M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).

  42. 42.

    Constantino, S. M. & Daw, N. D. Learning the opportunity cost of time in a patch-foraging task. Cogn. Affect. Behav. Neurosci. 15, 837–853 (2015).

  43. 43.

    Pearson, J. M., Watson, K. K. & Platt, M. L. Decision making: the neuroethological turn. Neuron 82, 950–965 (2014).

  44. 44.

    Gigerenzer, G. & Gaissmaier, W. Heuristic decision making. Annu. Rev. Psychol. 62, 451–482 (2011).

  45. 45.

    Gu, X. & FitzGerald, T. H. B. Interoceptive inference: homeostasis and decision-making. Trends Cogn. Sci. 18, 269–270 (2014).

  46. 46.

    Fawcett, T. W. et al. The evolution of decision rules in complex environments. Trends Cogn. Sci. 18, 153–161 (2014).

  47. 47.

    Dayan, P. Rationalizable irrationalities of choice. Top. Cogn. Sci. 6, 204–228 (2014).

  48. 48.

    Tovote, P., Fadok, J. P. & Lüthi, A. Neuronal circuits for fear and anxiety. Nat. Rev. Neurosci. 16, 317–331 (2015).

  49. 49.

    Jimenez, J. C. et al. Anxiety cells in a hippocampal–hypothalamic circuit. Neuron 97, 670–683.e6 (2018).

  50. 50.

    Blanchard, D. C. Translating dynamic defense patterns from rodents to people. Neurosci. Biobehav. Rev. 76, 22–28 (2017).

  51. 51.

    Payzan-LeNestour, E., Dunne, S., Bossaerts, P. & O’Doherty, J. The neural representation of unexpected uncertainty during value-based decision making. Neuron 79, 191–201 (2013).

  52. 52.

    Rigoli, F., Michely, J., Friston, K. J. & Dolan, R. J. The role of the hippocampus in weighting expectations during inference under uncertainty. Cortex 115, 1–14 (2019).

  53. 53.

    Harrison, L. M., Duggins, A. & Friston, K. J. Encoding uncertainty in the hippocampus. Neural Netw. 19, 535–546 (2006).

  54. 54.

    Strange, B. A., Duggins, A., Penny, W., Dolan, R. J. & Friston, K. J. Information theory, novelty and hippocampal responses: unpredicted or unpredictable? Neural Netw. 18, 225–230 (2005).

  55. 55.

    Lee, S. W., Shimojo, S. & O’Doherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687–699 (2014).

  56. 56.

    Boureau, Y.-L., Sokol-Hessner, P. & Daw, N. D. Deciding how to decide: self-control and meta-decision making. Trends Cogn. Sci. 19, 700–710 (2015).

  57. 57.

    Laux, L., Glanzmann, P., Schaffner, P. & Spielberger, C. D. STAI—State-Trait-Angstinventar. Theoretische Grundlagen und Handanweisung in German (Beltz Test GmbH, 1981).

  58. 58.

    Bless, H., Wänke, M., Bohner, G., Fellhauer, R. F. & Schwarz, N. Need for cognition: Eine Skala zur Erfassung von Engagement und Freude bei Denkaufgaben. Z. Sozialpsychol. 25 , 147–154 (1994).

  59. 59.

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).

  60. 60.

    Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies - revisited. Neuroimage 84, 971–985 (2014).

  61. 61.

    Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J. & Friston, K. J. Bayesian model selection for group studies. Neuroimage 46, 1004–1017 (2009).

  62. 62.

    Penny, W. D. Comparing dynamic causal models using AIC, BIC and free energy. Neuroimage 59, 319–330 (2012).

  63. 63.

    Baayen, R. H., Davidson, D. J. & Bates, D. M. Mixed-effects modeling with crossed random effects for subjects and items. J. Mem. Lang. 59, 390–412 (2008).

  64. 64.

    Hutton, C. et al. Image distortion correction in fMRI: a quantitative evaluation. Neuroimage 16, 217–240 (2002).

  65. 65.

    Ashburner, J. & Friston, K. J. Unified segmentation. Neuroimage 26, 839–851 (2005).

  66. 66.

    Eklund, A., Nichols, T. E. & Knutsson, H. Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates. Proc. Natl Acad. Sci. USA 113, 7900–7905 (2016).

Download references

Acknowledgements

We thank G. Castegnetti, S. Khemka, M. Staib, A. Tzovara and C. Ioan for discussions and help with data acquisition. The Wellcome Trust Centre for Neuroimaging is supported by a strategic grant from the Wellcome Trust (091593/Z/10/Z). C.W.K. was supported by two grants from the German Research Foundation (DFG) during the final stages of manuscript preparation: the collaborative research centre SFB TRR 169 and an Emmy Noether Research Group (392443797). The funders had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

C.W.K. and D.R.B. designed the experiment, developed the analysis procedures and wrote the paper. C.W.K. collected and analysed the data.

Competing interests

The authors declare no competing interests.

Correspondence to Christoph W. Korn.

Supplementary information

Supplementary Information

Supplementary Figs. 1–10, Supplementary Tables 1–17, Supplementary References and Supplementary Notes 1–3.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Further reading

Fig. 1: Task outline.
Fig. 2: Models of choice data in the fMRI sample.
Fig. 3: fMRI results during the choice phase.
Fig. 4: Visualization of the clusters in the hippocampus extending into the amygdala.