Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Surprise, value and control in anterior cingulate cortex during speeded decision-making

Nature Human Behaviour volume 4, pages 412–422 (2020)Cite this article

Subjects

Abstract

Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlighting surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive predictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Theoretical predictions and experimental paradigm.
Fig. 2: Behavioural strategies and reactive and proactive control processes.
Fig. 3: fMRI and behavioural performance.
Fig. 4: Feedback and reward-related surprise.
Fig. 5: Whole-brain polynomial fits.

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author on request.

Code availability

Detailed modelling methods are included in Supplementary Methods. All scripts used to derive predictions from the various computational models are available online at https://github.com/modelbrains/EVC-Simulations

References

  1. Ebitz, R. B. & Hayden, B. Y. Dorsal anterior cingulate: a Rorschach test for cognitive neuroscience. Nat. Neurosci. 19, 1278–1279 (2016).

    Article  CAS  PubMed  Google Scholar 

  2. Vassena, E., Holroyd, C. B. & Alexander, W. H. Computational models of anterior cingulate cortex: at the crossroads between prediction and effort. Front. Neurosci. 11, 316 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S. & Cohen, J. D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

    Article  CAS  PubMed  Google Scholar 

  4. Holroyd, C. B. & Coles, M. G. H. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  PubMed  Google Scholar 

  5. Brown, J. W. & Braver, T. S. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).

    Article  CAS  PubMed  Google Scholar 

  6. Yee, D. M. & Braver, T. S. Interactions of motivation and cognitive control. Curr. Opin. Behav. Sci. 19, 83–90 (2018).

    Article  PubMed  Google Scholar 

  7. Botvinick, M. & Braver, T. Motivation and cognitive control: from behavior to neural mechanism. Annu. Rev. Psychol. 66, 83–113 (2015).

    Article  PubMed  Google Scholar 

  8. Inzlicht, M., Shenhav, A. & Olivola, C. Y. The effort paradox: effort is both costly and valued. Trends Cogn. Sci. 22, 337–349 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Verguts, T., Vassena, E. & Silvetti, M. Adaptive effort investment in cognitive and physical tasks: a neurocomputational model. Front. Behav. Neurosci. 9, 57 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Apps, M. A. J., Grima, L. L., Manohar, S. & Husain, M. The role of cognitive effort in subjective reward devaluation and risky decision-making. Sci. Rep. 5, 16880 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Aarts, E. & Roelofs, A. Attentional control in anterior cingulate cortex based on probabilistic cueing. J. Cogn. Neurosci. 23, 716–727 (2010).

    Article  PubMed  Google Scholar 

  12. Vassena, E. et al. Overlapping neural systems represent cognitive effort and reward anticipation. PLoS One 9, e91008 (2014).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  13. Vassena, E., Krebs, R. M., Silvetti, M., Fias, W. & Verguts, T. Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice. Neuropsychologia 59, 112–123 (2014).

    Article  PubMed  Google Scholar 

  14. Shenhav, A., Botvinick, M. M. & Cohen, J. D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Alexander, W. H. & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nat. Neurosci. 14, 1338–1344 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Kolling, N. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat. Neurosci. 19, 1280–1285 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. di Pellegrino, G., Ciaramelli, E. & Làdavas, E. The regulation of cognitive control following rostral anterior cingulate cortex lesion in humans. J. Cogn. Neurosci. 19, 275–286 (2007).

    Article  PubMed  Google Scholar 

  18. Rushworth, M. F. S., Hadland, K. A., Gaffan, D. & Passingham, R. E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).

    Article  CAS  PubMed  Google Scholar 

  19. Ridderinkhof, K. R., van den Wildenberg, W. P., Segalowitz, S. J. & Carter, C. S. Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain Cogn. 56, 129–140 (2004).

    Article  PubMed  Google Scholar 

  20. Sheth, S. A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioral adaptation. Nature 488, 218–221 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Vassena, E., Deraeve, J. & Alexander, W. H. Predicting motivation: computational models of PFC can explain neural coding of motivation and effort-based decision-making in health and disease. J. Cogn. Neurosci. 29, 1633–1645 (2017).

    Article  PubMed  Google Scholar 

  22. Klein-Flügge, M. C., Kennerley, S. W., Friston, K. & Bestmann, S. Neural signatures of value comparison in human cingulate cortex during decisions requiring an effort-reward trade-off. J. Neurosci. 36, 10002–10015 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  23. Shenhav, A., Straccia, M. A., Cohen, J. D. & Botvinick, M. M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Lin, H., Saunders, B., Hutcherson, C. A. & Inzlicht, M. Midfrontal theta and pupil dilation parametrically track subjective conflict (but also surprise) during intertemporal choice. Neuroimage 172, 838–852 (2018).

    Article  PubMed  Google Scholar 

  25. Krajbich, I., Lu, D., Camerer, C. & Rangel, A. The attentional drift-diffusion model extends to simple purchasing decisions. Front. Psychol. 3, 193 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Mormann, M. M., Malmaud, J., Huth, A., Koch, C. & Rangel, A. The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgm. Decis. Mak., 5, 437–449 (2010).

    Google Scholar 

  27. Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Article  Google Scholar 

  28. Cavanagh, J. F. & Frank, M. J. Frontal theta as a mechanism for cognitive control. Trends Cogn. Sci. 18, 414–421 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Shenhav, A., Straccia, M. A., Botvinick, M. M. & Cohen, J. D. Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice. Cogn. Affect. Behav. Neurosci. 16, 1127–1139 (2016).

    Article  PubMed  Google Scholar 

  30. Alexander, W. H. & Brown, J. W. A general role for medial prefrontal cortex in event prediction. Front. Comput. Neurosci. 8, 69 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Braver, T. S. The variable nature of cognitive control: a dual mechanisms framework. Trends Cogn. Sci. 16, 106–113 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Wessel, J. R. & Aron, A. R. On the globality of motor suppression: unexpected events and their influence on behavior and cognition. Neuron 93, 259–280 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Wiecki, T. V., Sofer, I. & Frank, M. J. HDDM: hierarchical bayesian estimation of the drift-diffusion model in python. Front. Neuroinform. 7, 14 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Spiegelhalter, D. J., Best, N. G., Carlin, B. P. & Linde, A. V. D. Bayesian measures of model complexity and fit. J. R. Stat. Soc. B 64, 583–639

  35. Murphy, P. R., Boonstra, E. & Nieuwenhuis, S. Global gain modulation generates time-dependent urgency during perceptual choice in humans. Nat. Commun. 7, 13526 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Downar, J., Crawley, A. P., Mikulis, D. J. & Davis, K. D. A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities. J. Neurophysiol. 87, 615–620 (2002).

    Article  PubMed  Google Scholar 

  37. Knight, R. T. & Nakada, T. Cortico-limbic circuits and novelty: a review of EEG and blood flow data. Rev. Neurosci. 9, 57–70 (1998).

    Article  CAS  PubMed  Google Scholar 

  38. O’Reilly, J. X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl Acad. Sci. USA 110, E3660–E3669 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Grabenhorst, F. & Rolls, E. T. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn. Sci. 15, 56–67 (2011).

    Article  PubMed  Google Scholar 

  40. Brown, J. W. & Alexander, W. H. Foraging value, risk avoidance, and multiple control signals: how the ACC controls value-based decision-making. J. Cogn. Neurosci. 29, 1656–1673 (2017).

  41. Alexander, W. H. & Brown, J. W. Hierarchical error representation: a computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Comput. 27, 2354–2410 (2015).

    Article  PubMed  Google Scholar 

  42. Alexander, W. H. & Brown, J. W. Frontal cortex function as derived from hierarchical predictive coding. Sci. Rep. 8, 3843 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  43. Steenbergen, H., van, Band, G. P. H., Hommel, B., Rombouts, S. A. R. B. & Nieuwenhuis, S. Hedonic hotspots regulate cingulate-driven adaptation to cognitive demands. Cereb. Cortex 25, 1746–1756 (2015).

    Article  PubMed  Google Scholar 

  44. Alexander, W. H., Vassena, E., Deraeve, J. & Langford, Z. D. Integrative modeling of pFC. J. Cogn. Neurosci. 29, 1674–1683 (2017).

  45. Badre, D. & Nee, D. E. Frontal cortex and the hierarchical control of behavior. Trends Cogn. Sci. 22, 170–188 (2018).

    Article  PubMed  Google Scholar 

  46. Koechlin, E. & Summerfield, C. An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. 11, 229–235 (2007).

    Article  PubMed  Google Scholar 

  47. Nee, D. E. & Brown, J. W. Rostral–caudal gradients of abstraction revealed by multi-variate pattern analysis of working memory. NeuroImage 63, 1285–1294 (2012).

    Article  PubMed  Google Scholar 

  48. Nee, D. E. & D’Esposito, M. The hierarchical organization of the lateral prefrontal cortex. eLife https://doi.org/10.7554/eLife.12112 (2016).

  49. Silvetti, M., Seurinck, R. & Verguts, T. Value and prediction error in medial frontal cortex: integrating the single-unit and systems levels of analysis. Front. Hum. Neurosci. 5, 75 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Silvetti, M., Alexander, W., Verguts, T. & Brown, J. W. From conflict management to reward-based decision making: actors and critics in primate medial frontal cortex. Neurosci. Biobehav. Rev. 46, 44–57 (2014).

    Article  PubMed  Google Scholar 

  51. Alexander, W. H., Brown, J. W., Collins, A. G. E., Hayden, B. Y. & Vassena, E. Prefrontal cortex in control: broadening the scope to identify mechanisms. J. Cogn. Neurosci. 30, 1061–1065 (2018).

  52. Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of foraging. Science 336, 95–98 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Wagenmakers, E.-J. & Farrell, S. AIC model selection using Akaike weights. Psychon. Bull. Rev. 11, 192–196 (2004).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank C. Holroyd, J. Brown, T. Verguts, Z. Langford, J. Maynard Keenan and M. Pessiglione for useful discussions. W.H.A. was supported by FWO-Flanders Odysseus II Award (no. G.OC44.13N) and by start-up funds provided by Florida Atlantic University. E.V. was supported by the Marie Sklodowska-Curie actions with a standard IF-EF fellowship within the H2020 framework (H2020-MSCA-IF2015, grant no. 705630). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Donders Institute for Brain Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands

    Eliana Vassena

  2. Department of Experimental Psychology, Ghent University, Ghent, Belgium

    Eliana Vassena, James Deraeve & William H. Alexander

  3. Center for Complex Systems & Brain Sciences, Florida Atlantic University, Boca Raton, FL, USA

    William H. Alexander

Authors
  1. Eliana Vassena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Deraeve
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William H. Alexander
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.V. and W.H.A. conceived the study and designed the work. E.V., W.H.A. and J.D. performed acquisition, analysis and interpretation of data and drafted the manuscript. E.V. and W.H.A. revised the manuscript.

Corresponding author

Correspondence to William H. Alexander.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary handling Editor: Marike Schiffer

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Factors influencing Akaike Weight estimates.

Our analysis of fMRI data using polynomial functions fit to beta estimates from BOLD data is substantially different from classical or model-based fMRI analyses. In order to explore what factors may influence our results, as well as how nested polynomial functions independently account for observed patterns of activity in our data, we carried out additional simulations and analyses. First, we asked what factors affect the ability of our analysis approach to identify quartic effects using Akaike Weights. To answer this, we conducted six simulations of synthetic data generated by a quartic polynomial equation while varying the noise, number of subjects, and shape of the function. 10,000 simulation runs were conducted for each condition, and Akaike Weights calculated to obtain a distribution of the likelihood of Akaike Weights. The results of these simulations suggest that, as in classical univariate analyses, the likelihood of obtaining an Akaike Weight >0.999 for the quartic polynomial function (when the data was in fact generated by a quartic polynomial function) depends both on the quantity and noisiness of the data itself. That is, with less data and more noise, it becomes less likely that our analyses will assign the quartic polynomial function an Akaike Weight of 0.999 or higher. Additionally, changes in the shape of the quartic function that render it more similar to other (quadratic) functions reduce the likelihood of calculating an Akaike Weight >0.999 for the quartic function. These results suggest our analysis approach parallels typical fMRI analyses in terms of the factors that influence the likelihood of observing significant effects.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Figs. 1–7 and Supplementary References.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vassena, E., Deraeve, J. & Alexander, W.H. Surprise, value and control in anterior cingulate cortex during speeded decision-making. Nat Hum Behav 4, 412–422 (2020). https://doi.org/10.1038/s41562-019-0801-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-019-0801-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing