Surprise, value and control in anterior cingulate cortex during speeded decision-making

Abstract

Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlighting surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive predictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Theoretical predictions and experimental paradigm.
Fig. 2: Behavioural strategies and reactive and proactive control processes.
Fig. 3: fMRI and behavioural performance.
Fig. 4: Feedback and reward-related surprise.
Fig. 5: Whole-brain polynomial fits.

Data availability

The data that support the findings of this study are available from the corresponding author on request.

Code availability

Detailed modelling methods are included in Supplementary Methods. All scripts used to derive predictions from the various computational models are available online at https://github.com/modelbrains/EVC-Simulations

References

  1. 1.

    Ebitz, R. B. & Hayden, B. Y. Dorsal anterior cingulate: a Rorschach test for cognitive neuroscience. Nat. Neurosci. 19, 1278–1279 (2016).

  2. 2.

    Vassena, E., Holroyd, C. B. & Alexander, W. H. Computational models of anterior cingulate cortex: at the crossroads between prediction and effort. Front. Neurosci. 11, 316 (2017).

  3. 3.

    Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S. & Cohen, J. D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

  4. 4.

    Holroyd, C. B. & Coles, M. G. H. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

  5. 5.

    Brown, J. W. & Braver, T. S. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).

  6. 6.

    Yee, D. M. & Braver, T. S. Interactions of motivation and cognitive control. Curr. Opin. Behav. Sci. 19, 83–90 (2018).

  7. 7.

    Botvinick, M. & Braver, T. Motivation and cognitive control: from behavior to neural mechanism. Annu. Rev. Psychol. 66, 83–113 (2015).

  8. 8.

    Inzlicht, M., Shenhav, A. & Olivola, C. Y. The effort paradox: effort is both costly and valued. Trends Cogn. Sci. 22, 337–349 (2018).

  9. 9.

    Verguts, T., Vassena, E. & Silvetti, M. Adaptive effort investment in cognitive and physical tasks: a neurocomputational model. Front. Behav. Neurosci. 9, 57 (2015).

  10. 10.

    Apps, M. A. J., Grima, L. L., Manohar, S. & Husain, M. The role of cognitive effort in subjective reward devaluation and risky decision-making. Sci. Rep. 5, 16880 (2015).

  11. 11.

    Aarts, E. & Roelofs, A. Attentional control in anterior cingulate cortex based on probabilistic cueing. J. Cogn. Neurosci. 23, 716–727 (2010).

  12. 12.

    Vassena, E. et al. Overlapping neural systems represent cognitive effort and reward anticipation. PLoS One 9, e91008 (2014).

  13. 13.

    Vassena, E., Krebs, R. M., Silvetti, M., Fias, W. & Verguts, T. Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice. Neuropsychologia 59, 112–123 (2014).

  14. 14.

    Shenhav, A., Botvinick, M. M. & Cohen, J. D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

  15. 15.

    Alexander, W. H. & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nat. Neurosci. 14, 1338–1344 (2011).

  16. 16.

    Kolling, N. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat. Neurosci. 19, 1280–1285 (2016).

  17. 17.

    di Pellegrino, G., Ciaramelli, E. & Làdavas, E. The regulation of cognitive control following rostral anterior cingulate cortex lesion in humans. J. Cogn. Neurosci. 19, 275–286 (2007).

  18. 18.

    Rushworth, M. F. S., Hadland, K. A., Gaffan, D. & Passingham, R. E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).

  19. 19.

    Ridderinkhof, K. R., van den Wildenberg, W. P., Segalowitz, S. J. & Carter, C. S. Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain Cogn. 56, 129–140 (2004).

  20. 20.

    Sheth, S. A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioral adaptation. Nature 488, 218–221 (2012).

  21. 21.

    Vassena, E., Deraeve, J. & Alexander, W. H. Predicting motivation: computational models of PFC can explain neural coding of motivation and effort-based decision-making in health and disease. J. Cogn. Neurosci. 29, 1633–1645 (2017).

  22. 22.

    Klein-Flügge, M. C., Kennerley, S. W., Friston, K. & Bestmann, S. Neural signatures of value comparison in human cingulate cortex during decisions requiring an effort-reward trade-off. J. Neurosci. 36, 10002–10015 (2016).

  23. 23.

    Shenhav, A., Straccia, M. A., Cohen, J. D. & Botvinick, M. M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).

  24. 24.

    Lin, H., Saunders, B., Hutcherson, C. A. & Inzlicht, M. Midfrontal theta and pupil dilation parametrically track subjective conflict (but also surprise) during intertemporal choice. Neuroimage 172, 838–852 (2018).

  25. 25.

    Krajbich, I., Lu, D., Camerer, C. & Rangel, A. The attentional drift-diffusion model extends to simple purchasing decisions. Front. Psychol. 3, 193 (2012).

  26. 26.

    Mormann, M. M., Malmaud, J., Huth, A., Koch, C. & Rangel, A. The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgm. Decis. Mak., 5, 437–449 (2010).

  27. 27.

    Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

  28. 28.

    Cavanagh, J. F. & Frank, M. J. Frontal theta as a mechanism for cognitive control. Trends Cogn. Sci. 18, 414–421 (2014).

  29. 29.

    Shenhav, A., Straccia, M. A., Botvinick, M. M. & Cohen, J. D. Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice. Cogn. Affect. Behav. Neurosci. 16, 1127–1139 (2016).

  30. 30.

    Alexander, W. H. & Brown, J. W. A general role for medial prefrontal cortex in event prediction. Front. Comput. Neurosci. 8, 69 (2014).

  31. 31.

    Braver, T. S. The variable nature of cognitive control: a dual mechanisms framework. Trends Cogn. Sci. 16, 106–113 (2012).

  32. 32.

    Wessel, J. R. & Aron, A. R. On the globality of motor suppression: unexpected events and their influence on behavior and cognition. Neuron 93, 259–280 (2017).

  33. 33.

    Wiecki, T. V., Sofer, I. & Frank, M. J. HDDM: hierarchical bayesian estimation of the drift-diffusion model in python. Front. Neuroinform. 7, 14 (2013).

  34. 34.

    Spiegelhalter, D. J., Best, N. G., Carlin, B. P. & Linde, A. V. D. Bayesian measures of model complexity and fit. J. R. Stat. Soc. B 64, 583–639

  35. 35.

    Murphy, P. R., Boonstra, E. & Nieuwenhuis, S. Global gain modulation generates time-dependent urgency during perceptual choice in humans. Nat. Commun. 7, 13526 (2016).

  36. 36.

    Downar, J., Crawley, A. P., Mikulis, D. J. & Davis, K. D. A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities. J. Neurophysiol. 87, 615–620 (2002).

  37. 37.

    Knight, R. T. & Nakada, T. Cortico-limbic circuits and novelty: a review of EEG and blood flow data. Rev. Neurosci. 9, 57–70 (1998).

  38. 38.

    O’Reilly, J. X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl Acad. Sci. USA 110, E3660–E3669 (2013).

  39. 39.

    Grabenhorst, F. & Rolls, E. T. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn. Sci. 15, 56–67 (2011).

  40. 40.

    Brown, J. W. & Alexander, W. H. Foraging value, risk avoidance, and multiple control signals: how the ACC controls value-based decision-making. J. Cogn. Neurosci. 29, 1656–1673 (2017).

  41. 41.

    Alexander, W. H. & Brown, J. W. Hierarchical error representation: a computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Comput. 27, 2354–2410 (2015).

  42. 42.

    Alexander, W. H. & Brown, J. W. Frontal cortex function as derived from hierarchical predictive coding. Sci. Rep. 8, 3843 (2018).

  43. 43.

    Steenbergen, H., van, Band, G. P. H., Hommel, B., Rombouts, S. A. R. B. & Nieuwenhuis, S. Hedonic hotspots regulate cingulate-driven adaptation to cognitive demands. Cereb. Cortex 25, 1746–1756 (2015).

  44. 44.

    Alexander, W. H., Vassena, E., Deraeve, J. & Langford, Z. D. Integrative modeling of pFC. J. Cogn. Neurosci. 29, 1674–1683 (2017).

  45. 45.

    Badre, D. & Nee, D. E. Frontal cortex and the hierarchical control of behavior. Trends Cogn. Sci. 22, 170–188 (2018).

  46. 46.

    Koechlin, E. & Summerfield, C. An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. 11, 229–235 (2007).

  47. 47.

    Nee, D. E. & Brown, J. W. Rostral–caudal gradients of abstraction revealed by multi-variate pattern analysis of working memory. NeuroImage 63, 1285–1294 (2012).

  48. 48.

    Nee, D. E. & D’Esposito, M. The hierarchical organization of the lateral prefrontal cortex. eLife https://doi.org/10.7554/eLife.12112 (2016).

  49. 49.

    Silvetti, M., Seurinck, R. & Verguts, T. Value and prediction error in medial frontal cortex: integrating the single-unit and systems levels of analysis. Front. Hum. Neurosci. 5, 75 (2011).

  50. 50.

    Silvetti, M., Alexander, W., Verguts, T. & Brown, J. W. From conflict management to reward-based decision making: actors and critics in primate medial frontal cortex. Neurosci. Biobehav. Rev. 46, 44–57 (2014).

  51. 51.

    Alexander, W. H., Brown, J. W., Collins, A. G. E., Hayden, B. Y. & Vassena, E. Prefrontal cortex in control: broadening the scope to identify mechanisms. J. Cogn. Neurosci. 30, 1061–1065 (2018).

  52. 52.

    Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of foraging. Science 336, 95–98 (2012).

  53. 53.

    Wagenmakers, E.-J. & Farrell, S. AIC model selection using Akaike weights. Psychon. Bull. Rev. 11, 192–196 (2004).

Download references

Acknowledgements

We thank C. Holroyd, J. Brown, T. Verguts, Z. Langford, J. Maynard Keenan and M. Pessiglione for useful discussions. W.H.A. was supported by FWO-Flanders Odysseus II Award (no. G.OC44.13N) and by start-up funds provided by Florida Atlantic University. E.V. was supported by the Marie Sklodowska-Curie actions with a standard IF-EF fellowship within the H2020 framework (H2020-MSCA-IF2015, grant no. 705630). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

E.V. and W.H.A. conceived the study and designed the work. E.V., W.H.A. and J.D. performed acquisition, analysis and interpretation of data and drafted the manuscript. E.V. and W.H.A. revised the manuscript.

Correspondence to William H. Alexander.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary handling Editor: Marike Schiffer

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Factors influencing Akaike Weight estimates.

Our analysis of fMRI data using polynomial functions fit to beta estimates from BOLD data is substantially different from classical or model-based fMRI analyses. In order to explore what factors may influence our results, as well as how nested polynomial functions independently account for observed patterns of activity in our data, we carried out additional simulations and analyses. First, we asked what factors affect the ability of our analysis approach to identify quartic effects using Akaike Weights. To answer this, we conducted six simulations of synthetic data generated by a quartic polynomial equation while varying the noise, number of subjects, and shape of the function. 10,000 simulation runs were conducted for each condition, and Akaike Weights calculated to obtain a distribution of the likelihood of Akaike Weights. The results of these simulations suggest that, as in classical univariate analyses, the likelihood of obtaining an Akaike Weight >0.999 for the quartic polynomial function (when the data was in fact generated by a quartic polynomial function) depends both on the quantity and noisiness of the data itself. That is, with less data and more noise, it becomes less likely that our analyses will assign the quartic polynomial function an Akaike Weight of 0.999 or higher. Additionally, changes in the shape of the quartic function that render it more similar to other (quadratic) functions reduce the likelihood of calculating an Akaike Weight >0.999 for the quartic function. These results suggest our analysis approach parallels typical fMRI analyses in terms of the factors that influence the likelihood of observing significant effects.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Figs. 1–7 and Supplementary References.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vassena, E., Deraeve, J. & Alexander, W.H. Surprise, value and control in anterior cingulate cortex during speeded decision-making. Nat Hum Behav (2020). https://doi.org/10.1038/s41562-019-0801-5

Download citation