Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Value, search, persistence and model updating in anterior cingulate cortex

Nature Neuroscience volume 19, pages 1280–1285 (2016)Cite this article

Subjects

Abstract

Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Comparing dACC and pgACC in humans and macaques.
Figure 2: Derivation of value signals in dACC and presence of model-updating signals in dACC.
Figure 3: Comparing multiple value signals in vmPFC (top) and dACC (bottom) and decision-making processes.

Similar content being viewed by others

References

  1. Neubert, F.X., Mars, R.B., Sallet, J. & Rushworth, M.F. Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex. Proc. Natl. Acad. Sci. USA 112, E2695–E2704 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Behrens, T.E., Fox, P., Laird, A. & Smith, S.M. What is the most interesting part of the brain? Trends Cogn. Sci. 17, 2–4 (2013).

    Article  PubMed  Google Scholar 

  3. Kolling, N., Behrens, T.E., Mars, R.B. & Rushworth, M.F. Neural mechanisms of foraging. Science 336, 95–98 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Wittmann, M. et al. Predicting how your luck will change: decision making driven by multiple time-linked reward representations in anterior cingulate cortex. Nat. Commun. http://dx.doi.org/10.1038/ncomms12327 (2016).

  6. Bernacchia, A., Seo, H., Lee, D. & Wang, X.J. A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 14, 366–372 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Tervo, D.G. et al. Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014).

    Article  CAS  PubMed  Google Scholar 

  8. Karlsson, M.P., Tervo, D.G. & Karpova, A.Y. Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty. Science 338, 135–139 (2012).

    Article  CAS  PubMed  Google Scholar 

  9. O'Reilly, J.X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 110, E3660–E3669 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

    Article  CAS  PubMed  Google Scholar 

  11. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  PubMed  Google Scholar 

  12. Quilodran, R., Rothé, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).

    Article  CAS  PubMed  Google Scholar 

  13. Narayanan, N.S., Cavanagh, J.F., Frank, M.J. & Laubach, M. Common medial frontal mechanisms of adaptive control in humans and rodents. Nat. Neurosci. 16, 1888–1895 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  PubMed  Google Scholar 

  15. Warren, C.M., Hyman, J.M., Seamans, J.K. & Holroyd, C.B. Feedback-related negativity observed in rodent anterior cingulate cortex. J. Physiol. Paris 109, 87–94 (2015).

    Article  PubMed  Google Scholar 

  16. Phillips, J.M. & Everling, S. Event-related potentials associated with performance monitoring in non-human primates. Neuroimage 97, 308–320 (2014).

    Article  PubMed  Google Scholar 

  17. Scholl, J. et al. The good, the bad, and the irrelevant: neural mechanisms of learning real and hypothetical rewards and effort. J. Neurosci. 35, 11233–11251 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Shenhav, A., Botvinick, M.M. & Cohen, J.D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Shenhav, A., Straccia, M.A., Cohen, J.D. & Botvinick, M.M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Cole, M.W., Yeung, N., Freiwald, W.A. & Botvinick, M. Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 32, 566–574 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Amiez, C. & Petrides, M. Neuroimaging evidence of the anatomo-functional organization of the human cingulate motor areas. Cereb. Cortex 24, 563–578 (2014).

    Article  PubMed  Google Scholar 

  22. Procyk, E. et al. Midcingulate motor map and feedback detection: converging data from humans and monkeys. Cereb. Cortex 26, 467–476 (2016).

    PubMed  Google Scholar 

  23. O'Reilly, J.X. et al. Causal effect of disconnection lesions on interhemispheric functional connectivity in rhesus monkeys. Proc. Natl. Acad. Sci. USA 110, 13982–13987 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Kolling, N., Wittmann, M. & Rushworth, M.F. Multiple neural mechanisms of decision making and their competition under changing risk pressure. Neuron 81, 1190–1202 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Boorman, E.D., Rushworth, M.F. & Behrens, T.E. Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice. J. Neurosci. 33, 2242–2253 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Neubert, F.X., Mars, R.B., Thomas, A.G., Sallet, J. & Rushworth, M.F. Comparison of human ventral frontal cortex areas for cognitive control and language with areas in monkey frontal cortex. Neuron 81, 700–713 (2014).

    Article  CAS  PubMed  Google Scholar 

  27. Vogt, B.A. in Cingulate Neurobiology and Disease (ed. Vogt, B.A.) (Oxford Univ. Press, 2009).

  28. Passingham, R.E. & Wise, S.P. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight (Oxford Univ. Press, 2012).

  29. Strick, P.L., Dum, R.P. & Picard, N. Motor areas on the medial wall of the hemisphere. Novartis Found. Symp. 218, 64–75, discussion 75–80, 104–108 (1998).

    CAS  PubMed  Google Scholar 

  30. Van Hoesen, G.W., Morecraft, R.J. & Vogt, B.A. in Neurobiology of Cingulate Cortex and Limbic Thalamus (eds. Vogt, B.A. & Gabriel, M.) 249–284 (Birkhauser, 1993).

  31. Haber, S.N., Kim, K.S., Mailly, P. & Calzavara, R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368–8376 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Porrino, L.J. & Goldman-Rakic, P.S. Brainstem innervation of prefrontal and anterior cingulate cortex in the rhesus monkey revealed by retrograde transport of HRP. J. Comp. Neurol. 205, 63–76 (1982).

    Article  CAS  PubMed  Google Scholar 

  33. Eblen, F. & Graybiel, A.M. Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J. Neurosci. 15, 5999–6013 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Friedman, A. et al. A corticostriatal path targeting striosomes controls decision-making under conflict. Cell 161, 1320–1333 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Kennerley, S.W., Behrens, T.E. & Wallis, J.D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Rudebeck, P.H., Mitz, A.R., Chacko, R.V. & Murray, E.A. Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron 80, 1519–1531 (2013).

    Article  CAS  PubMed  Google Scholar 

  38. Hayden, B.Y., Pearson, J.M. & Platt, M.L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933–939 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Blanchard, T.C. & Hayden, B.Y. Neurons in dorsal anterior cingulate cortex signal postdecisional variables in a foraging task. J. Neurosci. 34, 646–655 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791–3808 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Luk, C.H. & Wallis, J.D. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J. Neurosci. 33, 1864–1871 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).

    Article  CAS  PubMed  Google Scholar 

  43. Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

    Article  PubMed  Google Scholar 

  44. Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).

    Article  PubMed  Google Scholar 

  45. Ebitz, R.B. & Platt, M.L. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628–640 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Sheth, S.A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioural adaptation. Nature 488, 218–221 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. McGuire, J.T. & Kable, J.W. Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence. Nat. Neurosci. 18, 760–766 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Cai, X. & Padoa-Schioppa, C. Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation. Neuron 81, 1140–1151 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Boorman, E.D., Behrens, T.E., Woolrich, M.W. & Rushworth, M.F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).

    Article  CAS  PubMed  Google Scholar 

  50. Strait, C.E., Blanchard, T.C. & Hayden, B.Y. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82, 1357–1366 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Amemori, K. & Graybiel, A.M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Wittmann, M.K. et al. Self-other mergence in frontal cortex during cooperation and competition. Neuron 91, 482–493 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. De Martino, B., Fleming, S.M., Garrett, N. & Dolan, R.J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).

    Article  CAS  PubMed  Google Scholar 

  54. Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).

    Article  CAS  PubMed  Google Scholar 

  55. Mars, R.B. et al. Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict. J. Neurosci. 29, 6926–6931 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Hare, T.A., Schultz, W., Camerer, C.F., O'Doherty, J.P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. USA 108, 18120–18125 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Wang, X.J. Decision making in recurrent neuronal circuits. Neuron 60, 215–234 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Hunt, L.T. et al. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476, S1–S3 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Freidin, E. & Kacelnik, A. Rational choice, context dependence, and the value of information in European starlings (Sturnus vulgaris). Science 334, 1000–1002 (2011).

    Article  CAS  PubMed  Google Scholar 

  61. Pearson, J.M., Watson, K.K. & Platt, M.L. Decision making: the neuroethological turn. Neuron 82, 950–965 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Logiaco, L., Quilodran, R., Procyk, E. & Arleo, A. Spatiotemporal spike coding of behavioral adaptation in the dorsal anterior cingulate cortex. PLoS Biol. 13, e1002222 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  63. Voloh, B., Valiante, T.A., Everling, S. & Womelsdorf, T. Theta-gamma coordination between anterior cingulate and prefrontal cortex indexes correct attention shifts. Proc. Natl. Acad. Sci. USA 112, 8457–8462 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M. & Rushworth, M.F. Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168 (2006).

    Article  CAS  PubMed  Google Scholar 

  65. Posner, M.I. & Petersen, S.E. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42 (1990).

    Article  CAS  PubMed  Google Scholar 

  66. Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control? Brain 128, 788–796 (2005).

    Article  PubMed  Google Scholar 

  67. Rushworth, M.F.S., Hadland, K.A., Gaffan, D. & Passingham, R.E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).

    Article  CAS  PubMed  Google Scholar 

  68. Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).

    Article  CAS  PubMed  Google Scholar 

  69. Mansouri, F.A., Buckley, M.J. & Tanaka, K. Mnemonic function of the dorsolateral prefrontal cortex in conflict-induced behavioral adjustment. Science 318, 987–990 (2007).

    Article  CAS  PubMed  Google Scholar 

  70. Camille, N., Tsuchida, A. & Fellows, L.K. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J. Neurosci. 31, 15048–15052 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Buckley, M.J. et al. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52–58 (2009).

    Article  CAS  PubMed  Google Scholar 

  72. Thaler, D., Chen, Y.C., Nixon, P.D., Stern, C.E. & Passingham, R.E. The functions of the medial premotor cortex. I. Simple learned movements. Exp. Brain Res. 102, 445–460 (1995).

    Article  CAS  PubMed  Google Scholar 

  73. Stoll, F.M., Fontanier, V. & Procyk, E. Specific frontal neural dynamics contribute to decisions to check. Nat. Commun. http://dx.doi.org/10.1038/ncomms11990 (2016).

  74. Meder, D. et al. Tuning the brake while raising the stake: network dynamics during sequential decision-making. J. Neurosci. 36, 5417–5426 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Klein-Flügge, M.C., Barron, H.C., Brodersen, K.H., Dolan, R.J. & Behrens, T.E. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J. Neurosci. 33, 3202–3211 (2013).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Experimental Psychology, University of Oxford, Oxford, UK

    Nils Kolling, Marco K Wittmann, Rogier B Mars & Matthew F S Rushworth

  2. Nuffield Department of Clinical Neurosciences, Centre for Functional MRI of the Brain (MRI), John Radcliffe Hospital, University of Oxford, Oxford, UK

    Tim E J Behrens, Erie D Boorman, Rogier B Mars & Matthew F S Rushworth

  3. Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, the Netherlands

    Rogier B Mars

Authors
  1. Nils Kolling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco K Wittmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim E J Behrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erie D Boorman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rogier B Mars
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthew F S Rushworth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nils Kolling.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kolling, N., Wittmann, M., Behrens, T. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat Neurosci 19, 1280–1285 (2016). https://doi.org/10.1038/nn.4382

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4382

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing