Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Value, search, persistence and model updating in anterior cingulate cortex

Nature Neuroscience volume 19, pages 1280–1285 (2016)Cite this article

Subjects

Abstract

Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.

Your institute does not have access to this article

Relevant articles

Open Access articles citing this article.

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

Buy

All prices are NET prices.

Figure 1: Comparing dACC and pgACC in humans and macaques.
Figure 2: Derivation of value signals in dACC and presence of model-updating signals in dACC.
Figure 3: Comparing multiple value signals in vmPFC (top) and dACC (bottom) and decision-making processes.

References

  1. Neubert, F.X., Mars, R.B., Sallet, J. & Rushworth, M.F. Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex. Proc. Natl. Acad. Sci. USA 112, E2695–E2704 (2015).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  2. Behrens, T.E., Fox, P., Laird, A. & Smith, S.M. What is the most interesting part of the brain? Trends Cogn. Sci. 17, 2–4 (2013).

    PubMed  Article  Google Scholar 

  3. Kolling, N., Behrens, T.E., Mars, R.B. & Rushworth, M.F. Neural mechanisms of foraging. Science 336, 95–98 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  4. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  5. Wittmann, M. et al. Predicting how your luck will change: decision making driven by multiple time-linked reward representations in anterior cingulate cortex. Nat. Commun. http://dx.doi.org/10.1038/ncomms12327 (2016).

  6. Bernacchia, A., Seo, H., Lee, D. & Wang, X.J. A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 14, 366–372 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  7. Tervo, D.G. et al. Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014).

    CAS  PubMed  Article  Google Scholar 

  8. Karlsson, M.P., Tervo, D.G. & Karpova, A.Y. Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty. Science 338, 135–139 (2012).

    CAS  Article  PubMed  Google Scholar 

  9. O'Reilly, J.X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 110, E3660–E3669 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

    CAS  PubMed  Article  Google Scholar 

  11. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    CAS  Article  PubMed  Google Scholar 

  12. Quilodran, R., Rothé, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).

    CAS  Article  PubMed  Google Scholar 

  13. Narayanan, N.S., Cavanagh, J.F., Frank, M.J. & Laubach, M. Common medial frontal mechanisms of adaptive control in humans and rodents. Nat. Neurosci. 16, 1888–1895 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  14. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    PubMed  Article  Google Scholar 

  15. Warren, C.M., Hyman, J.M., Seamans, J.K. & Holroyd, C.B. Feedback-related negativity observed in rodent anterior cingulate cortex. J. Physiol. Paris 109, 87–94 (2015).

    PubMed  Article  Google Scholar 

  16. Phillips, J.M. & Everling, S. Event-related potentials associated with performance monitoring in non-human primates. Neuroimage 97, 308–320 (2014).

    PubMed  Article  Google Scholar 

  17. Scholl, J. et al. The good, the bad, and the irrelevant: neural mechanisms of learning real and hypothetical rewards and effort. J. Neurosci. 35, 11233–11251 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  18. Shenhav, A., Botvinick, M.M. & Cohen, J.D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  19. Shenhav, A., Straccia, M.A., Cohen, J.D. & Botvinick, M.M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 1249–1254 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  20. Cole, M.W., Yeung, N., Freiwald, W.A. & Botvinick, M. Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 32, 566–574 (2009).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  21. Amiez, C. & Petrides, M. Neuroimaging evidence of the anatomo-functional organization of the human cingulate motor areas. Cereb. Cortex 24, 563–578 (2014).

    PubMed  Article  Google Scholar 

  22. Procyk, E. et al. Midcingulate motor map and feedback detection: converging data from humans and monkeys. Cereb. Cortex 26, 467–476 (2016).

    PubMed  Google Scholar 

  23. O'Reilly, J.X. et al. Causal effect of disconnection lesions on interhemispheric functional connectivity in rhesus monkeys. Proc. Natl. Acad. Sci. USA 110, 13982–13987 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  24. Kolling, N., Wittmann, M. & Rushworth, M.F. Multiple neural mechanisms of decision making and their competition under changing risk pressure. Neuron 81, 1190–1202 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  25. Boorman, E.D., Rushworth, M.F. & Behrens, T.E. Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice. J. Neurosci. 33, 2242–2253 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  26. Neubert, F.X., Mars, R.B., Thomas, A.G., Sallet, J. & Rushworth, M.F. Comparison of human ventral frontal cortex areas for cognitive control and language with areas in monkey frontal cortex. Neuron 81, 700–713 (2014).

    CAS  PubMed  Article  Google Scholar 

  27. Vogt, B.A. in Cingulate Neurobiology and Disease (ed. Vogt, B.A.) (Oxford Univ. Press, 2009).

  28. Passingham, R.E. & Wise, S.P. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight (Oxford Univ. Press, 2012).

  29. Strick, P.L., Dum, R.P. & Picard, N. Motor areas on the medial wall of the hemisphere. Novartis Found. Symp. 218, 64–75, discussion 75–80, 104–108 (1998).

    CAS  PubMed  Google Scholar 

  30. Van Hoesen, G.W., Morecraft, R.J. & Vogt, B.A. in Neurobiology of Cingulate Cortex and Limbic Thalamus (eds. Vogt, B.A. & Gabriel, M.) 249–284 (Birkhauser, 1993).

  31. Haber, S.N., Kim, K.S., Mailly, P. & Calzavara, R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368–8376 (2006).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  32. Porrino, L.J. & Goldman-Rakic, P.S. Brainstem innervation of prefrontal and anterior cingulate cortex in the rhesus monkey revealed by retrograde transport of HRP. J. Comp. Neurol. 205, 63–76 (1982).

    CAS  PubMed  Article  Google Scholar 

  33. Eblen, F. & Graybiel, A.M. Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J. Neurosci. 15, 5999–6013 (1995).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  34. Friedman, A. et al. A corticostriatal path targeting striosomes controls decision-making under conflict. Cell 161, 1320–1333 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  35. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2009).

    PubMed  PubMed Central  Article  Google Scholar 

  36. Kennerley, S.W., Behrens, T.E. & Wallis, J.D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  37. Rudebeck, P.H., Mitz, A.R., Chacko, R.V. & Murray, E.A. Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron 80, 1519–1531 (2013).

    CAS  Article  PubMed  Google Scholar 

  38. Hayden, B.Y., Pearson, J.M. & Platt, M.L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933–939 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  39. Blanchard, T.C. & Hayden, B.Y. Neurons in dorsal anterior cingulate cortex signal postdecisional variables in a foraging task. J. Neurosci. 34, 646–655 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  40. Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791–3808 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  41. Luk, C.H. & Wallis, J.D. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J. Neurosci. 33, 1864–1871 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  42. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).

    CAS  Article  PubMed  Google Scholar 

  43. Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

    PubMed  Article  Google Scholar 

  44. Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).

    PubMed  Article  Google Scholar 

  45. Ebitz, R.B. & Platt, M.L. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628–640 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  46. Sheth, S.A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioural adaptation. Nature 488, 218–221 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  47. McGuire, J.T. & Kable, J.W. Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence. Nat. Neurosci. 18, 760–766 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  48. Cai, X. & Padoa-Schioppa, C. Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation. Neuron 81, 1140–1151 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  49. Boorman, E.D., Behrens, T.E., Woolrich, M.W. & Rushworth, M.F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).

    CAS  Article  PubMed  Google Scholar 

  50. Strait, C.E., Blanchard, T.C. & Hayden, B.Y. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82, 1357–1366 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  51. Amemori, K. & Graybiel, A.M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  52. Wittmann, M.K. et al. Self-other mergence in frontal cortex during cooperation and competition. Neuron 91, 482–493 (2016).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  53. De Martino, B., Fleming, S.M., Garrett, N. & Dolan, R.J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).

    CAS  PubMed  Article  Google Scholar 

  54. Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).

    CAS  PubMed  Article  Google Scholar 

  55. Mars, R.B. et al. Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict. J. Neurosci. 29, 6926–6931 (2009).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  56. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  57. Hare, T.A., Schultz, W., Camerer, C.F., O'Doherty, J.P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. USA 108, 18120–18125 (2011).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  58. Wang, X.J. Decision making in recurrent neuronal circuits. Neuron 60, 215–234 (2008).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  59. Hunt, L.T. et al. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476, S1–S3 (2012).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  60. Freidin, E. & Kacelnik, A. Rational choice, context dependence, and the value of information in European starlings (Sturnus vulgaris). Science 334, 1000–1002 (2011).

    CAS  PubMed  Article  Google Scholar 

  61. Pearson, J.M., Watson, K.K. & Platt, M.L. Decision making: the neuroethological turn. Neuron 82, 950–965 (2014).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  62. Logiaco, L., Quilodran, R., Procyk, E. & Arleo, A. Spatiotemporal spike coding of behavioral adaptation in the dorsal anterior cingulate cortex. PLoS Biol. 13, e1002222 (2015).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  63. Voloh, B., Valiante, T.A., Everling, S. & Womelsdorf, T. Theta-gamma coordination between anterior cingulate and prefrontal cortex indexes correct attention shifts. Proc. Natl. Acad. Sci. USA 112, 8457–8462 (2015).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  64. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M. & Rushworth, M.F. Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168 (2006).

    CAS  Article  PubMed  Google Scholar 

  65. Posner, M.I. & Petersen, S.E. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42 (1990).

    CAS  Article  PubMed  Google Scholar 

  66. Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control? Brain 128, 788–796 (2005).

    PubMed  Article  Google Scholar 

  67. Rushworth, M.F.S., Hadland, K.A., Gaffan, D. & Passingham, R.E. The effect of cingulate cortex lesions on task switching and working memory. J. Cogn. Neurosci. 15, 338–353 (2003).

    CAS  PubMed  Article  Google Scholar 

  68. Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).

    CAS  Article  PubMed  Google Scholar 

  69. Mansouri, F.A., Buckley, M.J. & Tanaka, K. Mnemonic function of the dorsolateral prefrontal cortex in conflict-induced behavioral adjustment. Science 318, 987–990 (2007).

    CAS  PubMed  Article  Google Scholar 

  70. Camille, N., Tsuchida, A. & Fellows, L.K. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J. Neurosci. 31, 15048–15052 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  71. Buckley, M.J. et al. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52–58 (2009).

    CAS  PubMed  Article  Google Scholar 

  72. Thaler, D., Chen, Y.C., Nixon, P.D., Stern, C.E. & Passingham, R.E. The functions of the medial premotor cortex. I. Simple learned movements. Exp. Brain Res. 102, 445–460 (1995).

    CAS  PubMed  Article  Google Scholar 

  73. Stoll, F.M., Fontanier, V. & Procyk, E. Specific frontal neural dynamics contribute to decisions to check. Nat. Commun. http://dx.doi.org/10.1038/ncomms11990 (2016).

  74. Meder, D. et al. Tuning the brake while raising the stake: network dynamics during sequential decision-making. J. Neurosci. 36, 5417–5426 (2016).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  75. Klein-Flügge, M.C., Barron, H.C., Brodersen, K.H., Dolan, R.J. & Behrens, T.E. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J. Neurosci. 33, 3202–3211 (2013).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Experimental Psychology, University of Oxford, Oxford, UK

    Nils Kolling, Marco K Wittmann, Rogier B Mars & Matthew F S Rushworth

  2. Nuffield Department of Clinical Neurosciences, Centre for Functional MRI of the Brain (MRI), John Radcliffe Hospital, University of Oxford, Oxford, UK

    Tim E J Behrens, Erie D Boorman, Rogier B Mars & Matthew F S Rushworth

  3. Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, the Netherlands

    Rogier B Mars

Authors
  1. Nils Kolling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco K Wittmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim E J Behrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erie D Boorman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rogier B Mars
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthew F S Rushworth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nils Kolling.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kolling, N., Wittmann, M., Behrens, T. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat Neurosci 19, 1280–1285 (2016). https://doi.org/10.1038/nn.4382

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4382

Further reading

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing