Dorsal anterior cingulate cortex and the value of control

Journal name:
Nature Neuroscience
Volume:
19,
Pages:
1286–1291
Year published:
DOI:
doi:10.1038/nn.4384
Received
Accepted
Published online

Abstract

Debates over the function(s) of dorsal anterior cingulate cortex (dACC) have persisted for decades. So too have demonstrations of the region's association with cognitive control. Researchers have struggled to account for this association and, simultaneously, dACC's involvement in phenomena related to evaluation and motivation. We describe a recent integrative theory that achieves this goal. It proposes that dACC serves to specify the currently optimal allocation of control by determining the overall expected value of control (EVC), thereby licensing the associated cognitive effort. The EVC theory accounts for dACC's sensitivity to a wide array of experimental variables, and their relationship to subsequent control adjustments. Finally, we contrast our theory with a recent theory proposing a primary role for dACC in foraging-like decisions. We describe why the EVC theory offers a more comprehensive and coherent account of dACC function, including dACC's particular involvement in decisions regarding foraging or otherwise altering one's behavior.

At a glance

Figures

  1. dACC's proposed role in control allocation based on EVC.
    Figure 1: dACC's proposed role in control allocation based on EVC.

    Top: the EVC theory proposes that dACC monitors for information relevant to evaluating EVC and specifies the optimal control allocation to downstream regions. dACC is shown in the central panel, based on a Neurosynth meta-analysis of 428 human neuroimaging studies associated with cognitive control (http://neurosynth.org/analyses/terms/cognitive%20control/). Example input and output structures are shown at the left and right sides of the panel. Bottom: Previous findings suggest a role for dACC in using each of the monitoring signals listed (for example, errors, conflict) as the basis for one or more subsequent adjustments in control (for example, adjustment in one's speed-accuracy tradeoff or attentional focus). As indicated in the central panel, the EVC theory proposes that dACC evaluates the expected future benefits of applying varying intensities of control (arrow in each gauge) for each candidate control signal (different gauges; for example, different rules and the degree to which they could be attended) and subtracts from this the intrinsic cost of applying a given control intensity. This results in an estimate of the EVC. dACC then selects for execution the control signal settings that maximize EVC, projecting information about these signals to relevant downstream regions responsible for implementing the corresponding signals. Elsewhere we have provided a formal description of this process that aligns EVC and its components with ideas from artificial intelligence and control theory4, 74 and shown how a computational implementation of the theory can account for observed influences of incentives on control adjustments (and associated behavior)74. OFC: orbitofrontal cortex; STN: subthalamic nucleus; mPFC: medial prefrontal cortex; PFC: prefrontal cortex. Center panel adapted with permission from ref. 4, Elsevier.

  2. Decisions about engaging in a current task versus an alternate task from the perspective of FVT and EVC.
    Figure 2: Decisions about engaging in a current task versus an alternate task from the perspective of FVT and EVC.

    Left: FVT focuses on the expected reward for remaining on the current task (the default) and for switching to the alternate, nondefault task (including the time and resources consumed during the switch). (Note that these are not argued to be the only signals in dACC but to comprise the key foraging decision-related components of the dACC signal.) Right: by contrast, EVC considers the degree to which allocating different amounts of control to each task (separately or jointly) will accrue reward and effort-like costs. EVC also considers the switch costs in terms of both time and the demands of adjusting control signals, as well as the cognitive demands associated with decision conflict (as when the two tasks are mutually exclusive and close in value). Differences are highlighted in bold.

  3. Disentangling potential explanations for dACC involvement in foraging settings.
    Figure 3: Disentangling potential explanations for dACC involvement in foraging settings.

    (a) For foraging situations in which the value of the current or default behavior diminishes in a progressive manner (for example, through patch depletion, satiety, etc.), the marginal value theorem (MVT)66 predicts that it is optimal to switch when the value of the current behavior eventually reaches the value of switching (as estimated by the average expected value of the environment); this corresponds to the point of maximum decision difficulty (when the stay/switch options are closest in value), which is likely to increase the demands for control1, 15, 21 (for example, to support higher-fidelity judgments75 or adjustments in other default decision parameters such as response threshold15). Real-world foraging decisions therefore populate the area to the left of this indifference point. Foraging value signals can be examined past this indifference point, but care must be taken to estimate difficulty ideographically and to include sufficient choices on either side of the indifference point so that trials with high foraging values are neither especially difficult nor especially rare (i.e., potentially surprising)67. (b) We found that human dACC activity was best explained by an inverse U, with activity maximal around indifference and decreasing with less difficult choices. (c) Across two studies67, 68 we found that dACC was reliably engaged by the difficulty of foraging choices but saw no evidence of overlapping or adjacent regions signaling the value of foraging per se, even when excluding the easiest choices. Activations reflect liberally thresholded t-statistics (t ≥ 2.0) but dACC's choice-difficulty responses are robust to standard statistical thresholds. Panels adapted from ref. 67, Nature Publishing Group, and ref. 68, Psychonomic Society, Inc. Error bars reflect s.e.m.

References

  1. Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624652 (2001).
  2. Holroyd, C.B. & Yeung, N. Motivation of extended behaviors by anterior cingulate cortex. Trends Cogn. Sci. 16, 122128 (2012).
  3. Rushworth, M.F.S., Kolling, N., Sallet, J. & Mars, R.B. Valuation and decision-making in frontal cortex: one or many serial or parallel systems? Curr. Opin. Neurobiol. 22, 946955 (2012).
  4. Shenhav, A., Botvinick, M.M. & Cohen, J.D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217240 (2013).
  5. Heilbronner, S.R. & Hayden, B.Y. Dorsal anterior cingulate cortex: a bottom-up view. Annu. Rev. Neurosci. 39, 149170 (2016).
  6. Shackman, A.J. et al. The integration of negative affect, pain and cognitive control in the cingulate cortex. Nat. Rev. Neurosci. 12, 154167 (2011).
  7. Cole, M.W., Yeung, N., Freiwald, W.A. & Botvinick, M. Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 32, 566574 (2009).
  8. Vogt, B.A. Midcingulate cortex: Structure, connections, homologies, functions and diseases. J. Chem. Neuroanat. 74, 2846 (2016).
  9. Procyk, E. et al. Midcingulate motor map and feedback detection: converging data from humans and monkeys. Cereb. Cortex 26, 467476 (2016).
  10. Petersen, S.E. & Posner, M.I. The attention system of the human brain: 20 years after. Annu. Rev. Neurosci. 35, 7389 (2012).
  11. Botvinick, M.M. & Cohen, J.D. The computational and neural basis of cognitive control: charted territory and new frontiers. Cogn. Sci. 38, 12491285 (2014).
  12. Stuss, D.T. & Alexander, M.P. Is there a dysexecutive syndrome? Phil. Trans. R. Soc. Lond. B 362, 901915 (2007).
  13. Wallis, J.D. & Kennerley, S.W. Contrasting reward signals in the orbitofrontal cortex and anterior cingulate cortex. Ann. NY Acad. Sci. 1239, 3342 (2011).
  14. Ullsperger, M., Danielmeier, C. & Jocham, G. Neurophysiology of performance monitoring and adaptive behavior. Physiol. Rev. 94, 3579 (2014).
  15. Cavanagh, J.F. & Frank, M.J. Frontal theta as a mechanism for cognitive control. Trends Cogn. Sci. 18, 414421 (2014).
  16. Cohen, J.D., Dunbar, K. & McClelland, J.L. On the control of automatic processes: a parallel distributed processing account of the Stroop effect. Psychol. Rev. 97, 332361 (1990).
  17. Botvinick, M. & Braver, T. Motivation and cognitive control: from behavior to neural mechanism. Annu. Rev. Psychol. 66, 83113 (2015).
  18. Westbrook, A. & Braver, T.S. Cognitive effort: a neuroeconomic approach. Cogn. Affect. Behav. Neurosci. 15, 395415 (2015).
  19. Hare, T.A., Schultz, W., Camerer, C.F., O'Doherty, J.P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. USA 108, 1812018125 (2011).
  20. Hillman, K.L. & Bilkey, D.K. Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. J. Neurosci. 30, 77057713 (2010).
  21. Venkatraman, V. & Huettel, S.A. Strategic control in decision-making under uncertainty. Eur. J. Neurosci. 35, 10751082 (2012).
  22. Shen, C. et al. Anterior cingulate cortex cells identify process-specific errors of attentional control prior to transient prefrontal-cingulate inhibition. Cereb. Cortex 25, 22132228 (2015).
  23. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120122 (2003).
  24. Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884908 (2005).
  25. Narayanan, N.S., Cavanagh, J.F., Frank, M.J. & Laubach, M. Common medial frontal mechanisms of adaptive control in humans and rodents. Nat. Neurosci. 16, 18881895 (2013).
  26. Phillips, J.M. & Everling, S. Event-related potentials associated with performance monitoring in non-human primates. Neuroimage 97, 308320 (2014).
  27. Warren, C.M., Hyman, J.M., Seamans, J.K. & Holroyd, C.B. Feedback-related negativity observed in rodent anterior cingulate cortex. J. Physiol. Paris 109, 8794 (2015).
  28. Amiez, C., Sallet, J., Procyk, E. & Petrides, M. Modulation of feedback related activity in the rostral anterior cingulate cortex during trial and error exploration. Neuroimage 63, 10781090 (2012).
  29. Quilodran, R., Rothé, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314325 (2008).
  30. Ebitz, R.B. & Platt, M.L. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron 85, 628640 (2015).
  31. Sheth, S.A. et al. Human dorsal anterior cingulate cortex neurons mediate ongoing behavioural adaptation. Nature 488, 218221 (2012).
  32. Michelet, T. et al. Electrophysiological correlates of a versatile executive control system in the monkey anterior cingulate cortex. Cereb. Cortex 26, 16841697 (2016).
  33. Oehrn, C.R. et al. Neural communication patterns underlying conflict detection, resolution, and adaptation. J. Neurosci. 34, 1043810452 (2014).
  34. Tang, H. et al. Cascade of neural processing orchestrates cognitive control in human frontal cortex. eLife 5, 4671 (2016).
  35. McGuire, J.T., Nassar, M.R., Gold, J.I. & Kable, J.W. Functionally dissociable influences on learning rate in a dynamic environment. Neuron 84, 870881 (2014).
  36. O'Reilly, J.X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 110, E3660E3669 (2013).
  37. Kouneiher, F., Charron, S. & Koechlin, E. Motivation and cognitive control in the human prefrontal cortex. Nat. Neurosci. 12, 939945 (2009).
  38. Kaping, D., Vinck, M., Hutchison, R.M., Everling, S. & Womelsdorf, T. Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biol. 9, e1001224 (2011).
  39. Cavanagh, J.F. & Shackman, A.J. Frontal midline theta reflects anxiety and cognitive control: meta-analytic evidence. J. Physiol. Paris 109, 315 (2015).
  40. Mulder, M.J., Wagenmakers, E.-J., Ratcliff, R., Boekel, W. & Forstmann, B.U. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. J. Neurosci. 32, 23352343 (2012).
  41. Payzan-LeNestour, E., Dunne, S., Bossaerts, P. & O'Doherty, J.P. The neural representation of unexpected uncertainty during value-based decision making. Neuron 79, 191201 (2013).
  42. Ma, L., Hyman, J.M., Phillips, A.G. & Seamans, J.K. Tracking progress toward a goal in corticostriatal ensembles. J. Neurosci. 34, 22442253 (2014).
  43. Womelsdorf, T., Johnston, K., Vinck, M. & Everling, S. Theta-activity in anterior cingulate cortex predicts task rules and their adjustments following errors. Proc. Natl. Acad. Sci. USA 107, 52485253 (2010).
  44. Johnston, K., Levin, H.M., Koval, M.J. & Everling, S. Top-down control-signal dynamics in anterior cingulate and prefrontal cortex neurons following task switching. Neuron 53, 453462 (2007).
  45. Metzler-Baddeley, C. et al. Cingulum microstructure predicts cognitive control in older age and mild cognitive impairment. J. Neurosci. 32, 1761217619 (2012).
  46. de Wit, S. et al. Corticostriatal connectivity underlies individual differences in the balance between habitual and goal-directed action control. J. Neurosci. 32, 1206612075 (2012).
  47. Hochman, E.Y., Wang, S., Milner, T.E. & Fellows, L.K. Double dissociation of error inhibition and correction deficits after basal ganglia or dorsomedial frontal damage in humans. Neuropsychologia 69, 130139 (2015).
  48. Newman, L.A., Creer, D.J. & McGaughy, J.A. Cognitive control and the anterior cingulate cortex: how conflicting stimuli affect attentional control in the rat. J. Physiol. Paris 109, 95103 (2015).
  49. Kennerley, S.W., Walton, M.E., Behrens, T.E.J., Buckley, M.J. & Rushworth, M.F.S. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940947 (2006).
  50. Chudasama, Y. et al. The role of the anterior cingulate cortex in choices based on reward value and reward contingency. Cereb. Cortex 23, 28842898 (2013).
  51. Gläscher, J. et al. Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex. Proc. Natl. Acad. Sci. USA 109, 1468114686 (2012).
  52. Gaymard, B. et al. Effects of anterior cingulate cortex lesions on ocular saccades in humans. Exp. Brain Res. 120, 173183 (1998).
  53. Phillips, J.M., Johnston, K. & Everling, S. Effects of anterior cingulate microstimulation on pro- and antisaccades in nonhuman primates. J. Cogn. Neurosci. 23, 481490 (2011).
  54. Reinhart, R.M.G. & Woodman, G.F. Causal control of medial-frontal cortex governs electrophysiological and behavioral indices of performance monitoring and learning. J. Neurosci. 34, 42144227 (2014).
  55. Reinhart, R.M.G., Zhu, J., Park, S. & Woodman, G.F. Synchronizing theta oscillations with direct-current stimulation strengthens adaptive control in the human brain. Proc. Natl. Acad. Sci. USA 112, 94489453 (2015).
  56. Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control? Brain 128, 788796 (2005).
  57. Tolomeo, S. et al. A causal role for the anterior mid-cingulate cortex in negative affect and cognitive control. Brain 139, 18441854 (2016).
  58. McGuire, J.T. & Botvinick, M.M. Prefrontal cortex, cognitive control, and the registration of decision costs. Proc. Natl. Acad. Sci. USA 107, 79227926 (2010).
  59. Magno, E., Foxe, J.J., Molholm, S., Robertson, I.H. & Garavan, H. The anterior cingulate and error avoidance. J. Neurosci. 26, 47694773 (2006).
  60. Spunt, R.P., Lieberman, M.D., Cohen, J.R. & Eisenberger, N.I. The phenomenology of error processing: the dorsal ACC response to stop-signal errors tracks reports of negative affect. J. Cogn. Neurosci. 24, 17531765 (2012).
  61. Botvinick, M.M., Huffstetler, S. & McGuire, J.T. Effort discounting in human nucleus accumbens. Cogn. Affect. Behav. Neurosci. 9, 1627 (2009).
  62. Cavanagh, J.F., Masters, S.E., Bath, K. & Frank, M.J. Conflict acts as an implicit cost in reinforcement learning. Nat. Commun. 5, 5394 (2014).
  63. Hosking, J.G., Cocker, P.J. & Winstanley, C.A. Dissociable contributions of anterior cingulate cortex and basolateral amygdala on a rodent cost-benefit decision-making task of cognitive effort 39, 15581567 (2014).
  64. Parvizi, J., Rangarajan, V., Shirer, W.R., Desai, N. & Greicius, M.D. The will to persevere induced by electrical stimulation of the human cingulate gyrus. Neuron 80, 13591367 (2013).
  65. Kolling, N., Behrens, T.E.J., Mars, R.B. & Rushworth, M.F.S. Neural mechanisms of foraging. Science 336, 9598 (2012).
  66. Charnov, E.L. Optimal foraging, the marginal value theorem. Theor. Popul. Biol. 9, 129136 (1976).
  67. Shenhav, A., Straccia, M.A., Cohen, J.D. & Botvinick, M.M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nat. Neurosci. 17, 12491254 (2014).
  68. Shenhav, A., Straccia, M.A., Botvinick, M.M. & Cohen, J.D. Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice. Cogn. Affect. Behav. Neurosci. http://dx.doi.org/10.3758/s13415-016-0458-8 (2016).
  69. Schlund, M.W. et al. The tipping point: value differences and parallel dorsal-ventral frontal circuits gating human approach-avoidance behavior. Neuroimage 136, 94105 (2016).
  70. Kolling, N., Behrens, T., Wittmann, M.K. & Rushworth, M. Multiple signals in anterior cingulate cortex. Curr. Opin. Neurobiol. 37, 3643 (2016).
  71. Schall, J.D. & Emeric, E.E. Conflict in cingulate cortex function between humans and macaque monkeys: more apparent than real. Brain Behav. Evol. 75, 237238 (2010).
  72. Alexander, W.H. & Brown, J.W. Hierarchical error representation: a computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Comput. 27, 23542410 (2015).
  73. Westbrook, A. & Braver, T.S. Dopamine does double duty in motivating cognitive effort. Neuron 89, 695710 (2016).
  74. Musslick, S., Shenhav, A., Botvinick, M.M. & Cohen, J.D. A computational model of control allocation based on the expected value of control. in The 2nd Multidisciplinary Conference on Reinforcement Learning and Decision Making http://rldm.org/wp-content/uploads/2015/06/RLDM15AbstractsBooklet.pdf (2015).
  75. Manohar, S.G. et al. Reward pays the cost of noise reduction in motor and cognitive control. Curr. Biol. 25, 17071716 (2015).

Download references

Author information

Affiliations

  1. Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA.

    • Amitai Shenhav,
    • Jonathan D Cohen &
    • Matthew M Botvinick
  2. Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, Rhode Island, USA.

    • Amitai Shenhav
  3. Brown Institute for Brain Science, Brown University, Providence, Rhode Island, USA.

    • Amitai Shenhav
  4. Department of Psychology, Princeton University, Princeton, New Jersey, USA.

    • Jonathan D Cohen &
    • Matthew M Botvinick
  5. Google DeepMind, London, UK.

    • Matthew M Botvinick
  6. Gatsby Computational Neuroscience Unit, University College London, London, UK.

    • Matthew M Botvinick

Competing financial interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to:

Author details

Additional data