Article | Published:

Medial prefrontal cortex as an action-outcome predictor

Nature Neuroscience volume 14, pages 13381344 (2011) | Download Citation

Abstract

The medial prefrontal cortex (mPFC) and especially anterior cingulate cortex is central to higher cognitive function and many clinical disorders, yet its basic function remains in dispute. Various competing theories of mPFC have treated effects of errors, conflict, error likelihood, volatility and reward, using findings from neuroimaging and neurophysiology in humans and monkeys. No single theory has been able to reconcile and account for the variety of findings. Here we show that a simple model based on standard learning rules can simulate and unify an unprecedented range of known effects in mPFC. The model reinterprets many known effects and suggests a new view of mPFC, as a region concerned with learning and predicting the likely outcomes of actions, whether good or bad. Cognitive control at the neural level is then seen as a result of evaluating the probable and actual outcomes of one's actions.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    , , & Anterior cingulate cortex activity and impaired self-monitoring of performance in patients with schizophrenia: an event-related fMRI study. Am. J. Psychiatry 158, 1423–1428 (2001).

  2. 2.

    , , & The error-related negativity: An event-related potential accompanying errors. Psychophysiology 27, S34 (1990).

  3. 3.

    , , & Effects of crossmodal divided attention on late ERP components: II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78, 447–455 (1991).

  4. 4.

    et al. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749 (1998).

  5. 5.

    , , , & Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

  6. 6.

    & Neuronal activity related to rule and conflict in macaque supplementary eye field. Physiol. Behav. 77, 663–670 (2002).

  7. 7.

    , , & Performance monitoring by anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

  8. 8.

    & Role of cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998).

  9. 9.

    , & Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301, 229–232 (2003).

  10. 10.

    , , & Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

  11. 11.

    , & Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 21, 3447–3452 (2005).

  12. 12.

    & Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. J. Exp. Psychol. Hum. Percept. Perform. 26, 141–151 (2000).

  13. 13.

    & The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

  14. 14.

    & Risk prediction and aversion by anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 266–277 (2007).

  15. 15.

    & Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).

  16. 16.

    , , & Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

  17. 17.

    , & Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265 (2004).

  18. 18.

    et al. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28, 13775–13785 (2008).

  19. 19.

    , , & Cingulate cortex: diverging data from humans and monkeys. Trends Neurosci. 32, 566–574 (2009).

  20. 20.

    , , & BOLD fMRI activation for anti-saccades in nonhuman primates. Neuroimage 45, 470–476 (2009).

  21. 21.

    Human volition: towards a neuroscience of will. Nat. Rev. Neurosci. 9, 934–946 (2008).

  22. 22.

    , & Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. J. Neurosci. 28, 4671–4678 (2008).

  23. 23.

    Conflict effects without conflict in anterior cingulate cortex: multiple response effects and context specific representations. Neuroimage 47, 334–341 (2009).

  24. 24.

    , , & Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2009).

  25. 25.

    , , , & Effort-based cost-benefit valuation and the human brain. J. Neurosci. 29, 4531–4541 (2009).

  26. 26.

    , , & Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855–867 (2003).

  27. 27.

    et al. Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 327–336 (2007).

  28. 28.

    , & Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84, 2166–2170 (2000).

  29. 29.

    , & Functional heterogeneity of conflict, error, task-switching, and unexpectedness effects within medial prefrontal cortex. Neuroimage 54, 528–540 (2011).

  30. 30.

    , & Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat. Neurosci. 3, 502–508 (2000).

  31. 31.

    & Reward prediction error signals associated with a modified time estimation task. Psychophysiology 44, 913–917 (2007).

  32. 32.

    , & Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a 'generic' neural system for error-detection. J. Cogn. Neurosci. 9, 788–798 (1997).

  33. 33.

    & Dissociating response conflict and error likelihood in anterior cingulate cortex. J. Neurosci. 29, 14506–14510 (2009).

  34. 34.

    , , , & Error negativity does not reflect conflict: a reappraisal of conflict monitoring and anterior cingulate cortex activity. J. Cogn. Neurosci. 20, 1637–1655 (2008).

  35. 35.

    , & Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040–1055 (2006).

  36. 36.

    , & Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).

  37. 37.

    Multiple cognitive control effects of error likelihood and conflict. Psychol. Res. 73, 744–750 (2009).

  38. 38.

    , & Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).

  39. 39.

    , & Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J. Cogn. Neurosci. 19, 1994–2004 (2007).

  40. 40.

    , & Error effects in anterior cingulate cortex reverse when error likelihood is high. J. Neurosci. 30, 3467–3472 (2010).

  41. 41.

    & Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

  42. 42.

    , & Explaining the many varieties of working memory variation: dual mechanisms of cognitive control. in Variation of Working Memory (eds. Conway, C.J.A., Kane, M., Miyake, A. & Towse, J.) 76–106 (Oxford University Press, 2007).

  43. 43.

    & Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).

  44. 44.

    , , & A mechanism for error detection in speeded response time tasks. J. Exp. Psychol. Gen. 134, 163–191 (2005).

  45. 45.

    & Reinforcement learning with replacing eligibility traces. Mach. Learn. 22, 123–158 (1996).

  46. 46.

    & Dayan, P. Q-learning. Mach. Learn. 8, 279–292 (1992).

  47. 47.

    , , & Multiple Model-Based Reinforcement Learning. Neural Comput. 14, 1347–1369 (2002).

  48. 48.

    , , & States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).

  49. 49.

    & A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532–552 (1980).

  50. 50.

    et al. Dorsal anterior cingulate cortex: a role in reward-based decision making. Proc. Natl. Acad. Sci. USA 99, 523–528 (2002).

Download references

Acknowledgements

We thank L. Pessoa, S. Padmala, T. Braver, V. Stuphorn, A. Krawitz, D. Nee and the J. Schall laboratory for critical feedback. Supported in part by Air Force Office of Scientific Research FA9550-07-1-0454 (J.W.B.), R03 DA023462 (J.W.B.), R01 DA026457 (J.W.B.), a NARSAD Young Investigator Award (J.W.B.) and the Sidney R. Baer Jr. Foundation (J.W.B.). Supported by the Intelligence Advanced Research Projects Activity (IARPA) through Department of the Interior (DOI) contract D10PC20023. The US Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DOI or the US Government.

Author information

Affiliations

  1. Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana, USA.

    • William H Alexander
    •  & Joshua W Brown

Authors

  1. Search for William H Alexander in:

  2. Search for Joshua W Brown in:

Contributions

J.W.B. and W.H.A. conceptualized the model. W.H.A. implemented the model and ran the simulations. J.W.B. and W.H.A. wrote the manuscript.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Joshua W Brown.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figure 1, Supplementary Note and Supplementary Discussion

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.2921

Further reading