Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Medial prefrontal cell activity signaling prediction errors of action values

Nature Neuroscience volume 10, pages 647–656 (2007)Cite this article

Abstract

To adapt behavior to a changing environment, one must monitor outcomes of executed actions and adjust subsequent actions accordingly. Involvement of the medial frontal cortex in performance monitoring has been suggested, but little is known about neural processes that link performance monitoring to performance adjustment. Here, we recorded from neurons in the medial prefrontal cortex of monkeys learning arbitrary action-outcome contingencies. Some cells preferentially responded to positive visual feedback stimuli and others to negative feedback stimuli. The magnitude of responses to positive feedback stimuli decreased over the course of behavioral adaptation, in correlation with decreases in the amount of prediction error of action values. Therefore, these responses in medial prefrontal cells may signal the direction and amount of error in prediction of values of executed actions to specify the adjustment in subsequent action selections.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Task design and behavioral results.
Figure 2: Recording regions in two monkeys.
Figure 3: The representation of the type (positive versus negative) of feedback by medial PFC cells.
Figure 4: Effects of stimulus novelty examined in responses in visual blocks.
Figure 5: Changes of neuronal responses along the course of correct-action learning: single cell examples.
Figure 6: Changes of neuronal responses along the course of correct-action learning: population responses.
Figure 7: Relationship between neuronal responses and prediction errors.

Similar content being viewed by others

References

  1. Woodworth, R.S. Dynamics of Behavior (Holt, New York, 1958).

    Google Scholar 

  2. Daw, N.D. & Doya, K. The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16, 199–204 (2006).

    Article  CAS  Google Scholar 

  3. Matsumoto, K. & Tanaka, K. The role of the medial prefrontal cortex in achieving goals. Curr. Opin. Neurobiol. 14, 178–185 (2004).

    Article  CAS  Google Scholar 

  4. Rushworth, M.F., Walton, M.E., Kennerley, S.W. & Bannerman, D.M. Action sets and decisions in the medial frontal cortex. Trends Cogn. Sci. 8, 410–417 (2004).

    Article  CAS  Google Scholar 

  5. Falkenstein, M., Hohnsbein, J., Hoormann, J. & Blanke, L. Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78, 447–455 (1991).

    Article  CAS  Google Scholar 

  6. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E. & Donchin, E. A neural system for error detection and compensation. Psychol. Sci. 4, 385–390 (1993).

    Article  Google Scholar 

  7. Miltner, W.H.R., Braun, C.H. & Coles, M.G.H. Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. J. Cogn. Neurosci. 9, 788–798 (1997).

    Article  CAS  Google Scholar 

  8. Carter, C.S. et al. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749 (1998).

    Article  CAS  Google Scholar 

  9. Ullsperger, M. & von Cramon, D.Y. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J. Neurosci. 23, 4308–4314 (2003).

    Article  CAS  Google Scholar 

  10. Holroyd, C.B. et al. Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat. Neurosci. 7, 497–498 (2004).

    Article  CAS  Google Scholar 

  11. Mars, R.B. et al. Neural dynamics of error processing in medial frontal cortex. Neuroimage 28, 1007–1013 (2005).

    Article  Google Scholar 

  12. Niki, H. & Watanabe, M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 171, 213–224 (1979).

    Article  CAS  Google Scholar 

  13. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

    Article  CAS  Google Scholar 

  14. Bartholow, B.D. et al. Strategic control and medial frontal negativity: beyond errors and response conflict. Psychophysiology 42, 33–42 (2005).

    Article  Google Scholar 

  15. Pailing, P.E. & Segalowitz, S.J. The effects of uncertainty in error monitoring on associated ERPs. Brain Cogn. 56, 215–233 (2004).

    Article  Google Scholar 

  16. Vidal, F., Burle, B., Bonnet, M., Grapperon, J. & Hasbroucq, T. Error negativity on correct trials: a reexamination of available data. Biol. Psychol. 64, 265–282 (2003).

    Article  CAS  Google Scholar 

  17. Knutson, B., Westdorp, A., Kaiser, E. & Hommer, D. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12, 20–27 (2000).

    Article  CAS  Google Scholar 

  18. Walton, M.E., Devlin, J.T. & Rushworth, M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265 (2004).

    Article  CAS  Google Scholar 

  19. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  Google Scholar 

  20. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  21. Frank, M.J., Woroch, B.S. & Curran, T. Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47, 495–501 (2005).

    Article  CAS  Google Scholar 

  22. Holroyd, C.B., Nieuwenhuis, S., Yeung, N. & Cohen, J.D. Errors in reward prediction are reflected in the event-related brain potential. Neuroreport 14, 2481–2484 (2003).

    Article  Google Scholar 

  23. Amiez, C., Joseph, J.P. & Procyk, E. Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 21, 3447–3452 (2005).

    Article  Google Scholar 

  24. Gehring, W.J. & Knight, R.T. Prefrontal-cingulate interactions in action monitoring. Nat. Neurosci. 3, 516–520 (2000).

    Article  CAS  Google Scholar 

  25. Watkins, C. & Dayan, P. Q-learning. Mach. Learn. 8, 279–292 (1992).

    Google Scholar 

  26. Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  27. Yeung, N. & Sanfey, A.G. Independent coding of reward magnitude and valence in the human brain. J. Neurosci. 24, 6258–6264 (2004).

    Article  CAS  Google Scholar 

  28. Donkers, F.C., Nieuwenhuis, S. & van Boxtel, G.J. Mediofrontal negativities in the absence of responding. Brain Res. Cogn. Brain Res. 25, 777–787 (2005).

    Article  Google Scholar 

  29. Lewis, D.A., Foote, S.L., Goldstein, M. & Morrison, J.H. The dopaminergic innervation of monkey prefrontal cortex: a tyrosine hydroxylase immunohistochemical study. Brain Res. 449, 225–243 (1988).

    Article  CAS  Google Scholar 

  30. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

    Article  CAS  Google Scholar 

  31. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  32. Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci. 23, 9913–9923 (2003).

    Article  CAS  Google Scholar 

  33. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).

    Article  Google Scholar 

  34. Robbins, T.W. Chemistry of the mind: neurochemical modulation of prefrontal cortical function. J. Comp. Neurol. 493, 140–146 (2005).

    Article  CAS  Google Scholar 

  35. Ullsperger, M. & von Cramon, D.Y. The role of intact frontostriatal circuits in error processing. J. Cogn. Neurosci. 18, 651–664 (2006).

    Article  Google Scholar 

  36. Fantino, E. Choice and rate of reinforcement. J. Exp. Anal. Behav. 12, 723–730 (1969).

    Article  CAS  Google Scholar 

  37. Skinner, B.F. Science and Human Behavior (Macmillan, New York, 1953).

    Google Scholar 

  38. Williams, B.A. Conditioned reinforcement: experimental and theoretical issues. Behav. Anal. 17, 261–285 (1994).

    Article  CAS  Google Scholar 

  39. Parkinson, J.A. et al. The role of the primate amygdala in conditioned reinforcement. J. Neurosci. 21, 7770–7780 (2001).

    Article  CAS  Google Scholar 

  40. Allport, G.W. Pattern and Growth in Personality (Holt, Rinehart and Winston, New York, 1961).

    Google Scholar 

  41. Maslow, A.H. Motivation and Personality (Harper & Row, New York, 1970).

  42. Schultz, W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 1, 199–207 (2000).

    Article  CAS  Google Scholar 

  43. Blatter, K. & Schultz, W. Rewarding properties of visual stimuli. Exp. Brain Res. 168, 541–546 (2006).

    Article  Google Scholar 

  44. Carmichael, S.T. & Price, J.L. Architectonic subdivision of the orbital and medial prefrontal cortex in the macaque monkey. J. Comp. Neurol. 346, 366–402 (1994).

    Article  CAS  Google Scholar 

  45. Barbas, H. & Pandya, D.N. Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 286, 353–375 (1989).

    Article  CAS  Google Scholar 

  46. Walker, A.E. A cytoarchitectural study of the prefrontal area of the macaque monkey. J. Comp. Neurol. 73, 59–86 (1940).

    Article  Google Scholar 

  47. Shima, K. et al. Two movement-related foci in the primate cingulate cortex observed in signal-triggered and self-paced forelimb movements. J. Neurophysiol. 65, 188–202 (1991).

    Article  CAS  Google Scholar 

  48. Procyk, E., Tanaka, Y.L. & Joseph, J.P. Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat. Neurosci. 3, 502–508 (2000).

    Article  CAS  Google Scholar 

  49. Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

    Article  Google Scholar 

  50. Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J. Neurophysiol. 93, 884–908 (2005).

    Article  Google Scholar 

Download references

Acknowledgements

This research was partly supported by the Grant-in-Aid for Scientific Research on Priority Areas (17022047) from the Ministry of Education, Culture, Sports, Science and Technology of Japan. We thank W. Schultz for advice about task design, S. Shimamune and K. Murayama for discussion, R. A. Waggoner for taking MRI images, A. Phillips for developing a program for presenting visual stimuli, J. Helen for improving the English, and M. Tomonaga, H. Nakahara and W. Schultz for comments on an early manuscript.

Author information

Author notes

  1. Madoka Matsumoto, Kenji Matsumoto and Hiroshi Abe: These authors contributed equally to this work.

Authors and Affiliations

  1. Cognitive Brain Mapping Laboratory, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako, 351-0198, Saitama, Japan

    Madoka Matsumoto, Kenji Matsumoto, Hiroshi Abe & Keiji Tanaka

  2. Department of Behavioral and Brain Sciences, Primate Research Institute, Kyoto University, Kanrin, Inuyama, 484-8506, Aichi, Japan

    Madoka Matsumoto

  3. Graduate School of Decision Science and Technology, Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, 152-8552, Tokyo, Japan

    Hiroshi Abe

  4. Graduate School of Science and Engineering, Saitama University, 255 Shimoohkubo, Sakura-ku, Saitama, 338-8570, Saitama, Japan

    Keiji Tanaka

Authors
  1. Madoka Matsumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenji Matsumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiroshi Abe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Keiji Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenji Matsumoto.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Recording areas in two monkeys. (PDF 974 kb)

Supplementary Fig. 2

Learning curves of the monkeys and models, in action-learning blocks. (PDF 1181 kb)

Supplementary Fig. 3

Relationship between neuronal responses and prediction errors in each monkey. (PDF 1092 kb)

Supplementary Fig. 4

Distribution of positive-feedback preferring cells and negative-feedback preferring cells in the medial PFC. (PDF 1176 kb)

Supplementary Fig. 5

Comparison of the medial PFC cells' responses between visual blocks and action-learning blocks. (PDF 1055 kb)

Supplementary Fig. 6

Responses determined by the feedback type (positive/negative) but not by stay/shift of actions. (PDF 1539 kb)

Supplementary Table 1

Values of model parameters with which each model best fit the actual performance of monkeys. (PDF 802 kb)

Supplementary Methods (PDF 79 kb)

Supplementary Note (PDF 195 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Matsumoto, M., Matsumoto, K., Abe, H. et al. Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10, 647–656 (2007). https://doi.org/10.1038/nn1890

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1890

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing