Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Reward prediction based on stimulus categorization in primate lateral prefrontal cortex

Abstract

To adapt to changeable or unfamiliar environments, it is important that animals develop strategies for goal-directed behaviors that meet the new challenges. We used a sequential paired-association task with asymmetric reward schedule to investigate how prefrontal neurons integrate multiple already-acquired associations to predict reward. Two types of reward-related neurons were observed in the lateral prefrontal cortex: one type predicted reward independent of physical properties of visual stimuli and the other encoded the reward value specific to a category of stimuli defined by the task requirements. Neurons of the latter type were able to predict reward on the basis of stimuli that had not yet been associated with reward, provided that another stimulus from the same category was paired with reward. The results suggest that prefrontal neurons can represent reward information on the basis of category and propagate this information to category members that have not been linked directly with any experience of reward.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The sequential paired-association task with an asymmetric reward schedule and monkeys' behavioral performance.
Figure 2: An example of a reward-type cell and population activity.
Figure 3: An example of a stimulus reward–type cell and population activity.
Figure 4: An example of a reward-type neuron and population activities in three sequences.
Figure 5: A typical stimulus reward–type neuron and population activities in three sequences.

Similar content being viewed by others

References

  1. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  2. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    CAS  PubMed  Google Scholar 

  3. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

    Article  CAS  PubMed  Google Scholar 

  4. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

    Article  CAS  PubMed  Google Scholar 

  6. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    Article  CAS  PubMed  Google Scholar 

  7. Miller, E.K. & Cohen, J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    Article  CAS  PubMed  Google Scholar 

  8. Sakagami, M., Pan, X. & Uttl, B. Behavioral inhibition and prefrontal cortex in decision-making. Neural Netw. 19, 1255–1265 (2006).

    Article  PubMed  Google Scholar 

  9. Webster, M.J., Bachevalier, J. & Ungerleider, L.G. Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys. Cereb. Cortex 4, 470–483 (1994).

    Article  CAS  PubMed  Google Scholar 

  10. Petrides, M. & Pandya, D.N. Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. J. Comp. Neurol. 228, 105–116 (1984).

    Article  CAS  PubMed  Google Scholar 

  11. Petrides, M. & Pandya, D.N. Dorsolateral prefrontal cortex: Comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur. J. Neurosci. 11, 1011–1036 (1999).

    Article  CAS  PubMed  Google Scholar 

  12. Miyachi, S. et al. Organization of multisynaptic inputs from prefrontal cortex to primary motor cortex as revealed by retrograde transneuronal transport of rabies virus. J. Neurosci. 25, 2547–2556 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nat. Rev. Neurosci. 2, 820–829 (2001).

    Article  CAS  PubMed  Google Scholar 

  14. Fuster, J.M. The Prefrontal Cortex: Anatomy, Physiology, and Neurophysiology of the Frontal Lobe (Lippincott-Raven, New York, 1997).

    Google Scholar 

  15. Goldman-Rakic, P.S. The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. Phil. Trans. R. Soc. Lond. B 351, 1445–1453 (1996).

    Article  CAS  Google Scholar 

  16. Passingham, R.E. Attention to action. in The Prefrontal Cortex: Executive and Cognitive Functions (eds. Roberts, A.C, Robbins, T.W. & Weiskrantz, L.) 131–143 (Oxford University Press, New York, 1998).

    Chapter  Google Scholar 

  17. Miller, E.K. The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1, 59–65 (2000).

    Article  CAS  PubMed  Google Scholar 

  18. Sakagami, M. & Niki, H. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp. Brain Res. 97, 423–436 (1994).

    Article  CAS  PubMed  Google Scholar 

  19. Sakagami, M. & Tsutsui, K. The hierarchical organization of decision making in primate prefrontal cortex. Neurosci. Res. 34, 79–89 (1999).

    Article  CAS  PubMed  Google Scholar 

  20. Hoshi, E. Functional specialization within the dorsolateral prefrontal cortex: a review of anatomical and physiological studies of nonhuman primates. Neurosci. Res. 54, 73–83 (2006).

    Article  PubMed  Google Scholar 

  21. White, I.M. & Wise, S.P. Rule-dependent neuronal activity in the prefrontal cortex. Exp. Brain Res. 126, 315–335 (1999).

    Article  CAS  PubMed  Google Scholar 

  22. Wallis, J.D., Anderson, K.C. & Miller, E.K. Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953–956 (2001).

    Article  CAS  PubMed  Google Scholar 

  23. Genovesio, A., Brasted, P.J., Mitz, A.R. & Wise, S.P. Prefrontal cortex activity related to abstract response strategies. Neuron 47, 307–320 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001).

    Article  CAS  PubMed  Google Scholar 

  25. Shima, K., Isoda, M., Mushiake, H. & Tanji, J. Categorization of behavioral sequences in the prefrontal cortex. Nature 445, 315–318 (2007).

    Article  CAS  PubMed  Google Scholar 

  26. O'Doherty, J., Critchley, H., Deichmann, R. & Dolan, R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).

    Article  CAS  PubMed  Google Scholar 

  28. Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).

    Article  CAS  PubMed  Google Scholar 

  29. Leon, M.I. & Shadlen, M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).

    Article  CAS  PubMed  Google Scholar 

  30. Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).

    Article  PubMed  Google Scholar 

  31. Roesch, M.R. & Olson, C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).

    Article  PubMed  Google Scholar 

  32. Kobayashi, S. et al. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51, 861–870 (2006).

    Article  CAS  PubMed  Google Scholar 

  33. Sakagami, M. & Watanabe, M. Integration of cognitive and motivational information in the primate lateral prefrontal cortex. Ann. NY Acad. Sci. 1104, 89–107 (2007).

    Article  PubMed  Google Scholar 

  34. Watanabe, M. Role of anticipated reward in cognitive behavioral control. Curr. Opin. Neurobiol. 17, 213–219 (2007).

    Article  CAS  PubMed  Google Scholar 

  35. Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).

    Article  CAS  PubMed  Google Scholar 

  36. McGonigle, B.O. & Chalmers, M. Are monkeys logical? Nature 267, 694–696 (1977).

    Article  CAS  PubMed  Google Scholar 

  37. Bunsey, M. & Eichenbaum, H. Conservation of hippocampal memory function in rats and humans. Nature 379, 255–257 (1996).

    Article  CAS  PubMed  Google Scholar 

  38. Blaisdell, A.P., Sawa, K., Leising, K.J. & Waldmann, M.R. Causal reasoning in rats. Science 311, 1020–1022 (2006).

    Article  CAS  PubMed  Google Scholar 

  39. Hauser, M. & Spaulding, B. Wild rhesus monkeys generate causal inferences about possible and impossible physical transformations in the absence of experience. Proc. Natl. Acad. Sci. USA 103, 7181–7185 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Herrnstein, R.J. Acquisition, generalization, and discrimination reversal of a natural concept. J. Exp. Psychol. Anim. Behav. Process. 5, 116–129 (1979).

    Article  CAS  PubMed  Google Scholar 

  41. Zentall, T.R. Symbolic representation in animals: emergent stimulus relations in conditional discrimination learning. Anim. Learn. Behav. 27, 363–377 (1998).

    Article  Google Scholar 

  42. Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J. Neurosci. 23, 5235–5246 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Zentall, T.R., Galizio, M. & Critchfield, T.S. Categorization, concept learning and behavior analysis: an introduction. J. Exp. Anal. Behav. 78, 237–248 (2002).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell signaling prediction errors of action values. Nat. Neurosci. 10, 647–655 (2007).

    Article  CAS  PubMed  Google Scholar 

  45. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Hampton, A.N., Bossaerts, P. & O'Doherty, J.P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360–8367 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We are grateful to M. Koizumi, K. Nomoto, A. Noritake, and S. Kobayashi for technical assistance and J. Lauwereyns for insightful comments and discussion on the manuscript. This work was supported by the Human Frontier Science Program, Precursory Research for Embryonic Science and Technology, Japan Science and Technology Corporation, Grant-in-Aid for Scientific Research on Priority Areas and Tamagawa University Center of Excellence from the Ministry of Education, Culture, Sports, Science and Technology. (M.S.).

Author information

Authors and Affiliations

Authors

Contributions

M.S., I.T., M.T. and X.P. designed the task. X.P. and K.S. conducted the experiments and data analyses. X.P. and M.S. wrote the manuscript.

Corresponding author

Correspondence to Masamichi Sakagami.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8 and Supplementary Tables 1 and 2 (PDF 3432 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pan, X., Sawa, K., Tsuda, I. et al. Reward prediction based on stimulus categorization in primate lateral prefrontal cortex. Nat Neurosci 11, 703–712 (2008). https://doi.org/10.1038/nn.2128

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2128

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing