Abstract
To adapt to changeable or unfamiliar environments, it is important that animals develop strategies for goal-directed behaviors that meet the new challenges. We used a sequential paired-association task with asymmetric reward schedule to investigate how prefrontal neurons integrate multiple already-acquired associations to predict reward. Two types of reward-related neurons were observed in the lateral prefrontal cortex: one type predicted reward independent of physical properties of visual stimuli and the other encoded the reward value specific to a category of stimuli defined by the task requirements. Neurons of the latter type were able to predict reward on the basis of stimuli that had not yet been associated with reward, provided that another stimulus from the same category was paired with reward. The results suggest that prefrontal neurons can represent reward information on the basis of category and propagate this information to category members that have not been linked directly with any experience of reward.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).
Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).
Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).
O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).
Miller, E.K. & Cohen, J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).
Sakagami, M., Pan, X. & Uttl, B. Behavioral inhibition and prefrontal cortex in decision-making. Neural Netw. 19, 1255–1265 (2006).
Webster, M.J., Bachevalier, J. & Ungerleider, L.G. Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys. Cereb. Cortex 4, 470–483 (1994).
Petrides, M. & Pandya, D.N. Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. J. Comp. Neurol. 228, 105–116 (1984).
Petrides, M. & Pandya, D.N. Dorsolateral prefrontal cortex: Comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur. J. Neurosci. 11, 1011–1036 (1999).
Miyachi, S. et al. Organization of multisynaptic inputs from prefrontal cortex to primary motor cortex as revealed by retrograde transneuronal transport of rabies virus. J. Neurosci. 25, 2547–2556 (2005).
Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nat. Rev. Neurosci. 2, 820–829 (2001).
Fuster, J.M. The Prefrontal Cortex: Anatomy, Physiology, and Neurophysiology of the Frontal Lobe (Lippincott-Raven, New York, 1997).
Goldman-Rakic, P.S. The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. Phil. Trans. R. Soc. Lond. B 351, 1445–1453 (1996).
Passingham, R.E. Attention to action. in The Prefrontal Cortex: Executive and Cognitive Functions (eds. Roberts, A.C, Robbins, T.W. & Weiskrantz, L.) 131–143 (Oxford University Press, New York, 1998).
Miller, E.K. The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1, 59–65 (2000).
Sakagami, M. & Niki, H. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp. Brain Res. 97, 423–436 (1994).
Sakagami, M. & Tsutsui, K. The hierarchical organization of decision making in primate prefrontal cortex. Neurosci. Res. 34, 79–89 (1999).
Hoshi, E. Functional specialization within the dorsolateral prefrontal cortex: a review of anatomical and physiological studies of nonhuman primates. Neurosci. Res. 54, 73–83 (2006).
White, I.M. & Wise, S.P. Rule-dependent neuronal activity in the prefrontal cortex. Exp. Brain Res. 126, 315–335 (1999).
Wallis, J.D., Anderson, K.C. & Miller, E.K. Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953–956 (2001).
Genovesio, A., Brasted, P.J., Mitz, A.R. & Wise, S.P. Prefrontal cortex activity related to abstract response strategies. Neuron 47, 307–320 (2005).
Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001).
Shima, K., Isoda, M., Mushiake, H. & Tanji, J. Categorization of behavioral sequences in the prefrontal cortex. Nature 445, 315–318 (2007).
O'Doherty, J., Critchley, H., Deichmann, R. & Dolan, R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939 (2003).
Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).
Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).
Leon, M.I. & Shadlen, M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).
Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).
Roesch, M.R. & Olson, C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).
Kobayashi, S. et al. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51, 861–870 (2006).
Sakagami, M. & Watanabe, M. Integration of cognitive and motivational information in the primate lateral prefrontal cortex. Ann. NY Acad. Sci. 1104, 89–107 (2007).
Watanabe, M. Role of anticipated reward in cognitive behavioral control. Curr. Opin. Neurobiol. 17, 213–219 (2007).
Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).
McGonigle, B.O. & Chalmers, M. Are monkeys logical? Nature 267, 694–696 (1977).
Bunsey, M. & Eichenbaum, H. Conservation of hippocampal memory function in rats and humans. Nature 379, 255–257 (1996).
Blaisdell, A.P., Sawa, K., Leising, K.J. & Waldmann, M.R. Causal reasoning in rats. Science 311, 1020–1022 (2006).
Hauser, M. & Spaulding, B. Wild rhesus monkeys generate causal inferences about possible and impossible physical transformations in the absence of experience. Proc. Natl. Acad. Sci. USA 103, 7181–7185 (2006).
Herrnstein, R.J. Acquisition, generalization, and discrimination reversal of a natural concept. J. Exp. Psychol. Anim. Behav. Process. 5, 116–129 (1979).
Zentall, T.R. Symbolic representation in animals: emergent stimulus relations in conditional discrimination learning. Anim. Learn. Behav. 27, 363–377 (1998).
Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J. Neurosci. 23, 5235–5246 (2003).
Zentall, T.R., Galizio, M. & Critchfield, T.S. Categorization, concept learning and behavior analysis: an introduction. J. Exp. Anal. Behav. 78, 237–248 (2002).
Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell signaling prediction errors of action values. Nat. Neurosci. 10, 647–655 (2007).
Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).
Hampton, A.N., Bossaerts, P. & O'Doherty, J.P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360–8367 (2006).
Acknowledgements
We are grateful to M. Koizumi, K. Nomoto, A. Noritake, and S. Kobayashi for technical assistance and J. Lauwereyns for insightful comments and discussion on the manuscript. This work was supported by the Human Frontier Science Program, Precursory Research for Embryonic Science and Technology, Japan Science and Technology Corporation, Grant-in-Aid for Scientific Research on Priority Areas and Tamagawa University Center of Excellence from the Ministry of Education, Culture, Sports, Science and Technology. (M.S.).
Author information
Authors and Affiliations
Contributions
M.S., I.T., M.T. and X.P. designed the task. X.P. and K.S. conducted the experiments and data analyses. X.P. and M.S. wrote the manuscript.
Corresponding author
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–8 and Supplementary Tables 1 and 2 (PDF 3432 kb)
Rights and permissions
About this article
Cite this article
Pan, X., Sawa, K., Tsuda, I. et al. Reward prediction based on stimulus categorization in primate lateral prefrontal cortex. Nat Neurosci 11, 703–712 (2008). https://doi.org/10.1038/nn.2128
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.2128
This article is cited by
-
Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons
Nature Communications (2023)
-
Proselfs depend more on model-based than model-free learning in a non-social probabilistic state-transition task
Scientific Reports (2023)
-
Brain works principle followed by neural information processing: a review of novel brain theory
Artificial Intelligence Review (2023)
-
Chemogenetic inactivation reveals the inhibitory control function of the prefronto-striatal pathway in the macaque brain
Communications Biology (2021)