Reward prediction based on stimulus categorization in primate lateral prefrontal cortex

Article metrics

Abstract

To adapt to changeable or unfamiliar environments, it is important that animals develop strategies for goal-directed behaviors that meet the new challenges. We used a sequential paired-association task with asymmetric reward schedule to investigate how prefrontal neurons integrate multiple already-acquired associations to predict reward. Two types of reward-related neurons were observed in the lateral prefrontal cortex: one type predicted reward independent of physical properties of visual stimuli and the other encoded the reward value specific to a category of stimuli defined by the task requirements. Neurons of the latter type were able to predict reward on the basis of stimuli that had not yet been associated with reward, provided that another stimulus from the same category was paired with reward. The results suggest that prefrontal neurons can represent reward information on the basis of category and propagate this information to category members that have not been linked directly with any experience of reward.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: The sequential paired-association task with an asymmetric reward schedule and monkeys' behavioral performance.
Figure 2: An example of a reward-type cell and population activity.
Figure 3: An example of a stimulus reward–type cell and population activity.
Figure 4: An example of a reward-type neuron and population activities in three sequences.
Figure 5: A typical stimulus reward–type neuron and population activities in three sequences.

References

  1. 1

    Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

  2. 2

    Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

  3. 3

    Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

  4. 4

    Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

  5. 5

    Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).

  6. 6

    O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

  7. 7

    Miller, E.K. & Cohen, J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

  8. 8

    Sakagami, M., Pan, X. & Uttl, B. Behavioral inhibition and prefrontal cortex in decision-making. Neural Netw. 19, 1255–1265 (2006).

  9. 9

    Webster, M.J., Bachevalier, J. & Ungerleider, L.G. Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys. Cereb. Cortex 4, 470–483 (1994).

  10. 10

    Petrides, M. & Pandya, D.N. Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. J. Comp. Neurol. 228, 105–116 (1984).

  11. 11

    Petrides, M. & Pandya, D.N. Dorsolateral prefrontal cortex: Comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur. J. Neurosci. 11, 1011–1036 (1999).

  12. 12

    Miyachi, S. et al. Organization of multisynaptic inputs from prefrontal cortex to primary motor cortex as revealed by retrograde transneuronal transport of rabies virus. J. Neurosci. 25, 2547–2556 (2005).

  13. 13

    Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nat. Rev. Neurosci. 2, 820–829 (2001).

  14. 14

    Fuster, J.M. The Prefrontal Cortex: Anatomy, Physiology, and Neurophysiology of the Frontal Lobe (Lippincott-Raven, New York, 1997).

  15. 15

    Goldman-Rakic, P.S. The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. Phil. Trans. R. Soc. Lond. B 351, 1445–1453 (1996).

  16. 16

    Passingham, R.E. Attention to action. in The Prefrontal Cortex: Executive and Cognitive Functions (eds. Roberts, A.C, Robbins, T.W. & Weiskrantz, L.) 131–143 (Oxford University Press, New York, 1998).

  17. 17

    Miller, E.K. The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1, 59–65 (2000).

  18. 18

    Sakagami, M. & Niki, H. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp. Brain Res. 97, 423–436 (1994).

  19. 19

    Sakagami, M. & Tsutsui, K. The hierarchical organization of decision making in primate prefrontal cortex. Neurosci. Res. 34, 79–89 (1999).

  20. 20

    Hoshi, E. Functional specialization within the dorsolateral prefrontal cortex: a review of anatomical and physiological studies of nonhuman primates. Neurosci. Res. 54, 73–83 (2006).

  21. 21

    White, I.M. & Wise, S.P. Rule-dependent neuronal activity in the prefrontal cortex. Exp. Brain Res. 126, 315–335 (1999).

  22. 22

    Wallis, J.D., Anderson, K.C. & Miller, E.K. Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953–956 (2001).

  23. 23

    Genovesio, A., Brasted, P.J., Mitz, A.R. & Wise, S.P. Prefrontal cortex activity related to abstract response strategies. Neuron 47, 307–320 (2005).

  24. 24

    Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001).

  25. 25

    Shima, K., Isoda, M., Mushiake, H. & Tanji, J. Categorization of behavioral sequences in the prefrontal cortex. Nature 445, 315–318 (2007).

  26. 26

    O'Doherty, J., Critchley, H., Deichmann, R. & Dolan, R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939 (2003).

  27. 27

    Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).

  28. 28

    Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).

  29. 29

    Leon, M.I. & Shadlen, M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).

  30. 30

    Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).

  31. 31

    Roesch, M.R. & Olson, C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).

  32. 32

    Kobayashi, S. et al. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51, 861–870 (2006).

  33. 33

    Sakagami, M. & Watanabe, M. Integration of cognitive and motivational information in the primate lateral prefrontal cortex. Ann. NY Acad. Sci. 1104, 89–107 (2007).

  34. 34

    Watanabe, M. Role of anticipated reward in cognitive behavioral control. Curr. Opin. Neurobiol. 17, 213–219 (2007).

  35. 35

    Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).

  36. 36

    McGonigle, B.O. & Chalmers, M. Are monkeys logical? Nature 267, 694–696 (1977).

  37. 37

    Bunsey, M. & Eichenbaum, H. Conservation of hippocampal memory function in rats and humans. Nature 379, 255–257 (1996).

  38. 38

    Blaisdell, A.P., Sawa, K., Leising, K.J. & Waldmann, M.R. Causal reasoning in rats. Science 311, 1020–1022 (2006).

  39. 39

    Hauser, M. & Spaulding, B. Wild rhesus monkeys generate causal inferences about possible and impossible physical transformations in the absence of experience. Proc. Natl. Acad. Sci. USA 103, 7181–7185 (2006).

  40. 40

    Herrnstein, R.J. Acquisition, generalization, and discrimination reversal of a natural concept. J. Exp. Psychol. Anim. Behav. Process. 5, 116–129 (1979).

  41. 41

    Zentall, T.R. Symbolic representation in animals: emergent stimulus relations in conditional discrimination learning. Anim. Learn. Behav. 27, 363–377 (1998).

  42. 42

    Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J. Neurosci. 23, 5235–5246 (2003).

  43. 43

    Zentall, T.R., Galizio, M. & Critchfield, T.S. Categorization, concept learning and behavior analysis: an introduction. J. Exp. Anal. Behav. 78, 237–248 (2002).

  44. 44

    Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell signaling prediction errors of action values. Nat. Neurosci. 10, 647–655 (2007).

  45. 45

    Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

  46. 46

    Hampton, A.N., Bossaerts, P. & O'Doherty, J.P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360–8367 (2006).

Download references

Acknowledgements

We are grateful to M. Koizumi, K. Nomoto, A. Noritake, and S. Kobayashi for technical assistance and J. Lauwereyns for insightful comments and discussion on the manuscript. This work was supported by the Human Frontier Science Program, Precursory Research for Embryonic Science and Technology, Japan Science and Technology Corporation, Grant-in-Aid for Scientific Research on Priority Areas and Tamagawa University Center of Excellence from the Ministry of Education, Culture, Sports, Science and Technology. (M.S.).

Author information

M.S., I.T., M.T. and X.P. designed the task. X.P. and K.S. conducted the experiments and data analyses. X.P. and M.S. wrote the manuscript.

Correspondence to Masamichi Sakagami.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8 and Supplementary Tables 1 and 2 (PDF 3432 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Pan, X., Sawa, K., Tsuda, I. et al. Reward prediction based on stimulus categorization in primate lateral prefrontal cortex. Nat Neurosci 11, 703–712 (2008) doi:10.1038/nn.2128

Download citation

Further reading