Article | Published:

Prefrontal cortex and decision making in a mixed-strategy game

Nature Neuroscience volume 7, pages 404410 (2004) | Download Citation

Subjects

Abstract

In a multi-agent environment, where the outcomes of one's actions change dynamically because they are related to the behavior of other beings, it becomes difficult to make an optimal decision about how to act. Although game theory provides normative solutions for decision making in groups, how such decision-making strategies are altered by experience is poorly understood. These adaptive processes might resemble reinforcement learning algorithms, which provide a general framework for finding optimal strategies in a dynamic environment. Here we investigated the role of prefrontal cortex (PFC) in dynamic decision making in monkeys. As in reinforcement learning, the animal's choice during a competitive game was biased by its choice and reward history, as well as by the strategies of its opponent. Furthermore, neurons in the dorsolateral prefrontal cortex (DLPFC) encoded the animal's past decisions and payoffs, as well as the conjunction between the two, providing signals necessary to update the estimates of expected reward. Thus, PFC might have a key role in optimizing decision-making strategies.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    & Theory of Games and Economic Behavior (Princeton Univ. Press, New Jersey, 1944).

  2. 2.

    & Game Theory (MIT Press, Cambridge, Massachusetts, 1991).

  3. 3.

    & Games of Strategy (Norton, New York, 1999).

  4. 4.

    Decisions, Uncertainty, and the Brain (MIT Press, Cambridge, Massachusetts, 2003).

  5. 5.

    Equilibrium points in n-person games. Proc. Natl. Acad. Sci. USA 36, 48–49 (1950).

  6. 6.

    An iterative method of solving a game. Ann. Math. 54, 296–301 (1951).

  7. 7.

    , & Does minimax work? An experimental study. Econ. J. 111, 445–464 (2001).

  8. 8.

    Decision processes, expectations, and adoption of strategies in zero-sum games. Hum. Relat. 27, 813–824 (1974).

  9. 9.

    Nonmetric test of the minimax theory of two-person zerosum games. Proc. Natl. Acad. Sci. USA 84, 2106–2109 (1987).

  10. 10.

    & Testing the minimax hypothesis: a re-examination of O'Neill's game experiment. Econometrica 58, 1065–1085 (1990).

  11. 11.

    & Mixed strategies in strictly competitive games: a further test of the minimax hypothesis. Games Econ. Behav. 4, 261–283 (1992).

  12. 12.

    & Generation of random series in two-person strictly competitive games. J. Exp. Psychol. Gen. 121, 352–363 (1992).

  13. 13.

    & Subjective randomization in one- and two-person games. J. Behav. Dec. Making 7, 261–278 (1994).

  14. 14.

    Games with unique, mixed strategy equilibria: an experimental study. Games Econ. Behav. 10, 202–217 (1995).

  15. 15.

    & Predicting how people play games: a simple dynamic model of choice. Games Econ. Behav. 34, 104–122 (2001).

  16. 16.

    Mixed strategy play and the minimax hypothesis. J. Econ. Theory 104, 189–226 (2002).

  17. 17.

    & Minimax play at Wimbledon. Am. Econ. Rev. 91, 1521–1538 (2001).

  18. 18.

    & Theory of Learning in Games (MIT Press, Cambridge, Massachusetts, 1998).

  19. 19.

    & Learning behavior in an experimental matching pennies game. Games Econ. Behav. 7, 62–91 (1994).

  20. 20.

    & Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848–881 (1998).

  21. 21.

    Behavioral Game Theory (Princeton Univ. Press, Princeton, New Jersey, 2003).

  22. 22.

    Models of Man (Wiley, New York, 1957).

  23. 23.

    , & Judgement Under Uncertainty: Heuristics and Biases (Cambridge Univ. Press, Cambridge, UK, 1982).

  24. 24.

    Comments on Brown and Rosenthal's reexamination. Econometrica 59, 503–507 (1991).

  25. 25.

    & Randomization in individual choice behavior. Psychol. Rev. 104, 603–617 (1997).

  26. 26.

    , & Boundedly rational Nash equilibrium: a probabilistic choice approach. Games Econ. Behav. 18, 32–54 (1997).

  27. 27.

    Modeling Bounded Rationality (MIT Press, Cambridge, Massachusetts, 1998).

  28. 28.

    & Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

  29. 29.

    Markov games as a framework for multi-agent reinforcement learning. Machine Learning: Proc. 11th Int. Conf. pp. 157–163 (Morgan Kaufmann, San Francisco, California, 1994).

  30. 30.

    Log-linear Models and Logistic Regression edn. 2 (Springer-Verlag, New York, 1997).

  31. 31.

    & Model Selection and Multimodel Inference 2nd edn. (Springer-Verlag, New York, 2002).

  32. 32.

    & Machiavellian Intelligence: Social Expertise and the Evolution of Intellect in Monkeys, Apes and Humans (Oxford Univ. Press, Oxford, UK, 1988).

  33. 33.

    & Machiavellian Intelligence II: Extensions and Evaluations (Cambridge Univ. Press, Cambridge, UK, 1997).

  34. 34.

    Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).

  35. 35.

    Brain and Emotion (Oxford Univ. Press, Oxford, UK, 1999).

  36. 36.

    , & Reward-predicting and reward detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84, 2166–2170 (2000).

  37. 37.

    , & Performance monitoring by the supplementary eye field. Nature 408, 857–860 (2000).

  38. 38.

    , , & Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

  39. 39.

    , & Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998).

  40. 40.

    & Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).

  41. 41.

    & Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

  42. 42.

    & Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39, 693–700 (2003).

  43. 43.

    , , , & Saccade reward signals in posterior cingulate cortex. Neuron 40, 1031–1040 (2003).

  44. 44.

    Reward expectancy in primate prefrontal cortex. Nature 382, 629–632 (1996).

  45. 45.

    & Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).

  46. 46.

    , , , & Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).

  47. 47.

    & Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).

  48. 48.

    & Neuronal representation of response outcome in the primate prefrontal cortex. Cereb. Cortex 14, 47–55 (2004).

  49. 49.

    Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

  50. 50.

    , , & Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. J. Neurophysiol. 54, 714–734 (1985).

Download references

Acknowledgements

We thank L. Carr, R. Farrell, B. McGreevy and T. Twietmeyer for their technical assistance, J. Swan-Stone for programming, X.-J. Wang for discussions, and B. Averbeck and J. Malpeli for critically reading the manuscript. This work was supported by the James S. McDonnell Foundation and the National Institutes of Health (NS44270 and EY01319).

Author information

Affiliations

  1. Department of Brain and Cognitive Sciences, Center for Visual Science, University of Rochester, Rochester, New York 14627, USA.

    • Dominic J Barraclough
    • , Michelle L Conroy
    •  & Daeyeol Lee

Authors

  1. Search for Dominic J Barraclough in:

  2. Search for Michelle L Conroy in:

  3. Search for Daeyeol Lee in:

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Daeyeol Lee.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn1209

Further reading