Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

The neurobiology of punishment

Nature Reviews Neuroscience volume 8, pages 300–311 (2007)Cite this article

Key Points

  • Punishment is common across species, where it often serves to protect the personal interests of the animal concerned. In humans, it seems to promote and preserve cooperative behaviour, according to culturally acquired (moral) norms of behaviour. This extends to the existence of altruistic punishment, in which the act of punishing is personally costly to the punisher, but protects the cooperative interests of the group.

  • An account of the proximate basis of punishment focuses attention on the behavioural and neurobiological basis of motivation, and an understanding how basic learning and action selection systems deal with outcomes that involve other individuals.

  • Appetitive and aversive systems motivate actions through Pavlovian, habit-based and goal-directed actions, which are acquired in specific ways. Particularly important for social and altruistic actions may be learning through observation, in which naïve observers learn actions from others. This reflects the cultural acquisition of behaviour, and may be an underlying component of social norm development.

  • In social decision-making situations, individuals often benefit from cooperation. However, many cooperative situations give rise to the temptation to free-ride on the cooperative behaviour of others. This can be deterred by punishing non-cooperative free-riding behaviour. This pays off in the long run if the punisher interacts with the reformed free-rider again, but is costly if they do not.

  • Altruistic punishment appears to be common across diverse human cultures. It extends to third-party situations, in which passive observers punish those that they witness acting unfairly. It may be especially powerful in promoting cooperation when combined with other forms of punishment that arise from direct and indirect modes of reciprocity.

  • In the brain, recent functional MRI studies have highlighted key areas involved in the recognition and representation of unfairness (anterior insula), in the establishment of retributive goals (orbitofrontal cortex), and in the execution of punishing actions (striatum).

  • We outline a neurobiological model of punishment, in which an amygdala-centred pathway mediates impulsive, retaliative punishment, and a striatal system mediates instrumental punishment. This latter system may mediate goal-directed punishing actions that involve forward planning (that is, reciprocity-based punishment), as well as more habit-like punishing actions as experience becomes more extensive.

  • Consideration of the learning systems that underlie punishment predict that selfish reciprocity is likely to generalize to altruistic (strong) reciprocity. Accordingly, it may not be necessary to assume that altruistic retributive goals are inherited unconditioned appetitive stimuli.

  • Future research needs to focus on exploring more precisely the type actions that underlie altruistic punishment (at a proximate level), and understanding how learning and evolution interact in shaping cooperative behaviour (at an ultimate level).

Abstract

Animals, in particular humans, frequently punish other individuals who behave negatively or uncooperatively towards them. In animals, this usually serves to protect the personal interests of the individual concerned, and its kin. However, humans also punish altruistically, in which the act of punishing is personally costly. The propensity to do so has been proposed to reflect the cultural acquisition of norms of behaviour, which incorporates the desire to uphold equity and fairness, and promotes cooperation. Here, we review the proximate neurobiological basis of punishment, considering the motivational processes that underlie punishing actions.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Appetitive and aversive excitators and inhibitors.
Figure 2: Different mechanisms of learning and action.
Figure 3: Punishment in a public goods game.
Figure 4: Putative neurobiological substrates of punishment.
Figure 5: Fairness related outcome representations.

Similar content being viewed by others

References

  1. Clutton-Brock, T. H. & Parker, G. A. Punishment in animal societies. Nature 373, 209–216 (1995). A definitive review of the ecology and economics of punishment in animals.

    Article  CAS  PubMed  Google Scholar 

  2. Boyd, R. & Richerson, P. J. Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethol. Sociobiol. 13, 171–195 (1992).

    Article  Google Scholar 

  3. Boyd, R., Gintis, H., Bowles, S. & Richerson, P. J. The evolution of altruistic punishment. Proc. Natl Acad. Sci. USA 100, 3531–3535 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Fehr, E. & Fischbacher, U. The nature of human altruism. Nature 425, 785–791 (2003).

    Article  CAS  PubMed  Google Scholar 

  5. Bowles, S. & Gintis, H. The evolution of strong reciprocity: cooperation in heterogeneous populations. Theor. Popul. Biol. 65, 17–28 (2004).

    Article  PubMed  Google Scholar 

  6. Fehr, E. & Fischbacher, U. Social norms and human cooperation. Trends Cogn. Sci. 8, 185–190 (2004).

    Article  PubMed  Google Scholar 

  7. Gintis, H. The hitchhiker's guide to altruism: gene–culture coevolution, and the internalization of norms. J. Theor. Biol. 220, 407–418 (2003).

    Article  PubMed  Google Scholar 

  8. Moja, E. A., Cipolla, P., Castoldi, D. & Tofanetti, O. Dose–response decrease in plasma tryptophan and in brain tryptophan and serotonin after tryptophan-free amino acid mixtures in rats. Life Sci. 44, 971–976 (1989).

    Article  CAS  PubMed  Google Scholar 

  9. Mackintosh N. J. Conditioning and Associative Learning. Oxford Univ. Press, New York 1983). This classic text provides an essential and comprehensive introduction to animal learning theory.

    Google Scholar 

  10. Gray, J. A. Problems in the Behavioural Sciences 2nd edn Vol. 5 (Cambridge Univ. Press, Cambridge, 1991).

    Google Scholar 

  11. Bouton, M. E. Learning and Behavior: A Contemporary Synthesis (Sinauer, Sunderland, Massachusetts, 2006).

    Google Scholar 

  12. Montague, P. R. & Berns, G. S. Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002).

    Article  CAS  PubMed  Google Scholar 

  13. Dickinson, A. & Dearing M. F. Appetitive–aversive interactions and inhibitory processes in Mechanisms of Learning and Motivation (eds Dickinson, A. & Boakes, R. A.) 203–231 (Erlbaum, Hillsdale, New Jersey, 1979).

    Google Scholar 

  14. Pavlov, I. P. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (Oxford Univ. Press, London, 1927).

    Google Scholar 

  15. Rescorla, R. A. Probability of shock in the presence and absence of CS in fear conditioning. J. Comp. Physiol. Psychol. 66, 1–5 (1968).

    Article  CAS  PubMed  Google Scholar 

  16. Konorski, J. Integrative Activity of the Brain: An Interdisciplinary Approach (Chicago Univ. Press, Chicago, 1967).

    Google Scholar 

  17. Tremblay, L. & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708 (1999).

    Article  CAS  PubMed  Google Scholar 

  18. Schultz, W. Multiple reward signals in the brain. Nature Rev. Neurosci. 1, 199–207 (2000).

    Article  CAS  Google Scholar 

  19. O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J. & Andrews, C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neurosci. 4, 95–102 (2001).

    Article  CAS  PubMed  Google Scholar 

  20. Tobler, P. N., Fiorillo, C. D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).

    Article  CAS  PubMed  Google Scholar 

  21. Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Choosing the greater of two goods: neural currencies for valuation and decision making. Nature Rev. Neurosci. 6, 363–375 (2005).

    Article  CAS  Google Scholar 

  22. Ursu, S. & Carter, C. S. Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: implications for neuroimaging studies of decision-making. Brain Res. Cogn. Brain Res. 23, 51–60 (2005).

    Article  PubMed  Google Scholar 

  23. Nieuwenhuis, S. et al. Activity in human reward-sensitive brain areas is strongly context dependent. Neuroimage 25, 1302–1309 (2005).

    Article  PubMed  Google Scholar 

  24. Rolls, E. T. The orbitofrontal cortex and reward. Cereb. Cortex 10, 284–294 (2000).

    Article  CAS  PubMed  Google Scholar 

  25. Roesch, M. R. & Olson, C. R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).

    Article  CAS  PubMed  Google Scholar 

  26. Milad, M. R. & Quirk, G. J. Neurons in medial prefrontal cortex signal memory for fear extinction. Nature 420, 70–74 (2002).

    Article  CAS  PubMed  Google Scholar 

  27. Baxter, M. G. & Murray, E. A. The amygdala and reward. Nature Rev. Neurosci. 3, 563–573 (2002).

    Article  CAS  Google Scholar 

  28. Calder, A. J., Lawrence, A. D. & Young, A. W. Neuropsychology of fear and loathing. Nature Rev. Neurosci. 2, 352–363 (2001).

    Article  CAS  Google Scholar 

  29. Craig, A. D. How do you feel? Interoception: the sense of the physiological condition of the body. Nature Rev. Neurosci. 3, 655–666 (2002).

    Article  CAS  Google Scholar 

  30. Paulus, M. P. & Stein, M. B. An insular view of anxiety. Biol. Psychiatry 60, 383–387 (2006).

    Article  PubMed  Google Scholar 

  31. Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C. & Jones-Gotman, M. Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124, 1720–1733 (2001).

    Article  CAS  PubMed  Google Scholar 

  32. Seymour, B. et al. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nature Neurosci. 8, 1234–1240 (2005).

    Article  CAS  PubMed  Google Scholar 

  33. Jensen, J. et al. Separate brain regions code for salience vs. valence during reward prediction in humans. Hum. Brain Mapp. 15 Jun 2006 (doi: 10.1002/hbm.20274).

    Article  PubMed  Google Scholar 

  34. Nitschke, J. B., Sarinopoulos, I., Mackiewicz, K. L., Schaefer, H. S. & Davidson, R. J. Functional neuroanatomy of aversion and its anticipation. Neuroimage 29, 106–116 (2006).

    Article  PubMed  Google Scholar 

  35. Sarinopoulos, I., Dixon, G. E., Short, S. J., Davidson, R. J. & Nitschke, J. B. Brain mechanisms of expectation associated with insula and amygdala response to aversive taste: implications for placebo. Brain Behav. Immun. 20, 120–132 (2006).

    Article  PubMed  Google Scholar 

  36. Mesulam, M. M. & Mufson, E. J. Insula of the old world monkey. I. Architectonics in the insulo-orbito-temporal component of the paralimbic brain. J. Comp. Neurol. 212, 1–22 (1982).

    Article  CAS  PubMed  Google Scholar 

  37. Mufson, E. J., Mesulam, M. M. & Pandya, D. N. Insular interconnections with the amygdala in the rhesus monkey. Neuroscience 6, 1231–1248 (1981).

    Article  CAS  PubMed  Google Scholar 

  38. Hutchinson, R. R., Azrin, N. H. & Hunt, G. M. Attack produced by intermittent reinforcement of a concurrent operant response. J. Exp. Anal. Behav. 11, 489–495 (1968).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Ulrich, R. E. & Azrin, N. H. Reflexive fighting in response to aversive stimulation. J. Exp. Anal. Behav. 5, 511–520 (1962).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Fendt, M. & Fanselow, M. S. The neuroanatomical and neurochemical basis of conditioned fear. Neurosci. Biobehav. Rev. 23, 743–760 (1999).

    Article  CAS  PubMed  Google Scholar 

  41. Adams, D. B. Brain mechanisms of aggressive behavior: an updated review. Neurosci. Biobehav. Rev. 30, 304–318 (2006).

    Article  PubMed  Google Scholar 

  42. Thorndike, E. L. Animal Intelligence (Macmillan, New York, 1911).

    Google Scholar 

  43. Azrin, N. H. Some effects of two intermittent schedules of immediate and non-immediate punishment. J. Psychol. 42, 3–21 (1956).

    Article  Google Scholar 

  44. Church, R. M., Raymond, G. A. & Beauchamp, R. D. Response suppression as a function of intensity and duration of a punishment. J. Comp. Physiol. Psychol. 1, 39–44 (1967).

    Article  Google Scholar 

  45. Camp, D. S., Raymond, G. A. & Church, R. M. Temporal relationship between response and punishment. J. Exp. Psychol. 74, 114–123 (1967).

    Article  CAS  PubMed  Google Scholar 

  46. Azrin, N. H. Effects of punishment intensity during variable-interval reinforcement. J. Exp. Anal. Behav. 3, 123–142 (1960).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Solomon, R. L., Turner, L. H. & Lessac, M. S. Some effects of delay of punishment on resistance to temptation in dogs. J. Pers. Soc. Psychol. 8, 233–238 (1968).

    Article  CAS  PubMed  Google Scholar 

  48. Atnip, G. W. Stimulus and response reinforcer contingencies in autoshaping, operant, classical and omission training procedures in rats. J. Exp. Anal. Behav. 28, 56–69 (1977).

    Article  Google Scholar 

  49. Baron, A. Delayed punishment of a runway response J. Comp. Physiol. Psychol. 60, 131–134 (1965).

    Article  CAS  PubMed  Google Scholar 

  50. Walters, G. C. & Grusec, J. E. Punishment (W. H. Freeman, San Francisco, 1977).

    Google Scholar 

  51. Solomon, R. L., Turner, L. H. & Lessac, M. S. Some effects of delay of punishment on resistance to temptation in dogs. J. Pers. Soc. Psychol. 8, 233–238 (1968).

    Article  CAS  PubMed  Google Scholar 

  52. Azrin, N. H., Holz, W. C. & Hutchinson, R. R. Fixed-ratio escape reinforcement. J. Exp. Anal. Behav. 6, 141–148 (1963).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Boe, E. E. & Church, R. M. Permanent effects of punishment during extinction. J. Comp. Physiol. Psychol. 63, 486–492 (1967).

    Article  CAS  PubMed  Google Scholar 

  54. Estes, W. K. & Skinner, B. F. Some quantitative properties of anxiety. J. Exp. Psychol. 29, 390–400 (1941).

    Article  Google Scholar 

  55. Church, R. M. in Punishment and Aversive Behavior (eds Campbell, B. A. & Church, R. M.) (Appleton, New York, 1969).

    Google Scholar 

  56. Bolles, R. C., Holtz, R., Dunn, T. & Hill, W. Comparison of stimulus learning and response learning in a punishment situation. Learn. Motiv. 11, 78–96 (1980).

    Article  Google Scholar 

  57. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997). Comprehensively describes the theoretical basis of reinforcement learning models of dopamine in monkeys.

    Article  CAS  PubMed  Google Scholar 

  58. Wise, R. A. Dopamine, learning and motivation. Nature Rev. Neurosci. 5, 483–494 (2004).

    Article  CAS  Google Scholar 

  59. Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J. & Frith, C. D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Yin, H. H. & Knowlton, B. J. The role of the basal ganglia in habit formation. Nature Rev. Neurosci. 7, 464–476 (2006).

    Article  CAS  Google Scholar 

  61. Bar-Gad, I., Morris, G. & Bergman, H. Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol. 71, 439–473 (2003).

    Article  PubMed  Google Scholar 

  62. Niv, Y., Joel, D. & Dayan, P. A normative perspective on motivation. Trends Cogn. Sci. 10, 375–381 (2006).

    Article  PubMed  Google Scholar 

  63. Dickinson, A. & Balleine, B. W. in Steven's Handbook of Experimental Psychology 3rd edn Vol. 3 (ed. Gallistel, C. R.) 497–533 (John Wiley & Sons, New York, 2002).

    Google Scholar 

  64. Balleine, B. W. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol. Behav. 86, 717–730 (2005).

    Article  CAS  PubMed  Google Scholar 

  65. Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neurosci. 8, 1704–1711 (2005).

    Article  CAS  PubMed  Google Scholar 

  66. Blaisdell, A. P., Sawa, K., Leising, K. J. & Waldmann, M. R. Causal reasoning in rats. Science 311, 1020–1022 (2006).

    Article  CAS  PubMed  Google Scholar 

  67. Raby, C. R., Alexis, D. M., Dickinson, A. & Clayton, N. S. Planning for the future by western scrub-jays. Nature 445, 919–921 (2007). This remarkable study shows that scrub-jays can plan for future goals in a manner independent from their current motivational state.

    Article  CAS  PubMed  Google Scholar 

  68. Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nature Rev. Neurosci. 2, 820–829 (2001).

    Article  CAS  Google Scholar 

  69. Koechlin, E., Ody, C. & Kouneiher, F. The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181–1185 (2003).

    Article  CAS  PubMed  Google Scholar 

  70. Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    Article  CAS  PubMed  Google Scholar 

  71. Yoshida, W. & Ishii, S. Resolution of uncertainty in prefrontal cortex. Neuron 50, 781–789 (2006). Uses sophisticated behavioural modelling to show that the human prefrontal cortex is engaged to solve partially observable decision tasks, one of the central computational problems underlying multiagent games.

    Article  CAS  PubMed  Google Scholar 

  72. Hampton, A. N., Bossaerts, P. & O'Doherty, J. P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360–8367 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  73. Hershberger, W. A. An approach through the looking-glass. Anim. Learn. Behav. 14, 443–451 (1986). This famous experiment shows that chicks cannot learn to retreat from a food cart that moves in the same direction as them but at twice the speed.

    Article  Google Scholar 

  74. Dayan, P., Niv, Y., Seymour, B. & Daw, D. The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153–1160 (2006).

    Article  PubMed  Google Scholar 

  75. Fowler, H. & Miller, N. E. Facilitation and inhibition of runway performance by hind- and forepaw shock of various intensities. J. Comp. Physiol. Psychol. 56, 801–805 (1963).

    Article  CAS  PubMed  Google Scholar 

  76. Anson, J. E., Bender, L. & Melvin, K. B. Sources of reinforcement in establishment of self-punitive behavior. J. Comp. Physiol. Psychol. 67, 376–380 (1969).

    Article  CAS  PubMed  Google Scholar 

  77. Melvin, K. B. & Anson, J. E. Facilitative effects of punishment on aggressive behavior in Siamese fighting fish. Psychon. Sci. 14, 89–90 (1969).

    Article  Google Scholar 

  78. Morse, W. H., Mead, R. N. & Kelleher, R. T. Modulation of elicited behavior by a fixed-interval schedule of electric shock presentation. Science 157, 215–217 (1967).

    Article  CAS  PubMed  Google Scholar 

  79. Lieberman, D. A. Learning: Behavior and Cognition (Wadsworth, Belmont, California, 1999). Considers the efficacy of various forms of punishment in children, including omission of privileges and reinforcing good behaviour.

    Google Scholar 

  80. Cinyabuguma, M., Page, T. & Putterman, L. Cooperation under the threat of expulsion in a public goods experiment. J. Pub. Econ. 89, 1421–1435 (2005).

    Article  Google Scholar 

  81. Williams, K. D. Ostracism. Annu. Rev. Psychol. 58, 425–452 (2007).

    Article  PubMed  Google Scholar 

  82. Damato, M. R., Fazzaro, J. & Etkin, M. Anticipatory responding and avoidance discrimination as factors in avoidance conditioning. J. Exp. Psychol. 77, 41–47 (1968).

    Article  CAS  Google Scholar 

  83. Bolles, R. C. & Grossen, N. E. Effects of an informational stimulus on acquisition of avoidance behavior in rats. J. Comp. Physiol. Psychol. 68, 90–99 (1969).

    Article  Google Scholar 

  84. Starr, M. D. & Mineka, S. Determinants of fear over course of avoidance-learning. Learn. Motiv. 8, 332–350 (1977).

    Article  Google Scholar 

  85. Crawford, M., Masterson, F. & Wilson, D. Species-specific defense reactions in escape-from-fear situations. Anim. Learn. Behav. 5, 63–72 (1977).

    Article  Google Scholar 

  86. Dickinson, A. Contemporary Animal Learning Theory (Cambridge Univ. Press, Cambridge, UK, 1980).

    Google Scholar 

  87. Dinsmoor, J. A. Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing. J. Exp. Anal. Behav. 75, 311–333 (2001).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  88. Hendersen, R. W. & Graham, J. Avoidance of heat by rats — effects of thermal context on rapidity of extinction. Learn. Motiv. 10, 351–363 (1979).

    Article  Google Scholar 

  89. Kirkby, R. J. & Kimble, D. P. Avoidance and escape behavior following striatal lesions in the rat. Exp. Neurol. 20, 215–227 (1968).

    Article  CAS  PubMed  Google Scholar 

  90. White, I. M. & Rebec, G. V. Responses of rat striatal neurons during performance of a lever-release version of the conditioned avoidance response task. Brain Res. 616, 71–82 (1993).

    Article  CAS  PubMed  Google Scholar 

  91. Killcross, S., Robbins, T. W. & Everitt, B. J. Different types of fear-conditioned behaviour mediated by separate nuclei within amygdala. Nature 388, 377–380 (1997).

    Article  CAS  PubMed  Google Scholar 

  92. Kim, H., Shimojo, S. & O'Doherty, J. P. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol. 4, 1453–1461 (2006).

    CAS  Google Scholar 

  93. Cook, L. & Catania, A. C. Effects of drugs on avoidance and escape behaviour. Fed. Proc. 23, 818–835 (1964).

    CAS  PubMed  Google Scholar 

  94. Bandura, A. Social Learning Theory (General Learning, New York, 1977).

    Google Scholar 

  95. Gergely, G. & Watson, J. in Early social cognition: Understanding Others in the First Months of Life (ed. Rochat, P.) 101–136 (Lawrence Erlbaum, Mawah, New Jersey, 1999).

    Google Scholar 

  96. Galef, B. G. & Whiskin, E. E. Social transmission of information about multiflavored foods. Anim. Learn. Behav. 20, 56–62 (1992).

    Article  Google Scholar 

  97. Stoinski, T. S., Wrate, J. L., Ure, N. & Whiten, A. Imitative learning by captive western lowland gorillas (Gorilla gorilla gorilla) in a simulated food-processing task. J. Comp. Psychol. 115, 272–281 (2001).

    Article  CAS  PubMed  Google Scholar 

  98. Whiten, A. & Boesch, C. The cultures of chimpanzees. Sci. Am. 284, 60–67 (2001).

    Article  CAS  PubMed  Google Scholar 

  99. Tomasello, M., Kruger, A. & Ratner, H. Cultural learning. Behav. Brain Sci. 16, 495–552 (1993).

    Article  Google Scholar 

  100. Galef, B. G. & Laland, K. N. Social learning in animals: empirical studies and theoretical models. Bioscience 55, 489–499 (2005).

    Article  Google Scholar 

  101. Mineka, S. & Cook, M. Mechanisms involved in the observational conditioning of fear. J. Exp. Psychol. Gen. 122, 23–38 (1993).

    Article  CAS  PubMed  Google Scholar 

  102. Wohlschlager, A., Gattis, M. & Bekkering, H. Action generation and action perception in imitation: an instance of the ideomotor principle. Philos. Trans. R. Soc. Lond. B Biol. Sci. 358, 501–515 (2003).

    Article  PubMed  PubMed Central  Google Scholar 

  103. Heyes, C. & Ray, E. What is the significance of imitation in animals? Adv. Study Behav. 29, 215–245 (2000).

    Article  Google Scholar 

  104. Heyes, C. Causes and consequences of imitation. Trends Cogn. Sci. 5, 253–261 (2001).

    Article  CAS  PubMed  Google Scholar 

  105. Gergely, G., Bekkering, H. & Kiraly, I. Rational imitation in preverbal infants. Nature 415, 755 (2002).

    Article  CAS  PubMed  Google Scholar 

  106. Heyes, C., Bird, G., Johnson, H. & Haggard, P. Experience modulates automatic imitation. Brain Res. Cogn. Brain Res. 22, 233–240 (2005).

    Article  PubMed  Google Scholar 

  107. Bellman, R. Dynamic Programming (Princeton Univ. Press, Princeton, 1957).

    Google Scholar 

  108. Solomon, R. L., Kamin, L. J. & Wynne, L. C. Traumatic avoidance learning — the outcomes of several extinction procedures with dogs. J. Abnorm. Soc. Psychol. 48, 291–302 (1953).

    Article  CAS  Google Scholar 

  109. Rizzolatti, G., Fadiga, L., Gallese, V. & Fogassi, L. Premotor cortex and the recognition of motor actions. Cogn. Brain Res. 3, 131–141 (1996).

    Article  CAS  Google Scholar 

  110. Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192 (2004).

    Article  CAS  PubMed  Google Scholar 

  111. Fogassi, L. et al. Parietal lobe: from action organization to intention understanding. Science 308, 662–667 (2005).

    Article  CAS  PubMed  Google Scholar 

  112. Iacoboni, M. et al. Grasping the intentions of others with one's own mirror neuron system. PLoS Biol. 3, e79 (2005).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  113. Tomasello, M. & Call, J. Primate Cognition (Oxford Univ. Press, Oxford,1970).

    Google Scholar 

  114. Warneken, F. & Tomasello, M. Altruistic helping in human infants and young chimpanzees. Science 311, 1301–1303 (2006).

    Article  CAS  PubMed  Google Scholar 

  115. Thornton, A. & McAuliffe, K. Teaching in wild meerkats. Science 313, 227–229 (2006).

    Article  CAS  PubMed  Google Scholar 

  116. Yamagishi, T. in Advances in Group Processes Vol. 3 (ed. Lawler E. J.) 51–87 (Elsevier Science, Greenwich, Connecticut 1986).

    Google Scholar 

  117. Kaelbling, L. P., Littman, M. L. & Moore, A. W. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996).

    Article  Google Scholar 

  118. Sutton, R. S. & Barto, A. G. Reinforcement Learning. An Introduction (MIT press, Cambridge, Massachusetts, 1998).

    Book  Google Scholar 

  119. Bayer, H. M. & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  120. O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H. & Dolan, R. J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    Article  CAS  PubMed  Google Scholar 

  121. Tanaka, S. C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neurosci. 7, 887–893 (2004).

    Article  CAS  PubMed  Google Scholar 

  122. Haruno, M. et al. A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J. Neurosci. 24, 1660–1665 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  123. Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664–667 (2004).

    Article  CAS  PubMed  Google Scholar 

  124. Tanaka, S. C. et al. Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics. Neural Netw. 19, 1233–1241 (2006).

    Article  PubMed  Google Scholar 

  125. Littman, M. L. in Proceedings of the Eleventh International Conference on Machine Learning, 157—163 (Morgan Kaufmann, San Francisco, California, 1994). (1994).

    Google Scholar 

  126. Hu, J. L. & Wellman, M. P. Nash Q-learning for general-sum stochastic games. J. Mach. Learn. Res. 4, 1039–1069 (2004).

    Google Scholar 

  127. Claus, C. & Boutilier, C. The dynamics of reinforcement learning in cooperative multiagent systems. Proc. Natl Conf. Artif. Intell. 15, 746–752 (1998).

    Google Scholar 

  128. Ng, Y. N. & Russell, S. Algorithms for inverse reinforcement learning. Proc. Seventeenth Int. Conf. Mach. Learn. 663–670 (2000). Provides a thorough analysis of the inverse reinforcement learning problem.

  129. Abbeel, P. & Ng, A. Y. Apprenticeship learning via inverse reinforcement learning. ACM Int. Conf. Proc. Series pp 1–8 (2004).

  130. Yamagishi, T. & Sato, K. Motivational basis of the public goods problem. J. Pers. Soc. Psychol. 50, 67–73 (1986).

    Article  Google Scholar 

  131. Yamagishi, T. The provision of a sanctioning system as a public good. J. Pers. Soc. Psychol. 51, 110–116 (1986). One of the first experimental demonstrations that punishment increases cooperation.

    Article  Google Scholar 

  132. Fehr, E. & Gachter, S. Altruistic punishment in humans. Nature 415, 137–140 (2002). The first demonstration of altruistic punishment in humans.

    Article  CAS  PubMed  Google Scholar 

  133. Boyd, R. & Richerson, P. J. The evolution of reciprocity in sizable groups. J. Theor. Biol. 132, 337–356 (1988).

    Article  CAS  PubMed  Google Scholar 

  134. Gintis, H. Strong reciprocity and human sociality. J. Theor. Biol. 206, 169–179 (2000).

    Article  CAS  PubMed  Google Scholar 

  135. Stevens, J. R. & Hauser, M. D. Why be nice? Psychological constraints on the evolution of cooperation. Trends Cogn. Sci. 8, 60–65 (2004). Discusses some of the underlying cognitive components involved in reciprocity.

    Article  PubMed  Google Scholar 

  136. Stevens, J. R. The selfish nature of generosity: harassment and food sharing in primates. Proc. Biol. Sci. 271, 451–456 (2004).

    Article  PubMed  PubMed Central  Google Scholar 

  137. De Waal, F. B. M. Chimpanzee Politics: Power and Sex Among Apes (Johns Hopkins Univ. Press, Baltimore, Maryland, 1998).

    Google Scholar 

  138. Reeve, H. K. Queen activation of lazy workers in colonies of the eusocial naked mole-rat. Nature 358, 147–149 (1992).

    Article  CAS  PubMed  Google Scholar 

  139. Shinada, M., Yamagishi, T. & Ohmura, Y. False friends are worse than bitter enemies: 'altruistic' punishment of in-group members. Evol. Hum. Behav. 25, 379–393 (2004).

    Article  Google Scholar 

  140. Henrich, J. et al. In search of Homo economicus: Behavioral experiments in 15 small-scale societies. Am. Econ. Rev. 91, 73–78 (2001).

    Article  Google Scholar 

  141. Henrich, J. Cooperation, punishment, and the evolution of human institutions. Science 312, 60–61 (2006).

    Article  CAS  PubMed  Google Scholar 

  142. Fehr, E. & Gachter, S. Cooperation and punishment in public goods experiments. Am. Econ. Rev. 90, 980–994 (2000).

    Article  Google Scholar 

  143. Gurerk, O., Irlenbusch, B. & Rockenbach, B. The competitive advantage of sanctioning institutions. Science 312, 108–111 (2006).

    Article  PubMed  CAS  Google Scholar 

  144. Andreoni, J., Harbaugh, W. & Vesterlund, L. The carrot or the stick: rewards, punishments, and cooperation. Am. Econ. Rev. 93, 893–902 (2003).

    Article  Google Scholar 

  145. Bendor, J. & Swistak, P. The evolution of norms. Am. J. Sociol. 106, 1493–1545 (2001).

    Article  Google Scholar 

  146. Fehr, E. & Fischbacher, U. Third-party punishment and social norms. Evol. Hum. Behav. 25, 63–87 (2004).

    Article  Google Scholar 

  147. Rockenbach, B. & Milinski, M. The efficient interaction of indirect reciprocity and costly punishment. Nature 444, 718–723 (2006).

    Article  CAS  PubMed  Google Scholar 

  148. Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E. & Cohen, J. D. The neural basis of economic decision-making in the Ultimatum Game. Science 300, 1755–1758 (2003).

    Article  CAS  PubMed  Google Scholar 

  149. Knoch, D., Pascual-Leone, A., Meyer, K., Treyer, V. & Fehr, E. Diminishing reciprocal fairness by disrupting the right prefrontal cortex. Science 314, 829–832 (2006).

    Article  CAS  PubMed  Google Scholar 

  150. Singer, T. et al. Empathic neural responses are modulated by the perceived fairness of others. Nature 439, 466–469 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  151. de Quervain, D. J. et al. The neural basis of altruistic punishment. Science 305, 1254–1258 (2004).

    Article  CAS  PubMed  Google Scholar 

  152. Moll, J., Zahn, R., Oliveira-Souza, R., Krueger, F. & Grafman, J. The neural basis of human moral cognition. Nature Rev. Neurosci. 6, 799–809 (2005).

    Article  CAS  Google Scholar 

  153. Brunet, E., Sarfati, Y., Hardy-Bayle, M. C. & Decety, J. A PET investigation of the attribution of intentions with a nonverbal task. Neuroimage 11, 157–166 (2000).

    Article  CAS  PubMed  Google Scholar 

  154. Gallagher, H. L. & Frith, C. D. Functional imaging of 'theory of mind'. Trends Cogn. Sci. 7, 77–83 (2003).

    Article  PubMed  Google Scholar 

  155. Gallagher, H. L., Jack, A. I., Roepstorff, A. & Frith, C. D. Imaging the intentional stance in a competitive game. Neuroimage 16, 814–821 (2002).

    Article  PubMed  Google Scholar 

  156. Tomlin, D. et al. Agent-specific responses in the cingulate cortex during economic exchanges. Science 312, 1047–1050 (2006).

    Article  CAS  PubMed  Google Scholar 

  157. Vlaev, I. & Chater, N. Game relativity: how context influences strategic decision making. J. Exp. Psychol. Learn. Mem. Cogn. 32, 131–149 (2006).

    Article  PubMed  Google Scholar 

  158. Ackley, D. H. & Littman, M. L. in Artificial Life II, SFI Studies in the Sciences of Complexity, Vol. X (eds Langton, C. G., Taylor, C., Farmer, J. D. & Rasmussen, S.) 487–509 (Addison-Wesley, Reading, Massachusetts, 1991). Provides an excellent introduction to theory surrounding the evolution of learning systems.

    Google Scholar 

  159. Kamin, L. J. in Miami Symposium on the Prediction of Behavior: Aversive Stimulation (ed. Jones, M. R.) 9–33 (Miami Univ. Press, Miami,1968).

    Google Scholar 

  160. Rescorla R. A. Variation in the effectiveness of reinforcement and non-reinforcement following proir inhibitory conditioning. Learn. Motiv. 2, 113–123 (1971).

    Article  Google Scholar 

  161. Bakal, C. W., Johnson, R. D. & Rescorla, R. A. The effect of change in US quality on the blocking effect. Pavlov. J. Biol. Sci. 9, 97–103 (1974).

    Article  CAS  PubMed  Google Scholar 

  162. Bull, J. A. & Overmier, J. B. Additive and subtractive properties of excitation and inhibition. J. Comp. Physiol. Psychol. 66, 511–514 (1968).

    Article  PubMed  Google Scholar 

  163. Camerer, C. F. Behavioural Game Theory: Experiments in Strategic Interaction (Princeton Univ. Press, Princeton, 2003).

    Google Scholar 

  164. Montague, P. R. et al. Hyperscanning: simultaneous fMRI during linked social interactions. Neuroimage 16, 1159–1164 (2002).

    Article  PubMed  Google Scholar 

  165. McCabe, K., Houser, D., Ryan, L., Smith, V. & Trouard, T. A functional imaging study of cooperation in two-person reciprocal exchange. Proc. Natl Acad. Sci. USA 98, 11832–11835 (2001).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  166. Singer, T., Kiebel, S. J., Winston, J. S., Dolan, R. J. & Frith, C. D. Brain responses to the acquired moral status of faces. Neuron 41, 653–662 (2004).

    Article  CAS  PubMed  Google Scholar 

  167. King-Casas, B. et al. Getting to know you: reputation and trust in a two-person economic exchange. Science 308, 78–83 (2005).

    Article  CAS  PubMed  Google Scholar 

  168. Rilling, J. K., Sanfey, A. G., Aronson, J. A., Nystrom, L. E. & Cohen, J. D. The neural correlates of theory of mind within interpersonal interactions. Neuroimage 22, 1694–1703 (2004).

    Article  PubMed  Google Scholar 

  169. Rilling, J. K., Sanfey, A. G., Aronson, J. A., Nystrom, L. E. & Cohen, J. D. Opposing BOLD responses to reciprocated and unreciprocated altruism in putative reward pathways. Neuroreport 15, 2539–2543 (2004).

    Article  PubMed  Google Scholar 

  170. Rilling, J. et al. A neural basis for social cooperation. Neuron 35, 395–405 (2002).

    Article  CAS  PubMed  Google Scholar 

  171. Bowles, S. & Gintis, H. Homo reciprocans. Nature 415, 125–128 (2002).

    Article  CAS  PubMed  Google Scholar 

  172. Martin, J. H. Neuroanatomy: Text and Atlas 2nd edn (Appleton & Lange, Stamford, Connecticut, 1996).

    Google Scholar 

Download references

Acknowledgements

B.S. and R.D. are supported by a Wellcome Trust Programme Grant to R.D. T.S. was supported by the Medical Research Council. The authors thank N. Daw, P. Dayan and the anonymous reviewers for comments on the manuscript.

Author information

Authors and Affiliations

  1. The Wellcome Trust Centre for Neuroimaging, Institute of Neurology, UCL, 12 Queen Square, London, WC1X 3BG, UK

    Ben Seymour & Ray Dolan

  2. Institute of Cognitive Neuroscience, 17 Queen Square, London WC1X 3BG, UK, and the Center for Social Neuroscience and Neuroeconomics, University of Zürich, Blümlisalpstrasse 10 8006 Zürich, Switzerland.,

    Tania Singer

Authors
  1. Ben Seymour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tania Singer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ray Dolan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Seymour.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Related links

Related links

FURTHER INFORMATION

Wellcome Trust Centre for Neuroimaging

Glossary

Kin selection

Evolutionary models which predict that animals should be motivated to protect their relatives, to ultimately preserve their genetic inheritance.

Direct reciprocity

So called 'tit-for-tat reciprocity', in which a beneficial act to another individual is likely to be reciprocated by that individual.

Indirect reciprocity

Sometimes to referred to as reputation formation, this involves acquiring a reputation as being beneficial towards others, and therefore being treated favourably by them.

Altruistic punishment

Punishing that involves a selfless personal cost to the punisher, which is never likely to be recovered.

Pavlovian learning

Learning that a previously neutral environmental cue predicts a motivational salient outcome.

Instrumental learning

Learning that a particular action predicts a motivational salient outcome.

Credit assignment problem

This refers to the problem of attributing value to intermediate states and actions in sequential learning.

Ultimate basis

The ultimate basis of an observed behaviour refers to the overall reason for the existence of the behaviour, typically approached in terms of its evolutionary basis.

Proximate basis

The proximate basis of an observed behaviour refers to its immediate cause, such as the underlying neurobiological process.

Conditioned reinforcement

The process by which a Pavlovian-acquired value can reinforce instrumental action.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Seymour, B., Singer, T. & Dolan, R. The neurobiology of punishment. Nat Rev Neurosci 8, 300–311 (2007). https://doi.org/10.1038/nrn2119

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn2119

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing