Review Article | Published:

Partners and rivals in direct reciprocity

Abstract

Reciprocity is a major factor in human social life and accounts for a large part of cooperation in our communities. Direct reciprocity arises when repeated interactions occur between the same individuals. The framework of iterated games formalizes this phenomenon. Despite being introduced more than five decades ago, the concept keeps offering beautiful surprises. Recent theoretical research driven by new mathematical tools has proposed a remarkable dichotomy among the crucial strategies: successful individuals either act as partners or as rivals. Rivals strive for unilateral advantages by applying selfish or extortionate strategies. Partners aim to share the payoff for mutual cooperation, but are ready to fight back when being exploited. Which of these behaviours evolves depends on the environment. Whereas small population sizes and a limited number of rounds favour rivalry, partner strategies are selected when populations are large and relationships stable. Only partners allow for evolution of cooperation, while the rivals’ attempt to put themselves first leads to defection.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Change history

  • 27 March 2018

    In the version of this Review Article originally published, in Fig. 4 an arrow pointing from ALLC to ALLD was mistakenly omitted. This has now been corrected in all versions of the Review Article.

References

  1. 1.

    von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, Princeton, NJ, 1944).

  2. 2.

    Nash, J. F. Equilibrium points in n-person games. Proc. Natl Acad. Sci. USA 36, 48–49 (1950).

  3. 3.

    Flood, M. M. Some experimental games. Manage. Sci. 5, 5–26 (1958).

  4. 4.

    Rapoport, A. & Chammah, A. M. Prisoner’s Dilemma (Univ. Michigan Press, Ann Arbor, MI, 1965).

  5. 5.

    Trivers, R. L. The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35–57 (1971).

  6. 6.

    Nowak, M. A. Evolutionary Dynamics (Harvard Univ. Press, Cambridge, MA, 2006).

  7. 7.

    Sigmund, K. The Calculus of Selfishness (Princeton Univ. Press, Princeton, NJ, 2010).

  8. 8.

    Nowak, M. A. Five rules for the evolution of cooperation. Science 314, 1560–1563 (2006).

  9. 9.

    Smale, S. The prisoner’s dilemma and dynamical systems associated to non-cooperative games. Econometrica 48, 1617–1634 (1980).

  10. 10.

    Mailath, G. J. & Samuelson, L. Repeated Games and Reputations (Oxford Univ. Press, Oxford, 2006). Extensive compendium on repeated games from an economics point of view, which gives an excellent overview on the folk theorem literature.

  11. 11.

    Abreu, D. Extremal equilibria of oligopolistic supergames. J. Econ. Theory 39, 191–225 (1986).

  12. 12.

    Bernheim, D. & Whinston, M. D. Multimarket contact and collusive behavior. Rand J. Econ. 21, 1–26 (1990).

  13. 13.

    Cable, D. M. & Shane, S. A prisoner’s dilemma approach to entrepreneur–venture capitalist relationships. Acad. Manage. Rev. 22, 142–176 (1997).

  14. 14.

    Majeski, S. J. Arms races as iterated prisoner’s dilemma games. Math. Soc. Sci. 7, 253–266 (1984).

  15. 15.

    Aumann, R. J. War and peace. Proc. Natl Acad. Sci. USA 103, 17075–17078 (2006).

  16. 16.

    Wilkinson, G. S. Reciprocal food-sharing in the vampire bat. Nature 308, 181–184 (1984).

  17. 17.

    Stephens, D. W., McLinn, C. M. & Stevens, J. R. Discounting and reciprocity in an iterated prisoner’s dilemma. Science 298, 2216–2218 (2002).

  18. 18.

    Milinski, M. Tit for tat in sticklebacks and the evolution of cooperation. Nature 325, 433–435 (1987).

  19. 19.

    Ben-Porath, E. The complexity of computing a best response automaton in repeated games with mixed strategies. Games Econ. Behav. 2, 1–12 (1990).

  20. 20.

    Papadimitriou, C. H. On players with a bounded number of states. Games Econ. Behav. 4, 122–131 (1992).

  21. 21.

    Friedman, J. A non-cooperative equilibrium for supergames. Rev. Econ. Stud. 38, 1–12 (1971).

  22. 22.

    Aumann, R. J. in Essays in Game Theory and Mathematical Economics in Honor of Oskar Morgenstern (eds Henn, R. & Moeschlin, O.) 11–42 (Wissenschaftsverlag, Mannheim, 1981).

  23. 23.

    Maynard Smith, J. Evolution and the Theory of Games (Cambridge Univ. Press, Cambridge, 1982).

  24. 24.

    Selten, R. & Hammerstein, P. Gaps in Harley’s argument on evolutionarily stable learning rules and in the logic of “Tit for Tat”. Behav. Brain Sci. 7, 115–116 (1984).

  25. 25.

    Boyd, R. & Lorberbaum, J. No pure strategy is evolutionary stable in the iterated prisoner’s dilemma game. Nature 327, 58–59 (1987).

  26. 26.

    Bendor, J. & Swistak, P. Types of evolutionary stability and the problem of cooperation. Proc. Natl Acad. Sci. USA 92, 3596–3600 (1995).

  27. 27.

    van Veelen, M., Garcia, J., Rand, D. G. & Nowak, M. A. Direct reciprocity in structured populations. Proc. Natl Acad. Sci. USA 109, 9929–9934 (2012).

  28. 28.

    Garcia, J. & van Veelen, M. In and out of equilibrium I: Evolution of strategies in repeated games with discounting. J. Econ. Theory 161, 161–189 (2016).

  29. 29.

    Boyd, R. Mistakes allow evolutionary stability in the repeated prisoner’s dilemma game. J. Theor. Biol. 136, 47–56 (1989).

  30. 30.

    Fudenberg, D. & Maskin, E. Evolution and cooperation in noisy repeated games. Am. Econ. Rev. 80, 274–279 (1990).

  31. 31.

    Nowak, M. A. & Sigmund, K. Chaos and the evolution of cooperation. Proc. Natl Acad. Sci. USA 90, 5091–5094 (1993).

  32. 32.

    Imhof, L. A., Fudenberg, D. & Nowak, M. A. Evolutionary cycles of cooperation and defection. Proc. Natl Acad. Sci. USA 102, 10797–10800 (2005).

  33. 33.

    Imhof, L. A. & Nowak, M. A. Stochastic evolutionary dynamics of direct reciprocity. Proc. R. Soc. London Ser. B 277, 463–468 (2010).

  34. 34.

    Nowak, M. A., Sasaki, A., Taylor, C. & Fudenberg, D. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646–650 (2004). Introduces finite population size to evolutionary game dynamics.

  35. 35.

    Imhof, L. A., Fudenberg, D. & Nowak, M. A. Tit-for-tat or win-stay, lose-shift? J. Theor. Biol. 247, 574–580 (2007).

  36. 36.

    Garcia, J. & Traulsen, A. The structure of mutations and the evolution of cooperation. PLoS ONE 7, e35287 (2012).

  37. 37.

    Kurokawa, S. & Ihara, Y. Emergence of cooperation in public goods games. Proc. R. Soc. London Ser. B 276, 1379–1384 (2009).

  38. 38.

    Martinez-Vaquero, L. A., Cuesta, J. A. & Sanchez, A. Generosity pays in the presence of direct reciprocity: a comprehensive study of 2×2 repeated games. PLoS ONE 7, e35135 (2012).

  39. 39.

    Baek, S. K., Jeong, H. C., Hilbe, C. & Nowak, M. A. Comparing reactive and memory-one strategies of direct reciprocity. Sci. Rep. 6, 25676 (2016).

  40. 40.

    Axelrod, R. The Evolution of Cooperation (Basic Books, New York, NY, 1984). Axelrod’s tournament and the success of tit-for-tat have been transformative for the field; this book contains a detailed analysis of the tournament’s results .

  41. 41.

    Molander, P. The optimal level of generosity in a selfish, uncertain environment. J. Conflict Resolut. 29, 611–618 (1985).

  42. 42.

    Nowak, M. A. & Sigmund, K. Tit for tat in heterogeneous populations. Nature 355, 250–253 (1992).

  43. 43.

    Nowak, M. A. & Sigmund, K. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the prisoner’s dilemma game. Nature 364, 56–58 (1993). Describes a simple yet surprisingly powerful strategy to maintain cooperation in noisy repeated games, win-stay, lose-shift.

  44. 44.

    Kraines, D. P. & Kraines, V. Y. Pavlov and the prisoner’s dilemma. Theory Decis. 26, 47–79 (1989).

  45. 45.

    Lindgren, K. in The Economy as an Evolving Complex System II (eds Arthur, W. B., Durlauf, S. N. & Lane, D. A.) 337–368 (Addison-Wesley, Reading, MA, 1997).

  46. 46.

    Hauert, C. & Schuster, H. G. Effects of increasing the number of players and memory size in the iterated prisoner’s dilemma: a numerical approach. Proc. R. Soc. London Ser. B 264, 513–519 (1997).

  47. 47.

    Pinheiro, F. L., Vasconcelos, V. V., Santos, F. C. & Pacheco, J. M. Evolution of all-or-none strategies in repeated public goods dilemmas. PLoS Comput. Biol. 10, e1003945 (2014).

  48. 48.

    Hilbe, C., Martinez-Vaquero, L. A., Chatterjee, K. & Nowak, M. A. Memory-n strategies of direct reciprocity. Proc. Natl Acad. Sci. USA 114, 4715–4720 (2017).

  49. 49.

    Fischer, I. et al. Fusing enacted and expected mimicry generates a winning strategy that promotes the evolution of cooperation. Proc. Natl Acad. Sci. USA 110, 10229–10233 (2013).

  50. 50.

    Yi, S. D., Baek, S. K. & Choi, J.-K. Combination with anti-tit-for-tat remedies problems of tit-for-tat. J. Theor. Biol. 412, 1–7 (2017).

  51. 51.

    Duersch, P., Oechssler, J. & Schipper, B. When is tit-for-tat unbeatable? Int. J. Game Theory 43, 25–36 (2013).

  52. 52.

    Rapoport, A., Seale, D. A. & Colman, A. M. Is tit-for-tat the answer? On the conclusions drawn from axelrod’s tournaments. PLoS ONE 10, e0134128 (2015).

  53. 53.

    Bendor, J. In good times and bad: reciprocity in an uncertain world. Am. J. Polit. Sci. 31, 531–558 (1987).

  54. 54.

    Nowak, M. A., Sigmund, K. & El-Sedy, E. Automata, repeated games and noise. J. Math. Biol. 33, 703–722 (1995).

  55. 55.

    Brandt, H. & Sigmund, K. The good, the bad and the discriminator — errors in direct and indirect reciprocity. J. Theor. Biol. 239, 183–194 (2006).

  56. 56.

    Hofbauer, J. & Sigmund, K. Evolutionary Games and Population Dynamics (Cambridge Univ. Press, Cambridge, 1998).

  57. 57.

    Cressman, R. Evolutionary Dynamics and Extensive Form Games (MIT Press, Cambridge, MA, 2003).

  58. 58.

    Nowak, M. A. & Sigmund, K. The evolution of stochastic strategies in the prisoner’s dilemma. Acta Appl. Math. 20, 247–265 (1990).

  59. 59.

    Press, W. H. & Dyson, F. Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent. Proc. Natl Acad. Sci. USA 109, 10409–10413 (2012). Using innovative mathematical concepts, Press and Dyson show there are strategies for the repeated prisoner’s dilemma that allow players to extort their opponents.

  60. 60.

    Chen, J. & Zinger, A. The robustness of zero-determinant strategies in iterated prisoner’s dilemma games. J. Theor. Biol. 357, 46–54 (2014).

  61. 61.

    Pan, L., Hao, D., Rong, Z. & Zhou, T. Zero-determinant strategies in iterated public goods game. Sci. Rep. 5, 13096 (2015).

  62. 62.

    McAvoy, A. & Hauert, C. Autocratic strategies for iterated games with arbitrary action spaces. Proc. Natl Acad. Sci. USA 113, 3573–3578 (2016).

  63. 63.

    McAvoy, A. & Hauert, C. Autocratic strategies for alternating games. Theor. Popul. Biol. 113, 13–22 (2016).

  64. 64.

    Ichinose, G. & Masuda, N. Zero-determinant strategies in finitely repeated games. J. Theor. Biol. 438, 61–77 (2018).

  65. 65.

    Hilbe, C., Nowak, M. A. & Sigmund, K. The evolution of extortion in iterated prisoner’s dilemma games. Proc. Natl Acad. Sci. USA 110, 6913–6918 (2013). Based on computer simulations, this article suggests that extortionate strategies can only succeed in small populations, or when two populations evolve at different rates.

  66. 66.

    Hilbe, C., Nowak, M. A. & Traulsen, A. Adaptive dynamics of exortion and compliance. PLoS ONE 8, e77886 (2013).

  67. 67.

    Szolnoki, A. & Perc, M. Evolution of extortion in structured populations. Phys. Rev. E 89, 022804 (2014).

  68. 68.

    Szolnoki, A. & Perc, M. Defection and extortion as unexpected catalysts of unconditional cooperation in structured populations. Sci. Rep. 4, 5496 (2014).

  69. 69.

    Adami, C. & Hintze, A. Evolutionary instability of zero-determinant strategies demonstrates that winning is not everything. Nat. Commun. 4, 2193 (2013).

  70. 70.

    Wu, Z.-X. & Rong, Z. Boosting cooperation by involving extortion in spatial prisoner’s dilemma games. Phys. Rev. E 90, 062102 (2014).

  71. 71.

    Hilbe, C., Röhl, T. & Milinski, M. Extortion subdues human players but is finally punished in the prisoner’s dilemma. Nat. Commun. 5, 3976 (2014).

  72. 72.

    Xu, B., Zhou, Y., Lien, J. W., Zheng, J. & Wang, Z. Extortion can outperform generosity in iterated prisoner’s dilemma. Nat. Commun. 7, 11125 (2016).

  73. 73.

    Milinski, M., Hilbe, C., Semmann, D., Sommerfeld, R. D. & Marotzke, J. Humans choose representatives who enforce cooperation in social dilemmas through extortion. Nat. Commun. 7, 10915 (2016).

  74. 74.

    Hilbe, C., Hagel, K. & Milinski, M. Asymmetric power boosts extortion in an economic experiment. PLoS ONE 11, e0163867 (2016).

  75. 75.

    Hilbe, C., Wu, B., Traulsen, A. & Nowak, M. A. Cooperation and control in multiplayer social dilemmas. Proc. Natl Acad. Sci. USA 111, 16425–16430 (2014).

  76. 76.

    Stewart, A. J. & Plotkin, J. B. From extortion to generosity, evolution in the iterated prisoner’s dilemma. Proc. Natl Acad. Sci. USA 110, 15348–15353 (2013). This study shows that large evolving populations favour the emergence of generous strategies, and introduces the important concept of evolutionary robustness.

  77. 77.

    Stewart, A. J. & Plotkin, J. B. Collapse of cooperation in evolving games. Proc. Natl Acad. Sci. USA 111, 17558–17563 (2014). Describes all evolutionary robust strategies for iterated 2 × 2 games.

  78. 78.

    Stewart, A. J. & Plotkin, J. B. Small groups and long memories promote cooperation. Sci. Rep. 6, 26889 (2016).

  79. 79.

    Stewart, A. J., Parsons, T. L. & Plotkin, J. B. Evolutionary consequences of behavioral diversity. Proc. Natl Acad. Sci. USA 113, E7003–E7009 (2016).

  80. 80.

    Akin, E. in Ergodic Theory, Advances in Dynamics (ed. Assani, I.) 77–107 (de Gruyter, Berlin, 2016). Based on the mathematical formalism of zero-determinant strategies, Akin characterized all memory-1 partner strategies (called ‘good strategies’ in this article).

  81. 81.

    Akin, E. What you gotta know to play good in the iterated prisoner’s dilemma. Games 6, 175–190 (2015).

  82. 82.

    Akin, E. Good strategies for the iterated prisoner’s dilemma: Smale vs. Markov. J. Dynam. Games 4, 217–253 (2017).

  83. 83.

    Hilbe, C., Traulsen, A. & Sigmund, K. Partners or rivals? Strategies for the iterated prisoner’s dilemma. Games Econ. Behav. 92, 41–52 (2015). Introduces the notion of competitive rival strategies, and describes partner and rival strategies for the repeated prisoner’s dilemma with discounted payoffs.

  84. 84.

    van den Berg, P., Molleman, L. & Weissing, F. J. Focus on the success of others leads to selfish behavior. Proc. Natl Acad. Sci. USA 112, 2912–2917 (2015).

  85. 85.

    Dal Bó, P. Cooperation under the shadow of the future: experimental evidence from infinitely repeated games. Am. Econ. Rev. 95, 1594–1604 (2005).

  86. 86.

    Blonski, M., Ockenfels, P. & Spagnolo, G. Equilibrium selection in the repeated prisoner’s dilemma: axiomatic approach and experimental evidence. Am. Econ. J. Microecon 3, 164–192 (2011).

  87. 87.

    Fudenberg, D., Dreber, A. & Rand, D. G. Slow to anger and fast to forgive: cooperation in an uncertain world. Am. Econ. Rev. 102, 720–749 (2012).

  88. 88.

    Doebeli, M. & Hauert, C. Models of cooperation based on the prisoner’s dilemma and the snowdrift game. Ecol. Lett. 8, 748–766 (2005).

  89. 89.

    Nowak, M. A. Evolving cooperation. J. Theor. Biol. 299, 1–8 (2012).

  90. 90.

    Frean, M. R. The prisoner’s dilemma without synchrony. Proc. R. Soc. London Ser. B 257, 75–79 (1994).

  91. 91.

    Nowak, M. A. & Sigmund, K. The alternating prisoner’s dilemma. J. Theor. Biol. 168, 219–226 (1994).

  92. 92.

    Zagorsky, B. M., Reiter, J. G., Chatterjee, K. & Nowak, M. A. Forgiver triumphs in alternating prisoner’s dilemma. PLoS ONE 8, e80814 (2013).

  93. 93.

    Roberts, G. & Sherratt, T. N. Development of cooperative relationships through increasing investment. Nature 394, 175–179 (1998).

  94. 94.

    Wahl, L. M. & Nowak, M. A. The continuous prisoner’s dilemma: I. Linear reactive strategies. J. Theor. Biol. 200, 307–321 (1999).

  95. 95.

    Killingback, T. & Doebeli, M. The continuous prisoner’s dilemma and the evolution of cooperation through reciprocal altruism with variable investment. Am. Nat. 160, 421–438 (2002).

  96. 96.

    Gokhale, C. S. & Traulsen, A. Evolutionary games in the multiverse. Proc. Natl Acad. Sci. USA 107, 5500–5504 (2010).

  97. 97.

    Traulsen, A., Nowak, M. A. & Pacheco, J. M. Stochastic dynamics of invasion and fixation. Phys. Rev. E 74, 011909 (2006).

  98. 98.

    Fudenberg, D. & Imhof, L. A. Imitation processes with small mutations. J. Econ. Theory 131, 251–262 (2006).

  99. 99.

    Wu, B., Gokhale, C. S., Wang, L. & Traulsen, A. How small are small mutation rates? J. Math. Biol. 64, 803–827 (2012).

  100. 100.

    Boerlijst, M. C., Nowak, M. A. & Sigmund, K. Equal pay for all prisoners. Am. Math. Mon. 104, 303–307 (1997).

Download references

Acknowledgements

This work was supported by the European Research Council Start Grant 279307: Graph Games (to K.C.), Austrian Science Fund (FWF) Grant P23499-N23 (to K.C.), FWF NFN Grant S11407-N23 Rigorous Systems Engineering/Systematic Methods in Systems Engineering (to K.C.), Office of Naval Research Grant N00014-16-1- 2914 (to M.A.N.) and the John Templeton Foundation (M.A.N.). C.H. acknowledges generous support from the ISTFELLOW programme.

Author information

All authors conceived the study, performed the analysis and wrote the manuscript.

Competing interests

The authors declare no competing interests.

Correspondence to Christian Hilbe.

Supplementary information

Supplementary Information

Supplementary Figures 1 and 2

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Further reading

Fig. 1: Repeated interactions allow evolution of cooperation.
Fig. 2: Eight strategies for the repeated PD.
Fig. 3: Adaptive players versus ZD strategies.
Fig. 4: Partners and rivals.
Fig. 5: Evolution favours partners or rivals.