Abstract
Cooperation in collective action dilemmas usually breaks down in the absence of additional incentive mechanisms. This tragedy can be escaped if cooperators have the possibility to invest in reward funds that are shared exclusively among cooperators (prosocial rewarding). Yet, the presence of defectors who do not contribute to the public good but do reward themselves (antisocial rewarding) deters cooperation in the absence of additional countermeasures. A recent simulation study suggests that spatial structure is sufficient to prevent antisocial rewarding from deterring cooperation. Here we reinvestigate this issue assuming mixed strategies and weak selection on a gametheoretic model of social interactions, which we also validate using individualbased simulations. We show that increasing reward funds facilitates the maintenance of prosocial rewarding but prevents its invasion, and that spatial structure can sometimes select against the evolution of prosocial rewarding. Our results suggest that, even in spatially structured populations, additional mechanisms are required to prevent antisocial rewarding from deterring cooperation in public goods dilemmas.
Introduction
Explaining the evolution of cooperation has been a longstanding challenge in evolutionary biology and the social sciences^{1,2,3,4,5,6}. The problem is to explain how cooperators, whose contributions to the common good benefit everybody in a group, can prevent defectors from outcompeting them, leading to a tragedy of the commons where nobody contributes and no common good is created or maintained^{7}.
A solution to this problem is to provide individuals with additional incentives to contribute, thus making defection less profitable^{8, 9}. Incentives can be either negative (punishment) or positive (rewards). Punishment occurs when individuals are willing to spend resources in order for defectors to lose even more resources^{10}. Punishment can be stable against defection, since rare defectors are effectively punished^{11}. However, to initially invade and resist invasion by individuals who cooperate but refrain from investing into incentives, i.e., secondorder defectors, punishers must gain from punishing, for example, through reputational benefits in future interactions^{9, 12, 13}.
Defection in collective action problems can also be prevented via positive incentives. Using rewards, cooperators can pay to increase the payoff of other cooperators. While the emergence of such behavior is usually favored, as there are very few cooperators to reward when cooperators are rare, it becomes increasingly costly to sustain as cooperators become more abundant in the population^{14}. To resist secondorder defectors, nonrewarding players must benefit less from rewards^{15}. Alternatively, when both rewards and punishment are present, rewards can foster the emergence of punishment, which in turn can be stable provided secondorder punishment is available^{16}.
Individuals can either decide to impose incentives unilaterally, or they can pool their effort to impose incentives collectively. When acting collectively, individuals can be thought as investing into a fund used to either punish defectors or reward cooperators; in the latter scenario one speaks of “prosocial rewarding”. These collective mechanisms can be viewed as primitive institutions, as group members both design and enforce the rules to administer incentives to overcome social dilemmas^{17}. Pool rewards^{15, 18} are particularly interesting because they involve the creation of resources, as opposed to their destruction (as in punishment). Prosocial rewarding can favor cooperation only if nonrewarding players can be sufficiently prevented from accessing reward funds so that secondorder defectors benefit less from rewards than do rewarders^{15}. However, the presence of “antisocial rewarders”, i.e., individuals who do not contribute to the public good but reward themselves, destroys cooperation unless additional mechanisms, such as better rewarding abilities for prosocials, work in combination with exclusion^{19}.
Pool rewards can also be viewed as a second collective action dilemma played exclusively among those players who made a similar choice in the first public goods game, i.e., cooperators with each other, and defectors with each other. The nature of this secondary collective action is not necessarily similar to that of the first public goods game, and might, for example, involve nonlinear returns. This observation extends beyond human behavior, with situations where individuals are involved in different levels of social dilemmas being particularly likely in bacterial communities^{20}. Indeed, many species of bacteria secrete public good molecules (e.g., ironbinding siderophores and other signaling molecules), which are susceptible to exploitation from both their own and other strains^{21,22,23}. In addition, bacteria are also involved in withinspecies public goods games, as some of those public good molecules can also be strainspecific^{23, 24}. Thus, the relevance of poolreward mechanisms extends to nonhuman species.
A recent theoretical study by Szolnoki and Perc^{25} [hereafter, SP15] challenged the view that additional mechanisms are required to prevent antisocial rewarding from deterring cooperation in public goods games. Contrastingly, SP15 showed that, if individuals interact preferentially with neighbors in a spatially structured population, prosocial rewarding outcompetes antisocial rewarding and that increasing rewards is beneficial for prosocial rewarding. However, SP15 reached these conclusions by means of Monte Carlo simulations of a very specific model of spatial structure and evolutionary dynamics (a square lattice with overlapping groups and a Fermi update rule) where interacting groups are always equal to five. Hence, it remains unclear whether their results generalize to a broader range of spatial models.
Spatial structure can favor the evolution of cooperation. The main reason for this phenomenon is simple. When populations are spatially structured through limited dispersal, social interactions necessarily occur more often among relatives. Hence, kin selection is at work^{3, 26,27,28}. However, spatial structure also means that competitors are also more often kin^{29}. In certain models (such as an island model with WrightFisher demography^{30} or an evolutionary graph updated with a Moran birthdeath process^{31, 32}), these two effects cancel each other out and spatial structure has no effect on the evolution of cooperation. More generally, the net effect is not null, and its direction and magnitude can be often conveniently captured by a single “scaled relatedness coefficient”^{33}. The scaled relatedness coefficient depends on the demographic assumptions of a given model, including the “update rule” used to implement the evolutionary dynamics, but is otherwise independent of the payoffs from the game used to model social interactions^{33,34,35,36,37}.
Studies on spatial games and evolutionary graph theory have also investigated the effects of spatial structure on evolutionary game dynamics^{38,39,40,41}. Theses studies have shown how particular features of the graph used to represent the population and the update rules can promote or hinder the evolution of cooperation. In particular, it has been shown that, assuming weak selection on discrete strategies and additive effects, the interplay between graph topology and update rule can be captured by a single “structure coefficient” independent of the underlying game^{42, 43}. Importantly, the structure coefficient of evolutionary graph theory and the scaled relatedness coefficient of kin selection theory are connected by a simple transformation^{35}. Therefore, using scaled relatedness as a measure of spatial structure allows one to capture a large variety of spatial models, including spatial games and evolutionary graphs.
Here, we formulate a mathematical model that clarifies the role of spatial structure for cooperation to be favored through pool rewarding. In contrast to SP15 [which, in the tradition of spatial games and evolutionary graphs^{38, 40, 41, 44}, assume discrete strategies and strong selection] we assume continuous strategies and weak selection^{45, 46}. Our different assumptions allow us to build on existing theoretical work^{36} to analytically derive the conditions under which cooperation is favored and to write them as functions of the parameters of the game (including the group size) and of a single “scaled relatedness coefficient”^{33, 35, 36}, which serves as a natural measure of spatial structure. This allows us to make general predictions about the effect of spatial structure on cooperation, and to make connections between our results and the vast literature on social evolution theory^{3, 28, 47}.
Model
Public goods game with prosocial and antisocial reward funds
We consider a collective action problem with an incentive mechanism based on reward funds following the model of SP15. Individuals interact in groups of size n and play a linear public goods game (PGG) followed by a rewarding stage with nonlinear returns. There are two types of actions available to individuals: “rewarding cooperation” (RC, or “prosocial rewarding”), whereby a benefit r_{1}γ/n is provided to all group members (including the focal) at a cost γ, and “rewarding defection” (RD, or “antisocial rewarding”), whereby no benefit is provided and no cost is payed. The parameter r_{1} is the multiplication factor of the PGG, and it is such that 1 < r_{1} < n; this ensures that the first stage of the game is a prisoner’s dilemma, so that if rewards are absent rewarding cooperation is a dominated strategy.
Individuals choosing RC or RD also invest in their own reward funds. Each reward fund yields a per capita net reward r_{2} − γ (reward benefit r_{2} minus cost of contributing to the reward pool γ) provided there is at least another individual playing the same action among the n − 1 other group members, and zero otherwise (i.e., selfrewarding is not allowed and the cost γ is payed only if the rewarding institution is created). For example, a focal individual playing RC will pay the cost and receive the reward only if there is at least another RC among its n − 1 partners. This reflects a situation where reward funds yield nonlinear returns, and is reminiscent of those of a volunteer’s dilemma^{48}. Since the net reward r_{2} − γ does not depend on the group size n, r_{2} can in principle take any value greater than, or equal to γ. Note that individuals choosing the most common action are more likely to get the reward, even under random group formation. Hence, RC can prevail as long as its frequency in the global population is above one half and rewards outweigh the net cost of contributing to the PGG. However, if selfrewarding is allowed, cooperation is never favored even when all individuals play RC^{19}.
With the previous assumptions, and letting without loss of generality γ = 1, the payoffs for a focal individual choosing either RC or RD when k coplayers choose RC (and n − 1 − k coplayers choose RD) are respectively given by (cf. Equations 3.1, 3.2, and 3.3 in SP15):
and
Note that if everybody plays RC, everybody gets a payoff c_{ n−1} = r_{1} + r_{2} − 2. Instead, if everybody plays RD, everybody gets d_{0} = r_{2} − 1. Since r_{2} > 1, c_{ n−1} > d_{0} holds for all values of r_{1} and r_{2}, which means that full prosocial rewarding Pareto dominates full antisocial rewarding: Players are collectively better if all play prosocial rewarding with probability one rather than if all play antisocial rewarding with probability one. Therefore, despite the presence of rewards available to both cooperators and defectors, the game we study retains the characteristics of a typical social dilemma where full cooperation by all individuals in the group yields higher payoffs than full defection by all individuals in the group. In such situations, it is usually expected that spatial structure facilitates the evolution of RC. As we show below, this is not always the case.
Spatial structure and evolutionary dynamics
We consider a homogeneous spatially structured population of constant and finite size N_{T} where individuals interact with n − 1 other individuals according to the game described above. The exact type of spatial structure can follow any of a large family of models, including variants of the island model^{49} and transitive evolutionary graphs^{50}. In this last case, and for simplicity, we assume that individuals play a single (k + 1)player game with their k nearest neighbors, where k is the degree of the graph. All that is required for our analysis to be valid is that the selection gradient can be written in a form proportional to the gain function given by Eq. (3) below. We refer the interested reader to previous literature^{28, 33,34,35,36} for more details on this formalism and the models of spatial structure captured by our approach.
We assume that individuals implement mixed strategies, i.e., they play RC with probability z and RD with probability 1 − z, and investigate the evolutionary dynamics of the phenotype z. More specifically, we consider the fixation probability ρ(z, δ) of a single mutant playing z + δ in a resident population of phenotype z, take the phenotypic selection gradient $\mathcal{S}(z)={(d\rho /d\delta )}_{\delta =0}$ as a measure of evolutionary success^{35, 51}, and look into convergence stable strategies^{52} under trait substitution dynamics^{28}.
In order to evaluate the selection gradient, we make use of standard results regarding the evolution of a continuous phenotype in a spatially structured population^{28}. Denoting by z_{•} the phenotype of a focal individual, by z_{◦} the average phenotype of the individuals it socially interacts with, and by f(z_{•}, z_{◦}) the fecundity of the focal individual, and further assuming that fecundity is proportional to the expected payoffs from the game, the selection gradient $\mathcal{S}(z)$ takes the form^{34}
where $\mathcal{G}(z)$ is the “gain function”, which consists of three components: (i) the effect of the focal individual’s behavior on its fecundity (i.e., the “direct effect” $\U0001d4d2(z)$), (ii) the effect of the coplayers’ behavior on the focal individual’s fecundity (i.e., the “indirect effect” $\U0001d4d1(z)$), and (iii) a measure of relatedness between the focal individual and its neighbors, demographically scaled so as to capture the effects of local competition (i.e., the “scaled relatedness coefficient” κ).
For the matrix game with two pure strategies as the one we consider here, the direct and indirect effects appearing in Eq. (3) can be written, up to a constant factor, as^{36}
where
are the “direct gains from switching” recording the changes in payoff experienced by a focal if it unilaterally switches its action from RD to RC when k coplayers stick to RC and n − 1 − k stick to RD^{53}, and
are the “indirect gains from switching” recording the changes in the total payoff accrued by coplayers when the focal unilaterally switches its action from RD to RC^{36}. Eq. (4) expresses $\U0001d4d2(z)$ and $\U0001d4d1(z)$ as expected values of the direct and indirect gains from switching when the number of other individuals playing RC is distributed according to a binomial distribution with parameters n and z.
A necessary and sufficient condition for a mutant with phenotype z + δ to have a fixation probability greater than neutral when δ is vanishingly small is that $\mathcal{S}(z)>0$ and hence that $\U0001d4d2(z)+\kappa \U0001d4d1(z)>0$ holds. This condition can be interpreted as a scaled form of Hamilton’s rule^{33, 35}. Importantly, the gain function $\mathcal{G}(z)=\phantom{\rule{.25em}{0ex}}\U0001d4d2(z)+\kappa \U0001d4d1(z)$ allows one to identify “convergence stable” evolutionary equilibria^{54,55,56}; these are given either by singular strategies z^{*} (i.e., the zeros of the gain function) satisfying $\mathcal{G}\prime ({z}^{\u204e})<0$, or by the extreme points z = 0 (if $\mathcal{G}(0)<0$) and z = 1 (if $\mathcal{G}(1)>0$). Convergence stability is a standard way of characterizing longterm evolutionary attractors; a phenotype z^{*} is convergence stable if for resident phenotypes close to z^{*} mutants can invade only if mutants are closer to z^{*} than the resident^{52}.
Scaled relatedness
The coefficient κ appearing in Eq. (3) is the “scaled relatedness coefficient”, which balances the effects of both increased genetic relatedness and increased local competition characteristic of spatially structured populations^{29, 33,34,35, 37}. The scaled relatedness coefficient has been calculated for many models of spatial structure for which Eq. (3) applies, see Table 2 of ref. 33, Table 1 of ref. 34, Appendix A of ref. 36, and references therein for some examples. We also note that by identifying
where σ is the socalled “structure coefficient”, the right hand side of Eq. (3) recovers the “canonical equation of adaptive dynamics with interaction structure” (ref. 43, Eq. 5). Table 1 of ref. 43 provides several examples of models of spatial structure and their respective values of σ; transforming these values via Eq. (7), the scaled relatedness coefficients for such models can be obtained in a straightforward manner.
Usually, κ takes a value between −1 and 1 depending on the demographic assumptions of the model, but it is always such that the larger it is the less genetic relatedness is effectively reduced by the extent of local competition. Importantly, the larger the magnitude of scaled relatedness κ the more important the role of the indirect effect $\U0001d4d1(z)$ in the selection gradient. For this reason, we use scaled relatedness κ as a measure of spatial structure; we hence refer in the following to an increase in κ as an increase in spatial structure.
In the following, we present some examples to illustrate the connection between explicit spatial structure models and the formalism we use here, based on the scaled relatedness coefficient. For a wellmixed population or an island model with WrightFisher demography (such that generations are overlapping), the value of scaled relatedness is
(ref. 34, Eq. B.1). Contrastingly, in an island model with n_{d} demes with N individuals each (so that N_{T} = n_{d}N) and a Moran demography (where adults have a positive probability of surviving to the next generation) scaled relatedness becomes
where m is the migration rate (ref. 34, Eq. B.2). In this case, κ is inversely proportional to the migration rate m; it follows that an island model with less migration has, according to our definition, more spatial structure. As a final example, consider a transitive evolutionary graph of size N_{T} and degree k updated with a (deathbirth) Moran demography. In this case we have
(ref. 34, Eq. B.5 with m = 1; which can also be recovered from the value of σ given in Table 1 of ref. 43 after applying identity (7)). For this model of population structure, scaled relatedness is inversely proportional to the degree k. This means that, according to our terminology, graphs of larger degree (and hence more similar to a wellmixed population represented by a complete graph for which k = N_{T} − 1) are characterized by smaller spatial structure.
Results
Calculating the gains from switching by first replacing Eqs (1) and (2) into Eqs (5) and (6), then replacing the resulting expressions into Eqs (4a) and (4b), and simplifying, we obtain that the gain function for the PGG with reward funds can be written as (see Methods)
In the following, we identify convergence stable equilibria and characterize the evolutionary dynamics, first for wellmixed and then for spatially structured populations.
Infinitely large wellmixed populations
For wellmixed populations, and as N_{T} → ∞, the scaled relatedness coefficient reduces to zero (Eq. (8)). In this case, the gain function simplifies to $\mathcal{G}(z)=\phantom{\rule{.25em}{0ex}}\U0001d4d2(z)$ and we obtain the following characterization of the evolutionary dynamics (see Methods). If r_{1}/n + r_{2} ≤ 2, z = 0 is the only stable equilibrium, and RD dominates RC. Otherwise, if r_{1}/n + r_{2} > 2, both z = 0 and z = 1 are stable, and there is a unique z^{*} > 1/2 that is unstable. In this case, the evolutionary dynamics are characterized by bistability or positive frequency dependence, with the basin of attraction of full RD (z = 0) being always larger than the basin of attraction of full RC (z = 1). Moreover, z^{*} (and hence the basin of attraction of z = 0) decreases with increasing r_{1} and r_{2}. In particular, higher reward funds lead to less stringent conditions for RC to evolve. In any case, RC has to be initially common (z > 1/2) in order for full RC to be the final evolutionary outcome.
Spatially structured populations
Interactions in spatially structured populations (for which κ is not necessarily equal to zero) can dramatically alter the evolutionary dynamics of public goods with prosocial and antisocial rewards. In particular, we find that whether or not the extreme points z = 0 and z = 1 are stable depends on how the scaled relatedness coefficient κ compares to the critical values
and
which satisfy κ_{*} ≤ κ^{*}, in the following way (Fig. 1):

1.
For low values of κ (κ < κ_{*}), full RD (z = 0) is stable and full RC (z = 1) is unstable.

2.
For intermediate values of κ (κ_{*} < κ < κ^{*}), both full RD and full RC are stable.

3.
For large values of κ (κ > κ^{*}), full RC is stable and full RD is unstable.
For a given group size n and PGG multiplication factor r_{1}, κ_{*} = κ^{*} if and only if r_{2} = 1, i.e., if rewards are absent. In this case, full RD and full RC cannot be both stable.
Rewards have contrasting effects on κ_{*} (the critical scaled relatedness value below which full RC is unstable) and κ^{*} (the critical scaled relatedness value above which full RD is unstable). On the one hand, κ_{*} is decreasing in the reward benefit r_{2}, so larger rewards increase the parameter space where full RC is stable. If spatial structure is maximal, i.e., κ = 1, the condition for full RC to be stable is r_{1} + r_{2} > 2, which always holds. On the other hand, κ^{*} is an increasing function of r_{2}. Hence, larger rewards make it harder for spatial structure to destabilize the full RD equilibrium, and hence for RC to invade. For κ = 1, full RD is still stable whenever r_{1} < r_{2}. Contrastingly, full defection can never be stable if κ = 1 in the absence of rewards (i.e., r_{2} = 1) since, by definition, r_{1} > 1. From this analysis we can already conclude that even maximal spatial structure does not necessarily allow RC to invade and increase when rare. In addition, a minimum value of scaled relatedness is required for prosocial rewarding to be stable once it is fully adopted by the entire population.
Let us now investigate singular strategies. Depending on the parameter values, there can be either zero, one, or three interior points at which the gain function (and hence the selection gradient) vanishes (see Methods). If there is a unique singular point, then it is unstable while z = 0 and z = 1 are stable, and the evolutionary dynamics is characterized by bistability. If there are three singular points (probabilities z_{L}, z_{M}, and z_{R}, satisfying 0 < z_{L} < z_{M} < z_{R} < 1), then z = 0, z_{M}, and z = 1 are stable, while z_{L} and z_{R} are unstable. In this case RD and RC coexist at the convergence stable mixed strategy z_{M}; a necessary condition for this dynamical outcome is both relatively large reward benefits and relatively large scaled relatedness.
We calculated the singular strategies numerically, as the equation $\mathcal{G}(z)=0$ cannot be solved algebraically in the general case (Figs 2 and 3). Increasing scaled relatedness generally increases the parameter space where RC is favored. Yet, there are cases where increasing scaled relatedness can hinder the evolution of RC. Specifically, when the reward benefit is considerably larger than the public goods share, increasing scaled relatedness can increase the basin of attraction of the full RD equilibrium (Fig. 2c). Also, increasing rewards can be detrimental to RC in spatially structured populations by increasing the basin of attraction of full RD (Fig. 3c,f,h,i); this is never the case when there is no spatial structure (Fig. 3a,d,g). Finally, the best case scenario from the point of view of a rare mutant playing z = δ (where δ is vanishingly small) is in the absence of rewards (i.e., r_{2} = 1), because that is the case where the required threshold value of scaled relatedness to favor prosocial rewarding is the lowest (i.e., where κ^{*} attains its minimum value in Eq. (13)).
In order to understand why, contrary to naive expectations, increasing spatial structure might sometimes select against RC, note first that the derivative of the gain function with respect to κ is equal to the indirect effect $\U0001d4d1(z)$. This is nonnegative if
In the absence of rewards (i.e., r_{2} = 1), condition (14) always holds. That is, increasing scaled relatedness always promotes cooperation when there are no rewards. In addition, when 0 ≤ z ≤ 1/2, the function q(z) is nonnegative, so that condition (14) holds and $\U0001d4d1(z)$ is positive. Hence, increasing scaled relatedness is always beneficial for RC when such behavior is expressed less often than RD. However, increasing scaled relatedness might not always favor RC when such behavior is already common in the population, i.e., if z > 1/2. Indeed, when the multiplication factor of the PGG is relatively small and rewards are relatively large, condition (14) is not fulfilled for some z and $\U0001d4d1(z)$ is negative for some probability of playing RC (Fig. 4).
A closer look at the indirect gains from switching Θ_{ k } (Eq. 6) reveals why $\U0001d4d1(z)$, and hence the effect of scaled relatedness on the selection gradient, can be negative for some z. The indirect gains from switching are nonnegative for all k ≠ n − 2. For k = n − 2 and n ≥ 4 we have Θ_{ n−2} = (n − 1)r_{1}/n − r_{2} + 1, which can be negative if
holds. Inequality (15) is hence a necessary condition for $\U0001d4d1(z)$ to be negative for some z and for prosocial rewarding to fail to qualify as payoff cooperative or payoff altruistic [sensu ref. 36]. Indeed, when condition (15) holds and hence Θ_{ n−2} < 0, prosocial rewarding cannot be said to be altruistic according to the “focalcomplement” interpretation of altruism^{57, 58}. This is because the sum of the payoffs of the n − 1 coplayers of a given focal individual, out of which n − 2 play RC and one plays RD, is larger if the focal plays RD than if the focal plays RC. We also point out that RC is not altruistic according to an “individualcentered” interpretation^{58, 59} or “cooperative” [sensu ref. 60] if
since in this case the payoff to a focal individual playing RD as a function of the number of other players choosing RC in the group, d_{ k } (see Eq. (2)), is decreasing (and not increasing) with k at k = n − 2. Indeed, if condition (16) holds, players do not necessarily prefer other group members to play RC irrespective of their own strategy: a focal RD player would prefer one of its n − 1 coplayers to play RD rather than play RC. In the light of this analysis, it is perhaps less surprising that for some parameters increasing spatial structure can be detrimental to the evolution of prosocial rewarding, even if prosocial rewarding Pareto dominates antisocial rewarding.
Individualbased simulations
To test the validity of our mathematical model, we also ran individualbased simulations for a wellmixed population and a square lattice of size N_{T} = 400, both updated with a Moran deathbirth rule (see Methods for details). As predicted by our mathematical analysis, the evolutionary dynamics are often characterized by a single interior convergence unstable point z^{*} (Fig. 5). When the phenotypic value z_{0} of the initially monomorphic population is below such point, selection tends to disfavors rewarding cooperation and the population converges to full RD (z = 0). Contrastingly, when the population starts with a phenotypic value larger than z^{*}, selection tends to favor rewarding cooperation and the population converges to full RC (z = 1). In all cases, the convergence unstable strategy z^{*} resulting from our mathematical analysis are very good predictors of the point at which selection changes direction in our individualbased simulations.
Discussion
We have investigated the effect of spatial structure on the evolution of public goods cooperation with reward funds. Measuring spatial structure by means of a scaled relatedness coefficient allowed us to capture both the effects of increased genetic assortment and increased local competition that characterize evolution in spatially structured populations. We have found that (i) prosocial rewarding cannot invade full antisocial rewarding unless scaled relatedness is sufficiently large, but (ii) increasing scaled relatedness can be detrimental to prosocial rewarding in cases where rewards are considerably larger than the public goods share. We have also demonstrated the contrasting effects of increasing rewards, which (iii) only benefits prosocial rewarding in wellmixed populations, but (iv) can also benefit antisocial rewarding in spatially structured populations. These results illustrate how poolrewards introduce nonlinearities to the public goods game, which in turn lead to counterintuitive effects of population structure on the evolutionary dynamics.
We found that increasing spatial structure can sometimes be detrimental to prosocial rewarding. We confirmed this analytical result with individualbased simulations, where we showed that the basin of attraction of full prosocial rewarding can be greater in a wellmixed population than in a square lattice with a Moran deathbirth demography. This is a somewhat counterintuitive result, because spatial structure often favors the evolution of cooperation^{39, 41, 50, 60,61,62}. However, previous studies have shown that increasing population structure can sometimes have a negative effect on the evolution of cooperative strategies^{34, 63, 64}. For example, Hauert and Doebeli^{63} found that, when social interactions are modelled as a twoplayer snowdrift game between pure strategists under strong selection, wellmixed populations lead to higher levels of cooperation than square lattices. Peña et al.^{64} studied a multiplayer version of this game, but assuming weak selection and a deathbirth Moran process, and found similar results. In our case, spatial structure can oppose prosocial rewarding because the indirect gains from switching from antisocial to prosocial rewarding can be negative. In particular, if individuals interact in groups of size n ≥ 4 and exactly n − 2 coplayers choose rewarding cooperation while one coplayer chooses rewarding defection, playing rewarding defection (RD) rather than rewarding cooperation (RC) might increase (rather than decrease) the payoffs of coplayers. The reason is that, in this case, by choosing RD the focal player helps its RD coplayer getting the reward fund, while also allowing its RC coplayers to keep theirs, as the focal contribution is not critical to the creation of the prosocial reward fund. If the reward benefit is so large that Eq. (15) holds, the benefit to the single RD coplayer is greater than what everybody loses by the focal not contributing to the public good, and the sum of payoffs to coplayers is greater if the focal plays RD than if it plays RC. This implies that, although prosocial rewarding Pareto dominates antisocial rewarding, it does not strictly qualify as being payoff altruistic or payoff cooperative^{36}, hence the mixed effects of increasing spatial structure.
Our results also revealed the fact that higher values of the reward benefit r_{2} make it more difficult for prosocial rewarding to invade from rarity in spatially structured populations. Indeed, the critical value of scaled relatedness required for prosocial rewarding to be favored over antisocial rewarding is greater in the presence of rewards than in their absence. This is important because rewards are meant to be mechanisms incentivizing provision in public goods games^{15, 18}, rather than making collective action more difficult to emerge. Clearly, our result hinges on the assumption that prosocial rewarders and antisocial rewarders are both equally effective in rewarding themselves, i.e., that r_{2} is the same for both prosocial and antisocial rewarders. Challenging this assumption by making investments in rewards contingent on the production of the public good, or by increasing the ability of prosocials to reward each other relative to that of antisocials^{19}, will necessarily change this picture and promote prosocial rewarding in larger regions of the parameter space.
Although higher rewards prevent the initial invasion of prosocial rewarding, we have also shown that, once prosocial rewarding is common, higher rewards can further enhance the evolution of prosocial rewarding. These results are in line with the findings of SP15, who showed that when both spatial structure is sufficiently large (their spatial model supports cooperation even in the absence of rewards) and the initial frequency of prosocial rewarding is relatively high (i.e., 1/4 in their simulations), larger rewards promote prosocial rewarding.
In contrast to the original model by SP15, which considered discrete strategies and strong selection, we assumed continuous mixed strategies and weak selection^{36}. For wellmixed populations, it is well known that these different sets of assumptions lead to identical results under a suitable reinterpretation of the model variables^{53}. Thus, our result that in this case there is a unique convergence unstable z^{*} in mixed strategies also implies that the replicator dynamics for the twostrategy model will be characterized by an unstable rest point at a frequency z^{*} of prosocial rewarders, and corroborates the numerical results presented in section 3.a of SP15. By contrast, for structured populations the invasion and equilibrium conditions between discrete and mixedstrategy game models^{45, 46, 63} and between weak and strong selection models^{61} can differ. Hence, our results for spatially structured populations need not be identical to those reported in SP15. Importantly, our mathematical framework assumes that the population is essentially monomorphic. An initial state of, say, z = 1/3 in our Fig. 2 means that all individuals in the population play the same mixed strategy z = 1/3. Evolution then proceeds by means of a trait substitution sequence (TSS), whereby a single mutant (which we also assume plays a slightly different mixed strategy z + δ, where δ is small) will either become extinct or invade and replace the resident population^{65}. If the latter happens, the resident strategy is updated to z + δ and the process starts again, until a convergence stable state is reached. The TSS assumption, common in adaptive dynamics and related mathematical methods studying the evolution of continuous traits in spatially structured populations^{28, 66} is then in stark contrast to the numerical simulations used in SP15 and related studies^{41}, where evolution starts from a polymorphic population where a large number of mutants appear en masse either randomly or clustered together according to a given “prepared initial state”.
Finally, there is a slight difference in the way SP15 and our study (explicitly in our simulations and implicitly in our analytical model) implemented the evolutionary game on a graph. SP15 assume, in the tradition of other computational studies (e.g., refs 41, 67, 68), that a focal player’s total payoff is given by the sum of payoffs obtained in k + 1 different games, one “centered” on the focal player itself and the other k centered on its neighbors. As a result, a focal player interacts not only with firstorder but also with secondorder neighbors. An analytical treatment of such case would need an extension of our framework to incorporate an additional scaled relatedness coefficient to account for interactions with secondorder neighbors^{50}. Contrastingly, we assumed that a focal player obtains its payoff from a single multiplayer game with its k immediate neighbors^{64, 69}. This more parsimonious assumption allows us to analyze multiplayer interactions on graphs in a straightforward way (i.e., by allowing us to directly use the framework developed in ref. 36 requiring a single scaled relatedness coefficient) without importantly modifying the underlying evolutionary dynamics^{69}.
Our motivation for a set of assumptions different to those of SP15 was hence for both analytical tractability and wider applicability. An analytical solution of the model with discrete strategies (as in SP15) would require tracking higherorder genetic associations and effects of local competition, which can be a complicated task even in relatively simple models of spatial structure under weak selection^{61, 70,71,72,73}. By contrast, assuming continuous strategies (and a single game per player) allowed us to identify convergence stable levels of prosocial rewarding in a wide array of spatially structured populations, each characterized by a particular value of scaled relatedness. This way, we made analytical progress going beyond the numerical results on a particular type of population structure (a square lattice with overlapping groups of size n = 5) studied in SP15. Furthermore, treating scaled relatedness as an exogenous parameter independent of specific demographic assumptions, allows making more general predictions on the evolution of a trait, as well as relating results from different types of models^{74}. Understanding the effect of more specific demographic parameters (such as the dispersal rate, the update rule, or the degree of the network) can be achieved by determining how scaled relatedness can be expressed in terms of those parameters, as we have shown above.
Our analytical results are valid only to the first order of δ (the difference between the trait of mutants and the trait of residents). As a result, we cannot evaluate whether or not the singular strategies we identify as convergence stable are also “evolutionarily stable” or “locally uninvadable”, i.e., whether a population monomorphic for a singular value will resist invasion by mutants with traits close to the singular value^{37, 52, 54, 75}. This also means that our model does not allow one to check whether or not evolutionary branching (whereby a convergence stable but locally invadable population diversifies into differentiated coexisting morphs^{76}) might occur. We hasten to note that such drawback is not particular to our method^{28}. Evolutionary stability in spatially structured populations is significantly more challenging to characterize than convergence stability^{37, 77,78,79,80} and is thus beyond the scope of the present paper.
A related issue has to do with our assumption that individuals play mixed strategies and hence that payoffs are linear in the focal’s own strategy. For this kind of models, “a peculiar degeneracy raises its ugly head”^{81}, namely that the secondorder condition to evaluate evolutionary stability in a wellmixed population is null. In turn, this implies that phenotypic variants at a singular point that is convergence stable are strictly neutral. Such degeneracy is however restricted to wellmixed populations, and does not necessarily apply to spatially structured populations. Indeed, the condition for uninvadibility under weak selection in subdivided populations has been shown to depend also on mixed partial derivatives of the payoff function^{37}, which in general are not zero. All in all, our view is that assuming individuals play mixed strategies of a matrix game is not that problematic: For wellmixed populations (where the degeneracy raises its head), the convergence stable mixed strategies can be reinterpreted as evolutionarily stable points of a replicator dynamics in discrete strategies; for spatially structured populations, there is simply no degeneracy. Future work should explore the conditions under which convergence stable mixed strategies of the model presented here and other matrix games are locally uninvadable.
Both our model and that of SP15 have not considered the presence of individuals who are able to benefit from reward funds without contributing to them. In other words, secondorder defection is avoided by design. Allowing for secondorder defection makes cooperation through pool rewarding vulnerable, even in the absence of antisocial rewarding^{15}. Therefore, even though the conclusions of SP15 contradict the findings of dos Santos^{19}, namely that antisocial rewarding deters cooperation except in certain conditions (e.g., better rewarding abilities for prosocials), SP15 did not investigate standard poolrewarding^{15, 18, 19}. Hence, their claim that spatial structure prevents antisocial rewarding from deterring cooperation, while not always true as we have shown here, does not apply to the more general case of poolreward funds where secondorder defection is allowed. Exploring the effects of spatial structure in these more realistic cases remains an interesting line of research.
To conclude, we find that antisocial rewarding deters the invasion of cooperation unless scaled relatedness is sufficiently high and rewards are relatively low, or ideally absent. We argue that additional countermeasures, such as exclusion and better rewarding abilities for prosocials^{19}, are still required to (i) prevent antisocial rewarding from deterring cooperation between unrelated social partners, and (ii) allow prosocial rewarding to invade either when relatedness is low or when rewards are too large.
Methods
Gain function
To derive the gain function $\mathcal{G}(z)$ (Eq. (11)), we first calculate the direct and indirect gains from switching (Eqs (5) and (6)) associated to the payoffs of the game. We find that the gains from switching depend on the group size n in the following way.

1.
For n = 2: (Δ_{0}, Δ_{1}) = (r_{1}/2 − r_{2}, r_{1}/2 + r_{2} − 2) and (Θ_{0}, Θ_{1}) = (r_{1}/2 − r_{2} + 1, r_{1}/2 + r_{2} − 1).

2.
For n = 3: (Δ_{0}, Δ_{1}, Δ_{2}) = (r_{1}/3 − r_{2}, r_{1}/3 − 1, r_{1}/3 + r_{2} − 2) and (Θ_{0}, Θ_{1}, Θ_{2}) = (2r_{1}/3, 2r_{1}/3, 2r_{1}/3).

3.
For n = 4: (Δ_{0}, Δ_{1}, Δ_{2}, Δ_{3}) = (r_{1}/4 − r_{2}, r_{1}/4 − 1, r_{1}/4 − 1, r_{1}/4 + r_{2} − 2) and (Θ_{0}, Θ_{1}, Θ_{2}, Θ_{3}) = (3r_{1}/4, 3r_{1}/4 + r_{2} − 1, 3r_{1}/4 − r_{2} + 1, 3r_{1}/4).

4.
For n ≥ 5: (Δ_{0}, Δ_{1}, …, Δ_{ n−2}, Δ_{ n−1}) = (r_{1}/n − r_{2}, r_{1}/n − 1, …, r_{1}/n − 1, r_{1}/n + r_{2} − 2) and (Θ_{0}, Θ_{1}, Θ_{2}, …, Θ_{ n−3}, Θ_{ n−2}, Θ_{ n−1}) = ((n − 1)r_{1}/n, (n − 1)r_{1}/n + r_{2} − 1, (n − 1)r_{1}/n, …, (n − 1)r_{1}/n, (n − 1)r_{1}/n − r_{2} + 1, (n − 1)r_{1}/n).
Replacing the direct gains from switching Δ_{ k } into the expression for the direct effect $\U0001d4d2(z)$ (Eq. (4a)) and the indirect gains from switching Θ_{ k } into the expression for the indirect effect $\U0001d4d1(z)$ (Eq. (4b)), and simplifying, we obtain the gain function $\mathcal{G}(z)$ given in Eq. (11), which is valid for all n ≥ 2.
Evolutionary dynamics for κ = 0
For κ = 0, the gain function $\mathcal{G}(z)$ (Eq. (11)) reduces to $\U0001d4d2(z)$ (Eq. (4a)). This function is increasing and its endpoints are given by $\U0001d4d2(0)={r}_{1}/n{r}_{2}$ and $\mathcal{C}(1)={r}_{1}/n+{r}_{2}2$. Since 1 < r_{1} < n and r_{2} > 1, $\U0001d4d2(0)<0$ always holds and z = 0 is always stable. If r_{1}/n + r_{2} ≤ 2, $\U0001d4d2(z)$ is nonpositive for all z and z = 0 is the only stable equilibrium. If r_{1}/n + r_{2} > 2, $\U0001d4d2(1)>0$ and z = 1 is also stable. In this case, and since $\U0001d4d2(z)$ is increasing, $\U0001d4d2(z)$ has a single zero z^{*} in (0, 1) corresponding to an unstable equilibrium. Such zero is given by the unique solution to
$$\underset{p(z)}{\underset{\u23df}{{z}^{n1}{(1z)}^{n1}}}=\underset{\alpha}{\underset{\u23df}{\frac{n{r}_{1}}{n({r}_{2}1)}}}.$$Since p(z) is increasing in z, p(1/2) = 0, and α > 0 always holds, it follows that z^{*} > 1/2. Additionally, since α is decreasing in both r_{1} and r_{2}, z^{*} is increasing in both r_{1} and r_{2}.
Evolutionary dynamics for κ ≠ 0
Rearranging terms, the gain function $\mathcal{G}(z)$ given by Eq. (11) can be alternatively written as
$$\mathcal{G}(z)=\frac{{r}_{1}}{n}\left[1+\kappa (n1)\right]1+({r}_{2}1)\mathcal{P}(z)$$where
$$\mathcal{P}(z)=\sum _{k=0}^{n1}\left(\begin{array}{c}n1\\ k\end{array}\right){z}^{k}{(1z)}^{n1k}{\zeta}_{k}$$is a polynomial in Bernstein form^{53} of degree n − 1 with coefficients given by

1.
(ζ_{0}, ζ_{1}) = (−(1 + κ), 1 + κ) if n = 2.

2.
(ζ_{0}, ζ_{1}, ζ_{2}) = (−1, 0, 1) if n = 3.

3.
(ζ_{0}, ζ_{1}, ζ_{2}, ζ_{3}) = (−1, κ, −κ, 1) if n = 4.

4.
(ζ_{0}, ζ_{1}, ζ_{2}, …, ζ_{ n−3}, ζ_{ n−2}, ζ_{ n−1}) = (−1, κ, 0, …, 0, −κ, 1) if n ≥ 5.
The number of sign changes (and hence of singular points) of $\mathcal{G}(z)$ is bounded from above by the number of sign changes of $\mathcal{P}(z)$. Moreover, and by the variationdiminishing property of polynomials in Bernstein form^{53}, the number of sign changes of $\mathcal{P}(z)$ is equal to the number of sign changes of the sequence of coefficients (ζ_{0}, …, ζ_{ n−1}) minus an even integer. It then follows that the number of singular points is at most one if n ≤ 3 or if scaled relatedness is nonpositive, κ ≤ 0. In this case, the unique singular point z^{*} is convergence unstable. However, if n ≥ 4 and κ > 0, there could be up to three singular points z_{L}, z_{M}, and z_{R} satisfying 0 < z_{L} < z_{M} < z_{R} < 1 such that z_{L} and z_{R} are convergence unstable and z_{M} is convergence stable.
Computer simulations
We performed individualbased simulations for a population composed of N_{T} individuals, using NumPy version 1.11.1. Starting with a population monomorphic for a probability z = z_{0} of playing rewarding cooperation, we track the evolution of the phenotypic distribution as mutations of small effect continuously arise. Each individual j = 1, …, N_{T} is characterized by its probability z_{ j }. The average payoff of individual j is given by ${\pi}_{j}={\sum}_{k=0}^{n1}{\phi}_{k}\{{z}_{j}{c}_{k}+(1{z}_{j}){d}_{k}\}$ where the payoff sequences c_{ k } and d_{ k } are respectively given by Eqs (1) and (2), and ϕ_{ k } is the probability that exactly k out of n − 1 of j’s neighbors choose action RC. Technically, ϕ_{ k } is the probability mass function of a Poisson binomial distribution with parameters ${z}_{{j}_{1}},{z}_{{j}_{2}},\dots ,{z}_{{j}_{n1}}$, where ${j}_{\ell}$ represents the $\ell $th neighbor of individual j, and ${z}_{{j}_{\ell}}$ its phenotype. For simplicity, we approximate ϕ_{ k } by the probability mass function of a binomial random variable with parameters n and $p={\sum}_{\ell =1}^{n1}{z}_{{j}_{\ell}}/(n1)$; such approximation to the true probability mass function is very accurate when the probabilities ${z}_{{j}_{\ell}}$ are close to each other. We update the population via a deathbirth Moran process, i.e., we (i) randomly select one individual to die and one of its neighbors to give birth, and (ii) fill the vacated breeding spot with probability proportional to the average payoff. More specifically, assuming that j is chosen to die, its $\ell $th neighbor is given a probability $\mathrm{exp}(w{\pi}_{{j}_{\ell}})$ of replacing j, where w is the intensity of selection and ${\pi}_{{j}_{\ell}}$ is the payoff of the $\ell $th neighbor of j. The vacated breeding spot is filled with the same phenotypic value as the parent with probability 1 − μ and with a mutated phenotypic value with probability μ. If mutation occurs, the mutated phenotypic value is given by the parent value plus a small perturbation sampled from a normal distribution with mean zero and standard deviation ν. The resulting phenotypes are truncated so that they are numbers between 0 and 1. The previous procedure is repeated for a sufficiently large number of time steps.
We simulated two models of spatial population structure: a square lattice with periodic boundary conditions (i.e., a type of transitive graph with k = 4) of N_{T} = 400 nodes, and a wellmixed population of N_{T} = 400 individuals where a given focal individual is assigned four randomly chosen other individuals as neighbors each time step. Other parameter values used were: w = 10, μ = 0.01, ν = 0.005.
Additional Information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
 1.
Axelrod, R. The evolution of cooperation (Basic Books, New York, NY, 1984).
 2.
Sugden, R. The economics of rights, cooperation and welfare (Blackwell, Oxford and New York, 1986).
 3.
Frank, S. A. Foundations of social evolution (Princeton University Press, Princeton, NJ, 1998).
 4.
Lehmann, L. & Keller, L. The evolution of cooperation and altruism: a general framework and a classification of models. Journal of Evolutionary Biology 19, 1365–1376 (2006).
 5.
West, S. A., Griffin, A. S. & Gardner, A. Evolutionary explanations for cooperation. Current Biology 17, R661–R672 (2007).
 6.
Sigmund, K. The calculus of selfishness (Princeton University Press, 2010).
 7.
Hardin, G. The tragedy of the commons. Science 162, 1243–1248 (1968).
 8.
Oliver, P. Rewards and punishments as selective incentives for collective action: theoretical investigations. American Journal of Sociology 85, 1356–1375 (1980).
 9.
Hilbe, C. & Sigmund, K. Incentives and opportunism: from the carrot to the stick. Proceedings of the Royal Society B: Biological Sciences 277, 2427–2433 (2010).
 10.
Sigmund, K. Punish or perish? Retaliation and collaboration among humans. Trends in Ecology & Evolution 22, 593–600 (2007).
 11.
Boyd, R. & Richerson, P. Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethology and sociobiology 13, 171–195 (1992).
 12.
dos Santos, M., Rankin, D. J. & Wedekind, C. The evolution of punishment through reputation. Proceedings of the Royal Society B: Biological Sciences 278, 371–377 (2011).
 13.
Hilbe, C. & Traulsen, A. Emergence of responsible sanctions without second order free riders, antisocial punishment or spite. Scientific Reports 2, 458 (2012).
 14.
Hauert, C. Replicator dynamics of reward and reputation in public goods games. Journal of Theoretical Biology 267, 22–28 (2010).
 15.
Sasaki, T. & Unemi, T. Replicator dynamics in public goods games with reward funds. Journal of Theoretical Biology 287, 109–114 (2011).
 16.
Sasaki, T., Uchida, S. & Chen, X. Voluntary rewards mediate the evolution of pool punishment for maintaining public goods in large populations. Scientific Reports 5, 8917 (2015).
 17.
Ostrom, E. Governing the commons: the evolution of institutions for collective action (Cambridge University Press, Cambridge, 1990).
 18.
Sasaki, T. & Uchida, S. Rewards and the evolution of cooperation in public good games. Biology Letters 10, 20130903 (2014).
 19.
dos Santos, M. The evolution of antisocial rewarding and its countermeasures in public goods games. Proceedings of the Royal Society B: Biological Sciences 282, 20141994 (2015).
 20.
Hibbing, M. E., Fuqua, C., Parsek, M. R. & Peterson, S. B. Bacterial competition: surviving and thriving in the microbial jungle. Nature Reviews Microbiology 8, 15–25 (2010).
 21.
Griffin, A. S., West, S. A. & Buckling, A. Cooperation and competition in pathogenic bacteria. Nature 430, 1024–1027 (2004).
 22.
Khan, A. et al. Differential crossutilization of heterologous siderophores by nodule bacteria of Cajanus cajan and its possible role in growth under ironlimited conditions. Applied Soil Ecology 34, 19–26 (2006).
 23.
Hughes, D. P. & d’Ettorre, P. Sociobiology of communication: an interdisciplinary perspective (Oxford University Press, Oxford, 2008).
 24.
Hohnadel, D. & Meyer, J. M. Specificity of pyoverdinemediated iron uptake among fluorescent Pseudomonas strains. Journal of Bacteriology 170, 4865–4873 (1988).
 25.
Szolnoki, A. & Perc, M. Antisocial pool rewarding does not deter public cooperation. Proceedings of the Royal Society B: Biological Sciences 282, 20151975 (2015).
 26.
Hamilton, W. D. The genetical evolution of social behaviour. ii. Theoretical Population Biology 7, 17–52 (1964).
 27.
Hamilton, W. Selection of selfish and altruistic behavior in some extreme models. In Eisenberg, J. F. & Dillon, W. S. (eds) Man and Beast: Comparative Social Behavior, 57–91 (Smithsonian Press, Washington, 1971).
 28.
Rousset, F. Genetic structure and selection in subdivided populations. Monographs in Population Biology 40, 264 (2004).
 29.
Queller, D. C. Genetic relatedness in viscous populations. Evolutionary Ecology 8, 70–73 (1994).
 30.
Taylor, P. D. Altruism in viscous populations — an inclusive fitness model. Evolutionary Ecology 6, 352–356 (1992).
 31.
Grafen, A. An inclusive fitness analysis of altruism on a cyclical network. Journal of Evolutionary Biology 20, 2278–2283 (2007).
 32.
Taylor, P. D., Day, T. & Wild, G. Evolution of cooperation in a finite homogeneous graph. Nature 447, 469–472 (2007).
 33.
Lehmann, L. & Rousset, F. How life history and demography promote or inhibit the evolution of helping behaviours. Philosophical Transactions of the Royal Society B: Biological Sciences 365, 2599–2617 (2010).
 34.
Van Cleve, J. & Lehmann, L. Stochastic stability and the evolution of coordination in spatially structured populations. Theoretical Population Biology 89, 75–87 (2013).
 35.
Van Cleve, J. Social evolution and genetic interactions in the short and long term. Theoretical Population Biology 103, 2–26 (2015).
 36.
Peña, J., Nöldeke, G. & Lehmann, L. Evolutionary dynamics of collective action in spatially structured populations. Journal of Theoretical Biology 382, 122–136 (2015).
 37.
Mullon, C., Keller, L. & Lehmann, L. Evolutionary stability of jointly evolving traits in subdivided populations. The American Naturalist 188, 175–195 (2016).
 38.
Nowak, M. A. & May, R. M. Evolutionary games and spatial chaos. Nature 359, 826–829 (1992).
 39.
Ohtsuki, H., Hauert, C., Lieberman, E. & Nowak, M. A. A simple rule for the evolution of cooperation on graphs. Nature 441, 502–505 (2006).
 40.
Szabó, G. & Fath, G. Evolutionary games on graphs. Physics Reports 446, 97–216 (2007).
 41.
Perc, M., GómezGardeñes, J., Szolnoki, A., Floría, L. & Moreno, Y. Evolutionary dynamics of group interactions on structured populations: a review. Journal of the Royal Society Interface 10, 20120997 (2013).
 42.
Tarnita, C. E., Ohtsuki, H., Antal, T., Fu, F. & Nowak, M. A. Strategy selection in structured populations. Journal of Theoretical Biology 259, 570–581 (2009).
 43.
Allen, B., Nowak, M. A. & Dieckmann, U. Adaptive dynamics with interaction structure. The American Naturalist 181, E139–E163 (2013).
 44.
Perc, M. & Szolnoki, A. Coevolutionary games—a mini review. BioSystems 99, 109–125 (2010).
 45.
Grafen, A. The hawkdove game played between relatives. Animal Behaviour 27, 905–907 (1979).
 46.
Wild, G. & Traulsen, A. The different limits of weak selection and the evolutionary dynamics of finite populations. Journal of Theoretical Biology 247, 382–390 (2007).
 47.
Lehmann, L. & Rousset, F. The genetical theory of social behaviour. Philosophical Transactions of the Royal Society of London B: Biological Sciences 369 (2014).
 48.
Diekmann, A. Volunteer’s dilemma. Journal of Conflict Resolution 29, 605–610 (1985).
 49.
Wright, S. Evolution in mendelian populations. Genetics 16, 97–159 (1931).
 50.
Lehmann, L., Keller, L. & Sumpter, D. J. T. The evolution of helping and harming on graphs: the return of the inclusive fitness effect. Journal of Evolutionary Biology 20, 2284–2295 (2007).
 51.
Rousset & Billiard. A theoretical basis for measures of kin selection in subdivided populations: finite populations and localized dispersal. Journal of Evolutionary Biology 13, 814–825 (2000).
 52.
Geritz, S. A. H., Kisdi, E., Meszéna, G. & Metz, J. A. J. Evolutionarily singular strategies and the adaptive growth and branching of the evolutionary tree. Evolutionary Ecology Research 12, 35–57 (1998).
 53.
Peña, J., Lehmann, L. & Nöldeke, G. Gains from switching and evolutionary stability in multiplayer matrix games. Journal of Theoretical Biology 346, 23–33 (2014).
 54.
Eshel, I. Evolutionary and continuous stability. Journal of Theoretical Biology 103, 99–111 (1983).
 55.
Eshel, I. On the changing concept of evolutionary population stability as a reflection of a changing point of view in the quantitative theory of evolution. Journal of Mathematical Biology 34, 485–510 (1996).
 56.
Taylor, P. D. Evolutionary stability in oneparameter models under weak selection. Theoretical Population Biology 36, 125–143 (1989).
 57.
Matessi, C. & Karlin, S. On the evolution of altruism by kin selection. Proceedings of the National Academy of Sciences 81, 1754–1758 (1984).
 58.
Kerr, B., GodfreySmith, P. & Feldman, M. W. What is altruism? Trends in Ecology & Evolution 19, 135–140 (2004).
 59.
Uyenoyama, M. & Feldman, M. W. Theories of kin and group selection: a population genetics approach. Theoretical Population Biology 17, 380–414 (1980).
 60.
Peña, J., Wu, B. & Traulsen, A. Ordering structured populations in multiplayer cooperation games. Journal of the Royal Society Interface 13, 20150881 (2016).
 61.
Mullon, C. & Lehmann, L. The robustness of the weak selection approximation for the evolution of altruism against strong selection. Journal of Evolutionary Biology 27, 2272–2282 (2014).
 62.
Débarre, F., Hauert, C. & Doebeli, M. Social evolution in structured populations. Nature Communications 5, 4409 (2014).
 63.
Hauert, C. & Doebeli, M. Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428, 643–646 (2004).
 64.
Peña, J., Wu, B., Arranz, J. & Traulsen, A. Evolutionary games of multiplayer cooperation on graphs. PLOS Computational Biology 12, 1–15 (2016).
 65.
Champagnat, N. A microscopic interpretation for adaptive dynamics trait substitution sequence models. Stochastic Processes and their Applications 116, 1127–1160 (2006).
 66.
Metz, J., Nisbet, R. & Geritz, S. How should we define ‘fitness’ for general ecological scenarios? Trends in Ecology & Evolution 7, 198–202 (1992).
 67.
Santos, F. C., Santos, M. D. & Pacheco, J. M. Social diversity promotes the emergence of cooperation in public goods games. Nature 454, 213–216 (2008).
 68.
Peña, J. & Rochat, Y. Bipartite graphs as models of population structures in evolutionary multiplayer games. PLOS ONE 7, 1–13 (2012).
 69.
Szabó, G. & Hauert, C. Phase transitions and volunteering in spatial public goods games. Physical Review Letters 89, 118101 (2002).
 70.
Roze, D. & Rousset, F. Multilocus models in the infinite island model of population structure. Theoretical Population Biology 73, 529–542 (2008).
 71.
Ohtsuki, H. Evolutionary games in Wright’s island model: kin selection meets evolutionary game theory. Evolution 64, 3344–3353 (2010).
 72.
Wu, B., Traulsen, A. & Gokhale, C. S. Dynamic properties of evolutionary multiplayer games in finite populations. Games 4, 182–199 (2013).
 73.
Ohtsuki, H. Evolutionary dynamics of nplayer games played by relatives. Philosophical Transactions of the Royal Society B: Biological Sciences 369 (2014).
 74.
Gardner, A. & West, S. Demography, altruism, and the benefits of budding. Journal of Evolutionary Biology 19, 1707–1716 (2006).
 75.
Christiansen, F. B. On conditions for evolutionary stability for a continuously varying character. The American Naturalist 138, 37–50 (1991).
 76.
Doebeli, M., Hauert, C. & Killingback, T. The evolutionary origin of cooperators and defectors. Science 306, 859–862 (2004).
 77.
Day, T. Population structure inhibits evolutionary diversification under competition for resources. Genetica 112, 71–86 (2001).
 78.
Metz, J. A. J. & Gyllenberg, M. How should we define fitness in structured metapopulation models? including an application to the calculation of evolutionarily stable dispersal strategies. Proceedings of the Royal Society of London B: Biological Sciences 268, 499–508 (2001).
 79.
Ajar, É. Analysis of disruptive selection in subdivided populations. BMC Evolutionary Biology 3, 22 (2003).
 80.
Wakano, J. Y. & Lehmann, L. Evolutionary branching in demestructured populations. Journal of Theoretical Biology 351, 83–95 (2014).
 81.
Dieckmann, U. & Metz, J. A. Surprising evolutionary predictions from enhanced ecological realism. Theoretical Population Biology 69, 263–281 (2006).
Acknowledgements
We thank Pat Barclay, Laurent Lehmann, and Stu West for comments, and the Swiss National Science Foundation for funding (Grant P2LAP3–158669 to M.D.S.).
Author information
Affiliations
Department of Zoology, University of Oxford, Oxford, United Kingdom
 Miguel dos Santos
Department of Evolutionary Theory, Max Planck Institute for Evolutionary Biology, Plön, Germany
 Jorge Peña
GEOMAR Helmholtz Centre for Ocean Research Kiel, Kiel, Germany
 Jorge Peña
Authors
Search for Miguel dos Santos in:
Search for Jorge Peña in:
Contributions
Both authors contributed equally to this work. M.D.S. and J.P. conceived and designed the study, performed the analysis, and wrote the manuscript. Both authors gave final approval for submission.
Competing Interests
The authors declare that they have no competing interests.
Corresponding author
Correspondence to Miguel dos Santos.
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.