Evolutionary dynamics of N-person Hawk-Dove games

In the animal world, the competition between individuals belonging to different species for a resource often requires the cooperation of several individuals in groups. This paper proposes a generalization of the Hawk-Dove Game for an arbitrary number of agents: the N-person Hawk-Dove Game. In this model, doves exemplify the cooperative behavior without intraspecies conflict, while hawks represent the aggressive behavior. In the absence of hawks, doves share the resource equally and avoid conflict, but having hawks around lead to doves escaping without fighting. Conversely, hawks fight for the resource at the cost of getting injured. Nevertheless, if doves are present in sufficient number to expel the hawks, they can aggregate to protect the resource, and thus avoid being plundered by hawks. We derive and numerically solve an exact equation for the evolution of the system in both finite and infinite well-mixed populations, finding the conditions for stable coexistence between both species. Furthermore, by varying the different parameters, we found a scenario of bifurcations that leads the system from dominating hawks and coexistence to bi-stability, multiple interior equilibria and dominating doves.

the situation is described by the standard SG. In this case, both agents are faced with a common task. There are three possible outcomes: i) No one cooperates, and hence the task is not achieved with the consequent damage to each one. ii) Both individuals cooperate, and both benefit, sharing the work associated with the task. iii) If only one individual cooperates, both benefit from the task done despite the fact that the cooperator has paid the entire cost associated to the task 4 . If the benefit is greater than the cost, the ordering of payoffs coincides with that of the HDG. This means that in the case of only two individuals involved, both HDG and SG are equivalent. When more than two agents are involved, the conflict underlying the Snowdrift problem can be addressed through an N-person generalization of the standard SG 23 . The N-person SG considers a sample consisting of N agents and a task to be done. Every agent, regardless of whether he contributes to the task or not, will receive a fixed benefit if the task is performed by one or more agents within the sample. The total cost of performing the task is equally shared among those who perform it (the cooperators), while the defectors benefit without paying any cost [24][25][26][27][28] . On the other hand, the situation in the Hawk-Dove problem is of a very different nature compared to that underlying the Snowdrift. In the Hawk-Dove problem there is not a task to be done, but a resource susceptible to be shared. While in the Snowdrift problem every individual benefits from the task, if accomplished, in the Hawk-Dove problem only a kind of strategists (cooperators or defectors) benefits from the resource, excluding the opponent strategists from the distribution 17,18,29,30 . This different nature materializes when there are more than two individuals involved; in this general case, the mathematical formulation differs drastically between both situations, which call for a new approach to deal with the Hawk-Dove problem for many individuals.
Here, we propose a generalization of HDG for N agents, hereafter N-person HDG, with the aim of analyzing how the dynamics is affected when considering group interactions. In the N-person HDG, two kinds of strategists (hawks and doves) compete for a resource R. While hawks are willing to fight to get the whole resource, doves wish to share the resource equally, refusing the resource rather than fighting for it. Accordingly, when a set of individuals face a resource, the hawks fight among them, paying a cost c H , while doves retreat. Only in the case that there are no hawks competing for the resource, the doves will share it. The fitness of the individuals is determined by their payoff when getting involved in an N-person HDG. We study analytically the evolution of cooperative behavior within the framework of the replicator dynamics 1, 31 , by considering either a very large or an infinite population of hawks and doves. We derive an exact equation for the evolution of the fraction of doves in a well-mixed population, which can be solved numerically as a function of the parameters. We first study the infinite population size limit, and show the existence of a critical value for the hawks' cost c H above which cooperation is sustainable: while for values of c H , hawks and doves coexist in the stationary state, for lower values of c H the dynamics leads to the extinction of the doves, resulting in a population of only hawk strategists.
Furthermore, as a refinement of the model, we consider the situation in which the doves aggregate to defend the resource, not at all cost, but only in the case they are present in sufficient number to expel the hawks, i.e., there is a minimum threshold T that assures them to succeed. In that case, after expelling the hawks, paying a cost c D for it, doves share the resource without fighting among themselves. Otherwise, i.e., if the number of doves is below the threshold, they retreat and the hawks fight among themselves. It is worth noticing that although both hawks and doves can fight, a hawk will fight in any situation (unless all his opponents flee), while doves only fight against opposite strategists, and provided they are allowed to expel them. This refinement of the model, hereafter N-person HDG with threshold (HDG-T), captures the stress between gregarious and asocial behaviors. We show that, depending on the values of the parameters, the dynamics drives the system towards either one of the absorbing mono-strategic states (all hawks or all doves), or towards an interior equilibrium in which both strategies coexist. Subsequently, we study the dynamics in finite populations, finding results compatible with those obtained for the infinite size limit.

Results
Preliminary concepts. As mentioned earlier, the standard HDG is a two-player-two-strategies game that captures a conflict of interests in which the most advantageous strategy is the opposite to that of the opponent. There are two possible strategies, namely, hawkish behavior (H), that represents fighting, and dove behavior (D), that represents sharing. When faced with a common resource R, two doves equally share the resource obtaining each one R/2, while two hawks fight for it, paying a cost c that represents the damage caused by the fight, and therefore obtaining each one (R − c)/2. If a hawk meets a dove, the hawk obtains the whole resource R, while the dove retreats and gets nothing. The game can be expressed by means of its payoff matrix, where rows represent focal player's strategies, columns represent opponent's strategies, and the corresponding matrix element is the payoff received by the focal player: N-person HDG. The model we present here constitutes a generalization of the two-person HDG to an N-person game. Assume a sample of size N ≥ 2 with two kinds of strategists, the aggressive ones -like the hawks, and the cooperators -the doves. Let R be a resource such as food or water. As pointed out before, when the population is made up by all doves, they share the resource equally avoiding conflict. If doves meet hawks, all doves escape without fighting, while the remaining hawks fight for the resource paying a given cost. Their respective payoffs are: Hawks'Payoff: Scientific RepoRts | 7: 4800 | DOI:10.1038/s41598-017-04284-6 where n H and n D represent the numbers of hawks and doves in the sample, respectively, and c H the cost of injury. Thus, (n H − 1)c H represents the total injury suffered by all the hawks. It should be noted that doves only get payoffs when there are no hawks in the sample. For N = 2, the model recovers the standard two-person HDG. In order to study the effect of cost and sample size and composition, R is conventionally assumed to be 1. Let be a population of size Z, from which groups of size N are randomly sampled; let n H and n D denote the numbers of hawks and doves in the sample respectively, N = n H + n D . As a first approach to the problem, consider a very large population Z → ∞, composed of a fraction x of doves (x = n D /N), the rest (1 − x) being hawks. Social interactions take place in sample groups of size N, which are randomly selected from the whole population. The average fitness of hawks f H (x) and doves f D (x) is determined according to a binomial sampling, as shown in the methods section. We study the population dynamics in terms of the replicator Equation 1 , according to which the time evolution of x is given by: where  x represents the gradient of selection and 〈f(x)〉 stands for the average fitness of the whole population. Figure 1(a) shows the gradient of selection  x as a function of the density x of doves, for different values of c H and a simple size of N = 5. As shown, for a small cost (c H = 0.1), the gradient of selection is always negative and doves fail to survive regardless of any initial condition. Nevertheless, as the internal conflict among the hawks grows, and with it its associated cost c H , there appears a critical value, beyond which an internal equilibrium point indicates the coexistence of doves and hawks in the steady state. Furthermore, setting f D (x) = f H (x) and x = 0, the minimum value of c H for stable equilibrium to survive is c cv = R/(N − 1), being c cv the critical value. Figure 1(b) shows the fraction of doves ⁎ x in the equilibrium as a function of the cost c H , for different sample sizes N. As shown, the equilibrium fraction of doves increases with increasing cost of hawks and increasing sample size. The latter follows from the fact that the greater the sample size, the greater the number of hawks involved in the fight. In general, a higher competition of hawks (either due to an increase in sample size or to an increase in hawks' cost) entails a decrease in their average fitness, and thus the equilibrium frequency of doves increases. As c H > c cv implies > ⁎ x 0, it follows that c cv determines the value for the transition from a mono-species state to coexistence. At this point, let us assume a finite population of size Z, composed by k doves and (Z − k) hawks. As in the case of infinite populations, interactions take place in sample groups of size N, but in this case the mean fitness of hawks and doves is determined in accordance with a multivariate hypergeometric sampling, see the section Methods. Regarding the dynamics, we assume a stochastic birth-death process driven by a pairwise Fermi-like rule 32 . According to this rule, at each time step, two individuals (u, v) are randomly selected. The probability that a descendant of v replaces u is given by an increasing function of the payoff difference: u v where the inverse temperature w controls the noise. Low (high) w represents high (low) noise and, correspondingly, weak (strong) selection pressure. w → 0 leads to a probabilistic choice while w → ∞ and f u > f v lead to the deterministic imitation dynamics. Figure 2(a) shows the gradient of selection G(k) describing the evolutionary dynamics of N-person HDG in finite populations as a function of the fraction of doves k/Z, for different population sizes Z and costs c H and a sample size of N = 5. It should be noted that, for finite populations, the fraction of doves k/Z is not a continuous variable, but a discrete one. It is shown that, as in the previously discussed case of infinite populations, for a small cost c H the gradient of selection is always negative G(k) < 0, and therefore doves become extinct, regardless of their initial fraction in the population. For higher cost values c H , there exists a unique internal root k* making G(k) = 0. Therefore, a stable equilibrium arises with increasing c H and doves are promoted. In addition, both the gradient of selection G(k) and the internal stable equilibrium k* increase slightly as the group size Z increases, which implies that large populations facilitate the advantage of doves. Figure 2(b) depicts the equilibrium fractions of doves as a function of the cost c H , for different sample sizes N and a fixed population size Z = 100. Note that these results are compatible with those obtained for finite populations. Thus, when increasing c H , the regime in which the hawks dominate changes to a regime in which both species coexist. Moreover, increasing the sample size enhances the fitness of doves and, consequently, less hawks' cost c H is required for the existence of doves.
N-person HDG-T. In the above described N-person HDG, doves are assumed to be non-aggressive. As a refinement of the model, now we assume that doves can aggregate to protect the resource and thus avoid being plundered by hawks. This situation is very common in nature, where animals such as buffaloes aggregate to fight against predators. In this revised model, N-person HDG-T, we assume a threshold T. If the fraction of doves exceeds or equals the threshold (n D /N ≥ T), the doves expel the hawks, paying a cost c D for it, and then share the resource equally without fighting among themselves. Otherwise, if the number of doves is below the threshold (N D /N < T), doves retreat and hawks fight among themselves for the resource. The respective payoffs of doves and hawks in the N-person HDG-T are: Hawks'payoff: where n H (resp., n D ) stands for the number of hawks (resp., doves) in the sample of size N, c D for the cost of doves to protect the resource from being plundering by hawks, and c H for the cost of injury within hawks. Notice, once again, the different nature of both costs: while c H refers to intra-specific competition, c D refers to an interspecific (intergroup) conflict cost. For T = 1 the previous N-person HDG model is recovered.
As in the previous model, for the case of a very large population (Z → ∞) the population dynamics is studied according to the replicator equation. Figure 3 represents the frequency of doves in the equilibrium x* for different thresholds and costs in the limit of an infinite population. Upper panels show how the intra-specific conflict cost of hawks c H impacts the evolution of doves for different thresholds T = 0.2, 0.4, 0.6, 0.8 and for different fixed values of doves' cost c D . As shown in panel (a), when the threshold is small (T = 0.2), indicating that doves are more resourceful in intergroup competition, a slight cost of doves (c D = 0.2) drives the population to evolve into a full-dove state and no interior equilibrium exists. When c D is increased, an unstable equilibrium emerges and divides the system into two basins of attraction regardless of c H . Both full-dove state and full-hawk state can be reached from different initial conditions: for an initial frequency of doves x 0 > x*, the hawks vanish (f D > f H ), but for x 0 < x*, the doves are the ones that vanish (f D < f H ), indicating that both absorbing states are stable equilibria and the proposed game is transformed into a coordination game. Panels (b-d) show the behavior for greater thresholds T > 0.2. In these cases, for a small c H , an interior unstable equilibrium appears, whose value decreases for larger c H , indicating that increasing c H enlarges the basin of attraction of the full-dove steady state. With further increasing c H , multiple interior equilibria can occur and a new stable equilibrium emerges. Finally, for even higher values of c H , both unstable and stable equilibria converge to a saddle-node bifurcation beyond which the dynamics always leads to a full-dove state.
Bottom panels of Figure 3 show the effects of the intergroup conflict cost of doves c D . When the threshold is small (T = 0.2), as shown in panel (e), x* = 1 is the only steady state for low values of c D , which means that the dynamics always leads to a full-dove state regardless of the initial condition. With the increase of c D , an unstable equilibrium appears and the dynamics changes from dominating doves to bi-stability. However, for c H = 0.8, a saddle-node bifurcation emerges as c D is increased, but the stable equilibrium only exists for a very small range of values of c D , resulting in a transcritical bifurcation that leads to bi-stability. Panels (f-h) show the behavior for greater thresholds T > 0.2. In these cases, a small c H leads to an interior unstable equilibrium, while a large c H results in the domination of doves. Yet for a moderate c H , multiple interior equilibria including an unstable equilibrium and a stable equilibrium drive the system to either the domination of doves or the coexistence of both species under different initial conditions. Figure 4 shows a natural classification of the dynamics of N-person HDG-T that depicts three basic phase regimes. The condition that favors hawks is met when , indicating that a single hawk in a group of N − 1 doves is better off than a dove in a full-dove group. For T = 1 this condition becomes , which is always met, and hawks do not vanish (shown in Figure 1(b)). Besides, the condition for a successful invasion of doves is obtained when a single dove in a group of N − 1 hawks has a higher income than a full hawkish group, namely,  Within the bi-stability regime both full-dove state and full-hawk state can be reached from different initial conditions, while the bi-stability + stability regime corresponds to coexistence state together with full-dove state. In the doves regime the dynamics always leads to a full-dove state. The resource is taken R = 1.
Furthermore, T also has critical effects on the evolutionary dynamics. For instance, by increasing it, one can find regions where dominating doves give rise to bi-stability, and other changes such as from multiple interior equilibria to dominating doves, or from bi-stability to multiple interior equilibria and even to dominating doves. Additionally, increasing T suppresses doves even for a small c H , while there is an optimal T that facilitates doves.
H , increasing T not only strengthens the intergroup competitiveness of doves -there are less hawks to fight with, thus, the cost is smaller -when ≥ n N T / D , but also weakens the in-group competitiveness of hawks, since the decrease in the maximum number of hawks to win over doves results in less in-group injury when > n N T / H . As a consequence, an optimal T is found to promote doves. Particularly, for ≤ − c R N /( 1) H , the payoff of a single dove in a group of N − 1 hawks is lower than the payoff of a hawk in a full hawkish group , and therefore doves are suppressed. On the other hand, a small c H enhances the competitiveness of hawks in intergroup conflict. As a consequence, increasing T reduces the in-group conflict of hawks and finally favors hawks. Regarding the effects of sample size, regardless of costs, increasing N suppresses doves for T = 1/N, since the basin of attraction of the hawkish state increases, whereas doves are favored for T = 1 since stable equilibrium increases. This behavior is mainly attributed to the impact of sample size on in-group competition and resource allocation, benefiting the incomes of victorious agents from the reduced reduction.
As for finite populations, Figure 5 shows the equilibrium fraction of doves k*/Z as a function of the costs c H and c D , for different sample sizes N and a fixed population size Z = 100. The interior roots of G(k), varying with different costs and thresholds, are presented when the roots exist. These results are very similar to those corresponding to infinite populations shown in Figure 3. Particularly, since successful invasions can be directly obtained from P D (k) and P H (k), which are independent of binomial or hypergeometric samplings, the phase diagrams are extremely similar to those corresponding to infinite populations shown in Figure 4. We have also explored the effects of threshold and sample size, which in finite populations are not qualitatively different from those reported before for infinite populations. Moreover, by exploring the stationary distribution of doves, it is also found that both increasing c H and decreasing c D promote doves. Finally, although the equilibrium points show a small dependence on the population size Z, it vanishes as Z grows, finding that for sufficiently high values of Z the dynamics is independent of the population size, which means that for large enough populations ( ≥ Z N / 100) the infinite population limit is a fair approximation that captures the dynamics of the system.

Discussion
In summary, this work proposes a general framework of multi-player Hawk-Dove game that generalizes the traditional symmetric two-person Hawk-Dove game. Two different kinds of species, denoted as hawks and doves respectively, represent the competition to survive of hawkish and cooperative behaviors through group interactions. The hawkish behavior, a more competitive trait, wins intergroup conflict, but also bears the cost associated with intra-group conflict, whereas the cooperative behavior, overcoming intra-species conflict, may lose in interspecies conflict unless the fraction of doves exceeds a threshold T. By implementing the proposed model in well-mixed infinite and finite populations, we found that increasing the intra-group competition cost of hawks, as well as decreasing the intergroup competition cost of doves, promotes the evolution of doves. Furthermore, the threshold plays a key role in the model. As T decreases from T = 1, the phase regimes changes from dominating hawks and stability to bi-stability, multiple interior equilibria and dominating doves. Specially, if the intra-group competition cost of hawks is greater than R/(N − 1), an optimal threshold is found to optimize doves, otherwise increasing threshold suppresses them. Although two-person Hawk-Dove Games can mathematically be regarded as a Snowdrift Game, this equivalence breaks down when generalizing it to N persons. An N-person Snowdrift Game characterizes real-world situations in which a task needs to be done by cooperating, with the consequent benefit for the group, whereas the proposed model describes the competition for a resource between two species. The N-person Hawk-Dove here presented thus conceptualizes the dilemma that being an aggressive type can reward from interspecies competition but also incurs a high cost in terms of intra-species conflict, while cooperators aggregate and share the resource. We also hypothesize that, if the present model were to be applied to a real context in nature, the threshold needed for successful preservation of the doves type would be very high. Indeed, a high threshold is more realistic than the assumptions on which either doves obtain nothing unless no hawks are in the group or a low threshold that implies a fiercer intra-species conflict. Furthermore, group interactions can not be regarded as a set of independent pairwise encounters. Finally, we mention that in previous studies of a two-person Hawk-Dove Game, both an heterogeneous topology and different update rules have been proven to have an impact on cooperation 29,30 . It would thus be of further interest to explore how such factors change our findings for N-person hawk-dove games, and therefore gain more insights into our current understanding of cooperative behavior in social dilemmas.

Evolutionary dynamics in infinite populations. Consider a very large well-mixed population Z → ∞,
composed of a fraction x of doves and a fraction (1 − x) of hawks. Sample groups of size N are randomly selected from the population. Following a binomial sampling 33 , the mean fitness of hawks in the population is given by: ⁎ ⁎ f x f x ( ) ( ) 0 (10)

D H
As an illustrative example, let us consider the case T = N, corresponding to the N-person HDG. According to Equations (1 and 7), the mean fitness of hawks is given by: Hence, we get: Evolutionary dynamics in finite populations. In this subsection, we address the evolutionary dynamics of N-person HDG in finite populations of size Z. Unlike the case of infinite populations, where sampling followed a binomial distribution, the groups in finite populations are given by a multivariate hypergeometric sampling 34,35 . Let k be the number of doves in the total population. Accordingly, the number of hawks will be Z − k. Now, the fraction of doves (k/Z) is not a continuous variable, but a discrete one. The average fitness of hawks is thus given by: Assuming the Fermi-like rule 32 , at each elementary time step two individuals are chosen at random from the population. If they are of different species, then the probabilities that a dove replaces a hawk, and the opposite scenario, are given, respectively, as