Introduction

Understanding the evolution of cooperation is one of the major challenges in both social and biological sciences1. Cooperators benefit others at the costs of their own, while defectors may enjoy benefits brought by others without paying any cost themselves. It is well understood that, in a well-mixed population, cooperation cannot evolve because defectors are always better off than cooperators, unless special mechanisms are considered2. In contrast, it is also known that spatial structures generally promote the evolution of cooperation because spatial locality helps cooperators interact with other cooperators more often than with defectors3. This is called network (or spatial) reciprocity2. The evolution of cooperation has been considered on various types of networks4,5,6. Furthermore, network reciprocity has been combined with punishment7 or wisdom of group8, which are also known as promoting cooperative behavior. The effect of reciprocity is more apparent when vacant sites are allowed in space9. In those situations, Wang et al.10,11 found that cooperation was most promoted when the population density was close to the percolation threshold of underlying graph.

In a spatially extended setting, migration, a fundamental behavioral trait of animals and humans, can change dynamics of evolutionary games. Migration plays a key role in keeping biodiversity12,13,14 as well as the evolution of cooperation in the real world15. Researchers have thus considered the effect of migration in spatial structures on cooperative behavior. Moreover, it has been suggested that coevolution of strategies and mobilities strongly enhances cooperation. This is the important aspect of “coevolutionary games”16.

Earlier studies on random migration have shown that high migration rates generally prevent the evolution of cooperation since it brings the population closer to a well-mixed state, while cooperation may still be sustained if migration rates are moderate17,18,19,20. However, it has recently been shown that “contingent migration” has a different effect on the evolution of cooperation. In contingent migration models, each individual monitors its current environmental condition within its local neighborhood and migrates to another location if the condition is found to be undesirable. For instance, Jiang et al.21 studied a model of spatial games in which an individual moves to a nearby empty site with probability nD/4, where nD is the number of defectors in four adjacent neighbor sites. In other words, the individuals change their migratory decisions adaptively in response to the level of defectors' presence. They showed that this type of “adaptive migration” promotes cooperation, especially with an intermediate population density. Similarly, Yang et al.22 proposed a model with aspiration-induced migration in which an individual moves to a randomly chosen empty site within its four neighbors if the payoff of the individual falls below its aspiration level. They found that the optimal level of cooperation was achieved when both the aspiration level and the population density were moderate. Another movement from unfavorable locations was implemented by Chen et al.23, where, the lower the contribution from individuals was, the more individuals migrated from each location. Moreover, Helbing et al.24,25,26 showed that cooperation is even more facilitated if individuals can choose a preferred destination to move in the adaptive migration. These earlier studies have collectively illustrated that such adaptive migration strongly promotes the evolution of cooperation, allowing cooperative individuals to avoid exploitation by defectors.

One limitation in the earlier contingent migration studies is that migration was commonly assumed to be limited only to immediate neighbor sites of each individual. In the meantime, it has recently been shown that long-range migration can also promote cooperation if the strength of between-group selection is increased27. To our knowledge of the relevant literature, no existing work addressed the effect of adaptive long-distance migration on the evolution of cooperation, partly because the long-range migration has been generally regarded as harmful for cooperation as it would bring the population to a well-mixed state that favors defection. We believe, however, that adaptive long-range migration may be beneficial for cooperators because they may be able to escape from defector-dominated areas at once and potentially land in a community of other cooperators. This adaptive long-range migration is important because it is naturally observed in both humans and animals.

In this paper, we studied the effects of adaptive long-range migration on the evolution of cooperation. We develop an agent-based model in which individuals play the Prisoner's Dilemma (PD) game with local nearest neighbors on a regular spatial grid and may migrate to another site if the local situation is unfavorable. The key unique contribution of our work is to use an adaptive distance for long-range migration based on the number of defectors in the individual's vicinity, so that the possible distance of the adaptive migration is longer with more defectors in the area nearby.

Results

We conducted simulation experiments using the agent-based model described in the Methods. The parameter setting used for the experiments is as follows unless noted otherwise: L = 50, ρ = 0.3, n = 8 (Moore neighborhood). See the Methods section for the meaning of each parameter. We systematically varied values of α (sensitivity to defectors; note that smaller values of α means high sensitivity, except for α = 0) and d (migration range) to represent the following six different types of migration:

  • α = 0; d = 1: random short-range migration (RSM)

  • α = 0; d = 8 (>1): random long-range migration (RLM)

  • α = 0; d = 24 (~ L/2): random global migration (RGM; an extreme case of RLM)

  • α = 1/3, 1, or 3; d = 1: adaptive short-range migration (ASM)

  • α = 1/3, 1, or 3; d = 8 (>1): adaptive long-range migration (ALM)

  • α = 1/3, 1, or 3; d = 24 (~ L/2): adaptive global migration (AGM; an extreme case of ALM)

Figure 1 shows dmax(nD) in each case. dmax in random migration cases (denoted RXMs hereafter) do not depend on nD, while dmax in adaptive migration cases (denoted AXMs hereafter) monotonically increases along nD. RSM is a form of random migration to the immediate neighbors, which corresponds to typical assumptions adopted in earlier models. ASM corresponds to adaptive migration used in the recent literature. Figure 2 illustrates an example of ALM where the focal individual (located at the center) counts the number of defectors within the neighbors and tries to move to an empty site within a dmax range.

Figure 1
figure 1

Maximum migration distance dmax as a function of the number of defectors in the neighborhood (nD).

The Moore neighborhood is assumed (i.e., n = 8). Left: Short-range migration (d = 1). Center: Long-range migration (d = 8). Right: Global migration (d = 24 ~ L/2). Random migration models do not depend on nD (red lines at the top), while adaptive migration models (all the other curves) depend on nD.

Full size image
Figure 2
figure 2

An illustration of how ALM (Adaptive Long-range Migration) works (α = 1, d = 8).

In this example, there are three defectors within eight neighbors indicated by the red square (i.e., n = 8, nD = 3). The maximum migration distance is calculated as follows: dmax = round ((3/8)1 × 8) = 3. Therefore, the focal cooperator can migrate to another empty site within the 7-by-7 local region indicated in green.

Full size image

Other experimental parameters being varied included T, the temptation to defect and ρ, the population density, among others. The main experimental measurement obtained from each simulation run is the equilibrium fraction of cooperators, , after a certain time period elapses. This is obtained by averaging fc(t), the fraction of cooperators at generation t, for t = 4, 001 ~ 5, 000. The results are also averaged over 20 independent simulation runs for each set of parameters to check the robustness.

Effect of adaptive long-range migration under tempting conditions

First and foremost, we found that AXM strongly promoted cooperation compared to RXM, especially under conditions where the temptation to defect was considerably high. Figure 3 shows as a function of T. For the low values of T, all migration types sustained a high level of cooperation (except for α = 3 that made migration rather infrequent). For T = 1.5, ALM and AGM (α = 1/3) were slightly lower (about 0.92) than RSM (full cooperation; 1.0) because individuals did not move any more after the complete segregation between cooperators and defectors (see Fig. 4). In highly tempting conditions (T > 3), however, cooperators were easily exploited by defectors in RXMs. For T = 3.5, was 0.00 in RSM. In ASM (α = 1/3), cooperators were allowed to move in response to the presence of defectors, but they often remained near the defectors and were exploited by them. Therefore, cooperators were dominant up to T = 3.0, but no longer survived when T > 3.0. In ALM and AGM, in contrast, cooperators were able to secure a certain distance from defectors after the adaptive migration and thus cooperation was promoted even under highly tempting conditions. For instance, for T = 3.5, which was seemed to be extremely favorable for defectors, cooperators were still dominant in the cases of ALM (0.90) and AGM (0.92) for α = 1/3. Even complete segregation between cooperators and defectors was possible in ALM (for α = 1/3) and AGM (for α = 1/3, 1) (Fig. 4) (see also the typical movies of RXMs (Suppl. Video S1, S2 and S3) and AXMs (α = 1/3) (Suppl. Video S4, S5 and S6)). Moreover, under high tempting conditions, α = 1/3 only led to high cooperation in the adaptive long-range migration. This means that sensitivity to defectors is important in such situations (see Suppl. Fig. S1 for the effect of the α-d).

Figure 3
figure 3

Equilibrium fraction of cooperators () as a function of temptation to defect (T).

ρ = 0.3.

Full size image
Figure 4
figure 4

Typical snapshots of simulations after 10,000 generations.

Blue and red dots represent cooperators and defectors, respectively. Cooperators dominated in the population in ALM (Adaptive Long-range Migration; α = 1/3) and AGM (Adaptive Global Migration; α = 1/3, 1). In those cases, cooperators successfully maintained a certain distance with defectors. T = 3.5 and ρ = 0.3.

Full size image

Effect of density

We also investigated the effect of the population density on the evolution of cooperation. Figure 5 summarizes the results of another set of systematic simulations with varying ρ and T. AGM was used as the mode of migration. It can be seen in these plots that there is an optimal population density (e.g., around ρ = 0.3 for α = 1/3) that best promotes cooperation. If the density is extremely low, cooperators cannot form clusters, while if the density is too high, cooperators will likely meet other defectors even after adaptive migration. Our finding, that the moderate densities are suitable for enhancing cooperation, is in agreement with results reported in earlier literature.

Figure 5
figure 5

Equilibrium fraction of cooperators () as functions of population density (ρ) and temptation to defect (T).

AGM (Adaptive Global Migration) was used as the mode of adaptive migration. The results are obtained by averaging fc(t) for t = 4, 001 ~ 5, 000 (averaged over 10 independent simulation runs).

Full size image

Robustness against noise

We also tested the robustness of the results against noise, i.e., stochastic changes of individuals' strategies. This is an important issue because, without such noise, it would be rather trivial that the clusters of cooperators can sustain themselves forever once they are established separately from defectors. Realistically, however, such clusters would be susceptible to defectors emerging from inside, if strategic changes are allowed. To assess the impact of such strategic changes on the evolutionary sustainability of cooperation, we implemented random mutations of individual strategies. More specifically, after an individual imitates the highest strategy from its neighbors, the imitated strategy may flip to the opposite with probability pm ≪ 1.

Supplemental Fig. S2 shows typical evolutionary simulation runs with and without mutation (α = 1/3). Compared to no-mutation cases, conditions with mutation generally resulted in a lower level of cooperators because mutations often destroyed cooperative clusters, causing the continuously fluctuating fc. However, the impact of mutation was much less for long-range migration cases (ALM and AGM) than for ASM. Moreover, the sensitivity analysis over T revealed that ALM and AGM still maintained high levels of cooperation compared to other conditions (Fig. 6). These results confirmed that cooperation is promoted in ALM and AGM even under noisy conditions with strategic mutations.

Figure 6
figure 6

Equilibrium fraction of cooperators () as a function of the temptation to defect (T) with mutation.

pm = 0.01.

Full size image

Emergence of cooperation

Lastly, we considered the hardest situation for cooperation in which the initial condition is made of 100% defectors. In this setting, cooperators can still appear by mutation, but always as a minority surrounded by defectors. Supplemental Fig. S3 shows typical simulation runs with mutation starting from 100% defector initial populations. It is observed that cooperation did emerge by mutation and grew and maintained a certain level of proliferation (similar to those see in Suppl. Fig. S2) throughout the simulation runs.

We then conducted a sensitivity analysis of this result over varying mutation rate pm (Fig. 7). To our surprise, cooperation successfully emerged and dominated the population for extremely low mutation rates (pm < 10−3) especially in ALM and AGM (Fig. 7). We note that there are at least two important factors for understanding why cooperation can emerge and fixate with such low mutation rates. First, in our model setting, migration occurs before game play and strategy updating, so a mutant cooperator that emerged in a defector-dominated population in the previous generation may still be able to escape from the nearby defectors before strategy updating, if adaptive long-range migration is allowed. If the strategy updating were to take place before migration, the new cooperator would quickly become reverted back to a defector that is more successful in terms of payoffs. Second, our model assumed that strategic mutation would occur only immediately after an imitation, so a mutant cooperator can maintain the same cooperative strategy indefinitely as long as there are no neighbors around it. Due to this, mutant cooperators that luckily found an isolated empty space due to long-range migration can survive and once such cooperators come next to each other, cooperative behavior can spread quickly by clustering. Moreover, these clusters are less likely to be destroyed if the mutation rate is low.

Figure 7
figure 7

Robustness of cooperation against the mutation probability (pm).

α = 1/3, T = 3.0 and ρ = 0.3.

Full size image

Discussion

In this paper, we investigated the effect of adaptive long-range migration on the evolution of cooperation in a spatial population model. We found that adaptive long-range migration strongly enhanced the evolution and maintenance of cooperative behavior, especially under high tempting conditions where other forms of migration did not. We also found that the cooperation emerging with the adaptive long-range migration was considerably robust against stochastic mutations of strategies and the initial lack of cooperators in the population.

A possible caveat against this study is the robustness of cooperation may depend on the timing between migration and strategy updating. In other words, whether migration before strategy updating is appropriate or not. Based on the previous study21, we have assumed that migration occurs prior to strategy updating. This point should be discussed more in future works.

In the present study, we have not assumed individuals' global knowledge about the environment, in contrast to the previous studies such as Helbing et al.24,25,26 where individuals can know not only the quality of the present location but also the desire destination within the neighbors to move. For humans and animals, this situation may be hard to occur because new environments for individuals are often unpredictable in the real world. Therefore, the assumption in our model that individuals escape from the bad conditions but they randomly seek new environments is plausible.

Moreover, adaptive long-range migration is a naturally observed in human and animal behaviors. We can see that wild animals show a great migration when they fall into a starvation state. In addition, humans can easily detect unfavorable situations and then sometimes migrate globally by means of transportation. In those senses, adaptive long-range migration (but the destination is random) is appropriate for describing the migration of the species.

We believe that our model and findings provide a solid theoretical advancement for understanding the evolution of cooperation, which has still remained an open question in various scientific fields. Compared to earlier studies that assumed only short-range migration, our work illustrated that cooperative behavior may arise and proliferate more easily if one takes into account organisms' adaptive responses that scale with the strength of external stimuli – a typical behavioral trait commonly seen in animals and humans.

Methods

We develop an agent-based model in which individual agents distribute over a two-dimensional square lattice and play the PD game with their neighbors. The square lattice is composed of L × L sites with periodic boundary conditions. Each site is either empty or occupied by one individual. Empty sites represent spatial regions that individuals can migrate to. Initially, individuals are randomly distributed over the square lattice. One half of them are set to be cooperators while the other defectors. Their population density is given by ρ (i.e., the fraction of empty sites is 1 − ρ). The population density remains constant throughout a simulation run, since individuals will never die or be born.

Individuals are updated asynchronously in a randomized sequential order. The algorithm for updating an individual consists of the following three phases:

  1. 1

    Migration. Extending the adaptive migration model proposed by Jiang et al.21, our model incorporates adaptive long-range migration as follows. First, to decide the site to move into, the individual counts the number of defectors, nD, within its n neighbors. Using nD, the maximum migration distance is calculated as dmax = round((nD/n)αd), where α and d are experimental parameters. We call α sensitivity to defectors and d migration range, respectively. round(…) is used here because the space is discrete (see Fig. 1). Then the individual moves to a randomly selected empty site within a (2dmax + 1)-by-(2dmax + 1) square region around its original location (see the example in Fig. 2). This “adaptive long-range migration” enables individuals to migrate to a more distant site when the number of nearby defectors is larger. If there is no empty site within the region, the individual remains in the original location.

  2. 2

    Game play. After the migration, the individual plays the PD game with other individuals within its n nearest neighbor sites and accumulates the payoffs resulting from the games. If there are no other individuals within the neighborhood, no game is played. In each game, two individuals decide whether to cooperate or defect simultaneously based on their current strategies. They both obtain payoff R for mutual cooperation while P for mutual defection. If one selects cooperation while the other does defection, the former gets the sucker's payoff S while the latter gets the highest payoff T, the temptation to defect. The relationship of the four payoffs is usually T > R > P > S in PD games. Following the parameter settings used in the model by Nowak and May3, we used P = 0, R = 1 and S = 0, while T > 1, the temptation to defect, was varied as an experimental parameter.

  3. 3

    Strategy updating. Once all the games are over, the individual imitates the strategy of the other individual that achieved the highest total payoff among its neighbors (including itself; if there is a tie one individual is randomly selected). If there are no other individuals within the neighborhood, the individual maintains the same strategy.

When all individuals finish conducting the three phases above, it constitutes one time step of the simulation, which we refer to as a “generation” hereafter.