Abstract
Collective cooperation is essential for many social and biological systems, yet understanding how it evolves remains a challenge. Previous investigations report that the ubiquitous heterogeneous individual connections hinder cooperation by assuming individuals update strategies at identical rates. Here we develop a general framework by allowing individuals to update strategies at personalised rates, and provide the precise mathematical condition under which universal cooperation is favoured. Combining analytical and numerical calculations on synthetic and empirical networks, we find that when individuals’ update rates vary inversely with their number of connections, heterogeneous connections actually outperform homogeneous ones in promoting cooperation. This surprising property undercuts the conventional wisdom that heterogeneous structure is generally antagonistic to cooperation and, further helps develop an efficient algorithm OptUpRat to optimise collective cooperation by designing individuals’ update rates in any population structure. Our findings provide a unifying framework to understand the interplay between structural heterogeneity, behavioural rhythms, and cooperation.
Similar content being viewed by others
Introduction
Cooperative behaviour—in which individuals pay a cost to confer a benefit to others—is widely and deeply embedded in human and animal societies alike, and has attracted great research interests in studying the underlying mechanisms of favouring the emergence of cooperation1,2,3,4,5,6,7,8,9,10,11,12,13,14,15. Under the prominent metaphor of the prisoner’s dilemma16, without additional mechanisms including direct17,18 or indirect reciprocity19,20,21,22, and punishment23,24, unstructured populations—wherein everyone interacts with everyone else—are known to leave no opportunity for the survival of cooperators25,26. Thus in recent decades, researchers have been exploring evolutionary game dynamics in structured populations, where who interacts with whom is determined by a network or population structure, with links representing interactions between different individuals (nodes)4,5,6,27,28,29,30. The central question is: which population structures promote cooperation, and which hinder it?
In homogeneous networks—where all individuals basically have similar numbers of neighbours—a well-known finding is that cooperation is favoured if the ratio between the benefit (b) provided by a cooperator and the associated cost paid (c) exceeds the average number of neighbours 〈k〉, namely the simple rule4b/c > 〈k〉. Similar results can be found in the more general case: Allen et al. analytically calculated the critical benefit-to-cost ratio C*, above which cooperation is promoted for an arbitrary network topology5. Apart from confirming C* = 〈k〉 for homogeneous structures, this result informs a higher value of C* for heterogeneous structures31, wherein different individuals may have wildly different numbers of neighbours. Accordingly, although heterogeneous structures like scale-free networks32 are ubiquitous in real systems, they appear to hinder the emergence of cooperation compared to homogeneous structures31.
Despite remarkable advances in our understanding of the emergence of cooperation, many studies have confined that individuals update their strategy synchronously6,33,34,35—that all individuals update at exactly the same time. However, perfect synchronism is absent from the real world, and it has been shown that asynchronous updating—individuals are allowed to update at different time—can significantly alter the evolution of cooperation36,37,38,39,40. A typical asynchronous update rule is the death-birth update, where only a single individual is selected uniformly at random to die and their neighbours spread their strategies by competing for the vacant position at each time step4,5. Alternatively, individuals may change their strategies by mimicking that of their neighbours (imitation4, pairwise comparison12,41). All these important canonical updating rules have been based on a key assumption: that all individuals update their strategies at the same rate.
In reality, humans behave in more sophisticated ways in decision-making than simple identical updating. An empirical study on evolutionary games uncovered that individuals are observed to have many different possibilities for strategy updating in human behavioural experiments42. Indeed, both cognitive processing speed and personality traits can have an impact on the time of individual decision-making. Previous empirical studies have found that individuals vary significantly in cognitive processing speed43,44,45. For example, individuals with greater cognitive abilities have high information processing speed and display a short reaction time. On the other hand, many personality traits are also evidenced to correlate with the decision-making speed46. Taken together, the previous assumption of identical update rates for all individuals is too ideal to portray the update event and heterogeneous individual interaction rhythms in realistic scenarios47,48. This prompts us to ask how this dynamical heterogeneity might interact with structural heterogeneity to alter the evolution of cooperation.
Here we investigate evolutionary game dynamics under non-identical rates of strategy updating. Specifically, we consider the scenario where individuals are allowed to update their strategies at diverse individual rates. We find that non-uniform rates of strategy updating can have profound effects on the emergence of cooperation, especially on heterogeneous structures, and reveal a significant decrease in C* necessary to promote cooperation. Moreover, we develop an efficient algorithm OptUpRat to minimise the threshold for the emergence of cooperation by tuning the update rate of each individual on any network.
Results
We consider evolutionary game dynamics on a structured population of N players, whose interactions are represented by an undirected, unweighted network. At any given time, the state of each node (player) is characterised by a strategy of either cooperation (C) or defection (D) (Fig. 1a). In each round of the game, every node i plays the game pairwise with its immediate ki neighbours. Specifically, cooperators pay a cost c to provide a benefit b to each of their neighbours, while defectors pay nothing, and thus provide no benefit. In this way, each node i gains an average payoff fi, corresponding to the average benefits received (from neighbouring cooperators) minus its cost.
Traditionally, individuals are assumed to update their strategies following independent Poisson processes with identical rates. Here we depart from this practice: allowing each individual i to update its strategy with personalised rate λi (Fig. 1b). When an individual is chosen for an update, it does so by copying the strategy of one of its neighbours j, with probability proportional to the fitness of j, generally defined as Fj = 1 + δfj, where δ > 0 captures the intensity of selection4,5 (see Methods). For strong selection intensity, cooperation is disfavoured since the initial cooperator will not be able to survive or spread its strategy. Thus, to systematically uncover the effects of heterogeneous update rates on the fate of cooperators compared to existing findings, here we focus on the canonical case of weak selection.
To quantify the ability of cooperation to proliferate, we initialise our simulations with a single cooperator placed uniformly at random in a population among N − 1 defectors. The evolutionary game ends when a state with either all cooperators or all defectors is reached (Fig. 1c). We define the fixation probability of cooperation (ρC) as the probability of reaching the state of full cooperation over many realisations of this process. We can analogously define a probability ρD of reaching a full-defection state starting from a single defector planted of N − 1 cooperators. Our interest in this study is the condition under which cooperation is favoured to replace defection than vice versa4,5,26, namely ρC > ρD. This condition is equivalent to ρC > 1/N (Supplementary Note 1), namely that selection favours the emergence of cooperation relative to the neutral drift (δ = 0), in which neither cooperation nor defection is favoured (ρC = ρD = 1/N).
Evolutionary game dynamics on complex networks
First, we explore how the heterogeneous strategy updating affects the fate of cooperators on four commonly-studied population structures: lattice, small-world, Erdös-Rényi, and scale-free networks (Fig. 2). Under the traditional scenario of identical update rates (λi = 1 for all i), scale-free networks demand the largest critical benefit-to-cost ratio C*, above which cooperation is favoured among all the four structures, and the lattice structure the smallest (Fig. 2a), consistent with previous findings4,5. But surprisingly, when a node’s update rate varies inversely with its number of neighbours (λi = 1/ki), we find that this trend is reversed (Fig. 2b). Here, the scale-free network becomes the most amenable to cooperation, and lattice the least. Interestingly, we find that heterogeneous update rates can even improve upon the canonical threshold b/c > 〈k〉 (namely, C* = 〈k〉) for homogeneous populations4, allowing cooperation to emerge even when b/c < 〈k〉 (namely, C* < 〈k〉). Furthermore, we find that this pattern is strengthened when the update rate is inversely proportional to higher powers of ki (Fig. 2d). In contrast, when λi is positively related to ki, the ordering of C* over different structures matches the identical-rate case, but with the inhibition of cooperation fixation by heterogeneous networks amplified (Fig. 2c). We show the robustness of our results over different population sizes, average degrees and selection intensities in Supplementary Figs. 1–3.
We further shed light on our numerical findings by deriving a closed-form expression for the critical benefit-to-cost ratio C* as a function of the network structure (see Methods)
Here, ki = ∑jeij defines the number of neighbours (degree) of individual i, and eij = eji = 1 indicates that there is an edge between nodes i and j (eij = eji = 0 otherwise). The probability of a 1-step (n-step) random walk from i to j is denoted by pij (\({p}_{ij}^{(n)}\)), and ηij is the coalescence time49—the expected time for two random walks starting from nodes i and j to meet at a common node. As shown in Fig. 2e, all numerical results in Fig. 2a-d are in good agreement with the theoretical prediction of equation (1).
Role of network hubs
To intuitively understand why heterogeneous update rates can improve the fixation of cooperation in heterogeneous networks, we first consider how the evolutionary dynamics play out on a simple double-star structure (Fig. 3). When the fixation of cooperation occurs in this highly heterogeneous structure, it usually does so in four stages: (I) occupation of one of the hubs; (II) formation of a stable cluster of cooperators among that hub and its neighbours; (III) occupation of the other hub; and finally (IV) spread to the remaining nodes. As such, the ultimate triumph of cooperators can be thwarted if a hub imitates defection from even one of its (many) neighbours before stages (II) and (IV) are complete (Fig. 3c). There are ample opportunities for this to occur under the traditional setting of identical update rates (λi = 1), as illustrated in Fig. 3b. When λi = 1/ki however (Fig. 3a), hubs update relatively infrequently. As such, once a hub becomes a cooperator, it is effectively locked-in, giving time for its strategy to spread to the hub’s neighbours. Note that this lock-in effect can facilitate the formation of cooperative clusters to have higher payoffs to resist the invasion of defectors, yet defectors receive a lower payoff after driving their neighbours to defectors and further reduce their survival chances. By the same logic, the preferential updating of hubs (λi = ki) usually leads to the extinction of cooperation, as the formation of stable clusters of cooperators and the spread of cooperation is even harder than the traditional scenario of identical updating (Fig. 3c).
In Fig. 4, we illustrate the fundamental mechanism explaining why infrequent updates of hubs can facilitate cooperation. If an individual (grey node in Fig. 4a) decides to update its strategy, it will imitate the strategy of its neighbours according to their payoffs. The neighbouring cooperator obtains an average payoff PC = bqC∣C(〈k〉 − 1)/〈k〉 − c and the neighbouring defector obtains PD = bqC∣D(〈k〉 − 1)/〈k〉, where qC∣C (qC∣D) represents the conditional probability to find a cooperative neighbour for a given cooperator (defector). The contribution to the neighbouring cooperator and defector from the updating individual is excluded since they are equal. Thus the cooperator is favoured compared to the defector to disperse its strategy if PC > PD, namely
with Q = (qC∣C − qC∣D)(〈k〉 − 1) capturing the average number of cooperative neighbours that a cooperator has more than a defector. For the canonical setting with identical update rates (λi = 1), we know Q = 1 according to pair approximation (Supplementary Note 2), namely a cooperator has on average one more cooperative neighbour than a defector (Fig. 4a). This leads to the conclusion that cooperation is favoured when b/c > 〈k〉 (namely, C* = 〈k〉), which also degenerates to the simple rule4 for homogeneous networks where ki = 〈k〉.
Next we show how heterogeneous update rate alters the local dispersal of cooperation. When λi = 1/ki, we find that Q > 1 (Supplementary Note 2), indicating that the number of cooperative neighbours of a cooperator exceeds that of a defector by more than one (Fig. 4b). This implies that the net payoff of cooperators relative to defectors is further increased, giving cooperators more advantage in competition and dispersal. Therefore, the critical ratio for λi = 1/ki is smaller than the average degree 〈k〉 for a wide range of heterogeneous networks (C* < 〈k〉). In contrast, when λi = ki, the hubs update frequently and Q < 1 (Supplementary Note 2), indicating that on average, the number of cooperative neighbours of a cooperator exceeds that of a defector by less than one (Fig. 4c). This leads to a larger critical ratio (C* > 〈k〉) for promoting cooperation compared to the scenario with identical update rates shown in Fig. 4a.
We have numerically confirmed the above mechanism on larger scale-free networks. Figure 4d–f show the state of the hub, and the fraction of cooperators among the hub’s neighbours over the course of the game dynamics. For λi = 1/ki, we observe long-lasting periods of cooperation on the hub (Fig. 4e), with infrequent strategy switches from cooperation to defection, which results in the highest qC∣C for the hub (Fig. 4g) and in turn the highest qC∣C − qC∣D over all nodes with different degrees compared to other settings (Fig. 4h). In contrast, fast-updating hubs (λi = ki) have the lowest average fraction of cooperators among their neighbours (Fig. 4g), leading to a low fraction of cooperative neighbours for the cooperators relative to defectors over the whole network (Fig. 4h). This confirms that degree-inverse update rates promote cooperation on heterogeneous networks because a hub with a low update rate is more conducive to driving its neighbours to cooperation, which further enhances the local dispersal of cooperation among nodes with different degrees.
Furthermore, we find that infrequent updates of hubs can also bring long-term advantages to individuals. We could even consider the general evolutionary process with mutation, where a mutant appears with probability u when the population reaches full cooperation or full defection. Each individual accumulates long-term payoffs during a long period of time. Even with a high mutation rate (u = 1), we show that the inverse relationship between update rates and nodes’ degrees results in a higher long-term payoff for individuals than identical rates (λi = 1) (Fig. 5a). In contrast, frequent updates of hubs (λi = ki) lead to a lower payoff than the identical settings (Fig. 5a). This result is also robust over different mutation rates and selection intensities (Fig. 5b, c). Moreover, when the mutation is rare, the population is almost always in full cooperation or full defection, and the time spent in full cooperation (defection) is proportional to ρC (ρD)50. Therefore, the settings of update rates which promote cooperation further lead to a higher long-term average payoff, since individuals get b − c in full cooperation but 0 in full defection.
Theoretical analyses
In addition to the role of hubs that we uncover for the three specific update rate settings (λi = 1/ki, λi = 1, λi = ki), can we derive the general rule for promoting cooperation that also applies to other distributions of update rates? We next explore how different distributions of λi affect C* over five different synthetic networks: lattice, random regular, Erdös-Rényi, small-world, and scale-free. For a given network structure, we theoretically predict C* via equation (1) for uniform, normal, exponential and power-law distributions of the update rate. We find that the critical threshold of a typical homogeneous network—such as a lattice or random regular network—is almost unaffected by the choice of update rate distribution (Fig. 6a, Supplementary Fig. 4). In contrast, heterogeneous structures are quite sensitive, with scale-free networks presenting the most drastic variations in C* among the different update-rate distributions we consider. This malleability of C* in heterogeneous networks suggests the possibility of deliberately tuning the update rates to lower the barrier for the emergence of cooperation in a particular network. But to put this into practice, we must first overcome a computational hurdle.
In order to calculate C* using equation (1), one needs to solve a system of N(N − 1)/2 linear equations for the recurrence relations between the ηij (equation (7) in Methods). Unfortunately, this requires an overall complexity of \({{{{{{{\mathcal{O}}}}}}}}({N}^{6})\), rendering the problem intractable for large networks. To circumvent this, we offer an efficient approximation C* as
This expression obviates the need to solve large systems of linear equations and reduces the computational complexity to \({{{{{{{\mathcal{O}}}}}}}}({N}^{3})\). Here 〈k2〉 is the second moment of the degree distribution. We have \(\zeta={\sum }_{i,j}\frac{{k}_{i}{k}_{j}\Lambda }{N{K}^{2}({\lambda }_{i}+{\lambda }_{j})}\), where Λ = ∑iλi defines the total rate of update events and K = ∑iki is the summation of all nodes’ degrees. Finally, \({\Delta }_{{\lambda }^{(1)}},{\Delta }_{{\lambda }^{(2)}},{\Delta }_{{\widetilde{\eta }}_{n}}\) and \({\Delta }_{{\widetilde{\eta }}_{d}}\) are constants related to the heterogeneity of update rates and coalescence times, the expressions for which are given in Methods. When the update rates are identical, we have \({\Delta }_{{\lambda }^{(1)}}={\Delta }_{{\lambda }^{(2)}}={\Delta }_{{\widetilde{\eta }}_{n}}={\Delta }_{{\widetilde{\eta }}_{d}}=0\), and equation (3) recovers the previous results4,31.
Figure 6b compares the value of C* predicted by the approximation in equation (3) with that of numerical simulation on two empirical social networks51,52. We see that our approximation is remarkably accurate in both networks, regardless of the distribution of the update rates. Moreover, equation (3) offers intuition behind our previous observation that homogeneous structures are robust to different update rates (Fig. 6a). The high symmetry present in these networks means that heterogeneous update rates affect only a limited number of nodes. For such networks, we have \({\Delta }_{{\widetilde{\eta }}_{n}}\approx {\Delta }_{{\widetilde{\eta }}_{d}}\approx 0\), meaning that C* → 〈k〉 in the limit of large N. This coincides with the classical result4 (C* = 〈k〉) regardless of the distribution of update rates.
A simple condition for the emergence of cooperation
Starting from equation (3) (see Methods), we have the critical benefit-to-cost ratio for large heterogeneous networks
where 〈k〉 is the average degree and \({\Delta }_{{\widetilde{\eta }}^{(\infty )}}\approx \frac{\overline{\eta }}{{K}^{2}}{\sum }_{i < j}({k}_{i}-{k}_{j})({\lambda }_{i}-{\lambda }_{j}){e}_{ij}/({\lambda }_{i}+{\lambda }_{j})\). Note that \({\Delta }_{{\widetilde{\eta }}^{(\infty )}} < 0\) when any pair of nodes i and j satisfies the rule (ki − kj)(λi − λj) < 0. When the update rates are identical, we have \({\Delta }_{{\widetilde{\eta }}^{(\infty )}}=0\) and hence C* ≈ 〈k〉 as expected. In contrast, C* is smaller (larger) than 〈k〉 when \({\Delta }_{{\widetilde{\eta }}^{(\infty )}} < 0\) (\({\Delta }_{{\widetilde{\eta }}^{(\infty )}} > 0\)) (Supplementary Note 3.3). Table 1 summarises the values of C* predicted by equation (4) for the combinations of network structure/update-rate settings.
Taken together, we have theoretically motivated an efficient rule of thumb for lowering the threshold for the emergence of cooperation on large heterogeneous structures. Put simply, the order of any pair of nodes’ update rates (for example, λi > λj) should be reversed from the order of the nodes’ degrees (for example, ki < kj). That is, the one with larger degree should have smaller update rates and vice versa, as is demonstrated in Fig. 6c. In other words, the hubs in networks should update infrequently compared to their neighbours with fewer connections to promote the formation of cooperative clusters, which is consistent with the underlying mechanisms shown in Figs. 3 and 4. A simple but general realisation of this rule is \({\lambda }_{i}=1/{k}_{i}^{\gamma }(\gamma \, > \, 0)\) which we study numerically in Fig. 2d for different values of γ. This rule can achieve a lower critical ratio C* than identical update rates (γ = 0) on both synthetic heterogeneous (Fig. 2b) and empirical networks (γ = 1) (Fig. 6b and Supplementary Table 1 and Supplementary Figs. 6 and 7). Meanwhile, the contrary configuration of \({\lambda }_{i}={k}_{i}^{\gamma }\) leads to increases in C* on heterogeneous networks (Figs. 2c and 6b and Supplementary Figs. 8 and 9).
Moreover, we show that our conclusion can also be applied to other social dilemmas (Supplementary Note 4). For the general two-player game, a cooperator receives rewards R from mutual cooperation, while defectors obtain punishment P from mutual defection. A defector attempting to exploit a cooperator obtains T and leaves S to its opponent cooperator. We show that cooperation is favoured over defection when R > P + (T − S)(C* − 1)/(C* + 1), where a lower threshold for R can be achieved with a lower C*. Note that here C* is exactly the critical threshold under the donation game. This indicates our conclusion applies to other social dilemmas, such as the general prisoner’s dilemma (T > R > P > S)16, snowdrift game (T > R > S > P)3 and stag hunt game (R > T ≥ P > S)53.
The optimal update rate on any network
As an engineering application of designing unmanned and autonomous systems, can we adopt the simple heuristic to favour collective cooperation among agents? Specifically, can we find the optimal set of λi for a given networked system? To answer this question, we develop OptUpRat, an optimisation algorithm, to search for a set of λi that minimises C* (See Box 1, Supplementary Note 5 and Supplementary Fig. 10). Our algorithm OptUpRat is based on RMSProp (root mean square propagation), which is an optimisation algorithm designed for training neural networks54. Note that the settings of the learning rate ϵ, decay rate ρ and constant δopt parameters are the same as those in RMSprop—the learning rate ϵ controls the step size of the iteration; ρ controls the decay rate of the moving average; and δopt is a small constant added to the denominator to prevent division by zero (see the values of those parameters in Methods). To transform the constrained optimisation with λi > 0 for each individual i into an unconstrained optimisation problem, we define \({\lambda }_{i}=\exp ({\theta }_{i})\) to establish a function mapping from θi to C*. Then the optimal update rate and the corresponding C* can be obtained via iterative gradient descent, where the gradient is computed by solving a system of N(N − 1)/2 linear equations after taking the derivative with respect to θi on both sides of equation (7) in Methods.
Consistent with our rule, Fig. 7a shows the scale-free network is more flexible and attain a much smaller threshold at its optimal rate than the lattice. Moreover, the update rates of higher-degree nodes tend to decrease during the optimisation process, while those of smaller-degree nodes increase (Fig. 7b and Supplementary Fig. 11). Interestingly, we find that even on homogeneous structures such like lattices, a policy of identical update rates is not the best choice for promoting cooperation. Indeed, the final update rates deviate significantly from the initial conditions (Fig. 7c and Supplementary Fig. 12). Figure 7d shows that the optimal update rates for different network structures are consistent with our rules shown in Fig. 6c—namely that a node i’s update rate λi should vary inversely with its degree ki.
Discussion
Our findings reconcile the past conflicting results on how heterogeneous networks affect the evolution of cooperation. Studies that initialise evolutionary game dynamics with an equal number of cooperators and defectors have found that scale-free networks actually outperform homogeneous networks in promoting the evolution of cooperation, as measured by the average fraction of cooperators6. But from the perspective of fixation probability, heterogeneous structures impose a higher benefit-to-cost threshold for a single cooperator to take over a population of defectors, at least when all update rates are identical4,5,31. This predicts that heterogeneous network structures, despite their ubiquity in physical and social systems, tend to hinder the emergence of collective behaviour. By relaxing this assumption and allowing nodes to update their strategies at non-identical rates, we have shown that scale-free networks can in fact facilitate the fixation of cooperation. As such, degree-heterogeneous networks orchestrated by personalised update rates can be unambiguously conducive to cooperation, provided they are doubly heterogeneous—that is, also heterogeneous in update rate. Taken together, we argue that personalised interaction dynamics and network structure combine to shape the collective dynamics.
From the perspective of microscopic mechanism, we unveil that different update rules render the conflict results. Regarding the frequency of cooperators, previous canonical framework and update rule naturally lead to infrequent strategy switching (Supplementary Note 6)6,33,34. This facilitates the formation of cooperative clusters and leads to a high fraction of cooperators on heterogeneous networks. Previous findings are consistent with the underlying microscopic mechanism in our study, namely infrequent updates of hubs facilitate the emergence of cooperation. Indeed, by applying the canonical death-birth update with identical rates in the framework analysing the frequency of cooperators6,33,34, we find that heterogeneous networks impede the average frequency of cooperators compared to homogeneous scenarios (Supplementary Fig. 13).
Furthermore, we compare our results with experimental studies on cooperation in heterogeneous networks. Consistent with our theoretical findings, there is an insightful experimental study also reporting that heterogeneous networks do not promote cooperation in prisoner’s dilemmas55. In this behavioural experiment, a player’s decisions to cooperate or defect are relevant to the level of cooperation in their neighbourhoods, which renders the network irrelevant. Therefore, the main difference between this experimental finding and our study lies in the update rules. Specifically, players are more likely to imitate the strategy from neighbours with higher payoffs in our theoretical framework. To further uncover the behavioural dynamics from the perspective of fixation probability, a promising future application involves the design of human behavioural experiments starting from a single cooperator and ending with full cooperation or defection. Comparing the individual behavioural mode in experiments from these two perspectives will facilitate the understanding of the emergence of cooperation in realistic scenarios.
A natural extension of our findings is exploring the scenario with multiple strategies56,57,58. In this way, the diverse strategy update rhythms may couple multiple strategies with complex dynamics. In addition, our findings may contribute to the study of network formation, elucidating the factors influencing group formation, such as individuals’ propensity to establish connections with those who share similar rhythms. Specifically, discovering the scenarios wherein individuals with similar update rates are allowed to construct a group may provide valuable information regarding the optimal network configuration in the context of heterogeneity.
One promising direction for future research lies in evolutionary dynamics on temporal networks. Time-varying network structure is a recurring theme in social systems, encoding not only who interacts with whom but with when (and how often) these interactions happen59. It was recently discovered that temporal networks generally enhance the evolution of cooperation relative to comparable static networks12, yet the practical scenarios easily trigger the heterogeneous time rhythm of strategy updating. In real temporal networks, a node’s degree may vary drastically even over short time periods47,48,60. This—in tandem with other temporal effects such as burstiness and multi-frequency interactions47,61—may lead to more exotic evolutionary dynamics. By regarding a temporal network as a sequence of static snapshots, our theory might be adopted to further tailor individuals’ update rates in temporal evolutionary game dynamics.
Methods
Evolutionary process
In each round of the game, individuals interact with their neighbours and accumulate the payoffs accordingly. The payoff matrix of the game is given by
The state of network at any given time can be encoded by a binary vector x ∈ {0, 1}N, where xi = 1 denotes that the player i chooses strategy C, otherwise xi = 0 indicates strategy D. Using this representation of the network state x, i’s average payoff is fi(x) = − cxi + b∑jpijxj, where pij = eij/ki indicates the probability of a single step random walk from i to j on the network. For a node i with update rate λi, the probability to be chosen for a strategy update is λi/Λ, where Λ = ∑iλi defines the total rate of update events. It follows that at the end of each round, the probability for a player j to transmit its strategy to i is \({r}_{ji}({{{{{{{\bf{x}}}}}}}})=\frac{{\lambda }_{i}}{\Lambda }\frac{{e}_{ij}{F}_{j}({{{{{{{\bf{x}}}}}}}})}{{\sum }_{l}{e}_{il}{F}_{l}({{{{{{{\bf{x}}}}}}}})}\), where Fj(x) = 1 + δfj(x) indicates the fitness of individual j. Note that the fixation probability does not change when the rate of strategy updates for each individual is identical since the normalised update rates are the same.
Fixation probability
As shown in the Supplementary Note 1, the fixation probability of cooperation is derived by a first-order expression as the neutral fixation probability (1/N) plus a correction term due to weak selection, namely
where \(\widehat{\Delta }({{{{{{{\bf{x}}}}}}}})\) denotes the reproductive-value-weighted frequency change of cooperation, which is given by
Here \({\left\langle \varphi \right\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\) indicates the summation of the expectation of φ with φ(1)=φ(0)=0 under neutral drift through time step t = 0 to infinity, namely \({\langle \varphi ({{{{{{{\bf{x}}}}}}}})\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }=\mathop{\sum }\nolimits_{t=0}^{\infty }{\sum }_{{{{{{{{\bf{x}}}}}}}}\in {\{0,1\}}^{N}}{{\mathbb{P}}}_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\left[{{{{{{{\bf{X}}}}}}}}(t)={{{{{{{\bf{x}}}}}}}}\right]\varphi ({{{{{{{\bf{x}}}}}}}})\), where \({{\mathbb{P}}}_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\left[{{{{{{{\bf{X}}}}}}}}(t)={{{{{{{\bf{x}}}}}}}}\right]\) indicates the neutral probability of the system reaching state x at time step t starting from the initial state with a single uniformly selected cooperator in population with N − 1 defectors. Combining equations (5) and (6), the fixation probability can be expressed as
where \({\eta }_{ij}={\left\langle \widehat{x}-{x}_{i}{x}_{j}\right\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\), and \(\hat{x}={\sum }_{i}{\pi }_{i}{x}_{i}\) represents the reproductive-value-weighted frequency of cooperators, where πi is the reproductive value62,63,64 uniquely solved by Supplementary equation (2), quantifying the expected contribution of site i to the future gene pool under neutral drift. Here ηij satisfies the recurrence relation of
By letting ρC > 1/N, we obtain C* shown in equation (1).
Calculation of the critical ratio C *
We first define \({\eta }^{(n)}={\sum }_{i,j}{k}_{i}{p}_{ij}^{(n)}{\eta }_{ij}/K\), where K = ∑iki is the summation of all nodes’ degrees, then equation (1) can be rewritten as
From the recurrence relation of ηij in equation (7), we further derive the recurrence relation of η(n) with
where \({\widetilde{\eta }}^{(n+1)}={\sum }_{i,j,l}\frac{{k}_{i}}{K}{p}_{ij}^{(n)}\frac{2{\lambda }_{j}}{{\lambda }_{i}+{\lambda }_{j}}{p}_{jl}{\eta }_{il}\) and \({\eta }_{ii}^{+}=\frac{\Lambda }{2N{\lambda }_{i}}+{\sum }_{l}{p}_{il}{\eta }_{il}\).
By defining the difference \({\Delta }_{{\widetilde{\eta }}^{(n)}}:={\widetilde{\eta }}^{(n)}-{\eta }^{(n)}\) and using the recurrence relation of equation (8), we obtain the calculation of C* shown in equation (3) with mean-field approximation, with \({\Delta }_{{\widetilde{\eta }}_{n}}=-{\Delta }_{{\widetilde{\eta }}^{(2)}}+\frac{{K}^{2}}{{\sum }_{i}{k}_{i}^{2}}{\Delta }_{{\widetilde{\eta }}^{(\infty )}}\) and \({\Delta }_{{\widetilde{\eta }}_{d}}=-{\Delta }_{{\widetilde{\eta }}^{(2)}}-{\Delta }_{{\widetilde{\eta }}^{(3)}}+\frac{KN}{{\sum }_{i}{k}_{i}^{2}}{\Delta }_{{\widetilde{\eta }}^{(\infty )}}\) for simplification, where \({\Delta }_{{\lambda }^{(1)}}={\sum }_{i}\frac{{k}_{i}}{2K}\left(1-\frac{\Lambda }{N{\lambda }_{i}} \right)+{\sum }_{i,j}\frac{{k}_{i}}{2K}{p}_{ij}\left(1-\frac{2\Lambda }{N({\lambda }_{i}+{\lambda }_{j})}\right)\) and \({\Delta }_{{\lambda }^{(2)}}={\sum }_{i,j}\frac{{k}_{i}}{2K}({p}_{ij}+{p}_{ij}^{(2)})(1-\frac{2\Lambda }{N({\lambda }_{i}+{\lambda }_{j})})\). According to Supplementary Note 3, we further have \({\Delta }_{{\widetilde{\eta }}^{(2)}}\approx N{\Delta }_{{\widetilde{\eta }}^{(\infty )}}/\langle k\rangle\) and \({\Delta }_{{\widetilde{\eta }}^{(3)}}\approx N{\Delta }_{{\widetilde{\eta }}^{(\infty )}}/{\langle k\rangle }^{2}\) for large networks, and hence C* shown in equation (4) follows immediately.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
Source data are provided as a Source Data file. Data of empirical networks analysed in Fig. 6b are publicly available and can be found in the corresponding references51,52. Source data are provided with this paper.
Code availability
The codes are written using MathWorks MATLAB R2021a and Python 3.8.5. All source codes related to the work can be found at69 https://github.com/yaomeng1/PersonalizedStrategyUpdates.
References
Lieberman, E., Hauert, C. & Nowak, M. A. Evolutionary dynamics on graphs. Nature 433, 312–316 (2005).
Nowak, M. A. & May, R. M. Evolutionary games and spatial chaos. Nature 359, 826–829 (1992).
Hauert, C. & Doebeli, M. Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428, 643–646 (2004).
Ohtsuki, H., Hauert, C., Lieberman, E. & Nowak, M. A. A simple rule for the evolution of cooperation on graphs and social networks. Nature 441, 502–505 (2006).
Allen, B., Lippner, G., Chen, Y.-T., Fotouhi, B., Momeni, N., Yau, S.-T. & Nowak, M. A. Evolutionary dynamics on any population structure. Nature 544, 227–230 (2017).
Santos, F. C. & Pacheco, J. M. Scale-free networks provide a unifying framework for the emergence of cooperation. Phys. Rev. Lett. 95, 098104 (2005).
Sigmund, K.The Calculus of Selfishness (Princeton Univ. Press, 2010).
Szolnoki, A., Perc, M. & Szabó, G. Defense mechanisms of empathetic players in the spatial ultimatum game. Phys. Rev. Lett. 109, 078701 (2012).
Perc, M., Gómez-Gardenes, J., Szolnoki, A., Floría, L. M. & Moreno, Y. Evolutionary dynamics of group interactions on structured populations: a review. J. R. Soc. Interface 10, 20120997 (2013).
Perc, M. & Szolnoki, A. Coevolutionary games–a mini review. BioSystems 99, 109–125 (2010).
Levin, S. A. Collective cooperation: from ecological communities to global governance and back. In Unsolved Problems in Ecology, 311–317 (Princeton Univ. Press, 2020).
Li, A., Zhou, L., Su, Q., Cornelius, S. P., Liu, Y.-Y., Wang, L. & Levin, S. A. Evolution of cooperation on temporal networks. Nat. Commun. 11, 1–9 (2020).
Szolnoki, A., Perc, M. & Szabó, G. Topology-independent impact of noise on cooperation in spatial public goods games. Phys. Rev. E 80, 056109 (2009).
Amaral, M. A. & Javarone, M. A. Heterogeneity in evolutionary games: an analysis of the risk perception. Proc. R. Soc. A 476, 20200116 (2020).
Johnson, T. & Smirnov, O. Temporal assortment of cooperators in the spatial prisoner’s dilemma. Commun. Biol. 4, 1–10 (2021).
Rapoport, A., Chammah, A. M. & Orwant, C. J.Prisoner’s Dilemma: A study in Conflict and Cooperation (Michigan Univ. Press, 1965).
Nowak, M. A. & Sigmund, K. Tit for tat in heterogeneous populations. Nature 355, 250–253 (1992).
Press, W. H. & Dyson, F. J. Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent. Proc. Natl Acad. Sci. USA 109, 10409–10413 (2012).
Nowak, M. A. & Sigmund, K. Evolution of indirect reciprocity by image scoring. Nature 393, 573–577 (1998).
Wedekind, C. & Milinski, M. Cooperation through image scoring in humans. Science 288, 850–852 (2000).
Nowak, M. A. & Sigmund, K. Evolution of indirect reciprocity. Nature 437, 1291–1298 (2005).
Santos, F. P., Santos, F. C. & Pacheco, J. M. Social norm complexity and past reputations in the evolution of cooperation. Nature 555, 242–245 (2018).
Fehr, E. & Gächter, S. Altruistic punishment in humans. Nature 415, 137–140 (2002).
Hauert, C., Traulsen, A., Brandt, H., Nowak, M. A. & Sigmund, K. Via freedom to coercion: the emergence of costly punishment. Science 316, 1905–1907 (2007).
Hofbauer, J. & Sigmund, K.Evolutionary Games and Population Dynamics (Cambridge Univ. Press, 1998).
Nowak, M. A., Sasaki, A., Taylor, C. & Fudenberg, D. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646–650 (2004).
Maciejewski, W., Fu, F. & Hauert, C. Evolutionary game dynamics in populations with heterogenous structures. PLoS Comput. Biol. 10, e1003567 (2014).
Taylor, P. D., Day, T. & Wild, G. Evolution of cooperation in a finite homogeneous graph. Nature 447, 469–472 (2007).
Meng, Y., Broom, M. & Li, A. Impact of misinformation in the evolution of collective cooperation on networks. J. R. Soc. Interface 20, 20230295 (2023).
Wang, X., Zhou, L., McAvoy, A. & Li, A. Imitation dynamics on networks with incomplete information. Nat. Commun. 14, 7453 (2023).
Fotouhi, B., Momeni, N., Allen, B. & Nowak, M. A. Evolution of cooperation on large networks with community structure. J. R. Soc. Interface 16, 20180677 (2019).
Barabási, A.-L. & Albert, R. Emergence of scaling in random networks. Science 286, 509–512 (1999).
Santos, F. C., Pacheco, J. M. & Lenaerts, T. Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proc. Natl Acad. Sci. USA 103, 3490–3494 (2006).
Santos, F. C., Rodrigues, J. F. & Pacheco, J. M. Graph topology plays a determinant role in the evolution of cooperation. Proc. R. Soc. B 273, 51–55 (2006).
Santos, F. C., Santos, M. D. & Pacheco, J. M. Social diversity promotes the emergence of cooperation in public goods games. Nature 454, 213–216 (2008).
Grilo, C. & Correia, L. The influence of the update dynamics on the evolution of cooperation. Int. J. Comput. Intell. Syst. 2, 104–114 (2009).
Grilo, C. & Correia, L. Effects of asynchronism on evolutionary games. J. Theor. Biol. 269, 109–122 (2011).
Allen, J. M. & Hoyle, R. B. Asynchronous updates can promote the evolution of cooperation on multiplex networks. Physica A 471, 607–619 (2017).
Wang, D., Shuai, X., Pan, Q., Li, J., Lan, X. & He, M. Long deliberation times promote cooperation in the prisoner’s dilemma game. Physica A 537, 122719 (2020).
Zhang, J., Zhang, C., Cao, M. & Weissing, F. J. Crucial role of strategy updating for coexistence of strategies in interaction networks. Phys. Rev. E 91, 1–6 (2015).
Traulsen, A., Claussen, J. C. & Hauert, C. Coevolutionary dynamics: from finite to infinite populations. Phys. Rev. Lett. 95, 1–4 (2005).
Traulsen, A., Semmann, D., Sommerfeld, R. D., Krambeck, H. J. & Milinskib, M. Human strategy updating in evolutionary games. Proc. Natl Acad. Sci. USA 107, 2962–2966 (2010).
Jensen, A. R.Clocking the mind: Mental chronometry and individual differences (Elsevier, 2006).
Sheppard, L. D. & Vernon, P. A. Intelligence and speed of information-processing: A review of 50 years of research. Pers. Individ. Differ. 44, 535–551 (2008).
von Krause, M., Radev, S. T. & Voss, A. Mental speed is high until age 60 as revealed by analysis of over a million participants. Nat. Hum. Behav. 6, 700–708 (2022).
Block, J. & Peterson, P. Some personality correlates of confidence, caution, and speed in a decision situation. J. Abnorm. Psychol. 51, 34 (1955).
Barabási, A.-L. The origin of bursts and heavy tails in human dynamics. Nature 435, 207–211 (2005).
Karsai, M., Jo, H.-H. & Kaski, K.Bursty Human Dynamics (Springer, 2018).
Cox, J. T. Coalescing random walks and voter model consensus times on the torus in Zd. Ann. Probab. 17, 1333–1366 (1989).
McAvoy, A., Allen, B. & Nowak, M. A. Social goods dilemmas in heterogeneous societies. Nat. Hum. Behav. 4, 819–831 (2020).
Génois, M., Vestergaard, C. L., Fournet, J., Panisson, A., Bonmarin, I. & Barrat, A. Data on face-to-face contacts in an office building suggest a low-cost vaccination strategy based on community linkers. Netw. Sci. 3, 326–347 (2015).
Fournet, J. & Barrat, A. Contact patterns among high school students. PLoS One 9, e107878 (2014).
Skyrms, B. The stag hunt and the evolution of social structure (Cambridge Univ. Press, 2004).
Tieleman, T. & Hinton, G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw Mach. Learn. 4, 26–31 (2012).
Gracia-Lázaro, C., Ferrer, A., Ruiz, G., Tarancón, A., Cuesta, J. A., Sánchez, A. & Moreno, Y. Heterogeneous networks do not promote cooperation when humans play a prisoner’s dilemma. Proc. Natl Acad. Sci. USA 109, 12922–12926 (2012).
Gokhale, C. S. & Traulsen, A. Evolutionary games in the multiverse. Proc. Natl Acad. Sci. USA 107, 5500–5504 (2010).
Tarnita, C. E., Wage, N. & Nowak, M. A. Multiple strategies in structured populations. Proc. Natl Acad. Sci. USA 108, 2334–2337 (2011).
Venkateswaran, V. R. & Gokhale, C. S. Evolutionary dynamics of complex multiple games. Proc. R. Soc. B. 286, 20190900 (2019).
Holme, P. & Saramäki, J. Temporal networks. Phys. Rep. 519, 97–125 (2012).
Masuda, N. & Lambiotte, R.A Guide to Temporal Networks (World Scientific, 2016).
Unicomb, S., Iñiguez, G., Gleeson, J. P. & Karsai, M. Dynamics of cascades on burstiness-controlled temporal networks. Nat. Commun. 12, 133 (2021).
Fisher, R. A. The Genetical Theory of Natural Selection (Clarendon Press, 1930).
Taylor, P. D. Allele-frequency change in a class-structured population. Am. Nat. 135, 95–106 (1990).
Maciejewski, W. Reproductive value in graph-structured populations. J. Theor. Biol. 340, 285–293 (2014).
Erdös, P. & Rényi, A. On random graphs I. Publ. Math. (Debrecen) 6, 290–297 (1959).
Watts, D. J. & Strogatz, S. H. Collective dynamics of ‘small-world’networks. Nature 393, 440–442 (1998).
Molloy, M. & Reed, B. A critical point for random graphs with a given degree sequence. Random Struct. Algorithms 6, 161–180 (1995).
Barabási, A.-L., Albert, R. & Jeong, H. Mean-field theory for scale-free random networks. Physica A 272, 173–187 (1999).
Meng, Y., Cornelius, S. P., Liu, Y.-Y. & Li, A. Dynamics of collective cooperation under personalised strategy updates. GitHub https://doi.org/10.5281/zenodo.10828967 (2024).
Acknowledgements
Y.M. and A.L. are supported by National Key Research and Development Program of China (2022YFA1008400), National Natural Science Foundation of China (62173004), Beijing Nova Program (Z211100002121105) and SMP-IDATA Chenxing Youth Fund. Y.-Y.L. is supported by National Institutes of Health (R01AI141529, R01HD093761, RF1AG067744, UH3OD023268, U19AI095219, and U01HL089856).
Author information
Authors and Affiliations
Contributions
Y.M. and A.L. conceived, designed, and performed the research. All authors analysed the results. Y.M. performed mathematical calculations and numerical simulations under the direction of A.L. A.L., S.P.C. and Y.M. wrote the manuscript, and Y.-Y.L. edited the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks Marco Colnaghi and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Source data
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Meng, Y., Cornelius, S.P., Liu, YY. et al. Dynamics of collective cooperation under personalised strategy updates. Nat Commun 15, 3125 (2024). https://doi.org/10.1038/s41467-024-47380-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-024-47380-8
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.