Introduction

Cooperative behaviour—in which individuals pay a cost to confer a benefit to others—is widely and deeply embedded in human and animal societies alike, and has attracted great research interests in studying the underlying mechanisms of favouring the emergence of cooperation1,2,3,4,5,6,7,8,9,10,11,12,13,14,15. Under the prominent metaphor of the prisoner’s dilemma16, without additional mechanisms including direct17,18 or indirect reciprocity19,20,21,22, and punishment23,24, unstructured populations—wherein everyone interacts with everyone else—are known to leave no opportunity for the survival of cooperators25,26. Thus in recent decades, researchers have been exploring evolutionary game dynamics in structured populations, where who interacts with whom is determined by a network or population structure, with links representing interactions between different individuals (nodes)4,5,6,27,28,29,30. The central question is: which population structures promote cooperation, and which hinder it?

In homogeneous networks—where all individuals basically have similar numbers of neighbours—a well-known finding is that cooperation is favoured if the ratio between the benefit (b) provided by a cooperator and the associated cost paid (c) exceeds the average number of neighbours 〈k〉, namely the simple rule4b/c > 〈k〉. Similar results can be found in the more general case: Allen et al. analytically calculated the critical benefit-to-cost ratio C*, above which cooperation is promoted for an arbitrary network topology5. Apart from confirming C* = 〈k〉 for homogeneous structures, this result informs a higher value of C* for heterogeneous structures31, wherein different individuals may have wildly different numbers of neighbours. Accordingly, although heterogeneous structures like scale-free networks32 are ubiquitous in real systems, they appear to hinder the emergence of cooperation compared to homogeneous structures31.

Despite remarkable advances in our understanding of the emergence of cooperation, many studies have confined that individuals update their strategy synchronously6,33,34,35—that all individuals update at exactly the same time. However, perfect synchronism is absent from the real world, and it has been shown that asynchronous updating—individuals are allowed to update at different time—can significantly alter the evolution of cooperation36,37,38,39,40. A typical asynchronous update rule is the death-birth update, where only a single individual is selected uniformly at random to die and their neighbours spread their strategies by competing for the vacant position at each time step4,5. Alternatively, individuals may change their strategies by mimicking that of their neighbours (imitation4, pairwise comparison12,41). All these important canonical updating rules have been based on a key assumption: that all individuals update their strategies at the same rate.

In reality, humans behave in more sophisticated ways in decision-making than simple identical updating. An empirical study on evolutionary games uncovered that individuals are observed to have many different possibilities for strategy updating in human behavioural experiments42. Indeed, both cognitive processing speed and personality traits can have an impact on the time of individual decision-making. Previous empirical studies have found that individuals vary significantly in cognitive processing speed43,44,45. For example, individuals with greater cognitive abilities have high information processing speed and display a short reaction time. On the other hand, many personality traits are also evidenced to correlate with the decision-making speed46. Taken together, the previous assumption of identical update rates for all individuals is too ideal to portray the update event and heterogeneous individual interaction rhythms in realistic scenarios47,48. This prompts us to ask how this dynamical heterogeneity might interact with structural heterogeneity to alter the evolution of cooperation.

Here we investigate evolutionary game dynamics under non-identical rates of strategy updating. Specifically, we consider the scenario where individuals are allowed to update their strategies at diverse individual rates. We find that non-uniform rates of strategy updating can have profound effects on the emergence of cooperation, especially on heterogeneous structures, and reveal a significant decrease in C* necessary to promote cooperation. Moreover, we develop an efficient algorithm OptUpRat to minimise the threshold for the emergence of cooperation by tuning the update rate of each individual on any network.

Results

We consider evolutionary game dynamics on a structured population of N players, whose interactions are represented by an undirected, unweighted network. At any given time, the state of each node (player) is characterised by a strategy of either cooperation (C) or defection (D) (Fig. 1a). In each round of the game, every node i plays the game pairwise with its immediate ki neighbours. Specifically, cooperators pay a cost c to provide a benefit b to each of their neighbours, while defectors pay nothing, and thus provide no benefit. In this way, each node i gains an average payoff fi, corresponding to the average benefits received (from neighbouring cooperators) minus its cost.

Fig. 1: Illustration of the evolutionary process with identical versus personalised rates of strategy update.
figure 1

The interactions between four individuals are depicted in the example network structure in a, where individuals play games with their neighbours and gain the corresponding payoffs. The evolutionary process starts from a population of full defectors (red), and a cooperator (blue) invades the population via the top site. b The update event for each individual occurs as a Poisson process. We indicate on the timeline when each individual is chosen to update its strategy. The colour of the dot indicates the strategy after the update, which may be unchanged. When individuals' update rates are identical, they will have approximately the same number of strategy updates (numbers in orange, left panel), while for non-identical update rates, individuals with higher rates will update their strategies more often (right panel). The update rates for each individual in the right panel are λ1 = 1, λ2 = 1.25, λ3 = 1.75 and λ4 = 2, respectively. The change in the fraction of cooperation throughout the game is illustrated in c, and the evolutionary process ends when the population reaches a state of either full defection (left panel) or full cooperation (right panel).

Full size image

Traditionally, individuals are assumed to update their strategies following independent Poisson processes with identical rates. Here we depart from this practice: allowing each individual i to update its strategy with personalised rate λi (Fig. 1b). When an individual is chosen for an update, it does so by copying the strategy of one of its neighbours j, with probability proportional to the fitness of j, generally defined as Fj = 1 + δfj, where δ > 0 captures the intensity of selection4,5 (see Methods). For strong selection intensity, cooperation is disfavoured since the initial cooperator will not be able to survive or spread its strategy. Thus, to systematically uncover the effects of heterogeneous update rates on the fate of cooperators compared to existing findings, here we focus on the canonical case of weak selection.

To quantify the ability of cooperation to proliferate, we initialise our simulations with a single cooperator placed uniformly at random in a population among N − 1 defectors. The evolutionary game ends when a state with either all cooperators or all defectors is reached (Fig. 1c). We define the fixation probability of cooperation (ρC) as the probability of reaching the state of full cooperation over many realisations of this process. We can analogously define a probability ρD of reaching a full-defection state starting from a single defector planted of N − 1 cooperators. Our interest in this study is the condition under which cooperation is favoured to replace defection than vice versa4,5,26, namely ρC > ρD. This condition is equivalent to ρC > 1/N (Supplementary Note 1), namely that selection favours the emergence of cooperation relative to the neutral drift (δ = 0), in which neither cooperation nor defection is favoured (ρC = ρD = 1/N).

Evolutionary game dynamics on complex networks

First, we explore how the heterogeneous strategy updating affects the fate of cooperators on four commonly-studied population structures: lattice, small-world, Erdös-Rényi, and scale-free networks (Fig. 2). Under the traditional scenario of identical update rates (λi = 1 for all i), scale-free networks demand the largest critical benefit-to-cost ratio C*, above which cooperation is favoured among all the four structures, and the lattice structure the smallest (Fig. 2a), consistent with previous findings4,5. But surprisingly, when a node’s update rate varies inversely with its number of neighbours (λi = 1/ki), we find that this trend is reversed (Fig. 2b). Here, the scale-free network becomes the most amenable to cooperation, and lattice the least. Interestingly, we find that heterogeneous update rates can even improve upon the canonical threshold b/c > 〈k〉 (namely, C* = 〈k〉) for homogeneous populations4, allowing cooperation to emerge even when b/c < 〈k〉 (namely, C* < 〈k〉). Furthermore, we find that this pattern is strengthened when the update rate is inversely proportional to higher powers of ki (Fig. 2d). In contrast, when λi is positively related to ki, the ordering of C* over different structures matches the identical-rate case, but with the inhibition of cooperation fixation by heterogeneous networks amplified (Fig. 2c). We show the robustness of our results over different population sizes, average degrees and selection intensities in Supplementary Figs. 1–3.

Fig. 2: Effect of heterogeneous strategy update rates on the emergence of cooperation.
figure 2

We show the fixation probability of cooperation (ρC) as a function of the benefit-to-cost ratio (b/c) over different settings of the update rate (λi) of individual i, namely identical (λi = 1 for every individual in a) and heterogeneous (λi = 1/ki in b where ki is the number of neighbours of i, λi = ki in c) on lattice, Erdös-Rényi65 (ER), small-world66 (SW) and scale-free32 (SF) networks, respectively. The critical benefit-to-cost ratio C* above which the cooperation is favoured for each network occurs when the corresponding curve intersects the horizontal line representing the neutral-drift case (ρC = 1/N). C* for the scale-free case (purple) is marked. We demonstrate that the trend of C* reverses when the update rate varies inversely with ki in b, presenting the advantage of SF networks on favouring cooperation. d The ordering of C* for the four networks considered holds with \({\lambda }_{i}=1/{k}_{i}^{\gamma }\) (γ = 1, 2, 3, 4). Here we also show that SF networks are the most amenable to cooperation at non-identical update rates compared with other networks. e Simulation results on C* in a–d are in good agreement with our theoretical calculations shown in equation (1). Numerical values of ρC are obtained from the fraction of simulations in which the population reaches full cooperation out of 107 independent realisations on networks of 98 nodes for lattice and 100 for other networks with an average degree 〈k〉 = 6, and δ = 0.01. Source data are provided as a Source Data file.

Full size image

We further shed light on our numerical findings by deriving a closed-form expression for the critical benefit-to-cost ratio C* as a function of the network structure (see Methods)

$${C}^{*}=\frac{{\sum }_{i,j}{k}_{i}{p}_{ij}^{(2)}{\eta }_{ij}}{{\sum }_{i,j}{k}_{i}{p}_{ij}^{(3)}{\eta }_{ij}-{\sum }_{i,j}{k}_{i}{p}_{ij}{\eta }_{ij}}.$$
(1)

Here, ki = ∑jeij defines the number of neighbours (degree) of individual i, and eij = eji = 1 indicates that there is an edge between nodes i and j (eij = eji = 0 otherwise). The probability of a 1-step (n-step) random walk from i to j is denoted by pij (\({p}_{ij}^{(n)}\)), and ηij is the coalescence time49—the expected time for two random walks starting from nodes i and j to meet at a common node. As shown in Fig. 2e, all numerical results in Fig. 2a-d are in good agreement with the theoretical prediction of equation (1).

Role of network hubs

To intuitively understand why heterogeneous update rates can improve the fixation of cooperation in heterogeneous networks, we first consider how the evolutionary dynamics play out on a simple double-star structure (Fig. 3). When the fixation of cooperation occurs in this highly heterogeneous structure, it usually does so in four stages: (I) occupation of one of the hubs; (II) formation of a stable cluster of cooperators among that hub and its neighbours; (III) occupation of the other hub; and finally (IV) spread to the remaining nodes. As such, the ultimate triumph of cooperators can be thwarted if a hub imitates defection from even one of its (many) neighbours before stages (II) and (IV) are complete (Fig. 3c). There are ample opportunities for this to occur under the traditional setting of identical update rates (λi = 1), as illustrated in Fig. 3b. When λi = 1/ki however (Fig. 3a), hubs update relatively infrequently. As such, once a hub becomes a cooperator, it is effectively locked-in, giving time for its strategy to spread to the hub’s neighbours. Note that this lock-in effect can facilitate the formation of cooperative clusters to have higher payoffs to resist the invasion of defectors, yet defectors receive a lower payoff after driving their neighbours to defectors and further reduce their survival chances. By the same logic, the preferential updating of hubs (λi = ki) usually leads to the extinction of cooperation, as the formation of stable clusters of cooperators and the spread of cooperation is even harder than the traditional scenario of identical updating (Fig. 3c).

Fig. 3: Illustration of the role of hubs in the evolution of cooperation on a double-star structure.
figure 3

a The hubs, two centres of the double-star structure for example, have low update rates when λi = 1/ki (ki is the number of connections for each node), which facilitates the formation of local cluster of cooperation (blue dot, Stage II) once it is occupied by a cooperator (Stage I). Likewise, once the left hub spreads cooperation to the right hub (Stage III), the remaining nodes are quickly driven to cooperators (Stage IV). b When the update rates are identical (λi = 1), the hubs have many opportunities to change their strategy to defection before all neighbours become cooperators (Stage IV), making the fixation of cooperation less likely. c The hub switches its strategy quite frequently when λi = ki, which makes it hard to form even the left C-cluster (Stage II), to say nothing of spreading cooperation to the right centre.

Full size image

In Fig. 4, we illustrate the fundamental mechanism explaining why infrequent updates of hubs can facilitate cooperation. If an individual (grey node in Fig. 4a) decides to update its strategy, it will imitate the strategy of its neighbours according to their payoffs. The neighbouring cooperator obtains an average payoff PC = bqC∣C(〈k〉 − 1)/〈k〉 − c and the neighbouring defector obtains PD = bqC∣D(〈k〉 − 1)/〈k〉, where qC∣C (qC∣D) represents the conditional probability to find a cooperative neighbour for a given cooperator (defector). The contribution to the neighbouring cooperator and defector from the updating individual is excluded since they are equal. Thus the cooperator is favoured compared to the defector to disperse its strategy if PC > PD, namely

$$b({q}_{{{{{{{{\rm{C}}}}}}}}| {{{{{{{\rm{C}}}}}}}}}-{q}_{{{{{{{{\rm{C}}}}}}}}| {{{{{{{\rm{D}}}}}}}}})(\langle k\rangle -1)/\langle k\rangle -c \, > \, 0,$$
(2)

with Q = (qC∣C − qC∣D)(〈k〉 − 1) capturing the average number of cooperative neighbours that a cooperator has more than a defector. For the canonical setting with identical update rates (λi = 1), we know Q = 1 according to pair approximation (Supplementary Note 2), namely a cooperator has on average one more cooperative neighbour than a defector (Fig. 4a). This leads to the conclusion that cooperation is favoured when b/c > 〈k〉 (namely, C* = 〈k〉), which also degenerates to the simple rule4 for homogeneous networks where ki = 〈k〉.

Fig. 4: Mechanism for promoting collective cooperation with infrequent strategy updates of hubs.
figure 4

a–c Illustration on the scenario where a cooperator (blue dot) and a defector (red dot) compete to spread their strategy to the individual (grey dot) selected for strategy update under different update rates λi. Since behaviour dispersal occurs in the neighbourhood, the cooperator obtains on average bQ/〈k〉 − c more payoff than the defector (equation (2)), and the cooperator is favoured when the above expression is positive. a For identical updating (λi = 1), the cooperator has one more cooperative neighbour than the defector, therefore it receives b/〈k〉 more benefit than the defector at a cost of c. b When λi = 1/ki, the net benefit of the cooperator relative to the defector exceeds b/〈k〉 because the fraction of cooperative neighbours of the cooperator further increases compared to the defector, offering the cooperator a higher chance for dispersal. c We show that the fast strategy update of hubs (λi = ki) reduces the number of cooperative neighbours of the cooperator, which exceeds that of the defector by less than one. This lowers the benefit of the cooperator and reduces the chance to win the empty site. We further compare the state of the hub (grey lines) and the fraction of cooperation among its neighbours (blue lines) of a scale-free network with different settings of update rates (d–f). Generally, the hub imitates one of its cooperative neighbours and keeps cooperation for several rounds (light blue shaded region) before switching to defection (light red shaded region) in d. Statistically, we count the fraction of cooperators in the neighbourhood of a cooperative hub (qC∣C for the hub) throughout evolutionary process in g, and qC∣C − qC∣D for nodes with different degrees in h. Numerical calculations confirm the mechanism we present in a–c. Here, we use the same network parameters as Fig. 2. Source data are provided as a Source Data file.

Full size image

Next we show how heterogeneous update rate alters the local dispersal of cooperation. When λi = 1/ki, we find that Q > 1 (Supplementary Note 2), indicating that the number of cooperative neighbours of a cooperator exceeds that of a defector by more than one (Fig. 4b). This implies that the net payoff of cooperators relative to defectors is further increased, giving cooperators more advantage in competition and dispersal. Therefore, the critical ratio for λi = 1/ki is smaller than the average degree 〈k〉 for a wide range of heterogeneous networks (C* < 〈k〉). In contrast, when λi = ki, the hubs update frequently and Q < 1 (Supplementary Note 2), indicating that on average, the number of cooperative neighbours of a cooperator exceeds that of a defector by less than one (Fig. 4c). This leads to a larger critical ratio (C* > 〈k〉) for promoting cooperation compared to the scenario with identical update rates shown in Fig. 4a.

We have numerically confirmed the above mechanism on larger scale-free networks. Figure 4d–f show the state of the hub, and the fraction of cooperators among the hub’s neighbours over the course of the game dynamics. For λi = 1/ki, we observe long-lasting periods of cooperation on the hub (Fig. 4e), with infrequent strategy switches from cooperation to defection, which results in the highest qC∣C for the hub (Fig. 4g) and in turn the highest qC∣C − qC∣D over all nodes with different degrees compared to other settings (Fig. 4h). In contrast, fast-updating hubs (λi = ki) have the lowest average fraction of cooperators among their neighbours (Fig. 4g), leading to a low fraction of cooperative neighbours for the cooperators relative to defectors over the whole network (Fig. 4h). This confirms that degree-inverse update rates promote cooperation on heterogeneous networks because a hub with a low update rate is more conducive to driving its neighbours to cooperation, which further enhances the local dispersal of cooperation among nodes with different degrees.

Furthermore, we find that infrequent updates of hubs can also bring long-term advantages to individuals. We could even consider the general evolutionary process with mutation, where a mutant appears with probability u when the population reaches full cooperation or full defection. Each individual accumulates long-term payoffs during a long period of time. Even with a high mutation rate (u = 1), we show that the inverse relationship between update rates and nodes’ degrees results in a higher long-term payoff for individuals than identical rates (λi = 1) (Fig. 5a). In contrast, frequent updates of hubs (λi = ki) lead to a lower payoff than the identical settings (Fig. 5a). This result is also robust over different mutation rates and selection intensities (Fig. 5b, c). Moreover, when the mutation is rare, the population is almost always in full cooperation or full defection, and the time spent in full cooperation (defection) is proportional to ρC (ρD)50. Therefore, the settings of update rates which promote cooperation further lead to a higher long-term average payoff, since individuals get b − c in full cooperation but 0 in full defection.

Fig. 5: The advantage of infrequent strategy updates of hubs on individual long-term payoffs.
figure 5

a We calculate the average long-term individual payoff \({P}_{{\lambda }_{i}=1/{k}_{i}},{P}_{{\lambda }_{i}=1}\) and \({P}_{{\lambda }_{i}={k}_{i}}\) corresponding to different update rates (λi = 1/ki, λi = 1, λi = ki) for 50 individuals during the evolutionary process with mutation rate u = 1 on a scale-free network, which are presented as mean values +/- SD. b We present the payoff difference \({P}_{{\lambda }_{i}=1/{k}_{i}}-{P}_{{\lambda }_{i}=1}\) between λi = 1/ki and λi = 1 over different selection intensities and mutation rates. Analogously, the long-term payoff difference between identical rates and rates proportional to nodes' degrees \({P}_{{\lambda }_{i}=1}-{P}_{{\lambda }_{i}={k}_{i}}\) is shown in c. The long-term payoff is averaged over 103 independent samples, where the average payoff in each run is obtained over 106 rounds. Source data are provided as a Source Data file.

Full size image

Theoretical analyses

In addition to the role of hubs that we uncover for the three specific update rate settings (λi = 1/ki, λi = 1, λi = ki), can we derive the general rule for promoting cooperation that also applies to other distributions of update rates? We next explore how different distributions of λi affect C* over five different synthetic networks: lattice, random regular, Erdös-Rényi, small-world, and scale-free. For a given network structure, we theoretically predict C* via equation (1) for uniform, normal, exponential and power-law distributions of the update rate. We find that the critical threshold of a typical homogeneous network—such as a lattice or random regular network—is almost unaffected by the choice of update rate distribution (Fig. 6a, Supplementary Fig. 4). In contrast, heterogeneous structures are quite sensitive, with scale-free networks presenting the most drastic variations in C* among the different update-rate distributions we consider. This malleability of C* in heterogeneous networks suggests the possibility of deliberately tuning the update rates to lower the barrier for the emergence of cooperation in a particular network. But to put this into practice, we must first overcome a computational hurdle.

Fig. 6: The simple condition for promoting cooperation.
figure 6

a Illustration of C* above which cooperation is favoured under uniform (Uni.), normal (Nor.), exponential (Exp.) and power-law (Pow.) distributions of update rates on different structures of networks. Here each dot corresponds to a sample, and the error bars are plotted over 100 samples, indicating the mean values with +/- SD. The robustness of our results with different average degrees is shown in Supplementary Fig. 4. Note that the coupling of node degree and update rate will not bring quantitatively different results on the average C*. The consistent theoretical evidences and details are given in Supplementary Fig. 5. b C* obtained from our theory (equation (3)) with various update rate (λi) configurations (different markers) are well-matched with the numerical simulations on empirical networks corresponding to face-to-face contacts in an office building51 (Off.) and a high school52 (Stu.). Based on the analytical condition given in equation (4), we seek to reduce C* on large heterogeneous structures, specifically by letting nodes' update rates vary inversely to their degree ki as shown in c, where the size (colour) of nodes captures the magnitude of ki (λi). Source data are provided as a Source Data file.

Full size image

In order to calculate C* using equation (1), one needs to solve a system of N(N − 1)/2 linear equations for the recurrence relations between the ηij (equation (7) in Methods). Unfortunately, this requires an overall complexity of \({{{{{{{\mathcal{O}}}}}}}}({N}^{6})\), rendering the problem intractable for large networks. To circumvent this, we offer an efficient approximation C* as

$${C}^{*}\approx \frac{N{\langle k\rangle }^{2}\zeta /\langle {k}^{2}\rangle -1+{\Delta }_{{\lambda }^{(1)}}+{\Delta }_{{\widetilde{\eta }}_{n}}}{N\langle k\rangle \zeta /\langle {k}^{2}\rangle -1+{\Delta }_{{\lambda }^{(2)}}+{\Delta }_{{\widetilde{\eta }}_{d}}}.$$
(3)

This expression obviates the need to solve large systems of linear equations and reduces the computational complexity to \({{{{{{{\mathcal{O}}}}}}}}({N}^{3})\). Here 〈k2〉 is the second moment of the degree distribution. We have \(\zeta={\sum }_{i,j}\frac{{k}_{i}{k}_{j}\Lambda }{N{K}^{2}({\lambda }_{i}+{\lambda }_{j})}\), where Λ = ∑iλi defines the total rate of update events and K = ∑iki is the summation of all nodes’ degrees. Finally, \({\Delta }_{{\lambda }^{(1)}},{\Delta }_{{\lambda }^{(2)}},{\Delta }_{{\widetilde{\eta }}_{n}}\) and \({\Delta }_{{\widetilde{\eta }}_{d}}\) are constants related to the heterogeneity of update rates and coalescence times, the expressions for which are given in Methods. When the update rates are identical, we have \({\Delta }_{{\lambda }^{(1)}}={\Delta }_{{\lambda }^{(2)}}={\Delta }_{{\widetilde{\eta }}_{n}}={\Delta }_{{\widetilde{\eta }}_{d}}=0\), and equation (3) recovers the previous results4,31.

Figure 6b compares the value of C* predicted by the approximation in equation (3) with that of numerical simulation on two empirical social networks51,52. We see that our approximation is remarkably accurate in both networks, regardless of the distribution of the update rates. Moreover, equation (3) offers intuition behind our previous observation that homogeneous structures are robust to different update rates (Fig. 6a). The high symmetry present in these networks means that heterogeneous update rates affect only a limited number of nodes. For such networks, we have \({\Delta }_{{\widetilde{\eta }}_{n}}\approx {\Delta }_{{\widetilde{\eta }}_{d}}\approx 0\), meaning that C* → 〈k〉 in the limit of large N. This coincides with the classical result4 (C* = 〈k〉) regardless of the distribution of update rates.

A simple condition for the emergence of cooperation

Starting from equation (3) (see Methods), we have the critical benefit-to-cost ratio for large heterogeneous networks

$${C}^{*}\approx \langle k\rangle+\frac{{\langle k\rangle }^{2}\langle {k}^{2}\rangle {\Delta }_{{\widetilde{\eta }}^{(\infty )}}}{{\langle k\rangle }^{3}\zeta+({\langle k\rangle }^{3}-\langle k\rangle \langle {k}^{2}\rangle -\langle {k}^{2}\rangle ){\Delta }_{{\widetilde{\eta }}^{(\infty )}}},$$
(4)

where 〈k〉 is the average degree and \({\Delta }_{{\widetilde{\eta }}^{(\infty )}}\approx \frac{\overline{\eta }}{{K}^{2}}{\sum }_{i < j}({k}_{i}-{k}_{j})({\lambda }_{i}-{\lambda }_{j}){e}_{ij}/({\lambda }_{i}+{\lambda }_{j})\). Note that \({\Delta }_{{\widetilde{\eta }}^{(\infty )}} < 0\) when any pair of nodes i and j satisfies the rule (ki − kj)(λi − λj) < 0. When the update rates are identical, we have \({\Delta }_{{\widetilde{\eta }}^{(\infty )}}=0\) and hence C* ≈ 〈k〉 as expected. In contrast, C* is smaller (larger) than 〈k〉 when \({\Delta }_{{\widetilde{\eta }}^{(\infty )}} < 0\) (\({\Delta }_{{\widetilde{\eta }}^{(\infty )}} > 0\)) (Supplementary Note 3.3). Table 1 summarises the values of C* predicted by equation (4) for the combinations of network structure/update-rate settings.

Table 1 Critical benefit-to-cost ratio C* for the fixation of cooperation under different update rates and network structures
Full size table

Taken together, we have theoretically motivated an efficient rule of thumb for lowering the threshold for the emergence of cooperation on large heterogeneous structures. Put simply, the order of any pair of nodes’ update rates (for example, λi > λj) should be reversed from the order of the nodes’ degrees (for example, ki < kj). That is, the one with larger degree should have smaller update rates and vice versa, as is demonstrated in Fig. 6c. In other words, the hubs in networks should update infrequently compared to their neighbours with fewer connections to promote the formation of cooperative clusters, which is consistent with the underlying mechanisms shown in Figs. 3 and 4. A simple but general realisation of this rule is \({\lambda }_{i}=1/{k}_{i}^{\gamma }(\gamma \, > \, 0)\) which we study numerically in Fig. 2d for different values of γ. This rule can achieve a lower critical ratio C* than identical update rates (γ = 0) on both synthetic heterogeneous (Fig. 2b) and empirical networks (γ = 1) (Fig. 6b and Supplementary Table 1 and Supplementary Figs. 6 and 7). Meanwhile, the contrary configuration of \({\lambda }_{i}={k}_{i}^{\gamma }\) leads to increases in C* on heterogeneous networks (Figs. 2c and 6b and Supplementary Figs. 8 and 9).

Moreover, we show that our conclusion can also be applied to other social dilemmas (Supplementary Note 4). For the general two-player game, a cooperator receives rewards R from mutual cooperation, while defectors obtain punishment P from mutual defection. A defector attempting to exploit a cooperator obtains T and leaves S to its opponent cooperator. We show that cooperation is favoured over defection when R > P + (T − S)(C* − 1)/(C* + 1), where a lower threshold for R can be achieved with a lower C*. Note that here C* is exactly the critical threshold under the donation game. This indicates our conclusion applies to other social dilemmas, such as the general prisoner’s dilemma (T > R > P > S)16, snowdrift game (T > R > S > P)3 and stag hunt game (R > T ≥ P > S)53.

The optimal update rate on any network

As an engineering application of designing unmanned and autonomous systems, can we adopt the simple heuristic to favour collective cooperation among agents? Specifically, can we find the optimal set of λi for a given networked system? To answer this question, we develop OptUpRat, an optimisation algorithm, to search for a set of λi that minimises C* (See Box 1, Supplementary Note 5 and Supplementary Fig. 10). Our algorithm OptUpRat is based on RMSProp (root mean square propagation), which is an optimisation algorithm designed for training neural networks54. Note that the settings of the learning rate ϵ, decay rate ρ and constant δopt parameters are the same as those in RMSprop—the learning rate ϵ controls the step size of the iteration; ρ controls the decay rate of the moving average; and δopt is a small constant added to the denominator to prevent division by zero (see the values of those parameters in Methods). To transform the constrained optimisation with λi > 0 for each individual i into an unconstrained optimisation problem, we define \({\lambda }_{i}=\exp ({\theta }_{i})\) to establish a function mapping from θi to C*. Then the optimal update rate and the corresponding C* can be obtained via iterative gradient descent, where the gradient is computed by solving a system of N(N − 1)/2 linear equations after taking the derivative with respect to θi on both sides of equation (7) in Methods.

Consistent with our rule, Fig. 7a shows the scale-free network is more flexible and attain a much smaller threshold at its optimal rate than the lattice. Moreover, the update rates of higher-degree nodes tend to decrease during the optimisation process, while those of smaller-degree nodes increase (Fig. 7b and Supplementary Fig. 11). Interestingly, we find that even on homogeneous structures such like lattices, a policy of identical update rates is not the best choice for promoting cooperation. Indeed, the final update rates deviate significantly from the initial conditions (Fig. 7c and Supplementary Fig. 12). Figure 7d shows that the optimal update rates for different network structures are consistent with our rules shown in Fig. 6c—namely that a node i’s update rate λi should vary inversely with its degree ki.

Fig. 7: Designing the optimal update rates to promote cooperation on heterogeneous networks.
figure 7

a We present the convergence of the objective function C* for a scale-free network (purple) and a lattice (blue) over 103 iterations of our optimisation algorithm OptUpRat. b The corresponding evolution of the (tunable) λi for all nodes, which are divided into three categories (large, moderate and small), based on the range of degrees in the scale-free network. The mean update rate among individuals in each category is shown with the thicker line. We see that the optimal update rates tend to decrease for large nodes (orange) and generally increase for small nodes (green). c For the lattice, the optimal update rate also presents the deviations from the identical rate. Beyond presenting the detailed process for optimising C* in panels a–c we show the final λi compared to the nodes' degree for scale-free networks (generated by the configuration model67, Barabéasi-Albert model32), small-world network66 (rewiring probability 0.7) and networks constructed from a uniform attachment model68 in d, where we normalise the optimal update rate and the node degree. We again observe an inverse relationship between the final update rates and the corresponding nodes' degree, consistent with our rule shown in Fig. 6c. Here we use the same network parameters as Fig. 2. Source data are provided as a Source Data file.

Full size image

Discussion

Our findings reconcile the past conflicting results on how heterogeneous networks affect the evolution of cooperation. Studies that initialise evolutionary game dynamics with an equal number of cooperators and defectors have found that scale-free networks actually outperform homogeneous networks in promoting the evolution of cooperation, as measured by the average fraction of cooperators6. But from the perspective of fixation probability, heterogeneous structures impose a higher benefit-to-cost threshold for a single cooperator to take over a population of defectors, at least when all update rates are identical4,5,31. This predicts that heterogeneous network structures, despite their ubiquity in physical and social systems, tend to hinder the emergence of collective behaviour. By relaxing this assumption and allowing nodes to update their strategies at non-identical rates, we have shown that scale-free networks can in fact facilitate the fixation of cooperation. As such, degree-heterogeneous networks orchestrated by personalised update rates can be unambiguously conducive to cooperation, provided they are doubly heterogeneous—that is, also heterogeneous in update rate. Taken together, we argue that personalised interaction dynamics and network structure combine to shape the collective dynamics.

From the perspective of microscopic mechanism, we unveil that different update rules render the conflict results. Regarding the frequency of cooperators, previous canonical framework and update rule naturally lead to infrequent strategy switching (Supplementary Note 6)6,33,34. This facilitates the formation of cooperative clusters and leads to a high fraction of cooperators on heterogeneous networks. Previous findings are consistent with the underlying microscopic mechanism in our study, namely infrequent updates of hubs facilitate the emergence of cooperation. Indeed, by applying the canonical death-birth update with identical rates in the framework analysing the frequency of cooperators6,33,34, we find that heterogeneous networks impede the average frequency of cooperators compared to homogeneous scenarios (Supplementary Fig. 13).

Furthermore, we compare our results with experimental studies on cooperation in heterogeneous networks. Consistent with our theoretical findings, there is an insightful experimental study also reporting that heterogeneous networks do not promote cooperation in prisoner’s dilemmas55. In this behavioural experiment, a player’s decisions to cooperate or defect are relevant to the level of cooperation in their neighbourhoods, which renders the network irrelevant. Therefore, the main difference between this experimental finding and our study lies in the update rules. Specifically, players are more likely to imitate the strategy from neighbours with higher payoffs in our theoretical framework. To further uncover the behavioural dynamics from the perspective of fixation probability, a promising future application involves the design of human behavioural experiments starting from a single cooperator and ending with full cooperation or defection. Comparing the individual behavioural mode in experiments from these two perspectives will facilitate the understanding of the emergence of cooperation in realistic scenarios.

A natural extension of our findings is exploring the scenario with multiple strategies56,57,58. In this way, the diverse strategy update rhythms may couple multiple strategies with complex dynamics. In addition, our findings may contribute to the study of network formation, elucidating the factors influencing group formation, such as individuals’ propensity to establish connections with those who share similar rhythms. Specifically, discovering the scenarios wherein individuals with similar update rates are allowed to construct a group may provide valuable information regarding the optimal network configuration in the context of heterogeneity.

One promising direction for future research lies in evolutionary dynamics on temporal networks. Time-varying network structure is a recurring theme in social systems, encoding not only who interacts with whom but with when (and how often) these interactions happen59. It was recently discovered that temporal networks generally enhance the evolution of cooperation relative to comparable static networks12, yet the practical scenarios easily trigger the heterogeneous time rhythm of strategy updating. In real temporal networks, a node’s degree may vary drastically even over short time periods47,48,60. This—in tandem with other temporal effects such as burstiness and multi-frequency interactions47,61—may lead to more exotic evolutionary dynamics. By regarding a temporal network as a sequence of static snapshots, our theory might be adopted to further tailor individuals’ update rates in temporal evolutionary game dynamics.

Methods

Evolutionary process

In each round of the game, individuals interact with their neighbours and accumulate the payoffs accordingly. The payoff matrix of the game is given by

$$\begin{array}{c}\begin{array}{cc}\qquad C&\quad D\,\,\end{array}\\ \begin{array}{c}C\\ D\end{array}\left(\begin{array}{cc}b-c&-c\\ b&0\end{array}\right)\end{array} .$$

The state of network at any given time can be encoded by a binary vector x ∈ {0, 1}N, where xi = 1 denotes that the player i chooses strategy C, otherwise xi = 0 indicates strategy D. Using this representation of the network state x, i’s average payoff is fi(x) = − cxi + b∑jpijxj, where pij = eij/ki indicates the probability of a single step random walk from i to j on the network. For a node i with update rate λi, the probability to be chosen for a strategy update is λi/Λ, where Λ = ∑iλi defines the total rate of update events. It follows that at the end of each round, the probability for a player j to transmit its strategy to i is \({r}_{ji}({{{{{{{\bf{x}}}}}}}})=\frac{{\lambda }_{i}}{\Lambda }\frac{{e}_{ij}{F}_{j}({{{{{{{\bf{x}}}}}}}})}{{\sum }_{l}{e}_{il}{F}_{l}({{{{{{{\bf{x}}}}}}}})}\), where Fj(x) = 1 + δfj(x) indicates the fitness of individual j. Note that the fixation probability does not change when the rate of strategy updates for each individual is identical since the normalised update rates are the same.

Fixation probability

As shown in the Supplementary Note 1, the fixation probability of cooperation is derived by a first-order expression as the neutral fixation probability (1/N) plus a correction term due to weak selection, namely

$${\rho }_{C}=\frac{1}{N}+\delta {\left\langle {\left.\frac{d}{d\delta }\right| }_{\delta=0}\widehat{\Delta }({{{{{{{\bf{x}}}}}}}})\right\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }+O\left({\delta }^{2}\right),$$
(5)

where \(\widehat{\Delta }({{{{{{{\bf{x}}}}}}}})\) denotes the reproductive-value-weighted frequency change of cooperation, which is given by

$$\widehat{\Delta }({{{{{{{\bf{x}}}}}}}})=\mathop{\sum}\limits_{i}\frac{{k}_{i}}{{\lambda }_{i}{\sum }_{l}\frac{{k}_{l}}{{\lambda }_{l}}}\mathop{\sum}\limits_{j}\left({x}_{j}-{x}_{i}\right){r}_{ji}({{{{{{{\bf{x}}}}}}}}).$$
(6)

Here \({\left\langle \varphi \right\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\) indicates the summation of the expectation of φ with φ(1)=φ(0)=0 under neutral drift through time step t = 0 to infinity, namely \({\langle \varphi ({{{{{{{\bf{x}}}}}}}})\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }=\mathop{\sum }\nolimits_{t=0}^{\infty }{\sum }_{{{{{{{{\bf{x}}}}}}}}\in {\{0,1\}}^{N}}{{\mathbb{P}}}_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\left[{{{{{{{\bf{X}}}}}}}}(t)={{{{{{{\bf{x}}}}}}}}\right]\varphi ({{{{{{{\bf{x}}}}}}}})\), where \({{\mathbb{P}}}_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\left[{{{{{{{\bf{X}}}}}}}}(t)={{{{{{{\bf{x}}}}}}}}\right]\) indicates the neutral probability of the system reaching state x at time step t starting from the initial state with a single uniformly selected cooperator in population with N − 1 defectors. Combining equations (5) and (6), the fixation probability can be expressed as

$${\rho }_{C}=\frac{1}{N}+\frac{\delta }{\Lambda {\sum }_{i}\frac{{k}_{i}}{{\lambda }_{i}}}\left[-c\mathop{\sum}\limits_{i,j}{k}_{i}{p}_{ij}^{(2)}{\eta }_{ij}+b\left(\mathop{\sum}\limits_{i,j}{k}_{i}{p}_{ij}^{(3)}{\eta }_{ij}-\mathop{\sum}\limits_{i,j}{k}_{i}{p}_{ij}{\eta }_{ij}\right)\right]+O\left({\delta }^{2}\right),$$

where \({\eta }_{ij}={\left\langle \widehat{x}-{x}_{i}{x}_{j}\right\rangle }_{{{{{{{{\rm{u}}}}}}}}}^{\circ }\), and \(\hat{x}={\sum }_{i}{\pi }_{i}{x}_{i}\) represents the reproductive-value-weighted frequency of cooperators, where πi is the reproductive value62,63,64 uniquely solved by Supplementary equation (2), quantifying the expected contribution of site i to the future gene pool under neutral drift. Here ηij satisfies the recurrence relation of

$${\eta }_{ij}=\left\{\begin{array}{ll}\frac{\Lambda }{N({\lambda }_{i}+{\lambda }_{j})}+{\sum }_{k}\frac{{\lambda }_{i}}{{\lambda }_{i}+{\lambda }_{j}}{p}_{ik}{\eta }_{kj}+{\sum }_{k}\frac{{\lambda }_{j}}{{\lambda }_{i}+{\lambda }_{j}}{p}_{jk}{\eta }_{ki},\quad &i\ne j\\ 0,\hfill &i=j\end{array}\right..$$
(7)

By letting ρC > 1/N, we obtain C* shown in equation (1).

Calculation of the critical ratio C *

We first define \({\eta }^{(n)}={\sum }_{i,j}{k}_{i}{p}_{ij}^{(n)}{\eta }_{ij}/K\), where K = ∑iki is the summation of all nodes’ degrees, then equation (1) can be rewritten as

$${C}^{*}=\frac{{\eta }^{(2)}}{{\eta }^{(3)}-{\eta }^{(1)}}.$$

From the recurrence relation of ηij in equation (7), we further derive the recurrence relation of η(n) with

$${\eta }^{(n)}=\mathop{\sum}\limits_{i,j}\frac{{k}_{i}}{K}{p}_{ij}^{(n)}\frac{\Lambda }{N({\lambda }_{i}+{\lambda }_{j})}+{\widetilde{\eta }}^{(n+1)}-\mathop{\sum}\limits_{i}\frac{{k}_{i}}{K}{p}_{ii}^{(n)}{\eta }_{ii}^{+},$$
(8)

where \({\widetilde{\eta }}^{(n+1)}={\sum }_{i,j,l}\frac{{k}_{i}}{K}{p}_{ij}^{(n)}\frac{2{\lambda }_{j}}{{\lambda }_{i}+{\lambda }_{j}}{p}_{jl}{\eta }_{il}\) and \({\eta }_{ii}^{+}=\frac{\Lambda }{2N{\lambda }_{i}}+{\sum }_{l}{p}_{il}{\eta }_{il}\).

By defining the difference \({\Delta }_{{\widetilde{\eta }}^{(n)}}:={\widetilde{\eta }}^{(n)}-{\eta }^{(n)}\) and using the recurrence relation of equation (8), we obtain the calculation of C* shown in equation (3) with mean-field approximation, with \({\Delta }_{{\widetilde{\eta }}_{n}}=-{\Delta }_{{\widetilde{\eta }}^{(2)}}+\frac{{K}^{2}}{{\sum }_{i}{k}_{i}^{2}}{\Delta }_{{\widetilde{\eta }}^{(\infty )}}\) and \({\Delta }_{{\widetilde{\eta }}_{d}}=-{\Delta }_{{\widetilde{\eta }}^{(2)}}-{\Delta }_{{\widetilde{\eta }}^{(3)}}+\frac{KN}{{\sum }_{i}{k}_{i}^{2}}{\Delta }_{{\widetilde{\eta }}^{(\infty )}}\) for simplification, where \({\Delta }_{{\lambda }^{(1)}}={\sum }_{i}\frac{{k}_{i}}{2K}\left(1-\frac{\Lambda }{N{\lambda }_{i}} \right)+{\sum }_{i,j}\frac{{k}_{i}}{2K}{p}_{ij}\left(1-\frac{2\Lambda }{N({\lambda }_{i}+{\lambda }_{j})}\right)\) and \({\Delta }_{{\lambda }^{(2)}}={\sum }_{i,j}\frac{{k}_{i}}{2K}({p}_{ij}+{p}_{ij}^{(2)})(1-\frac{2\Lambda }{N({\lambda }_{i}+{\lambda }_{j})})\). According to Supplementary Note 3, we further have \({\Delta }_{{\widetilde{\eta }}^{(2)}}\approx N{\Delta }_{{\widetilde{\eta }}^{(\infty )}}/\langle k\rangle\) and \({\Delta }_{{\widetilde{\eta }}^{(3)}}\approx N{\Delta }_{{\widetilde{\eta }}^{(\infty )}}/{\langle k\rangle }^{2}\) for large networks, and hence C* shown in equation (4) follows immediately.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.