Self-regulation versus social influence for promoting cooperation on networks

Cooperation is a relevant and controversial phenomenon in human societies. Indeed, although it is widely recognized essential for tackling social dilemmas, finding suitable policies for promoting cooperation can be arduous and expensive. More often, it is driven by pre-established schemas based on norms and punishments. To overcome this paradigm, we highlight the interplay between the influence of social interactions on networks and spontaneous self-regulating mechanisms on individuals behavior. We show that the presence of these mechanisms in a prisoner’s dilemma game, may oppose the willingness of individuals to defect, thus allowing them to behave cooperatively, while interacting with others and taking conflicting decisions over time. These results are obtained by extending the Evolutionary Game Equations over Networks to account for self-regulating mechanisms. Specifically, we prove that players may partially or fully cooperate whether self-regulating mechanisms are sufficiently stronger than social pressure. The proposed model can explain unconditional cooperation (strong self-regulation) and unconditional defection (weak self-regulation). For intermediate self-regulation values, more complex behaviors are observed, such as mutual defection, recruiting (cooperate if others cooperate), exploitation of cooperators (defect if others cooperate) and altruism (cooperate if others defect). These phenomena result from dynamical transitions among different game structures, according to changes of system parameters and cooperation of neighboring players. Interestingly, we show that the topology of the network of connections among players is crucial when self-regulation, and the associated costs, are reasonably low. In particular, a population organized on a random network with a Scale-Free distribution of connections is more cooperative than on a network with an Erdös-Rényi distribution, and, in turn, with a regular one. These results highlight that social diversity, encoded within heterogeneous networks, is more effective for promoting cooperation.


The Evolutionary Game equation on Networks (EGN) and self games
Let V = {1, 2, . . . , N } be the set of players. Each player is placed in a vertex of an undirected graph, defined by the adjacency matrix A = {a v,w } ∈ {0, 1} N ×N with (v, w) ∈ V 2 . Specifically, a v,w = 1 when v is connected to w, 0 otherwise. It is also assumed that a v,v = 0. The degree of player v is defined as the cardinality of his neighborhood, namely: a v,w .
In the literature on evolutionary game theory, it is assumed that, at each time, one individual uses a pure strategy in a given set while playing games with connected individuals. These games are often modeled as Prisoner's dilemma games, where the set of pure strategies contains only two elements: cooperation (C) and defection (D). The outcome of those games is described by the following payoff matrix: where R is the reward when both players cooperate, T is the temptation to defect when the opponent cooperates, S is the sucker's payoff earned by a cooperative player when the opponent defects, and P is the punishment for mutual defection. More specifically, for a Prisoner's dilemma game, the reward is a better outcome than the punishment (R > P ), the temptation payoff is higher than the reward (T > R), and the punishment is preferred to the sucker's payoff (P > S). Without loss of generality, we assume R = 1 and P = 0, thereby normalizing the advantage of mutual cooperation over mutual defection to 1 [S1]. Under this assumption, T > 1 and S < 0. It is straightforward to note that T − 1 quantifies the temptation to defect, while −S is related to the disadvantage of being defected. Therefore, we can distinguish two cases: At each time instant, an individual v will play k v continuous prisoner's dilemma games with his neighbors [S2, S3, S4]; specifically, he chooses his own level of cooperation indicated by x v ∈ [0, 1]. Notice that discrete strategies C and D are special cases obtained for x v = 1 and x v = 0, respectively. When any two connected players v and w take part in a game, the payoff for v is defined by the continuous function φ : [0, 1] × [0, 1] → R (see [S4]): The total payoff φ v of player v is the sum of all outcomes of two-player games with neighbors. Formally, the payoff function φ v : [0, 1] N → R is defined as follows: where x is the vector of all the x v variables. Moreover, given the vector x, we define the following payoff of pure strategies C (x v = 1) and D (x v = 0): Following [S5, S6], the EGN equation for two-strategy games reads as follows: where It is clear that the level of cooperation of player v increases (decreases) when ∆p v (x) is positive (negative). In other words, the player v will be more cooperative over time as long as the payoff he can earn using the pure strategy C is better than the payoff he can earn using the pure strategy D.
This evaluation of the benefits provided by the available strategies can be formulated in an alternative way. Specifically, suppose that player v is able to appraise whether a change of his strategy x v produces an improvement of his payoff φ v . This means that, if the derivative of φ v with respect to x v is positive (negative), the player would like to increase (decrease) his level of cooperation. According to this idea, notice that: Thus, the EGN equation (2) can be rewritten as follows: Since in (3) x v (1 − x v ) ≥ 0, then the sign ofẋ v depends only on the term ∂φ v /∂x v , which involves the states x w of all neighbors, rather than the current state x v of player v himself. Then, if this term is positive (negative), player v would like to increase (decrease) his level of cooperation x v . Of course, when it is null, player v has no incentives to change his mind.
It is worthwhile to notice that, while the replicator equation is used to describe the dynamics of population where strategies correspond to the phenotypes of the individuals, its extension on graphs, the EGN equation, is suitable for analyzing the dynamics of individuals arranged on a network, and able to choose their strategies in the continuous set [0, 1].
The EGN equation (3), as well as most of the models presented in the literature, assumes that the strategy dynamics of a generic player v is driven only by external factors. Indeed, ∂φv ∂xv depends only on the state of neighboring players, not on the current state x v of player v himself. Inspired by mechanisms describing self-regulation in animal societies reported in [S7], we overcome this issue by introducing the Self-Regulated EGN equation (SR-EGN); this new model is obtained by adding a self-regulating term f v to the EGN equation, balancing the external feedback ∂φv ∂xv . The SR-EGN equation reads as: where the parameter β v is used to tune the effectiveness of the introduced self-regulation mechanism. Specifically, we assume that this self-regulation term embodies a game that a given individual plays against himself. To describe this self game, consider two generic players, choosing feasible strategies y and z. As already mentioned, the first player can assess whether a change of his strategy y can lead to an improvement of the payoff φ(y, z). In particular, the assessment is based on the sign of the partial derivative In the particular case of individuals representing both the first and second player at the same time, the derivative reads as follows: Therefore, the self-regulating term is defined as: Notice that function f v (x v ) in equation (4) depends on x v . Thus, the self game introduces a feedback mechanism regulated by the parameter β v ∈ R. In particular, in equation (4), β v > 0 represents a negative feedback, β v < 0 stands for a positive feedback, while β v = 0 refers to situations where the player v does not play a self game.

Steady states and linearization
A steady state x * is a solution of equation (4) satisfyingẋ v = 0 ∀v ∈ V. In order to be feasible, the components of a steady state must belong to the set [0, 1]. Formally, the set of feasible steady states is: It is clear that all points such that for all v, x * v = 0 or x * v = 1 are in the set Θ. We remark that set Θ may contain also other steady states, exhibiting partial levels of cooperation. Among the pure steady states, particularly relevant are the followings: , and x * ALLD = [0, 0, . . . , 0] . Indeed, they represent a population composed by full cooperators and full defectors, respectively, and thus they describe the spread of cooperation, or alternatively its extinction in a given population.
The dynamical properties of these two pure steady states is fundamental for the emergence of cooperation. In particular, their stability can be locally analyzed by linearizing system (4).
The Jacobian matrix of system (3), J(x) = {j v,w (x)}, is defined as follows: It is easy to show that the Jacobian matrix reduces to a diagonal one for both x * ALLC and x * ALLD . Moreover, observe that: Therefore: The system (4) may have steady states with some components in the set (0, 1). Consider a steady state x * and suppose that one component x * v belongs to the set (0, 1). Therefore: Notice that: where represents an equivalent player, incorporating the average decisions of all neighbors of player v.
For x = x * , we have that equation (5) can be rewritten as: The following theoretical results on the feasibility of x * v hold.
Secondly, notice that: Similarly, since β v > Q v , then ρQ v < ρβ v . Moreover, subtracting β v from both left and right side of the inequality, we get Dividing the last inequality by (ρ − 1)β v , we get: The proof is concluded by observing that: Therefore, inequalities (7) and (8) imply that: Proof. First of all, T > 1, S < 0 and 1 − T < −S (S-driven game), then ρ = S 1−T > 1.
Secondly, notice that: where the last inequality holds since The proof is concluded by observing that: Therefore, inequalities (10) and (11) imply that: x * v ∈ (0, 1).
The stability of a steady state x * v ∈ (0, 1) can be studied by observing that the sign ofẋ v in equation (4) depends only on the term ∂φv(x) ∂xv − β v f v (x v ); according to equation (6), this term is linear with respect to x v , and hence the stability of x * v depends on the slope of this straight line. Specifically, the slope is equal to Since β v > 0, then we can have two cases: • the slope is positive when the game is T-driven. In this case, the point x * v acts as a repeller; • the slope is negative when the game is S-driven. In this case, the point x * v acts as an attractor.
These results are reported in Figure 2 of the main paper.

Global emergence of cooperation in the EGN equation with self-regulations
The global emergence of cooperation is reached when all members of a social network turn their strategies to cooperation. Therefore, the asymptotic stability of x * ALLC , as well as the instability of x * ALLD , have a fundamental role in this context. In order to study the stability of steady states x * ALLC and x * ALLD , we start by analyzing their linear stability. Moreover, an appropriate Lyapunov function is found, for proving that, under certain hypotheses, x * ALLC is also globally asymptotically stable. Finally, different hypotheses are used for identifying a Lyapunov function proving that x * ALLD is globally asymptotically stable.

ALLC
Recall that the spectrum of J(x * ) characterizes the linear stability of any steady state x * [S8].
The following results hold.
Theorem 3. If β v > k v ∀v ∈ V, then x * ALLC is asymptotically stable. Proof. As shown before, the Jacobian matrix evaluated for x * ALLC is diagonal. Then, the elements on the diagonal of the Jacobian matrix correspond to its eigenvalues and they are defined as follows: Since β v > k v ∀v ∈ V and T > 1, all the eigenvalues are negative. Thus, x * ALLC is asymptotically stable.
Theorem 4. If ∃v ∈ V : β v > k v , then x * ALLD is unstable. Proof. The eigenvalues of the Jacobian matrix relative to the steady state x * ALLD are: If the hypothesis of the theorem are fulfilled, since S < 0, then there is at least one positive eigenvalue, implying that x * ALLD is an unstable steady state. These results are summarized as follows: defection dominates over cooperation. Then, if the system does not present any internal feedback mechanism (i.e. β v = 0 ∀v ∈ V), the whole social network will converge to x * ALLD (cooperation vanishes). Anyway, using β v > k v for all the members of the population, x * ALLD is destabilized and x * ALLC becomes attractive.

ALLC
Theorems 3 and 4 prove that under suitable condition, x * ALLC is asymptotically stable and x * ALLD is unstable. Anyway, this is not sufficient to prove the global emergence of cooperation. Indeed, there can be some other steady states in Θ which may be also attractive. Nevertheless, a Lyapunov function [S9] for the steady state x * ALLC on the set x ∈ (0, 1] N can be found.
Adapting the approach presented in [S10, S11] to the SR-EGN equation, we consider the following function: Moreover, the time derivative of V (x) is defined as follows: Clearly,V (x * ALLC ) = 0.
It is easy to show that: and hence: Interestingly, the parameter ρ is equal to the ratio between the maximum and minimum of two "driving forces", namely the temptation to defect (T − 1) and the disadvantage of being defected (−S). Therefore, we can distinguish two cases: • for a T-driven game, since T − 1 > −S, then ψ = 1 − T , ξ = S and ρ = 1 − T S ; • for a S-driven game, since T − 1 < −S, then ψ = S, ξ = 1 − T and ρ = S 1 − T .
The following result holds.
Theorem 5. If β v > ρk v ∀v ∈ V, then V (x) is a Lyapunov function.
Proof. This is a direct consequence of inequality (18).

ALLD
Consider the following function for the steady state x * ALLD : The time derivative of V (x) is defined as follows: Clearly,V (x * ALLD ) = 0.
Proof. This is a direct consequence of inequality (22).

Game transitions
The previous results highlight the effectiveness of the self-regulating term in promoting cooperation. In this Section we show that this fact is due to transitions between different games, occurring when the system parameters are changed. We start from the generic formulation of the replicator equation for the two-strategy case, as proposed in [S10]: where σ C and σ D are the elements of the diagonal payoff matrix The sign of the parameters σ C and σ D rules the dynamics of equation (23) as follows: • σ C < 0 and σ D > 0: Prisoner's dilemma game (PD), for which defection is the only dominant strategy; • σ C > 0 and σ D > 0: Stag Hunt game (SH), where cooperation is the best response to cooperation, and defection is the best response to defection; • σ C < 0 and σ D < 0: Chicken game (CH), where cooperation is the best response to defection, and vice versa; • σ C > 0 and σ D < 0: Harmony game (HA), for which cooperation is the only dominant strategy.
It is worthwhile to notice that σ C < 0 models the temptation to defect, while σ D > 0 models the fear to be betrayed.
We can rewrite the SR-EGN equation according to the structure of equation (23): . Then, we have the following relationships: σ v C and σ v D characterize the equivalent game of player v. They depend on game parameters T and S, degree k v and self-regulating factor β v . Interestingly, they depend also on the strategy of the equivalent player x v . This fact implies that the values of σ v C and σ v D change not only by changing the system's parameters, but also over time, according to the dynamics of x v , thus producing dynamical game transitions. Hereafter, we report the conditions for which these transitions occur.
2.4.1 T-driven case: σ v C We establish the conditions for which the sign of σ v C is independent or dependent on the on the level of cooperation of his neighborhood x v . According to equation (24), we have We distinguish three cases.
. Using equation (14), since in this case ψ = 1 − T , we have that: Then Since T > 1, we get the following result: Using equation (15), since in this case ξ = S, we have that: Since ρ = 1−T S , then: By hypothesis, S < 0, and hence we get the following result: 3. The sign of σ v C depends on x v when β v ∈ kv ρ , k v . Noticing that and dividing by S < 0, since ρ = 1−T S , we get: Summarizing: Additionally, by the definition of σ v C we have that then for the T-driven case σ v C decreases as x v increases.

D
We establish the conditions for which the sign of σ v D is independent or dependent on the on the level of cooperation of his neighborhood x v . According to equation (24), we have We distinguish three cases. (15), since ξ = S, we have that Then, Since S < 0, we get the following result: Using equation (14), since ψ = 1 − T we have that: Then, since ρ = 1−T S , we have: Since, moreover, S < 0, we get the following result: and dividing the inequality by S < 0, since ρ = 1−T S , we get: Summarizing: Additionally, by the definition of σ v D we have that: for the T-driven case, and hence σ v D increases as x v increases. Table 1 reports a summary of the results on game transitions for the T-driven case.
Condition We establish the conditions for which the sign of σ v C is independent or dependent on the on the level of cooperation of his neighborhood x v . According to equation (24), we have We distinguish three cases.
. Using equation (14), since ψ = S, we have Therefore, since ρ = S 1−T , we have Moreover, since T > 1, we get the following result: . Using equation (15), since ξ = 1 − T , we have Then: By hypothesis, T > 1, and hence we get the following result: , the sign of σ v C depends on x v . Starting from and dividing the inequality by 1 − T < 0, since ρ = S 1−T , we get: Summarizing: Moreover, since: for the S-driven case, then σ v C increases as x v increases.

S-driven case: σ v D
We establish the conditions for which the sign of σ v D is independent or dependent on the level of cooperation of his neighborhood x v . According to equation (24), we have We distinguish three cases.
Using equation (15), since ξ = 1 − T , we have Therefore, recalling that ρ = S 1−T , we get: Since T > 1, we get the following result: Using equation (14), since ψ = S, we have that: Then: Since S < 0, we get the following result: 3. For β v ∈ kv ρ , k v , the sign of σ v D depends on x v . Starting from and dividing the inequality by 1 − T < 0, since ρ = S 1−T , we get: Summarizing: Additionally, by the definition of σ v D we have that: for the T-driven case, and hence σ v D decreases as x v increases. Table 2 reports a summary of the results on game transitions for the S-driven case.