The evolution of conditional moral assessment in indirect reciprocity

Indirect reciprocity is a major mechanism in the maintenance of cooperation among unrelated individuals. Indirect reciprocity leads to conditional cooperation according to social norms that discriminate the good (those who deserve to be rewarded with help) and the bad (those who should be punished by refusal of help). Despite intensive research, however, there is no definitive consensus on what social norms best promote cooperation through indirect reciprocity, and it remains unclear even how those who refuse to help the bad should be assessed. Here, we propose a new simple norm called “Staying” that prescribes abstaining from assessment. Under the Staying norm, the image of the person who makes the decision to give help stays the same as in the last assessment if the person on the receiving end has a bad image. In this case, the choice about whether or not to give help to the potential receiver does not affect the image of the potential giver. We analyze the Staying norm in terms of evolutionary game theory and demonstrate that Staying is most effective in establishing cooperation compared to the prevailing social norms, which rely on constant monitoring and unconditional assessment. The application of Staying suggests that the strict application of moral judgment is limited.

Unconditionally applying the Scoring norm, yet, raises a key question: Is it morally or socially acceptable to refuse to help someone with a bad image?This point is Scoring's Achilles' heel.By definition, when a discriminator refuses to help a potential opponent with bad image, the discriminator's image decisions become clouded.Thus, in the Scoring norm, bad image is contagious.Although helping can redress the discriminator's image, a bad image may cause the discriminator to undergo rejection by other discriminators.Even slight, involuntary errors can damage a discriminator's image and thus payoff.The Scoring norm therefore results in mutual defection of all players 14,15 ; this is the 'Scoring dilemma'.
To address this issue, a previous study developed social norms that distinguish between justified and unjustified defection by accounting for the recipient's image 1 .Precisely, in case a bad recipient is refused help, such refusal should not damage the donor's image (i.e.justified defection) 1,14,15 .Indeed, the top eight social norms, identified from 4,096 candidate strategies by systematic research (called the 'leading eight') 19,20 share a common relevant feature: if a good donor refuses to help a bad recipient, the donor is assigned a good image (see Extended Data Table 1).
Although the prior leading social norms are highly sophisticated, they are thus cognitively costly.Indeed, all of the leading eight rely on (i) the donor's last action and (ii) the recipient's last image (i.e.second-order social norms), and six of them also rely on (iii) the donor's last image (i.e.third-order social norms) 21 .Accounting for the image of both players would be rational in theory yet may overtax individuals in practice.Empirical studies on indirect reciprocity games reported experimental results that many participants' assessments have appeared to rely only on the player's actions, not also on the images a player views 22 .Is a simpler recipe than that provided by the leading eight available?
To tackle the Scoring dilemma, we formulate an opposite approach from prior paradigms by not applying higher-order social norms.We consider the effects of conditional observation on the opponent's image 23,24 (Fig. 1).The assessment system abstains from observation and thus also from assessment of whether or not refusal of help to the bad recipient is justified.Conditional assessment can prevent damage to the donor's image, as conditional assessment substantially generalizes the standard framework of indirect reciprocity to a meta-choice of {Assess, Reserve}.In the Assess case, the assessment is made according to the specific social norm, whereas in the Reserve case, the pre-existing image of the focal player is kept as-is.The Reserve option is applicable for a broad range of assessment systems.We apply a conditional assessment for Scoring, leading to a new social norm: to perform assessment as Scoring when a potential recipient has a good image, or otherwise, to abstain from assessment; we call this new norm Staying.
We model our paradigm on the giving game, in which the donor player has an opportunity to help the recipient player at a personal cost c > 0; if the donor helps, then the recipient earns benefit b > 0, with b > c.For simplicity, we assume that all discriminators share the same information about all personal images, provided by this unique assessment system.We then consider both (i) implementation error 14,26 , by which intentional help involuntarily fails with probability e 1 , and (ii) assessment error 21,27 , by which the assessment system involuntarily assesses a donor who should have been good as bad, or vice versa, with probability e 2 .
We compare Staying with the four most prevailing social norms 9,16,25 : Scoring [11][12][13] , Simple-standing 1,14,15 , Stern-judging 3,16 , and Shunning 5,17 (Table 1).Simple-standing and Sternjudging are the only two second-order social norms among the leading eight.When a donor refuses to help a bad recipient, Simple-standing and Stern-judging assess the donor as good, whereas Scoring and Shunning evaluate the donor as bad.Simple-standing, the most tolerant norm, assigns a good image to a person helping an individual evaluated as bad, and under Shunning, the most strict norm, a bad image is assigned to a donor who does not help a potential recipient evaluated as bad.In contrast with the four most prevailing social norms, no assessment is performed under Staying; the player's pre-existing image is simply preserved.
To study the evolutionary effects of these different social norms, we assume distinct strategies in very large populations: cooperators, defectors, and discriminators.Cooperators unconditionally give help; defectors unconditionally refuse help; and discriminators, irrespective of the social norm, give and refuse help to good and bad recipients, respectively.In this study, we investigate the replicator dynamics describing the tendency whereby strategies that result in above-average earnings grow in frequency 12,28 .We further assume that the image updating is much faster than the time scale of game interactions, so that we can study the replicator dynamics at a stationary state of the image system 28 .More methodical detail is provided in the Methods.
We first describe the Staying paradigm (Fig. 2a).Staying results in defectordiscriminator mixed equilibrium R with the fraction of discriminators z R = e 2 c/[λ(b -c)], where λ = (1 -e 1 )(1 -2e 2 ) describes social visibility, a probability that a donor's intention to cooperate precisely gets through an observer.Equilibrium R is unstable and its z R value indicates the minimal frequency of discriminators required to invade then take over a population of defectors.For a sufficiently small assessment error e 2 , equilibrium R appears in the state space, and depending the initial conditions, the ending population consists exclusively of either defectors or discriminators.As the assessment error e 2 decreases to 0, the basin of attraction for the node of discriminators expands, and equilibrium R approaches the node (see the Methods for detailed analysis).
The other four rules examined are Scoring, Simple-standing, Stern-judging, and Shunning (Fig. 2b-e).In these cases, for a sufficiently small cost-to-benefit ratio c/b , the evolutionary dynamics result in an unstable mixed equilibrium R of defectors and discriminators, with the fraction of discriminators z R = c/(λb).It follows that for e 2 < (1 − c/b), the basin of attraction for discriminators between the two strategies is wider for Staying than for the other four norms.Even when the assessment error e 2 is infinitesimally small, a fraction of discriminators less than c/b is incapable of invading successfully (see the Methods for detailed analysis).In striking contrast to this, Staying can thrive even by rare mutants taking over the population of defectors, irrespective of the cost-to-benefit ratio.
Most theory on the evolution of cooperation by indirect reciprocity is based on unconditional assessment.Evolutionary study on conditional assessment has started mainly by individual-based simulations 23,24 .In this paper, we fully analyse Staying, which is characterized by conditional assessment, and reveal that Staying is superior to the leading eight social norms of indirect reciprocity, particularly Simple-standing and Stern-judging.In mutual defection, within the population of defectors, either Simple-standing or Stern-judging leads individuals evaluated as bad to look good and then to exploit help from other discriminators.In contrast, under the Scoring or Shunning norm, individuals are evaluated as bad as a result of interacting with defectors, leading to rejection by other discriminators; this is a main reason why the four social norms: Scoring, Simple-standing, Stern-judging, and Shunning, are unlikely to emerge.In contrast, Staying can leave the images of good discriminators and bad defectors intact; this enables discriminators to channel their cooperation and subvert the stalemate of mutual defection even with a small perturbation of the population state (which is on the order of assessment errors).
The advantages of Staying are not limited to the emergence of effectiveness.A thirdorder social norm in the leading eight, called Strict-standing 19 (or L7 28 ), can provide the same image dynamics as Staying (see Extended Data Table 1).This fact indicates that Strict-standing is as evolutionarily stable as Staying; however, Strict-standing and Staying are conceptually different.Staying preserves the donor's image as a result of abstention from observation; in contrast, Strict-standing reassigns the previous image during execution of the observation.The leading eight, which share the obligatory nature of unconditional observation, are less advantageous than Staying in terms of cognitive costs or errors.
Refusing help for the bad is difficult to morally assess.The situation causes a variety of controversial opinions.Previous study of indirect reciprocity answered it strictly with a clear standard of justice.In contrast to this, Staying suspends the application of a scoring rule to the situation.Staying can be seen as a social norm applied loosely to some extent.As is known, the application of the law is difficult and judicial discretion sometimes has to work.The controversy has been continuing between two principles.One recognizes that a law is sometimes forced to be applied loosely, and tolerates judgment based on the judge's belief 29 .The other inhibits the judicial discretion and requires a judge to apply a law strictly 30 .We unveiled the excellence of Staying in forming social order.The findings suggest the limitation of strict application of rules.In the sense our study implies that evolutionary study of indirect reciprocity can contribute to further understanding of social norms and the law.What image donor looks?

Model
Indirect reciprocity in the giving game.The main model is based on the standard framework for the evolution of indirect reciprocity [19][20][21] .Using this framework, discriminators are given a strategy by an assessment rule (called 'social norm') combined with an action rule.We base indirect reciprocity on the giving game, which is a two-player donation game in which one player acts as a donor and the other as a recipient.The donor is given the opportunity to choose to help the recipient at personal cost.The recipient can only receive help from the donor, if any is forthcoming.In other words, for simplicity, it is assumed that there is no option to reject help.
In this giving game, action rules prescribe to discriminators who are acting as potential donors how to respond to a potential recipient in a specific situation depending on the last image scores of both donor and recipient.We consider a simple model in which each individual is endowed with a binary image score of 'good' or 'bad'.The action rule we apply to discriminators is to give help to a good recipient or to refuse help to a bad one, unless otherwise specified.After observing every interaction in the giving game, discriminators assign the donor's image by following the specific social norm, which is a function of (i) the donor's last action, (ii) the recipient's last image, and (iii) the donor's last image.When depending only on (i), the rule is called 'first-order,' when depending on both (i) and (ii), it is called 'second-order,' and when depending on (i), (ii), and (iii), it is called 'third-order' 21 .

Conditional assessment.
In the present study, we examine conditional assessment 23,24 , which specifies a meta-choice of 'Assess' or 'Reserve.'When choosing Assess, discriminators assign either a good or a bad image to a potential donor; when choosing Reserve, discriminators abstain from assessment.Based on this concept of conditional assessment, we introduce 'Staying,' which is a new social norm specifying that when a potential recipient is good, the donor's image should be assessed, as is done under Scoring, and otherwise, the donor's image should be reserved (that is, left unchanged).
To understand Staying, we also comparatively explore all first-and second-order social norms that take into account the recipient's last image and the donor's last action 9,16,25 .In particular, we focus on the four most prevailing social norms: Scoring [11][12][13] , Simple-standing 1,14,15 , Stern-judging 3,16 , and Shunning 5,17 .Scoring is the best-known first-order social norm, and the other three are second-order social norms.Table 1 and Extended Data Table S1 provide full details of these social norms.When assessing use of Staying and the four norms, the consensus is that if a recipient has a good image, a good image should be assigned to those who helped and a bad image to those who did not help, the same as in the simplest case, Scoring.
Observation, information, and errors.In the model, we consider both public information and indirect observation.We assume indirect observation, by which players with the same social norm adopt and share equally the same image of a focal player, which has been provided by a representative observer.For simplicity, we also assume perfect information, in which the probability that players know the image of a potential recipient is 100%.In addition, we consider both implementation error 14,26 and assessment error 21,27 .We denote by e 1 the probability that a player who has intended to help involuntarily fails to do so.A player who has intended not to help, but, is not the case: the intentional non-help is executed with no error.That is, the implementation error is unilateral.We denote by e 2 the probability that an observer involuntarily assesses a donor who should have been good as bad, or a donor who should have been bad as good.Thus, it is natural to assume that 0 < e 1 < 1 and 0 < e 2 < 1/2.
Evolutionary dynamics.To study the evolutionary dynamics of discriminators, we respect a continuous-entry model: an individual's birth and death sometimes happen, and when they do, this changes the strategy distribution in the population 12,27 .We consider that in an individual's lifetime, that individual infinitely plays the one-round giving game with different opponents.We consider an infinitely large population to examine replicator dynamics 28 , which, in general, are described as dx S /dt = x S (P S -P), where x S denotes the relative frequency of strategy S; P S is the expected payoff for strategy S, given by the limit in the mean of the payoff per round for the strategy; and P is the average payoff over the population, given by ∑x S P S .We note that each homogeneous state with x S = 1 is a trivial equilibrium of the replicator dynamics.First, we examine three strategies: discriminators [S = Z], cooperators [X], and defectors [Y].Cooperators unconditionally intend to help a potential recipient, and in contrast, defectors unconditionally intend not to help a potential recipient.
Image dynamics.To describe the dynamics of image scores, both good and bad, we use g S to denote the frequency of individuals with a good image among colleagues adopting the same strategy S, and we use g to denote the average fraction of individuals with good images over the population; thus, g = x g X + y g Y + z g Z .In addition, we use g S,I to denote the probability that a good image is assigned to a potential donor who adopts strategy S and also faces a potential recipient with an image score I = good [G] or bad [B].The population size is very large, so we assume that the composition of the population does not change between consecutive, one-round giving games 12,28 .Thus, the frequencies of good players satisfy (1)

Analyses
Staying norm.We begin by analyzing the Staying norm, in which case equation ( 1) is described as (2)   where ε = (1 -e 1 )(1 -e 2 ) + e 1 e 2 .In equation ( 2), the first term, εg, in the sum for g X or g Z describes the probability that a cooperator or discriminator who faces a good recipient (probability g) is assigned a good image by giving help with no errors (probability (1 -e 1 )(1e 2 )) or by failing to give help with both the errors (probability e 1 e 2 ).For g Y , the term e 2 g in the sum expresses the probability that a defector who faces a good recipient (probability g) is assigned a good image through assessment errors (probability e 2 ).The second term in the sum, g S (1 -g), describes the probability that a donor who faces a bad recipient (probability 1 -g) is assigned a good image.In this case, according to the definition of Staying, the probability of finding a good discriminator should remain unchanged, as g S .Solving these equations leads to g Y = e 2 , g X = g Z = ε, and thus, g = e 2 y + ε(1 -y).
Then, the expected payoffs are given by (3) This yields For x = 0, that is, on edge YZ in Fig. 2a, we have This results in z = z R such that it satisfies P Z -P Y = 0, leading to Considering that 0 < e 2 < 1/2 and b > c, a boundary equilibrium R with (x, y, z) = (0, 1z R , z R ) enters the edge YZ for the typical parameter settings (b > c, and e 1 and e 2 are sufficiently small).Equilibrium R is repelling along the edge and divides the edge into the basins of attraction for the homogeneous states of defectors (y = 1) and discriminators (z = 1).
Next, we turn to the payoff difference between cooperators and discriminators, Considering g X = g Z = ε, it follows that where the average frequency of the good g is less than 1 when both the errors are non-zero.This means that for the typical parameter settings, cooperators are dominated by discriminators and defectors, thus leading the population to converge to edge YZ (x = 0).It follows that the homogeneous state of discriminators (z = 1) is evolutionarily stable for sufficiently small errors (see Fig. 2a) and becomes globally stable when there is no assessment error, e 2 = 0.
Second-order social norms.We then focus on the 16 second-order social norms (except for one that prescribes unconditional defection given by BBBB (to unconditionally assign bad using Table 1).Compared to the case of the Staying norm, any of these 16 norms is less likely to invade a population of defectors.We note that since discriminators and defectors intend to refuse to help a bad recipient, g Y,B = g Z,B holds.To analyse the replicator equations, we apply the expected payoffs, as in equation (3).For any of the 16 norms, equation (4) becomes We consider small but non-zero errors, which yield 0 < g < 1.When g Z,G -g Y,G ≤ 0, it follows that P Z -P Y < 0, and thus discriminators are dominated by defectors.When g Z,G -g Y,G > 0, this can result in, if any, a boundary equilibrium R with z = z R , such that it satisfies equation (10) implies that for all cases of the 16 second-order social norms, the threshold frequency for discriminators to successfully invade a population of defectors is at least the costto-benefit ratio c/b.
Scoring, Simple-standing, Stern-judging, and Shunning.We compare the effectiveness of Staying with the results of the most prevailing social norms.We specifically check the global dynamics of Scoring, Simple-standing, Stern-judging, and Shunning.First, we note that by definition the conditional probability that a donor is assessed as good when a potential recipient is good (that is, the first term in the sum in equation ( 1)) is the same as it is in equation ( 2) of Staying.The difference is in the second term in the sum.
For Scoring, we obtain Since Scoring is a first-order social norm that depends only on what a donor did, the degrees of goodness in cooperators and defectors, g X and g Y , are independent of the recipient's degree of goodness g.A discriminator is assessed as good for a good recipient, with probability ε, or for a bad recipient, with the probability that he intentionally defects yet with assessment error e 2 .
Next, for Simple-standing, Simple-standing is the most tolerant norm, which is to assign a good image to a donor, irrespective of his/her actions to a bad recipient.Thus, the second term in the sum is the same as (1 -e 2 )(1 -g) over g X , g Y , and g Z , in which case the donor is assessed as good only when no assessment error occurs.
Then, for Stern-judging, Stern-judging assigns a good image to those who refuse help for a bad recipient and a bad image to those who give help for a bad recipient.This leads to the second term in the sum for g X .When a recipient is bad, unintentionally refusing help with no assessment error or intentionally giving help with assessment errors are both assessed as good.This conditional probability is (1 -e 1 )e 2 + e 1 (1 -e 2 ) = 1 -ε.
Finally, for Shunning, Shunning is the strictest case, in which a bad image is assigned to a donor irrespective of his/her actions toward a bad recipient.Thus, the second term in the sum is the same as e 2 (1 -g) over g X , g Y , and g Z , in which case the donor is assessed as good only when assessment errors occur.
Substituting equation ( 11) (12), or (14) into equation ( 4), we obtain Considering g > 0, this results in a unique boundary equilibrium R with z = z R , such that it is a repelling point and satisfies This leads to that when (ε -e 2 )b > c, the unique equilibrium enters edge ZY (x = 0).Comparing equations ( 6) and ( 16), it is obvious that the basin of attraction for node Z (z = 1) is wider under Staying than under the other four cases.Indeed, as the degrees of error, e 1 and e 2 , move toward 0, the fraction necessary for discriminators to emerge, z R in equations ( 6) and ( 16), converges to 0 and c/b, respectively.Thus, under Scoring, Simple-standing, Stern-judging, or Shunning, a sufficiently small cost-to-benefit ratio c/b is required for rare mutants of discriminators to successfully invade a population of defectors.In striking contrast to this, under Staying, rare mutants of discriminators can invade as long as assessment errors e 2 are very small.
Next, we turn to the payoff difference between cooperators and discriminators in equation (7).For Scoring, substituting equation (11) yields Hence, there is a line consisting of fixed points with the same z-coordinate given by equation (16).This line connects boundary fixed points Q on edge ZX (y = 0) and R on edge ZY (x = 0).We note that in contrast to R, Q is attracting along edge ZX.In particular, node Z is a saddle point, and node Y is a unique equilibrium that is asymptotically stable (see Fig. 2b).More details of these global dynamics can be explored by applying analogous arguments from refs. 5 and 12.
For Simple-standing or Shunning, substituting equation ( 12) or ( 14) yields the same results as in equation ( 8): P Z − P X = (1− e 1 )c(1− g) ≥ 0. For Stern-judging, substituting equation (13) yields P Z − P X = (1− e 1 )[(ε − e 2 )bz + c](1− g) ≥ 0. Thus, for the typical parameter settings, similarly to Staying, cooperators are dominated by discriminators and defectors, leading the population to converge to edge YZ (x = 0).Thus, the global dynamics are qualitatively the same as those for Staying.However, the four social norms and the Staying norm differ quantitatively in the position of the repeller R, z R (see Fig. 2c-e).Extended Data Table 1.Social norms: Staying, Scoring, Shunning, and the leading eight.'G' and 'B' describe good image and bad image, respectively.'R' means that the focal player's image remains unchanged.With Scoring [11][12][13] , whether to help or not determines the donor's image.When a potential recipient has a good image, the leading eight strategies 19,20 all have the same assessment as Scoring.Coding of the leading eight, L1 to L8, is the same as assigned in ref.
28. L1 is the original 'Standing' 1,14,15 , which is viewed as a third-order social norm.Only rules L1 and L2 have different action rules, which prescribe cooperation when both donor and recipient have bad images.L1 (Standing) and L3 (Simple-standing) differ in that part of the action rule and also in the assessment of donors who refuse to help a bad recipient.Shunning 5,17 and Scoring do not belong to the leading eight, as Shunning could not attain a sufficiently high degree of cooperation at the equilibrium state 25 .

Figure 1 .
Figure 1.Conditional assessment in giving games.In the Assess and Reserve cases, the observer does and does not assess the donor's image score, respectively.

Figure 2 .
Figure 2. Evolution of indirect reciprocity with different social norms.The triangles describe a simplex of the state space {(x, y, z): x + y + z = 1} where x, y, z ≥ 0 denote the frequencies of cooperators, defectors, and discriminators, respectively.Each node (X, Y, or Z: x, y, or z = 1) of the triangle corresponds to the homogeneous state of each specific strategy.a, Under Staying, discriminators always are better off than cooperators.Thus, cooperators will vanish and then the population will eventually converge to either node Z or Y. b, Under Scoring, a continuum of equilibria connects boundary attractor Q and repeller R. The population drifts along the continuum and moves close to R, eventually attaining node Y. c-e, Under Simple-standing, Stern-judging, or Shunning, the dynamics are qualitatively similar to those in a.The basin of attraction for node Z is wider in a than in b-e.Parameters: c = 1, b = 1.5, e 1 = e 2 = .01.R corresponds approximately to z R = .02in a or = .66 in b-e.

Table 1 . How social norms make moral assessments in giving games
. 'G' and 'B' describe good and bad image, respectively.'R' means that the player's image remains unchanged.