## Abstract

Evolutionary games of cyclic competitions have been extensively studied to gain insights into one of the most fundamental phenomena in nature: biodiversity that seems to be excluded by the principle of natural selection. The Rock-Paper-Scissors (RPS) game of three species and its extensions [e.g., the Rock-Paper-Scissors-Lizard-Spock (RPSLS) game] are paradigmatic models in this field. In all previous studies, the intrinsic symmetry associated with cyclic competitions imposes a limitation on the resulting coexistence states, leading to only selective types of such states. We investigate the effect of nonuniform intraspecific competitions on coexistence and find that a wider spectrum of coexistence states can emerge and persist. This surprising finding is substantiated using three classes of cyclic game models through stability analysis, Monte Carlo simulations and continuous spatiotemporal dynamical evolution from partial differential equations. Our finding indicates that intraspecific competitions or alternative symmetry-breaking mechanisms can promote biodiversity to a broader extent than previously thought.

## Introduction

Fundamental to species coexistence and biodiversity are competitions. In ecosystems there are two types of competitions: interspecific (competitions among individuals from different species) and intraspecific (competitions among individuals in the same species), where both types can either promote or hinder species coexistence^{1, 2}. The purpose of this paper is to demonstrate, through a systematic study of several models of cyclic evolutionary game, that intraspecific competitions can induce unusual states of coexistence that have not been reported previously. Intraspecific competitions may thus be more fundamental to biodiversity than previously thought.

A natural and typical mechanism for interspecific competitions is predator-prey interaction, while intraspecific competitions occur because individuals in the same species compete for essential life-sustaining resources such as food, water, light, and opposite sex. A well known type of intraspecific competitions is cannibalism or intraspecific predation^{3,4,5,6,7,8,9,10}, which can occur with high likelihood especially when there is lack of sufficient resources. Such competitions can also occur when individuals fight each other for mating opportunities, which were observed for side-blotched lizards in California^{11}. In the past decade there were studies of the effect of intraspecific competitions on biodiversity^{12,13,14,15,16}, with results such as the experimental finding that the competitions tend to drive disruptive selection^{12}, enhanced host survival through intraspecific competition between co-infecting parasite strains^{13}, and directional selection of certain fish species^{15, 16}.

To understand coexistence and biodiversity, the approach of mathematical modeling has proven to be useful, providing fundamental insights into the various mechanisms underlying species coexistence at both the macroscopic, population^{17,18,19} and the microscopic, individual competition levels^{20,21,22,23}. Historically, the theoretical approach began with mathematically modeling growth and competitions through dynamical equations at the population level^{17,18,19}. In the past fifteen years or so, microscopic models at the level of individual interactions were extensively studied based on the mathematical paradigm of evolutionary games^{24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53}. A milestone result^{22, 26} is the elucidation of the role of species mobility in coexistence, which traditionally had been regarded as detrimental to coexistence. In particular, utilizing the framework of three cyclic competing species, the rock-paper-scissors (RPS) model, the authors^{22, 26} demonstrated robust coexistence in the weak mobility regime, providing a resolution to the paradox that macroscopic models exclude coexistence of mobile species but, in realistic ecological processes ranging from bacteria run and tumble to animal migration, coexistence is ubiquitous. The basic dynamical structure supporting the coexistence of mobile species was identified to be spiral wave patterns that emerge and evolve with time in the physical space^{22}, which are robust against noise^{26}. Other issues that have been investigated include the stability of spatial patterns^{24, 29, 39, 52, 54,55,56,57}, the role of conservation laws^{27, 44}, pairwise and group-level interactions^{53}, basins of the coexistence states^{30, 31}, and the effects of a wide array of behaviors/quantities on coexistence such as long range migration^{38, 45}, uniform intraspecific competition^{32}, local habitat suitability^{46}, multi-strategy competition^{49}, inhomogeneous reaction rates^{28, 33, 40, 47, 48}, epidemic spreading^{34}, and spatial extent and population size^{25, 36, 41}. While most of these works were for three cyclic competing species, there were studies extending the model to arbitrary number of species^{42, 43, 53} and addressing the role of competition at the mesoscopic (i.e., group) level in coexistence^{52}. Here extinction means no coexistence and only one surviving species in the system.

In previous works, a well established result is that only a certain type of coexistence states can exist. For example, in a system of three cyclically competing species (RPS game), the only coexistence state is one that involves all three species: it is not possible for a state of two coexisting species^{22, 26} to be stable. Likewise, in a cyclic system of five species, there are two distinct coexistence states with either three or five species - there cannot be coexistence states with two or four species^{52}. The reason for the selective coexistence states lies in the intrinsic symmetry of the system, as the competing species are at the same footing. Intuitively, symmetry breaking can possibly lead to more diverse coexistence states. The purpose of this paper is to demonstrate and establish that realistic intraspecific competitions that generically break the intrinsic symmetry of cyclic competitions, can lead to *all possible coexisting states*. In particular, in the real world the degree of intraspecific competition in general depends on the particular species. To describe symmetry-breaking in a concrete way, we use a parameter, e.g., the rate of intraspecific competition. The rate can then be nonuniform for different species, which can have a significant effect on the game dynamics^{33, 58,59,60,61}. At the microscopic level, the species are thus no longer on the equal footing - effectively introducing symmetry breaking into the system. As a result, coexisting states without a global symmetry at the macroscopic level can arise. We establish this striking result through extensive computations and mathematical analyses of cyclically competing systems with different number of species.

## Results

### Models and mathematical representations

To investigate the dynamical evolutions of cyclically competing species in the presence of intraspecific competitions, we study three game systems: the classic RPS model, the extended RPS (ERPS) model of five species, and the rock-paper-scissors-lizard-spock (RPSLS) model. The dynamical interacting rules of the three models are shown in Fig. 1. At the microscopic level, each model can be described by evolutionary dynamics on a lattice system, while at the macroscopic level the model can be approximated by a set of ordinary differential equations (ODEs). In addition, partial differential equations (PDEs) can be used to study the spatiotemporal evolution of the population densities.

For convenience, we employ a square lattice with periodic boundary conditions to host a cyclic game system at the microscopic level, where an individual occupies a lattice site. Given the interspecific competition rate *σ*, species reproduction rate *μ*, intraspecific competition rate *p*, and movement rate *ε*, interactions among the individuals can be described by

where *X* and *Y* represent two cyclically interacting species, \(\varnothing \) denotes an empty site, and *Z* stands for either an individual from the same species or an empty site, and the quantity *ε* is defined to be *ε* ≡ 2*MN* with *M* being the individual mobility parameter and *N* being the total number of individuals. In the evolutionary game system, each site can be occupied by an individual from one of the species or left empty. At each time step, a randomly selected individual can compete, reproduce, or move into one of its nearest neighbors at random, provided that the corresponding interaction rule as specified in Eq. (1) is satisfied.

Under the mean field approximation where the population size is large: *N* → ∞, the system can be described by the rate equations governing the time evolution of the species densities (ODEs). Let *a*(*t*), *b*(*t*), and *c*(*t*) be the densities of the three species *A*, *B*, and *C* at time *t*, respectively. The RPS game model can be described by

where *ρ*(*t*) ≡ *a*(*t*) + *b*(*t*) + *c*(*t*) is the total density of the three species. In each equation, the first and second terms describe reproduction of a species at rate *μ* and a decrease in the species density by invasion at rate *σ*, respectively. The third term represents the decrease in the density due to intraspecific competition of rates *p*
_{
a
}, *p*
_{
b
}, or *p*
_{
c
} for species *A*, *B* and *C*, respectively, where the factor 1/2 associated with the rate accounts for the two-way interactions between two individuals in the same species. (ODE models for ERPS and RPSLS games are provided in Supplementary Information).

The ODE model ignores the effects of the spatial domain in which the interactions occur. To take into account the spatial dimension, a PDE model can be derived. In particular, consider a square domain of linear size *L* with periodic boundary conditions, where *L*
^{2} = *N*. We normalize the domain to the unit square so that the distance between two nearest neighbors is \(\delta x=1/\sqrt{N}=(1/L)\). The densities of subpopulations *A*, *B*, and *C* at time *t* and site **x** = (*x*
_{1}, *x*
_{2}) are denoted as *a*(**x**, *t*), *b*(**x**, *t*), and *c*(**x**, *t*), respectively. For interspecific and intraspecific competitions as well as reproduction, the dynamical equations for these quantities only involve neighbors located at **x** ± *δx* · **e**
_{
i
}, where {**e**
_{
i
}}_{
i=1,2} are the base vectors of the lattice. For the RPS game, we obtain the following evolutionary equations:

where \(M=\frac{\varepsilon }{2}{(1/\sqrt{N})}^{2}\). (The corresponding PDEs for ERPS and RPSLS games are provided in Supplementary Information).

To be concrete, in this paper we fix the rates of interspecific competition and reproduction to be *σ* = 1 and *μ* = 1, respectively.

### Coexistence states in the RPS system

Figure 2 illustrates the possible coexistence states in the RPS system, where the middle column represents a state in which two species coexist - a previously unknown coexistence state. Specifically, a stability analysis of the corresponding ODE system with respect to systematic changes in a intraspecific competition parameter, say *p*
_{
a
}, with fixed *p*
_{
b
} = 1 and *p*
_{
c
} = 0.5, reveals the following phenomena. Firstly, for weak intraspecific competition, i.e., 0 < *p*
_{
a
} ≤ 1.5, coexistence is physically not possible due to the existence of a heteroclinic cycle, where only one species can survive. Secondly, for moderate intraspecific competition, i.e., 1.5 < *p*
_{
a
} ≤ 4, for each species reproduction and death are counter-balanced, so all three species can coexist, as indicated by the left column in Fig. 2. Thirdly, for stronger intraspecific competition, i.e., *p*
_{
a
} > 4, the new coexistence state of two species emerges, as indicated by the middle column in Fig. 2. The striking consequence is that, in this case, the nature of the interaction between the two surviving species has become the predator-prey type as a result of the balance between prey’s reproduction and predator’s death from competitions among its own individuals. In the limit of infinitely strong intraspecific competition, i.e., *p*
_{
a
} → ∞, the predator becomes extinct, leaving the prey as the only surviving species, as shown in the right column of Fig. 2. Figure 3(a) presents a bifurcation diagram of these behaviors (see Methods for more details).

The spatial patterns associated with distinct coexisting states can be revealed by numerical solutions of the lattice and PDE models, Eqs (1) and (3), respectively. Figure 2(c–e) show, for *M* = 10^{−3}, nine representative snapshots of the spatial patterns in the long time regime for different values of *p*
_{
a
}, where the three species are denoted by red, blue, and yellow, respectively. Specifically, the three columns (from left to right) are associated with *p*
_{
a
} = 2.5, 5.2 and 100, respectively, the two top rows [Fig. 2(c,d)] are from lattice models of size *L* = 100 × 100 and 500 × 500, respectively, and the bottom row is from the PDE model. We see that, all possible coexistence states can occur and, as the intraspecific competition parameter is increased, physical coexistence states for which the number of survival species is in the order 1 → 3 → 2 → 1 emerge, which is consistent with the results from the bifurcation analysis of the ODE model. Further, we find that the thresholds between different stable phases from lattice simulations are consistent with those from the bifurcation analysis of the ODE model, as supported by a calculation of the survival probability *P*
_{
surv
} from 100 sampled parameter values (see Fig. S9 in Supplementary Information). The middle column shows the patterns of the coexistence state of two species in the spatial domain, which has not been observed previously in the cyclic game of three species. Previous studies also revealed that the coexistence state of three species is supported by spiral waves in the domain^{22, 26}. In general, spiral waves can be stable, unstable or convectively unstable even in the absence of intraspecific competitions^{54,55,56}. However, with intraspecific competition induced symmetry breaking at the microscopic level, various coexistence states can arise but no spiral wave patterns can form.

### Coexistence states in the ERPS system

There are five competing species in the ERPS system, as shown in Fig. 1(b). To be concrete, we consider a fixed set of parameter values: *p*
_{
a
} = 1.9, *p*
_{
b
} = 2, *p*
_{
d
} = 1.3, and *p*
_{
e
} = 0.7. A bifurcation analysis of the underlying ODE model (Supplementary Information) leads to the following results on the role of intraspecific competition in coexistence. Firstly, for weak intraspecific competition, i.e., 0 ≤ *p*
_{
c
} < 2.6, mathematically all species can coexist but small perturbations can lead to extinction, physically excluding coexistence. Secondly, for a moderate level of intraspecific competition, i.e., 2.6 ≤ *p*
_{
c
} < 5.6, all species can physically coexist. Thirdly, in the regime of strong intraspecific competition, i.e., 5.6 < *p*
_{
c
} < 5.8, coexistence states of four species can emerge, breaking the cyclic symmetry. Fourthly, for stronger intraspecific competition, i.e., *p*
_{
c
} ≥ 5.8, three species can coexist, exhibiting a predator-prey relation. For *p*
_{
c
} → ∞, the predator populations diminish and only two species can coexist. Figure 3(b) presents a bifurcation diagram of these behaviors.

The emergence of all possible coexistence states, especially the unusual states of three and four coexisting species, can be substantiated by resorting to the lattice and PDE models (see Supplementary Information for the PDE model of the ERPS system). To be concrete, we set the simulation parameters to be *p*
_{
a
} = 1.9, *p*
_{
b
} = 2, *p*
_{
d
} = 1.3, *p*
_{
e
} = 0.7, and *M* = 10^{−3}. Figure 4(c–e) show fifteen snapshots of the spatial patterns in the long time regime from the lattice and PDE models for a number of different values of *p*
_{
c
}, where the five species are denoted by red, blue, green, yellow and pink, respectively. The panels are organized into rows and columns, where columns 1–4 (from left to right) are associated with *p*
_{
c
} = 3.3, 5.65, 6.5 and 100, and column 5 is for *p*
_{
c
} = 100 but with a different parameter setting (*p*
_{
a
} = 1.9, *p*
_{
b
} = 2, *p*
_{
d
} = 0.01, and *p*
_{
e
} = 0.7). The two top rows [Fig. 4(c,d)] are the results from lattices of size *L* = 100 × 100, 500 × 500, respectively, and the bottom row represents the results from the PDE model. These results are consistent with those from the stability analysis of the corresponding ODE system in that coexistence states of all possible numbers of species can occur. The most striking phenomenon is the coexistence of four and three species (corresponding to the second and third columns, respectively), which have not been reported previously for the ERPS system. Similar to the RPS system, with intraspecific competitions the coexistence states are not supported by spiral wave patterns in the spatial domain.

### Coexistence states in the RPSLS system

For the five-species RPSLS system, for a representative set of fixed parameter values, e.g., *p*
_{
a
} = 0.3, *p*
_{
b
} = 1.1, *p*
_{
c
} = 2.5, and *p*
_{
d
} = 0.7, but varying *p*
_{
e
}, a stability analysis reveals the following phenomena. Firstly, in the regime of weak intraspecific competition, i.e., 0 ≤ *p*
_{
e
} < 1.8, coexistence is physically not possible. Secondly, for 1.8 ≤ *p*
_{
e
} < 2.4, coexistence of all five species is physically realizable and robust. Thirdly, for 2.4 ≤ *p*
_{
e
} < 8, coexistence of five species is no longer possible. Instead, coexistence states of four species can emerge, breaking the cyclic symmetry. Fourthly, for *p*
_{
e
} ≥ 8, only three species, which do not exhibit a sub-cyclic structure, can coexist and, for *p*
_{
e
} → ∞, two of the three species can survive in a predator-prey relation. Figure 3(c) presents a bifurcation diagram of these behaviors. Detailed simulations from the lattice and PDE (Supplementary Information) models give consistent results. For example, for *p*
_{
a
} = 0.3, *p*
_{
b
} = 1.1, *p*
_{
c
} = 2.5, and *p*
_{
d
} = 0.7, various coexistence states can emerge for different values of *p*
_{
e
}, as shown by the spatial patterns in Fig. 5(c–e). The results shown in the second and fourth columns, which indicate the coexistence states of four and two species, are surprising as such states have not been uncovered previously in the study of RPSLS system^{52}. We also find that, associated with the coexistence of three species [c.f., Fig. 5(a)], there is absence of any cyclic interaction structure among the three survived species. This is in fact a *non*-*sub*-*cyclic interacting structure* which, to our knowledge, has not been reported previously in the studies of interspecific interaction models^{52, 62}. For the ten distinct cases of three survived species among five, the coexistence states with such a non-sub-cyclic structure are stable. Interestingly, the conventional coexistence states with a sub-cyclic structure among the three surviving species are unstable (See Supplementary Information for more details). It is also apparent that, dynamically, the coexistence states are not supported by spiral wave patterns.

### Role of intraspecific competition in promoting diverse coexistence states - a qualitative understanding

To understand the effect of intraspecific competition on coexistence qualitatively, we investigate the population change for each species *i* as a result of interspecific and intraspecific competitions as well as reproduction at time *t*, denoted as *C*
_{
i
}(*t*), *I*
_{
i
}(*t*) and *R*
_{
i
}(*t*) (scaled by the lattice size *N*), respectively. From the fact that the populations are determined by the balance between reproduction and competition, we find it useful to define two quantities: *H*
_{
i
}(*t*) = *R*
_{
i
}(*t*) − *C*
_{
i
}(*t*) and *S*
_{
i
}(*t*) = *R*
_{
i
}(*t*) − *C*
_{
i
}(*t*) − *I*
_{
i
}(*t*). We then have *H*
_{
i
}(*t*) − *S*
_{
i
}(*t*) = *I*
_{
i
}(*t*) ≥ 0 and the population at time *t* can be written as \({P}_{i}(t)={P}_{i}^{0}+{\int }_{0}^{t}\,{S}_{i}(k)dk\) for a given initial population \({P}_{i}^{0}\). For *S*
_{
i
}(*t*) > 0, species *i* can survive as its population tends to increase with time, which will cause a decrease in the population of the next species in the cycle (prey of species *i*) as a result of interspecific competitions, leading to possible extinction. For *S*
_{
i
}(*t*) < 0, the population of species *i* decreases and possibly becomes extinct. These simple observations imply that, in order for multiple species to survive, *S*
_{
i
}(*t*) must fluctuate about zero. Equivalently, for surviving species *i*, its population fluctuations can be described by a normal diffusion process: \(\langle {S}_{i}^{2}(t)\rangle \sim t\).

For the case of the coexistence of three species in the RPS system, Fig. 6(a) shows the time evolution of *H*
_{
i
}(*t*) and *S*
_{
i
}(*t*). We observe that *H*
_{
i
}(*t*) > 0 but *S*
_{
i
}(*t*) fluctuates about zero, which indicates that, without intraspecific competitions, the corresponding populations tend to increase with time due to *H*
_{
i
}(*t*) > 0. However, intraspecific competition can reduce the populations, because *S*
_{
i
} ≈ 0. This implies that a possible balance between the increasing and decreasing trends can be attained, stabilizing the populations. For the case of species extinction [Fig. 6(b,c)], we find that *H*
_{
i
} can be positive initially but becomes negative due to the decrease in the reproduction as a result of strong intraspecific competitions.

For the cyclic system of three species (*A* → *B* → *C* → *A*), we note that each pair of species constitutes a predator-prey system. To examine the process leading to the coexistence of two species, we assume a decrease in the population of predator (*A*) from its initial population due to strong intraspecific competitions, which will immediately reduce the interspecific competition with its prey (*B*). As a result, the prey population can increase due to reproduction. The change in the population of species (*B*) will in turn enhance the interspecific competition with its prey (*C*), leading to a decrease in its population and possibly to its extinction. This chain of interactions indicates that intraspecific competition of a species can have a dramatic effect on the populations of other interacting species, potentially generating a distinct equilibrium state. To provide theoretical support, we identify the critical level of intraspecific competition leading to qualitative changes in the species populations through a mathematical analysis of the existence and the stability conditions of the equilibrium states listed in Table 1 (see Methods and Supplementary Information).

In the three species system, the coexistence state of two species is characteristic of that of a predator-prey system. Intuitively, the classic Lotka-Volterra model can be used to describe such a situation, where the populations can exhibit periodic oscillations with time (or a limit cycle in the phase space). We find that, however, in the presence of intraspecific competitions, the populations of the coexisting species do not exhibit periodic oscillations but steady states, as shown in Fig. 2(b). Thus, intraspecific competitions can either enhance coexistence and induce diverse coexistence states, or break the equilibrium and lead to extinction.

## Discussion

In the classic three- or five-species cyclic games, the species are on the equal footing in the sense that no particular species is superior or inferior to any other species. This intrinsic symmetry imposes a restriction on the survival or coexistence states of the system. For example, for the classic RPS game, either only one species survives as a result of interspecific interactions in which the end result is the disappearance of such competitions, or all three species can sustain and coexist^{20,21,22,23}. For this reason the coexistence state of two species has never been reported before in the literature. Likewise, for the five-species RPSLS game, the known coexistence states contain a sole surviving species, three species, or all five species^{52}. It is conceivable that, when the intrinsic symmetry among the competing species is broken, more diverse types of coexistence states can emerge.

Nonuniform intraspecific competitions represent one mechanism that can break the symmetry: they put the species on unequal footings. For example, if competitions among individuals in a species are stronger than those in another species, the former requires more resources to reproduce in order to survive and is therefore effectively “inferior” to the latter. As a result, a predator-prey type of behavior can emerge on the macroscopic scale where intraspecific competitions associated with the predator are stronger than those with the prey to reach a new equilibrium state in which the two surviving species are no longer on equal footing. Mathematically, it is not necessary for such a state to possess any intrinsic symmetry. The consequence is that coexistence states involving any number of species (insofar the number is less than or equal to the total number of species in the system) can arise.

The contributions of this paper are mathematical analyses, physical understanding, and comprehensive numerical tests that symmetry breaking can lead to more diverse coexistence states than previously reported. To accomplish this goal, we systematically studied three classes of cyclic game systems with either three or five species, subject to nonuniform, species dependent, intraspecific competitions. For each system, we focus on the asymptotic dynamical states (i.e., the coexistence states) of the system utilizing three approaches: ODE based stability analysis, microscopic Monte-Carlo simulation of the lattice model, and spatiotemporal evolution of the corresponding PDE model. A detailed bifurcation analysis of the ODE model reveals that, as the strength of the intraspecific competition for one species is systematically varied (while keeping the intraspecific competition strengths for the other species fixed), coexistence states of any number of species can arise in different parameter regimes. The occurrence of these states is further supported by both lattice and PDE simulations. A distinct feature is that the coexistence states here are not associated with any spiral wave patterns that were previously established as the underlying spatiotemporal dynamical structure supporting the coexistence of multiple species in cyclic game systems^{22, 26}. Our findings suggest that symmetric breaking with nonuniform intraspecific competitions across the species may be more beneficial to biodiversity.

## Methods

### Numerical methods

All PDE models are solved by the standard spectral method and lattice simulations are of the Monte Carlo type. Lattice sizes vary from 100 × 100 to 500 × 500.

### Stability analysis of RPS game

In the classic RPS game in the absence of intraspecific competitions (i.e., *p*
_{
i
} = 0), there are two possible states: one in which all three species coexist and another with only one surviving species. In the presence of intraspecific competitions, a new type of states in which two species coexist can arise, which can be seen by finding the fixed points of the ODE model, Eq. (2), and analyzing their stabilities. The first type of fixed points, denoted by **p**
_{1}, correspond to an extinction state:

The second type **p**
_{2} is for states in which two species coexist and one species is extinct:

where *w*
_{1} = 2/(*p*
_{
a
}
*p*
_{
b
} + 2(*p*
_{
a
} + *p*
_{
b
}) − 4), *w*
_{2} = 2/(*p*
_{
a
}
*p*
_{
c
} + 2(*p*
_{
a
} + *p*
_{
c
}) − 4), and *w*
_{3} = 2/(*p*
_{
b
}
*p*
_{
c
} + 2(*p*
_{
b
} + *p*
_{
c
}) − 4). The existence and stability conditions of type **p**
_{2} fixed points are summarized in Table 1. The last type **p**
_{3} corresponds to the state in which all three species survive, i.e., (*a**, *b**, *c**), where

and Γ = *p*
_{
a
}
*p*
_{
b
}
*p*
_{
c
} + 8 + 2[*p*
_{
a
}
*p*
_{
b
} + *p*
_{
b
}
*p*
_{
c
} + *p*
_{
c
}
*p*
_{
a
} − 2(*p*
_{
a
} + *p*
_{
b
} + *p*
_{
c
}) + 12]. For a more detailed analysis of stability and existence of fixed points, see Supplementary Information.

To assess the stabilities of these different types of fixed points in a concrete way, we fix (*p*
_{
b
}, *p*
_{
c
}) = (1, 0.5) and vary the parameter *p*
_{
a
}. For *p*
_{
a
} ≥ 0, there are three fixed points of type **p**
_{1} and one fixed point of type **p**
_{3}. However, for *p*
_{
a
} ≥ 2, only one fixed point of type **p**
_{2} [Eq. (5)] exists. The three fixed points of type **p**
_{1} are saddles for *p*
_{
a
} ≥ 0. While for 0 < *p*
_{
a
} ≤ 1.5 all fixed points are unstable, and there is an asymptotically stable heteroclinic cycle constituting three heteroclinic orbits connecting any two saddle fixed points (all of the **p**
_{1} type). Since the cycle is arbitrarily close to the saddle fixed points, a small perturbation can cause a trajectory to diverge from the cycle, leading to extinction. For 1.5 < *p*
_{
a
} ≤ 4, the fixed point of the type **p**
_{3} [Eq. (8)] becomes stable, in which all species coexist. We note that, for *p*
_{
a
} ≥ 2, fixed points of the type **p**
_{2} [Eq. (5)] are created and are unstable. For *p*
_{
a
} > 4, the fixed points given by Eqs (5) and (8) become stable and unstable, respectively, indicating the emergence of the coexistence state of two species. In this case, the fixed point given by Eq. (5) is globally stable. As *p*
_{
a
} is increased further, this fixed point approaches an extinction state:

Figure 3(a) presents a bifurcation diagram of these behaviors.

## References

- 1.
Chesson, P. Mechanism of maintenance of species diversity.

*Annu. Rev. Ecol. Syst.***31**, 343–366 (2000). - 2.
Chesson, P. & Kuang, J. J. The interaction between predation and competition.

*Nature***456**, 235–238 (2008). - 3.
Fox, L. R. Cannibalism in natural populations.

*Annu. Rev. Ecol. Syst.***6**, 87–106 (1975). - 4.
Polis, G. A. The evolution and dynamics of intraspecific predation.

*Annu. Rev. Ecol. Syst.***12**, 225–251 (1981). - 5.
Birkhead, T. R., Lee, K. E. & Young, P. Sexual cannibalism in the praying mantis hierodula membranacea.

*Behaviour***106**, 112–118 (1988). - 6.
Kraak, S. B. M. Female preference and filial cannibalism in aidablennius sphynx (teleostei, blenniidae) - a combined field and laboratory study.

*Behav. Proc.***36**, 85–97 (1996). - 7.
Maxwell, M. R. Lifetime mating opportunities and male mating behaviour in sexually cannibalistic praying mantids.

*Anim. Behav.***55**, 1011–1028 (1998). - 8.
Rudolf, V. H. W. Consequences of stage-structured predators: Cannibalism, behavioral effects and trophic cascades.

*Ecology***88**, 2991–3003 (2007). - 9.
Santi, F. & Maini, S. Ladybirds mothers eating their eggs: is it cannibalism?

*B. Insectol.***60**, 89–91 (2007). - 10.
Prokop, P. & Václav, R. Seasonal aspects of sexual cannibalism in the praying mantis (mantis religiosa).

*J. Ethol.***26**, 213–218 (2008). - 11.
Sinervo, B. & Lively, C. M. The rock-paper-scissors game and the evolution of alternative male strategies.

*Nature***380**, 240–243 (1996). - 12.
Bolnick, D. I. Can intraspecific competition drive disruptive selection? An experimental test in natural populations of sticklebacks.

*Evolutions***58**, 608–618 (2004). - 13.
Balmer, O., Stearns, S. C., Schötzau, A. & Brun, R. Intraspecific competition between co-infecting parasite strains enhances host survival in African trypanosomes.

*Ecology***90**, 3367–3378 (2009). - 14.
Calsbeek, R. & Cox, R. M. Experimentally assessing the relative importance of predation and competition as agents of selection.

*Nature***465**, 613–616 (2010). - 15.
Albertson, R. C., Streelman, J. T. & Kocher, T. D. Directional selection has shaped the oral jaws of lake malawi cichlid fishes.

*Proc. Nat. Acad. Sci. (USA)***100**, 5252–5257 (2003). - 16.
Quinn, T. P., Hodgson, S., Flynn, L., Hilborn, R. & Rogers, D. E. Directional selection by fisheries and the timing of sockeye salmon (oncorhynchus nerka) migrations.

*Ecol. Appl.***17**, 733–739 (2007). - 17.
May, R. M.

*Stability and Complexity in Model Ecosystems*(Princeton University Press, Princeton, NJ, 1973). - 18.
May, R. M. Biological populations with nonoverlapping generations: stable points, stable cycles, and chaos.

*Science***186**, 645–647 (1974). - 19.
May, R. M. & Leonard, W. J. Nonlinear aspects of competition between three species.

*SIAM J. Appl. Math.***29**, 243–253 (1975). - 20.
Kerr, B., Riley, M. A., Feldman, M. W. & Bohannan, B. J. Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors.

*Nature***418**, 171–174 (2002). - 21.
Czárán, T. L., Hoekstra, R. F. & Pagie, L. Chemical warfare between microbes promotes biodiversity.

*Proc. Natl. Acad. Sci. (USA)***99**, 786–790 (2002). - 22.
Reichenbach, T., Mobilia, M. & Frey, E. Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games.

*Nature***448**, 1046–1049 (2007). - 23.
Szabó, G. & Fath, G. Evolutionary games on graphs.

*Phys. Rep.***446**, 97–216 (2007). - 24.
Frean, M. & Abraham, E. R. Rock–scissors–paper and the survival of the weakest.

*P. Roy. Soc. B-Biol. Sci.***268**, 1323–1327 (2001). - 25.
Traulsen, A., Claussen, J. C. & Hauert, C. Coevolutionary dynamics: from finite to infinite populations.

*Phys. Rev. Lett.***95**, 238701 (2005). - 26.
Reichenbach, T., Mobilia, M. & Frey, E. Noise and correlations in a spatial population model with cyclic competition.

*Phys. Rev. Lett.***99**, 238105 (2007). - 27.
Peltomäki, M. & Alava, M. Three-and four-state rock-paper-scissors games with diffusion.

*Phys. Rev. E***78**, 031906 (2008). - 28.
Szabó, G. & Szolnoki, A. Phase transitions induced by variation of invasion rates in spatial cyclic predator-prey models with four or six species.

*Phys. Rev. E***77**, 011906 (2008). - 29.
Berr, M., Reichenbach, T., Schottenloher, M. & Frey, E. Zero-one survival behavior of cyclically competing species.

*Phys. Rev. Lett.***102**, 048102 (2009). - 30.
Shi, H.-J., Wang, W.-X., Yang, R. & Lai, Y.-C. Basins of attraction for species extinction and coexistence in spatial rock-paper-scissors games.

*Phys. Rev. E***81**, 030901 (2010). - 31.
Ni, X., Yang, R., Wang, W.-X., Lai, Y.-C. & Grebogi, C. Basins of coexistence and extinction in spatially extended ecosystems of cyclically competing species.

*Chaos***20**, 045116 (2010). - 32.
Yang, R., Wang, W.-X., Lai, Y.-C. & Grebogi, C. Role of intraspecific competition in the coexistence of mobile populations in spatially extended ecosystems.

*Chaos***20**, 023113 (2010). - 33.
He, Q., Mobilia, M. & Täuber, U. C. Spatial rock-paper-scissors models with inhomogeneous reaction rates.

*Phys. Rev. E***82**, 051909 (2010). - 34.
Wang, W.-X., Lai, Y.-C. & Grebogi, C. Effect of epidemic spreading on species coexistence in spatial rock-paper-scissors games.

*Phys. Rev. E***81**, 046113 (2010). - 35.
Frey, E. Evolutionary game theory: Theoretical concepts and applications to microbial communities.

*Physica A***389**, 4265–4298 (2010). - 36.
Ni, X., Wang, W.-X., Lai, Y.-C. & Grebogi, C. Cyclic competition of mobile species on continuous space: Pattern formation and coexistence.

*Phys. Rev. E***82**, 066211 (2010). - 37.
Venkat, S. & Pleimling, M. Mobility and asymmetry effects in one-dimensional rock-paper-scissors games.

*Phys. Rev. E***81**, 021917 (2010). - 38.
Wang, W.-X., Ni, X., Lai, Y.-C. & Grebogi, C. Pattern formation, synchronization, and outbreak of biodiversity in cyclically competing games.

*Phys. Rev. E***83**, 011917 (2011). - 39.
Jiang, L.-L., Wang, W.-X., Lai, Y.-C. & Ni, X. Multi-armed spirals and multi-pairs antispirals in spatial rock-paper-scissors games.

*Phys. Lett. A***376**, 2292–2297 (2012). - 40.
Juul, J., Sneppen, K. & Mathiesen, J. Clonal selection prevents tragedy of the commons when neighbors compete in a rock-paper-scissors game.

*Phys. Rev. E***85**, 061924 (2012). - 41.
Lamouroux, D., Eule, S., Geisel, T. & Nagler, J. Discriminating the effects of spatial extent and population size in cyclic competition among species.

*Phys. Rev. E***86**, 021911 (2012). - 42.
Avelino, P., Bazeia, D., Losano, L., Menezes, J. & Oliveira, B. Junctions and spiral patterns in generalized rock-paper-scissors models.

*Phys. Rev. E***86**, 036112 (2012). - 43.
Avelino, P., Bazeia, D., Losano, L. & Menezes, J. von Neummann’s and related scaling laws in rock-paper-scissors-type games.

*Phys. Rev. E***86**, 031119 (2012). - 44.
Knebel, J., Krüger, T., Weber, M. F. & Frey, E. Coexistence and survival in conservative lotka-volterra networks.

*Phys. Rev. Lett.***110**, 168106 (2013). - 45.
Hua, D.-Y., Dai, L.-C. & Lin, C. Four-and three-state rock-paper-scissors games with long-range selection.

*Europhys. Lett.***101**, 38004 (2013). - 46.
Park, J., Do, Y., Huang, Z.-G. & Lai, Y.-C. Persistent coexistence of cyclically competing species in spatially extended ecosystems.

*Chaos***23**, 023128 (2013). - 47.
Juul, J., Sneppen, K. & Mathiesen, J. Labyrinthine clustering in a spatial rock-paper-scissors ecosystem.

*Phys. Rev. E***87**, 042702 (2013). - 48.
Kang, Y., Pan, Q., Wang, X. & He, M. A golden point rule in rock-paper-scissors-lizard-spock game.

*Physica A***392**, 2652–2659 (2013). - 49.
Dobrinevski, A., Alava, M., Reichenbach, T. & Frey, E. Mobility-dependent selection of competing strategy associations.

*Phys. Rev. E***89**, 012721 (2014). - 50.
Claudia, C. & Carletti, T. Stochastic patterns in a 1D rock-paper-scissor model with mutation.

*arXiv preprint arXiv*:*1401*.*4302*(2014). - 51.
Feng, S.-S. & Qiang, C.-C. Self-organization of five species in a cyclic competition game.

*Physica A***392**, 4675–4682 (2013). - 52.
Zheng, H.-Y.

*et al*. Mesoscopic interactions and species coexistence in evolutionary game dynamics of cyclic competitions.*Sci. Rep.***4**, 7486 (2014). - 53.
Szolnoki, A., Vukov, J. & Perc, M. From pairwise to group interactions in games of cyclic dominance.

*Phys. Rev. E***89**, 062125 (2014). - 54.
Szczesny, B., Mobilia, M. & Rucklidge, A. M. When does cyclic dominance lead to stable spiral waves?

*Europhys. Lett.***102**, 28012 (2013). - 55.
Szczesny, B., Mobilia, M. & Rucklidge, A. M. Characterization of spiraling patterns in spatial rock-paper-scissors games.

*Phys. Rev. E***90**, 032704 (2014). - 56.
Szczesny, B., Mobilia, M. & Rucklidge, A. M. The influence of mobility rate on spiral waves in spatial rock-paper-scissors games.

*Games***7**, 24 (2014). - 57.
Szolnoki, A.

*et al*. Cyclic dominance in evolutionary games: a review.*J. R. Soc. Interface***11**, 20140735 (2014). - 58.
Taylor, C. & Nowak, M. A. Evolutionary game dynamics with non-uniform interaction rates.

*Theor. Popul. Biol.***69**, 243–252 (2006). - 59.
He, Q., Mobilia, M. & Täuber, U. C. Coexistence in the two-dimensional May-Leonard model with random rates.

*J. Phy. Condens. Matt.***82**, 97–105 (2011). - 60.
Schreiber, S. J. & Killingback, T. P. Spatial heterogeneity promotes coexistence of rock-paper-scissors metacommunities.

*Theor. Popul. Biol.***86**, 1–11 (2013). - 61.
Laird, R. A. & Schamp, B. S. Competitive intransitivity, population interaction structure, and strategy coexistence.

*J. Theor. Biol.***365**, 149–158 (2015). - 62.
Vukov, J., Szolnoki, A. & Szabó, G. Diverging fluctuations in a spatial five-species cyclic dominance game.

*Phys. Rev. E***88**, 022123 (2013).

## Acknowledgements

Y.D., J.P. and B.J. was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIP) (No. NRF-2016R1A2B4011009, NRF-2016R1C1B1010340, NRF-2015S1A5A2A03049830). B.J. was also supported by the National Research Foundation of Korea (NRF) through Grant No. NRF-2016R1D1A1B03935514. Y.C.L. was supported by Army Research Office under Grant No. W911NF-14-1-0504. Y.C.L. would also like to acknowledge support from the Vannevar Bush Faculty Fellowship program sponsored by the Basic Research Office of the Assistant Secretary of Defense for Research and Engineering and funded by the Office of Naval Research through Grant No. N00014-16-1-2828.

## Author information

### Affiliations

### Contributions

J.P., B.J. and Y.D. devised the research project. J.P. performed numerical simulations. J.P., B.J., Y.D. and Y.C.L. analyzed the results and wrote the paper.

### Corresponding author

## Ethics declarations

### Competing Interests

The authors declare that they have no competing interests.

## Additional information

**Publisher's note:** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Electronic supplementary material

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Park, J., Do, Y., Jang, B. *et al.* Emergence of unusual coexistence states in cyclic game systems.
*Sci Rep* **7, **7465 (2017). https://doi.org/10.1038/s41598-017-07911-4

Received:

Accepted:

Published:

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.