Introduction

Sexual reproduction is a unifying feature of eukaryotes1 and yet it is extremely diverse2. Sexual systems (also known as “sexual patterns”), defined as the pattern of distribution of the male and female function among the individuals of a given species, vary from separate fixed sexes (known as gonochorism in animals and dioecy in plants) to simultaneous hermaphroditism (each individual produces both male and female gametes at the same time). These two sexual systems can be viewed as the extremes in a sexually plastic gradient3 of intermediate systems (sequential hermaphroditism) and mixed systems (coexistence of males and/or females with hermaphrodites)4,5. Sexual systems have a profound influence on individuals’ mating success and fitness6, population sex ratios and effective sizes7, as well as colonisation events and habitat use8. As a result, sexual systems influence the population dynamics and resilience to natural and anthropogenic stressors of ecologically and commercially important species that are often endangered or overexploited9.

Hermaphroditism is predominant in flowering plants (angiosperms)10, where 94% of the species have male and female sex organs in the same individual/flower, and it is widespread in invertebrates and teleost fish (the only vertebrates to exhibit hermaphroditism11), totalling 5% of animal species or up to ~30% if insects are excluded12. While this diversity suggests multiple evolutionary transitions between sexual systems in response to selection, current evolutionary models on the adaptive advantage of different sexual systems explain little about how and why sexual systems evolve and thus their large-scale distribution across the tree of life. This might indicate that adaptive predictions alone fail to fully explain why some sexual forms evolve in some taxa but not others (Williams’ paradox)4,13. Therefore, unravelling the evolutionary history of sexual systems and quantifying how frequently and in what direction transitions occur is key to revealing which sexual systems are evolutionarily labile or stable, elucidating how one changes into another over evolutionary time, and identifying the environmental, genetic and developmental drivers favouring or opposing these changes. Yet, our understanding of how sexual systems evolve is still limited, particularly in animals.

Theoretical models, initially developed for plants, suggest that simultaneous hermaphroditism and dioecy are evolutionary stable conditions that are retained over a long evolutionary time and unlikely lost once evolved, while mixed sexual systems represent evolutionary intermediate stages4,5,14 (Fig. 1).

Fig. 1: Theoretical framework for the evolution of sexual systems.
figure 1

Illustration of potential evolutionary transitions between gonochorism (in grey) and simultaneous hermaphroditism (in yellow) via mixed systems (mixed pathways) as described in plants and some animals; via sequential hermaphroditism (sequential pathways: protogyny in red and protandry in blue) as recently suggested4; or without intermediate states (direct pathways) as proposed for plants6. Double-headed arrows indicate theoretical pathways.

Full size image

Simultaneous hermaphroditism is likely the ancestral state in angiosperms from which dioecy, a rare sexual system in plants6, has evolved independently several times, possibly to avoid inbreeding15,16. Theoretical models predict that separate sexes in plants evolve from hermaphroditism in different ways: (1) primarily through the intermediate state of gynodioecy17, a common sexual system in plants that occurs when a male-sterile mutant invades a hermaphroditic population resulting in the coexistence of hermaphrodites and females; (2) through androdioecy, a less common system18,19 in which mutations resulting in female sterility lead to the coexistence of hermaphrodites and males; (3) via trioecy, i.e., the coexistence of hermaphrodites, males and females, which is very rare; and (4) less frequently, via a direct transition6,10 (Fig. 1). However, in animals, no evidence of a direct transition between hermaphroditism and gonochorism exists. Once gained, dioecy was believed to be an irreversible condition20, a conclusion based on the assumption that returning to a simultaneous expression of male- and female-specific genes would likely produce contrasting effects on sex-specific physiology. Recent studies, however, reject this claim in plants, as phylogenetic reconstructions of direct transitions from dioecy/gonochorism to simultaneous hermaphroditism have been documented10,21.

The same theoretical framework with mixed pathways has been proposed also for animals where, in contrast to plants, gonochorism is the most common sexual system, androdioecy is more common than gynodioecy5,14 and trioecy is very rare22. However, several reproductive characteristics in plants differ substantially from those in animals23, albeit similarities can be found in some invertebrates24; hence, different theoretical frameworks are required (Fig. 1). Furthermore, evolutionary transitions between sexual systems in teleost fish (~34000 species, comprising the overwhelming majority of the ray-finned fishes, Actinopterygii)25, might be less likely to occur via a mixed pathway (Fig. 1) given that in this group only a few killifish species of the genus Kryptolebias (formerly Rivulus) are truly androdioecious5,26,27. Beyond teleosts, the presence of gynodioecy and trioecy among vertebrates is still debated in the jawless hagfish Myxine glutinosa (Myxini)14,28. Recently, sequential hermaphroditism has been suggested as a possible intermediary state that may facilitate evolutionary changes between gonochorism and simultaneous hermaphroditism4 (Fig. 1). However, phylogenetic studies on the evolution of hermaphroditism at large scale do not typically discriminate between the different forms of hermaphroditism and treat the sexual systems as a binary trait29. Thus, we currently have no robust large-scale study on the evolution of sexual systems in animals and we do not know whether sequential hermaphroditism represents an evolutionary intermediate stage between gonochorism and simultaneous hermaphroditism, whether protogyny and protandry act equally as transitional forms between the two, and whether gonochorism and simultaneous hermaphroditism are evolutionary stable conditions in animals as they are in plants.

The evolution of hermaphroditism in animals has mostly been interpreted in the context of its adaptive advantages relative to gonochorism, as proposed by the low density and the size advantage models30. The former predicts that simultaneous hermaphroditism evolves under low population densities and/or low dispersal capacity as, in these conditions, individuals with this sexual system can maximise their chances of securing a mate compared to sex-changing or gonochoristic individuals31. Note, however, that the advantages of self-fertilizing simultaneous hermaphrodites may be offset by an increased risk of inbreeding. The size advantage model proposes adaptive explanations for the evolution of sequential hermaphroditism based on the relationship between size and fecundity30,32,33,34. Specifically, since most fish, including sequential hermaphrodites, have indeterminate growth35, i.e., they can keep growing as far as food resources and environment allow, sex change should be favoured when the reproductive value of an individual depends on size (and thus age), and this affects in particular one of the sexes. Thus, individuals change from smaller first sex to larger second sex and the direction of sex change depends on the sex that maximises its reproductive value with a larger size36. The size advantage model has been supported in crustaceans37, molluscs38, and teleost fish39.

The interdependence between size, fecundity and fitness is affected by a species’ mating system, defined as the pattern of sexual interactions given the number of reproducing males and females (Table 1). Therefore, among sequential hermaphrodites, protandry (male-to-female sex change) is usually expected in species that reproduce in monogamous or random pairs and where individuals switch from small males to large, highly fecund females, achieving higher reproductive potential. Conversely, protogyny (female-to-male sex change) is usually expected in polygynous/group-mating species, where small females become large dominant males that monopolise females, often grouped in harems (Table 1). In both systems, cases exist with a few individuals born directly as the second sex. Specifically, in digynic protandrous species, primary females directly develop as such and secondary females develop from males after sex change40. Likewise, in diandric protogynous species, primary males develop directly as such whereas secondary males develop from females after sex change41.

Table 1 Predictions of associations between most common sexual systems (pattern of distribution of the male and female function among the individuals of a given species), mating systems (pattern of sexual interactions that take place considering the number of males and females involved in reproduction), adult sex ratio, size of mates, and spawning behaviour (how the two sexes interact to release the gametes) in teleosts.
Full size table

Crucially, life-history traits underpin the formulation and assumptions of the size advantage model. Life history theory is central to the study of sexual systems evolution since it allows to derive clear predictions about why and when individuals should allocate energy among different life-history traits, including sexual functions, to optimise fitness42. However, life-history traits are surprisingly not explicitly and formally incorporated in the size advantage model, nor tested in empirical studies13. Longevity, maximum size and age/size at first maturity are key life-history traits because they determine individual fitness, influence demographic parameters of populations43 and impact populations’ genetic diversity44. These traits evolve and are under several selective forces at the population level, but differences in the intensity of selection among species can lead to large-scale diversity, thus allowing large-scale comparative studies to inform our understanding of how and why they evolved45. Since sequential hermaphrodites achieve higher fitness when reproducing as the second sex36, hence the advantage of changing sex, they should, on average, benefit more than gonochoristic and simultaneous hermaphroditic species from increased longevity (overall and/or as the second sex in particular), or larger size, especially in protandry, where females are the larger sex as size gives fecundity advantage. In general, larger females tend to produce more eggs than smaller ones both within and across species46, while larger males do not necessarily increase their sperm production with size. In males, larger size gives the advantage to secure dominance and increase fertilisation rates, but not necessarily fecundity. Alternatively, sequential hermaphrodites could mature, on average, earlier as the first sex compared to the same sex in gonochoristic species and capitalise on reproduction as the second sex. These predictions, however, remain to be tested.

Although exceptions occur, spawning behaviour, i.e., how the two sexes interact to release the gametes, can be broadly classified in fish as pair spawning, involving only two individuals at the time, and group spawning, comprising large breeding groups47 (Table 1). Mating system and spawning behaviour together determine the intensity of direct male-male competition and sperm competition (i.e., the competition between the sperm of two or more males for fertilisation of the same eggs), and thus the certainty of paternity. Sperm competition is a key selective force shaping male reproductive anatomy, physiology and behaviour across diverse animal groups48,49. In general, sperm competition is assumed to be low in haremic systems39,50,51 where large dominant males can better monopolise groups of females (pair spawning) with no or limited competition by other males52. Likewise, low sperm competition is expected under monogamy. Group spawning is commonly found in promiscuous mating, leading to intense sperm competition53 as many males try to fertilise the eggs of multiple females. The intensity of sperm competition has been incorporated in the size advantage model54 as it can play a significant role in the advantage of protogyny: changing sex from female-to-male should be more advantageous when paternity assurance is high due to reduced sperm competition55. Consistent with these predictions, the gonadosomatic index (GSI), defined as the percentage of body mass devoted to the gonads56 and a reliable indicator of the intensity of sperm competition57, is significantly lower in protogynous teleost species than in gonochoristic congeners51,52,58. However, protandrous teleost fish do not always conform to theoretical expectations, exhibiting higher GSI as males than expected52. We have recently proposed that, at least in the family Sparidae, high male GSI in protandrous fish can be explained not only by group spawning and high sperm competition in some species but also because high investment in the gonads can represent a compensatory mechanism that allows small males to fertilise highly fecund females much larger than themselves58.

Teleosts account for more than 50% of the extant species of vertebrates and are characterised not only by their extraordinary diversity in morphology, physiology, ecology and habitat but also by different sexual systems, including gonochorism, different forms of hermaphroditism —the only group among vertebrates— and unisexuality (all-female populations)11,59,60. Hermaphroditism in teleosts is broadly divided into simultaneous (synchronous) and sequential (consecutive) hermaphroditism, the latter in the form of protandry, protogyny and bidirectional sex change (Table 1). Thus, the remarkable diversity in sexual systems in teleost fish makes them an ideal group in which to study the evolution of different forms of hermaphroditism29. Here, we investigate the evolutionary origin and transitions among sexual systems across 4614 teleost species belonging to 49 orders and 293 families using a recent time-calibrated phylogeny61 and modern phylogenetic comparative approaches. Our large-scale approach allows us to fully unravel how sexual patterns evolved and identify which ones represent evolutionary stable conditions. We focus on gonochorism, protogyny, protandry and simultaneous hermaphroditism as these are the most common sexual systems in teleosts. For hermaphrodites, we only included species for which functional hermaphroditism could be confirmed by primary literature; all remaining species, following the sexual system obtained from FishBase62, were classified as gonochoristic, excluding the species with ambiguous information (see Methods and Supplementary Fig. 1). We do not distinguish digynic and diandric species (or populations) in this study because the number of sequentially hermaphroditic species in our dataset is not sufficient for splitting them into narrower categories. Thus, separating digynic and diandric species would lead to a small sample size per category while increasing the number of parameters to be estimated, ultimately eroding the power of the analysis. Likewise, unisexual species (“biotypes”, hybrid in origin)59,63 are too few to be incorporated in any formal analyses in our study.

Here, we demonstrate that gonochorism is the likely ancestral condition in teleosts and it is an evolutionarily stable state from which protogyny and protandry evolve at a moderate evolutionary rate. Consistent with theoretical predictions, we show that simultaneous hermaphroditism cannot evolve directly from gonochorism but rather through the intermediate stage of sequential hermaphroditism, most likely protandry. Further, we expand the theoretical framework of the evolution of hermaphroditism investigating how life-history traits and male GSI differ between sexual systems, as predicted by life history theory. In support of these predictions, we found evidence of a longer lifespan in protogynous species compared to gonochoristic and strong evidence of smaller GSI in protogynous males. However, contrary to predictions, we found no difference in maximum size and age or size at maturity across sexual systems. We discuss how our results should be incorporated into a broader framework with sex-determining mechanisms and gonadal plasticity as possible constraining and facilitating mechanisms, respectively, to gain a fuller understanding of the evolution of sexual systems and possibly resolve Williams’ paradox.

Results

Evolutionary history of sexual systems in teleosts

Our dataset includes 4614 extant teleost species, of which 294 are hermaphroditic (protogynous: n = 196; protandrous: n = 36; bidirectional sex changers: n = 16; simultaneous hermaphrodites: n = 46; Fig. 2; Methods, Data collection and verification; Supplementary Data 1). We used Discrete models of evolution to reconstruct the evolutionary history of sexual systems using Reversible Jump (RJ) Markov chain Monte Carlo (MCMC) in BayesTraits (Methods, Phylogenetic comparative analysis). Treating sexual systems as a two-character state (gonochoristic or hermaphroditic) our analysis reveals that gonochorism is the most likely ancestral character state in teleosts (Fig. 3a, b; Supplementary Table 1) and that hermaphroditism evolves slowly from, and reverts very quickly and multiple times back to, gonochorism (Fig. 3a, c; Supplementary Table 1). This indicates that gonochorism is an evolutionarily stable state in teleosts.

Fig. 2: Sexual systems of extant species of teleosts.
figure 2

Sexual systems are colour coded for gonochorism (n = 4320; grey), protogyny (n = 196; red), protandry (n = 36; blue), bidirectional sex change (n = 16; green) and simultaneous hermaphroditism (n = 46; yellow). Families (n = 32) with hermaphroditic species are labelled. Silhouettes have been obtained from fishualize108 or drawn by one of the authors (C.B.).

Full size image
Fig. 3: The evolutionary history of the sexual system in teleosts.
figure 3

a Visual summary of maximum likelihood ancestral state reconstruction as a two-character state (gonochorism or hermaphroditism) that best approximates results of our RJ-MCMC Multistate model. The sexual systems of extant species and their ancestors are colour coded for gonochorism (n = 4320; grey) and hermaphroditism (n = 294; magenta). b Density plots from RJ-MCMC Multistate models for the estimated probability of character state at the root of the phylogeny colour coded for gonochorism (mean = 66%; grey) and hermaphroditism (mean: 34%; magenta). c RJ-MCMC multistate posterior distributions of the transition rates from gonochorism to hermaphroditism (magenta) and from hermaphroditism to gonochorism (grey).

Full size image

Treating the sexual system as a four-character state (gonochoristic, protandric, protogynic, simultaneous hermaphroditic; Fig. 2) reveals that both types of sequential hermaphroditism evolve at a very low rate from gonochorism and revert very rapidly back to it (Fig. 4, Table 2). In contrast, direct transitions between gonochorism and simultaneous hermaphroditism are very slow if they happen at all, given that over 60% and 31% of the models estimate the transition from gonochorism to simultaneous hermaphroditism and the reversal, respectively, to be equal to zero. Our analysis also shows that protogyny evolves as slowly from gonochorism as it switches to protandry and simultaneous hermaphroditism, although 32% of models estimate the latter transition to be equal to zero. Conversely, protandry is lost quickly to protogyny and simultaneous hermaphroditism, and very rapidly to gonochorism. Therefore, simultaneous hermaphroditism evolves from sequential hermaphroditism, most likely from protandry, and is lost to gonochorism, protogyny and protandry at similarly low rates, although approximately half of the models estimate transition rates to sequential hermaphroditism to be equal to zero. Altogether, these results clearly indicate that in teleosts gonochorism is an evolutionarily stable state; protogyny is evolutionarily more stable than protandry, while simultaneous hermaphroditism evolves rarely, most like from protandry, and is evolutionarily stable being lost slowly to gonochorism, and less likely, to protogyny and protandry (Fig. 4, Table 2).

Fig. 4: Transitions rates between sexual systems in teleosts.
figure 4

Summary of RJ-MCMC Multistate analysis with density plots of the posterior distributions of the transition rates to gonochorism (grey), protogyny (red), protandry (blue), and simultaneous hermaphroditism (yellow). Gonochorism is the estimated likely ancestral condition. Note, only x axis, but not y axis, are the same for each pair of gain and loss between two-character states. The thickness of the arrows is roughly proportional to the mean magnitude of the transition rates from the posterior distribution. Dashed arrows indicate transition rates estimated to be equal to 0 in over 40% of the models in the posterior distributions. Sample sizes of extant species included in our analysis for each sexual system category are indicated between parentheses.

Full size image
Table 2 Results of the RJ-MCMC Multistate analysis in BayesTraits of sexual systems as a four-state categorical variable: gonochorism (G), protogyny (PG), protandry (PA), or simultaneous hermaphroditism (SH).
Full size table

Life-history traits and sexual systems in teleosts

Using life-history theory, we predicted that sequential hermaphrodites live longer and/or reach larger adult size and/or mature earlier as the first sex than gonochoristic species. The phylogenetic generalised least square (PGLS) analyses revealed that protogynous, but not protandrous, species live longer than gonochoristic species (Fig. 5a; Table 3). Larger species however might live longer, therefore we repeated the analysis controlling for allometry; even so, adding size (maximum length) as a covariate did not alter this result (Fig. 5b; Supplementary Table 2). Contrary to predictions, we did not find any significant size difference across sexual systems (Fig. 5c; Table 3). Female and male age at first maturity does not differ across species with different sexual systems (Fig. 5d, e; Table 3), even when accounting for allometry (length at maturity; Supplementary Table 4), nor does sex-specific length at maturity (Table 3). Finally, the PGLS revealed that protogynous males have lower GSI values than gonochoristic and protandrous ones, but GSI does not differ significantly between gonochoristic and protandric males (Fig. 5f; Table 3) even when considering allometry (Supplementary Table 2).

Fig. 5: Life history traits by sexual system in teleosts.
figure 5

Phylogenetic estimated mean and phylogenetic standard error from the PGLS results of: a longevity (year, log10 -transformed; G: n = 758; PG: n = 69; PA: n = 17); b longevity while controlling for maximum length (G: n = 575; PG: n = 61; PA: n = 8); c maximum length (cm, log10 -transformed; G: n = 2612; PG: n = 167; PA: n = 20); d male age at first maturity (year, log10 -transformed; G: n = 259; PG: n = 15; PA: n = 9); e female age at first maturity (year, log10 -transformed; G: n = 282; PG: n = 30; PA: n = 5); f male gonadosomatic index, GSI (log10 -transformed; G: n = 44; PG: n = 38; PA: n = 15). In all panels gonochorism (G) is depicted in grey, protogyny (PG) in red and protandry (PA) in blue. *P < 0.05; ***P < 0.001 (please refer to Table 3 for details). Source data are provided as a Source Data file.

Full size image
Table 3 Results of phylogenetic generalised least square (PGLS) model of longevity (year; log10 -transformed), maximum length (cm; log10 -transformed), age at first maturity (year; log10 -transformed), length at first maturity (cm; log10 -transformed) per each sex (♂: male; ♀: female), and male gonadosomatic index (GSI; log10 -transformed) across sexual systems: gonochorism (G); protogyny (PG); protandry (PA).
Full size table

Discussion

Our large-scale phylogenetic study has tested the theoretical predictions on how sexual systems evolve and has revealed the evolutionary origin of and transitions between different sexual systems in the highly diverse teleosts. We identify gonochorism and simultaneous hermaphroditism as stable conditions over evolutionary time. In support of recent theoretical models4, our study demonstrates that simultaneous hermaphroditism is unlikely to evolve directly from gonochorism and instead requires the intermediate step of sequential hermaphroditism, most likely protandry. We find support for the predictions derived from the life-history theory that protogynous species live longer than gonochoristic species but no evidence that sequential hermaphrodites attain a larger size or mature earlier than gonochoristic species. Finally, we find strong evidence that protogynous males invest the least in male gonad tissues (quantified by the gonadosomatic index) relative to gonochoristic and protandric males. Combined, these results suggest that the two forms of sequential hermaphroditism must be treated separately in theoretical and empirical studies as protandry and protogyny are characterised by distinct life history strategies36, even though they both entail sex change.

Using the largest dataset ever collected with four sexual systems in teleosts, our study reveals a complex and dynamic way through which sexual systems evolve and switch between one another. Sequential hermaphroditism can evolve slowly from gonochorism, the ancestral state in teleosts, but revert to gonochorism rapidly. Although gonochorism is an evolutionarily stable condition, gained faster than it is lost, these results refute the assumption that the transition to gonochorism is irreversible20 and represent another example64,65 against Dollo’s law of irreversibility66, as previously suggested29. Conversely, sequential hermaphroditism in teleosts, particularly protandry, is less evolutionarily stable than gonochorism. Our results however contradict Pennell et al.’s29 finding that the evolutionary transition from gonochorism to hermaphroditism occurs over twice as fast as the reverse, suggesting rapid evolution of hermaphroditism from gonochorism, a conclusion that the same authors acknowledge is counterintuitive. In contrast, we find that the evolutionary gain of hermaphroditism is slower than its loss to gonochorism, regardless of whether we treat sexual system as a binary trait (gonochorism vs hermaphroditism) or discriminate between different forms of hermaphroditism. Heterogeneity in the rate of gain and losses across large phylogenies can potentially bias the estimates of the faster transition rate for binary traits67. However, our analysis of four states reveals that protandry is lost rapidly to both protogyny and gonochorism, and to a lesser degree, to simultaneous hermaphroditism.

We note that Pennell et al.29 used a much smaller dataset biased towards a greater proportion of hermaphroditic than gonochoristic species than what is observed in teleosts and did not discriminate between different types of hermaphroditism. We have accepted the classification in FishBase62 for gonochoristic species (unless rejected or disputed by primary literature), without individually confirming their sexual system as done for the hermaphroditic species in our dataset. This is because gonochorism is rarely confirmed in primary sources even when present in fish. As a result, if we used only a few gonochoristic species for which sexual system is explicitly confirmed in the original sources, the dataset would be strongly biased against gonochorism and include an unrealistic small number of gonochoristic species, ultimately undermining the robustness of the results. However, we acknowledge that a few species currently classified as gonochoristic in our dataset might be hermaphroditic. Although it is not possible to predict how this could influence the outcome of the analysis, given that this depends on the number of affected species, their phylogenetic position and the sexual system of their closely related species, our results represent an accurate picture of the evolution of the sexual system in fish with the data currently available. Crucially, our results at four states indicate that rapid transition rates from hermaphroditism to gonochorism in our analysis at two states are robust and reveal that protandry and protogyny—but not simultaneous hermaphroditism—evolve much more slowly from gonochorism than the reverse.

Importantly, our study demonstrates that simultaneous hermaphroditism does not originate directly from gonochorism but rather through sequential hermaphroditism, most likely protandry. However, simultaneous hermaphroditism is lost preferentially to gonochorism than to either form of sequential hermaphroditism. Thus, our analyses demonstrate that an intermediate stage is required for the gain of simultaneous hermaphroditism from gonochorism but not the loss back to it. These results support theoretical predictions (Fig. 1) that sex-specific gene expression in gonochoristic species may prevent direct evolutionary transitions between gonochorism and simultaneous hermaphroditism, and intermediate stages, like sequential hermaphroditism, are required4. Overall, our study is consistent with suggestions that the complexity of sex-specific physiology and behaviour is likely to constrain some transitions between sexual systems. Androdioecy is considered an intermediate stage from simultaneous hermaphroditism to dioecy in plants and from gonochorism to hermaphroditism in some invertebrates5,14. However, this sexual system is extremely rare in fish and cannot explain the evolution of the diverse sexual systems in this vertebrate group, where instead sequential hermaphroditism seems to play a similar role. We suggest that future studies in other taxa may also consider sequential hermaphroditism (if present) together with other mixed systems as an important stepping stone for evolutionary changes between gonochorism and simultaneous hermaphroditism as we have found in teleosts.

According to life-history theory, sequential hermaphrodites should, on average, live longer, grow bigger and/or mature earlier as the first sex than gonochoristic species. We find that protogynous and protandrous species differ in their life history strategies: protogynous, but not protandrous species, live longer than gonochoristic species. These differences reflect the fact that protogyny and protandry maximise their fitness as the second sex36 which differs between the two systems. Therefore, the longer life in protogynous species favours large successful males (second sex) that can monopolise females in harems or in spawning grounds. Conversely, protandrous species benefit primarily by achieving a larger size, as larger females (second sex) are more fecund than smaller ones. In addition, male investment in gonad tissue (as quantified by the gonadosomatic index) is lower in protogyny, as expected by theory52,58, since large males can better monopolise mating opportunities and face low levels of sperm competition in harems and group spawning (Table 1). Small-sized protandrous males in group spawning instead need to boost their investment in the gonads but even in the absence of sperm competition (monogamy) they require large gonads to fertilise highly fecund females, larger than themselves58. Thus, sexual systems and mating strategies affect life-history traits differentially in protogynous and protandrous species. It is well known that in sequential hermaphrodites the second sex always matures later and is larger than the first sex, so it is not surprising that in protandrous species females are significantly larger than males when reaching maturity, while in protogynous species males are significantly larger than females55 (excluding the cases of primary females and primary males, respectively). Yet, no comparison has been made for size/age at first maturity for males and females across sexual systems. Life-history theory predicts that the first sex of sequential hermaphrodites matures earlier than the same sex in gonochorism, but, with the data currently available, we find no evidence for this.

Our study includes explicitly life-history traits into a theoretical framework for the evolution of sexual systems and provides some evidence in support of theoretical predictions. However, records on life-history traits for teleosts species in general, and hermaphroditic species in particular, are still scarce. Even less complete and reliable data are available on mating systems and spawning behaviours, which should be incorporated in future studies aiming at obtaining a more complete understanding of the role that life-history traits play. Particularly necessary to fully assess theoretical predictions are sex-specific data for size and time spent as females and as males in sequential hermaphrodites, and for investment in male vs female function in simultaneous hermaphrodites, for which currently little is known. Future studies should re-evaluate these relationships as more data become available for a large number of species. While we have shown that life history theory can provide a major contribution to our understanding of sexual system evolution, below we present a general model for studying sexual systems and propose that the highly dynamic picture revealed by this study should be expanded using a more comprehensive approach that includes not only selection and adaptation, but also sex-determining mechanisms and gonadal plasticity (Fig. 6).

Fig. 6: Theoretical framework for the study of the evolution of sexual systems in teleosts.
figure 6

Overview of parameters (with some examples) considered in the low density and the size advantage models (*), used in our analyses (**) and proposed in the present study (***).

Full size image

Sex determination in gonochoristic animals is determined either at fertilisation by different genetic mechanisms, including male (XX/XY) or female (ZW/ZZ) heterogamety with homomorphic or heteromorphic sex chromosomes and polygenic systems, or after conception by environmental factors, or by a combination of both68,69. Fish are characterised by an incredible diversity70,71,72 and plasticity29,73 of sex-determining mechanisms. Many fish do not have sex chromosomes71. When they are present, they might not always be clearly differentiated since sex-determining loci might not be easily identifiable74 and in some cases, the sex can be determined by a change in a single nucleotide71. High turnover of sex chromosomes has also been detected in some fish lineages (e.g., sticklebacks75,76), including reversal to autosomes29. It has been suggested that fixed, strongly canalised, genetic sex determination (culminating in the formation of fully differentiated and stable heteromorphic sex chromosomes) might constrain the evolution of hermaphroditism, acting as an evolutionary trap2,77,78,79. Even if this is not the case29,80, sequential hermaphrodites do not appear to have sexually differentiated chromosomes81, but data are currently scarce for formal analyses. Finally, sequential hermaphroditism can be regarded as a clear example of phenotypic plasticity, and since epigenetics underlies phenotypic plasticity, epigenetic mechanisms have been proposed to participate in the evolutionary transitions between different sexual systems and sex-determining mechanisms82. Therefore, although complete genetic control of hermaphroditism is common in plants83, a better knowledge of the genetic and epigenetic mechanisms of sex determination could be helpful to explain how hermaphroditism in teleosts has evolved in some taxonomic groups but not in others under similar ecological pressures.

Previous attempts to connect the distribution of sexual systems have invoked morphological84 and developmental85 aspects. Developmental plasticity is uniquely documented in teleosts via the bipotential nature of their gonads and gonoducts85. Thus, while in most vertebrate taxa gonads develop from two distinct germinal layers (medulla, endodermal in origin, which gives rise to the testes; and cortex, mesodermal in origin, which gives rise to the ovary), in teleosts the gonads consist entirely of the cortex homologue86. Moreover, teleosts are the only group of vertebrates where the Müllerian duct is absent, and the gonoduct has the same origin in both sexes, being the reproductive system completely independent of the excretory system85. Therefore, anatomically all teleosts could, in principle, be hermaphrodites85. Furthermore, the transition to protogyny may be favoured by the peculiarities of gonadal development in many gonochoristic teleost species, which develop a female gonad, complete with ovaries containing cysts of oocytes, during the initial stage of gonadal formation87,88. Only later testicular development is triggered and superimposed on this arrangement so that the individual ultimately matures functionally as a male89,90,91. Thus, protogyny might be favoured because female gonads are often the first to develop albeit temporarily, even in protandrous species91. Bidirectional sex change, a rarer system in teleosts (Table 1), further demonstrates the importance of gonadal plasticity. In most cases, the initial strategy is protogyny92, but after sex change adult males can revert back to females when triggered by new social conditions. The retention of some female gonadal tissue in males facilitates a new change of sex, if and when required93. The maintenance of both gonadal tissues could facilitate a transition to simultaneous hermaphroditism. Thus, the study of the evolution of sexual systems in fish (and possibly other taxa) could greatly benefit from taking into consideration the facilitating/constraining aspects linked to gonadal developmental plasticity and the existence of different sex-determining mechanisms.

In conclusion, our study reveals that gonochorism is the most likely ancestral state and the most evolutionary stable sexual system in teleosts. In support of theoretical predictions, we demonstrate that simultaneous hermaphroditism cannot evolve directly from gonochorism but requires an intermediate step, most likely through protandry. However, simultaneous hermaphroditism is more likely to be lost to gonochorism than to sequential hermaphroditism in teleosts. Overall, our study reveals that the evolution of sexual systems is evolutionarily more dynamic and complex than commonly assumed. Our results support theoretical assumptions that changes between sexual systems are likely constrained by sex-specific gene expression, physiology and behaviour. In addition, we propose that the adaptive advantage of different sexual systems is further underpinned in fish by their extraordinary and unique developmental plasticity94, including common and fast transitions among different sex-determining mechanisms29,68. Our study also reveals that different sexual systems exhibit different life-history strategies that allow species with sequential hermaphroditism to maximise fitness as the second sex36, particularly in protogynous species, and highlights the need for more sex-specific life-history data to gain a fuller and deeper understanding of the interplay between life-history strategies and sexual systems. Altogether we propose that a comprehensive framework that incorporates life-history traits, sex-determining mechanisms and gonadal plasticity into traditional theoretical models of sexual system adaptive value will be essential if we are to fully understand the evolution of sexual systems, their phylogenetic distribution and their implications for conservation and management.

Methods

Data collection and verification

We compiled the most comprehensive database on sexual systems in teleosts to date. A dataset was first extracted from FishBase62 for a total of 10914 actinopterygian species, of which 10875 were teleosts. Information on the sexual system was available for 9005 teleost species. Of these, 4740 species were included in the most recent and largest molecular phylogeny for the class61 (available at https://fishtreeoflife.org; Supplementary Fig. 1). Next, species were classed as hermaphroditic only if functional hermaphroditism could be confirmed by primary literature, as recently compiled elsewhere95, with further species confirmed from the primary literature. For the remaining species, we maintained the gonochoristic classification of FishBase62, unless recent literature stated otherwise. Indeed, gonochorism is rarely confirmed in literature even when present, so including gonochoristic only species for which this sexual pattern is confirmed would strongly bias the dataset against gonochorism, ultimately undermining the robustness of the analyses. Importantly, species for which there is contrasting information in the literature were discarded. Altogether our final dataset included 4614 teleosts, with 4320 gonochoristic and 294 hermaphrodite species (Supplementary Fig. 1), of which there were 196 protogynous, 36 protandrous, 16 bidirectional species and 46 simultaneous hermaphrodites (Supplementary Data 1). Unisexual species were not included in the analyses, due to their extremely low number and hybrid orgin59,63; we also did not have enough data (and power) to consider separately digynic and diandric species.

Life history traits (Supplementary Table 3) were also collected from the primary literature, FishBase62 and rFishBase96: longevity (in years), maximum length (total length, TL in cm); age (in years) and length at first maturity (in cm) of males and females; male GSI (the maximum value recorded, expected to coincide with the peak of the reproductive season). When more than one value was present for longevity for a given species, we used the maximum value reported in the wild. We controlled for allometry as follows: longevity was controlled for maximum length (available for both sexes combined); age at first maturity was controlled for length at first maturity (by sex). GSI was controlled for male length at first maturity (male-specific): in this case, we could not use maximum length, not sex-specific, which would give an incorrect length of males in protandric species, where the larger individuals are females.

Phylogenetic comparative analyses

We investigated the evolutionary history of sexual systems of 4614 teleost species using Multistate models in BayesTraits V.397,98 in a Bayesian framework. Multistate estimates instantaneous transition rates between alternative character states of a single categorical variable (i.e., the rate of change between states along the branches of a phylogeny), based on a continuous-time Markov model of evolution for discrete traits99,100. A high transition rate from one state to another indicates that the first state changes rapidly to the second state over evolutionary time. Therefore, a character state is evolutionarily stable when it is lost more slowly than it is gained101. Multistate also produces posterior distributions of the ancestral character state at the root of the phylogeny. We scaled the tree by a default constant (mean of 0.1) in all analyses101 and used an exponential prior whose mean was seeded from a uniform hyperprior ranging from 0 to 10 to reduce inherent uncertainty and biases of prior choice98. We ran all Multistate analyses with Reversible Jump (RJ) Markov chain Monte Carlo (MCMC) methods. MCMC samples models in direct proportion to their fit to the data, generating a posterior distribution of parameter estimates for each transition rate, and RJ sets some parameters equal to zero or equal to one another, thereby reducing model complexity and over-parametrization97,98,100. As a result, posterior distributions of parameter estimates may not be normal; we thus summarised results by presenting the mean and mode of the posterior distributions of each parameter estimate, 95% higher posterior density, and percentage of models with parameters estimated to be 0. We ran all MCMC chains for 320 million iterations in addition to a burn-in of half a million iterations, sampling every 200,000 iterations. All chains converged and showed good mixing as indicated by their effective sample sizes of 2000 and visual inspections of their traces in Tracer v1.6102. All analyses were run in triplicate and the three independent chains converged on very similar solutions, leading to qualitatively similar results. Here we present the results from the first chain. We ran RJ-MCMC Multistate analysis on the sexual system (Supplementary Table 4) as a binary state (gonochoristic or hermaphrodite) and as a four-state categorical variable (gonochorism, protandry, protogyny, simultaneous hermaphroditism). Bidirectional sequential hermaphrodites were excluded from the latter analysis as the sample size of extant species was too low so the chains failed to converge and mix properly when the sexual system was studied as a five-state categorical variable. For the analyses with a two-character state, we graphed the evolutionary history of sexual systems on the phylogeny using maximum likelihood (ML) in the R package ape v.5.3103, which provided a reasonably close approximation of the RJ-MCMC Multistate results (this was not the case with the four-character state analysis).

We used phylogenetic generalised least square (PGLS) models104,105,106 to test for the association of each life-history trait, entered as dependent variables, with sexual systems entered as an independent discrete variable with three possible states (gonochorism, protogyny, protandry), as not enough data were available for simultaneous hermaphroditic species and bidirectional sex changers. PGLS models were run with the R package caper107 in Maximum Likelihood. The parameter λ of PGLS models quantifies the strength of the phylogenetic signal in the model residuals104. λ ranges between zero (there is no phylogenetic structure in the data) and one (the species share similarity in trait values directly proportional to their common evolutionary time, under the Brownian motion model of evolution104,106). Continuous variables were log10-transformed to meet assumptions of normality. The statistical tests in PGLS were two-tailed.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.