Speciation across depth gradients in reef corals

Abstract

Ecological speciation through adaptation to different habitats can readily occur without strong geographic isolation when the same traits underlie both ecological divergence and reproductive isolation. In light-dependent corals with environment-mediated spawning, adaptation to varying light regimes across depths provides opportunities for such speciation. We show that depth-related distributions are common among sister lineages of corals within the photic zone. We then investigated molecular drivers of depth-associated adaptive divergence by analyzing sequence variation in proteins related to environmental sensing in depth-segregated and light-dependent lineages in the Orbicella species complex. Specifically, we analyzed 1) two genetically divergent ecotypes of Orbicella faveolata across a depth gradient, and 2) two depth-segregated sister species, O. annularis and O. franksi, with different spawning times following sunset. Genome-wide analyses indicate divergence across depths occurred through adaptation via positive selection on G-protein-coupled receptors (GPCRs). These molecules mediate chemo/photo/thermo-reception, enhancing physiological adaptation across environments, and are also involved in reproductive isolation via differences in spawning time. Our study proposes a molecular mechanism for the origin of depth-segregated coral species, common across anthozoans, in which ecological divergence operates at spatial scales smaller than their larval dispersal potential, and highlights avenues contributing to generating biodiversity in the sea.

Introduction

Ecological speciation, via adaptation to different habitats¹, has been proposed as a major mechanism for the origin of species², with many examples ranging from plants to vertebrates³. Ecological speciation is easier to identify when traits associated with habitat adaptation are also involved in reproductive isolation⁴, eliminating the need for linkage between genes related to habitat adaptation and those related to reproductive isolation⁵. Unveiling the molecular mechanisms behind ecology-driven reproductive isolation remains central to understanding this process⁶. Comparisons of recently diverged species (<10 K generations) facilitate the identification of genes driving ecological speciation because most alleles across their genomes have similar frequencies with little differentiation, with only a few regions, likely tied to speciation, exhibiting major shifts in allele frequencies and increased divergence⁷.

Segregation across depth between closely related lineages^8,9 is a promising area of study for understanding the role of ecological speciation in light-dependent corals (i.e., with photosynthetic symbionts). Corals living at different depths are exposed to distinct light levels, which affects not only photosynthetic physiology but also many aspects of gamete release and larval settlement cued by light, such as small differences in the timing of spawning, namely a narrow and discrete temporal window (i.e., minutes or hours apart) between species¹⁰, both of which are well documented for some groups^11,12,13. However, key remaining questions are: (1) whether divergent genes between nascent depth-segregated species are tied to adaptation to different depths, (2) whether these same genes are linked to reproduction contributing to reproductive isolation, (3) whether natural selection has shaped the evolution of these genes, and (4) whether processes operating within species fuel divergence between species.

We begin by documenting patterns of intraspecific genetic divergence along depth gradients across the coral tree of life to provide an estimate of how frequently closely related coral lineages differ in their depth distributions, ranging from just a few to several tens of meters. We then focus on genetic analyses of common Caribbean reef builder corals in the Orbicella species complex. In addition to differences in morphology^14,15, ecology¹⁶, and physiology¹⁷, these species have well-documented differences in depth distribution and reproductive biology¹⁸. The spawning differences are remarkably consistent within an evening, across years, and regions¹³, and are responsible for their reproductive isolation, although the precise mechanisms controlling reproductive barriers remain unknown.

Specifically, Orbicella annularis and O. franksi are the most closely related of the three described species in the genus^14,15,19; O. annularis occurs in shallower water than O. franksi and also spawns about 2 h later after sunset on the same day^13,16. The third species, O. faveolata, has a wider depth distribution that overlaps with both O. annularis and O. franksi; its spawning time largely overlaps with that of O. annularis, but the gametes of O. faveolata are generally incompatible with the gametes of the other two species^18,20. However, as we document below, O. faveolata itself consists of two cryptic lineages (i.e., genetically distinct but morphologically similar) that inhabit different depths and thus represent an additional (younger) example of depth-associated divergence parallel to that of O. annularis and O. franksi.

Light quantity and quality decrease rapidly with depth in shallow marine areas, while other abiotic factors remain relatively stable²¹ (e.g., temperature, oxygen, nutrients), thus acting as a major factor structuring shallow-water coral communities, given coral dependence on sunlight for most of their nutrition via endo-photosymbiotic microalgae^22,23. Here, we studied morphological and genome-wide variation across shallow depth gradients to test the hypothesis that the same genes that regulate environmental sensing and reproduction, related to adaptation to different light environments, also diverge early in the formation of new species of depth-segregated corals. We identify these genes as G-Protein-Coupled Receptors (GPCRs), known to serve as environmental sensors activated by chemical, thermal, and light cues that control calcium (Ca²⁺) and glutamate signaling, modulating intracellular signal transduction pathways, and are probably involved in differential timing of spawning and larval settlement preferences across depths under different photic environments^{24,25,26,27,28}. Thus, divergence between incipient species is likely driven by genes responding to varying environmental signals across depths. We build upon the current knowledge that these receptors are involved in spawning behavior and larvae photo- and chemosensitivity, and thus, we hypothesize that variation in these genes could drive reproductive isolation and speciation in light-dependent and depth-segregated corals.

Results

Divergence across depths is widespread across the coral tree of life

To characterize the generality of the divergence across depth gradients in corals within the photic zone (<150 m), we leveraged a review²⁹ of 102 publications and related references therein on cryptic lineages, defined as morphologically indistinguishable species complexes that overlap in range but that exhibit signatures of barriers to gene flow, across the coral tree of life. Out of 24 reported cases with substantial genetic divergence occurring in sympatry, at least 15 were associated, directly or indirectly, with depth differences across several geographical scales²⁹ (Fig. 1A and Supplementary Data 1). Divergence across depths has been uncovered in 13 genera of stony (Scleractinia) coral, and two genera of soft (Malacalcyonacea and Scleralcyonacea) coral species across both brooders and broadcast spawners (Fig. 1B, reviewed in ref. ²⁹), which differ in dispersal potential, thus influencing the distribution of genetic variation across the seascape.

Fig. 1: Geographic, phylogenetic, and reproductive evidence of depth-related divergence in shallow and mesophotic (<150 m) corals.

A World map showing the ocean basin where depth-segregated coral lineages (cryptic or not) have been reported based on multiple lines of evidence indicated by symbols (dot size indicates number of reports per geographic location) (reviewed in Grupstra et al.²⁹). Map generated using R package maps v 3.4.3; GPL-2 license. B Examples are plotted against a phylogenetic tree for all orders/families in the class Anthozoa (Phylum: Cnidaria), based on 1729 loci from 234 species (data from Quattrini et al.¹²³). Shown are 15 examples of depth-related genetic divergence across coral taxa worldwide (Caribbean and Gulf of Mexico; Western, Central, and Eastern Tropical Pacific). Reproductive mode is indicated by letters (Brooder-BS, Broadcast Spawner-BR). Marker types represent kind of molecular/biological evidence for divergence (SNPs, Microsatellites- Micro, Morphology- Morph, RFLSPs, and Phenology- Pheno). C Spawning times after sunset in Panama for Orbicella annularis and O. franksi, sister species that occur in shallower and deeper water, respectively (data from Levitan et al.¹³). Source data are provided as a Source Data File.

Differences in depth distribution ranges vary from a few meters, like in Favia fragum and Acropora spp. (Caribbean), to several meters, as in the rest of the species (Caribbean, Central, Eastern Tropical, and Western Pacific). Altogether, this illustrates the pervasiveness of depth divergence across the coral phylogeny^{30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51} within the photic zone and underscores the role of depth and light availability in frequently driving lineage divergence among ocean basins worldwide. Importantly, in the case of O. annularis and O. franksi, depth segregation across a light gradient determines different spawning time that maintain species boundaries through a fine temporal reproductive barrier between them despite occurring in sympatry and with inter-species crosses possible under lab settings¹⁸ (Fig. 1C).

Divergence across depths is widespread and occurs even in well-studied species

To understand the mechanisms driving speciation in shallow depth-segregated corals experiencing divergent light environments, we initially test for variation within species by sampling colonies of the common Caribbean reef builder O. faveolata across a depth (6–19 m) and steep light (~600 µmol m⁻² s⁻¹) gradient at Media Luna Reef (Puerto Rico, US). Genome-wide single nucleotide polymorphisms (SNPs = 12,859—unlinked markers) revealed that this well-studied species is composed of two previously unrecognized (i.e., cryptic) lineages that segregate across depths (Fig. 2A, B, Supplementary Fig. 1), with no signs of admixture between them (Global F_ST = 0.13; Fig. 2C and Supplementary Fig. 1 and 2A–E). Colonies at 6 and 9 m exclusively belonged to the Shallow lineage, all colonies at 19 m belonged to the Deep lineage, and both lineages co-occurred between 12 and 18 m (Fig. 2D). Mean polyp density differed significantly among depths (Kruskal–Wallis test on 103 colonies, X² (4) = 39.735, p ≤ 0.01). Pairwise polyp density comparisons (Nemenyi Post-Hoc) showed colonies to differ morphologically between 6 and 12 m (p ≤ 0.01), 6 and 19 m (p ≤ 0.05), 9 and 12 m (p ≤ 0.01), 9 and 18 m (p ≤ 0.01), and 9 and 19 m (p ≤ 0.01) depth. Yet, while the colonies of the Shallow lineage have higher polyp density than Deep ones (Wilcoxon test on 89 colonies −W = 413, p ≤ 0.01) (Fig. 2E), such differences decrease when comparing them at depths where the two lineages co-exist, 12 and 18 m (Supplementary Fig. 2F). This degree of morphological overlap highlights the challenge of distinguishing these lineages in the field, as corals exhibit a high degree of morphological plasticity. However, lineages could respond differentially to environmental factors between depths, as indicated by a broader displacement of mean polyp density of the Shallow lineage colonies from 6–9 m to 12–18 m when compared to that same displacement measure in the Deep lineage colonies from 19 m to 12–18 m (Supplementary Fig. 2G), where the light attenuation coefficient decreased by ~33% in just 1 m from 18 to 19 m.

Fig. 2: Evidence for depth-segregated cryptic lineages of Orbicella faveolata.

A Representative colonies of O. faveolata in the field found to belong to two distinct cryptic lineages (Shallow/Deep) at Media Luna Reef, La Parguera (Puerto Rico). B Average diffuse attenuation coefficient at Media Luna Reef. Light decrease from 0 to 20 m is ~600 µmol m⁻² s⁻¹. C Discriminant analysis of principal components (DAPC; optimal K = 2, and the number of PC = 1) inferred from 12,859 unlinked single nucleotide polymorphisms (SNPs). D Relative abundance of the Shallow lineage (n = 59) was almost twice that of the deep (n = 30), with the former found at 6 (n = 16), 9 (n = 21), 12 (n = 16), and 18 m (n = 6), while the latter distributed equally at 12 (n = 10), 18 (n = 10), and 19 m (n = 10). E Best-fit demographic model from GADMA analysis for the history of demography and genetic exchange between the Shallow and Deep lineages of O. faveolata. This best (IMi) model includes two epochs in each lineage, with asymmetric migration events (vertical black arrows), and exponential and linear population growth, in the Shallow and Deep lineages, respectively. Horizontal lines are separated by 13,360 individuals, that represent the estimated original effective population size. F Mean polyp density (1.5 cm⁻²) of colonies by lineage identified by DAPC and ADMIXTURE analyses. Sample size are as follows: Shallow lineages n = 59 colonies, Deep lineage n = 30 colonies. Polyp density was estimated per colony from four randomly selected grid squares per image. Boxplots show the median with quartiles and whiskers extending 1.5× the interquartile range. Statistical significance was assessed using two-sided Wilcoxon rank-sum tests between lineages. Source data are provided as a Source Data File.

The divergence between these depth-specialized lineages of O. faveolata is recent, around 212 Kyr or approximately 6000 generations, assuming a generational time of 35 years (Fig. 2F and Supplementary Fig. 3). Subsequently, both lineages experienced population expansions with episodic asymmetric events of gene flow or migration (vertical black arrows in Fig. 2F). Furthermore, a genome-wide phylogenetic reconstruction of the Orbicella genus and its closest living relatives (Cyphastrea serailia and C. microphtalma) placed these two lineages as more closely related to each other than other O. faveolata populations across the Caribbean (Supplementary Fig. 2H), but clearly within the O. faveolata clade.

Genes near outlier SNPs are environmental sensors involved in spawning behavior

To better identify loci with significantly higher genetic differentiation (i.e., putative outlier SNP loci) between Deep and Shallow lineages of O. faveolata, we scored genetic diversity across their genomes. We identified 253 outlier SNPs, with 17 consistently detected by three different methods (per-site F_ST, BayeScan, and PCAdapt; Fig. 3A) and therefore considered for further functional genomic analyses. These 17 loci lie close (within ±10 kb) to 112 annotated protein-coding genes, out of which 64 corresponded to unique annotations (Fig. 3B and Supplementary Data 2).

Fig. 3: Genome-wide outlier loci linked to environmental and reproductive signaling pathways.

A Venn Diagram of common outlier loci identified by three detection methods (Per site F_ST, BayeScan, and PCAdapt). B Manhattan plot of F_ST across the genome. The x-axis shows chromosomal positions, and the y-axis shows F_ST values. Alternating colors indicate different chromosomes. Green dots are 17 common outlier loci with protein-coding genes within ±10 Kb windows for each outlier SNP. Labeling (in red) indicates conserved protein domain families for top-result genes with functional annotations involved in GPCR signaling transduction pathways regulating coral reproductive behavior. For complete gene annotation, see Supplementary Data 3. Source data are provided as a Source Data File.

A rank-based Gene Ontology (GO) enrichment analysis, based on the above-mentioned genes, indicates that GO terms related to reproduction, neurotransmitters, and ion flux signaling pathways are significantly overrepresented (p < 0.01) (Supplementary Fig. 4). GO terms that best represent independent groups of significant contribution to each GO subdivision are ovarian follicle cell migration (GO:0007297; GO:0007298), cellular potassium ion homeostasis (GO:0030007), and amino acid homeostasis (GO:0080144) for Biological Processes (Supplementary Fig. 4A); Nicotinic acid adenine dinucleotide phosphate(NAADP)-sensitive calcium-release channel activity (GO:0072345), and P-type sodium potassium-exchanging transporter activity (GO:0086037; GO:1990239) for Molecular Function (Supplementary Fig. 4B); and lateral part of the cell (GO:0097574; GO:1990794) for Cellular Component (Supplementary Fig. 4C).

Further, we compared our results to studies of gene expression in corals during reproductive events^24,25,27,52. Importantly, we found that 46.8% (30/64 of unique annotations) of these genes are reported to be involved in environmental sensing during coral spawning or larval release events (Table 1 and Supplementary Data 3). Specifically, these genes regulate cell signaling pathways via GPCRs through cyclic adenosine monophosphate (cAMP) and phosphatidylinositol (PLC) enzymes, and glutamate and Ca²⁺ signaling. These GPCRs are intermembrane proteins that control voltage-gated channels, particularly those associated with Ca²⁺ ions, associated with rhodopsin-like molecules, which respond to chemical (e.g., neurotransmitters and hormones^28,52), light (e.g., red and blue wavelengths^24,25,26,27), and thermal cues^26,52 (Fig. 4; Table 1 and Supplementary Data 3). Together, these genes have been found to play a role in environmental sensing and signaling transduction pathways, mediated by Ca²⁺ as a central regulator, in cnidarian reproductive cycles^24,25,27.

Fig. 4: Molecular signaling pathway during coral spawning (after Rosenberg et al.²⁴).

All genes reported by the cited model are included in this diagram. Pathways with protein-coding genes within ±10 Kb windows for each outlier in this study are highlighted in red. Created in BioRender. https://BioRender.com/24xay7z. Source data are provided as a Source Data File.

Table 1 Molecular signaling pathway during coral spawning (after Rosenberg et al.²⁴)

These reproductive processes are triggered by the interaction of differential light wavelengths (e.g., moonlight), neuropeptides (e.g., glutamate or dopamine), and temperature variation (e.g., diel-light cycles), which excite chemo/photo/thermo-receptors (rhodopsin/melanopsin-like GPCRs), releasing glutamate, which modulates the transfer of Ca²⁺ across the cell membrane^24,25,27. This, in turn, activates Ca²⁺-modulated (CAMs) and protein kinase C (PKC) proteins, which act as transcriptional factors^24,25,26, regulating intracellular activity and affecting cellular pathways, such as apoptosis, cytoskeleton reorganization, and muscle fiber contraction, ultimately leading to spawning^{11,52,53,54,55} or larval release⁵² (Fig. 4).

Collectively, these results suggest that the genomic differentiation between Shallow and Deep lineages occurs mostly at loci encoding G-protein-coupled transmembrane receptors that respond to environmental signals such as luminous stimulus, neuropeptides, and temperature changes. These cellular sensors control cytoplasmic ion fluxes such as Ca²⁺, characteristic of phototransduction pathways in corals, and affect signaling pathways linked to synchronization of reproduction in both brooding and broadcast spawning corals, such as gonad and larval maturation, and finally, gamete spawning or larval release (Fig. 4).

Clines of SNPs tied to environmental sensing indicate strong natural selection

To quantify the strength of selection in genes putatively tied to speciation across depths in O. faveolata, we estimated allele frequencies across depths for those SNPs previously identified as outliers. We found pronounced steep clines, indicating strong spatial segregation for genetic regions under divergence in less than 100 linear meters (Fig. 5 and Supplementary Fig. 5). Such allele frequency changes coincide with a change in depth (<20 vertical meters) and light decrease (~600 µmol m⁻² s⁻¹)⁴⁴, which were also reflected in morphological differences and genetic segregation of colonies belonging to the Shallow and Deep lineages (Fig. 2). Estimations of the strength of selection on each locus, derived from the center and width of the cline, consistently showed that all outlier SNPs experienced strong coefficients of selection (s > 1%). Overall, selection coefficients were higher under the assumptions of heterozygous disadvantage (s = 8–16%) than in the ecotone model (s = 0.9–2%) (Supplementary Data 4).

Fig. 5: Characteristic cline shape estimates of outlier SNPs along the depth gradient of O. faveolata. Depiction of two characteristic cline shape estimates of the 17 outlier SNPs along the geographic and depth gradient of differentiation.

Representative gene names close to the SNPs under strong (s > 1%) natural selection. Solid lines depict the maximum-likelihood cline fit, and shaded error bands represent 95% confidence intervals around the fitted clines as estimated by HZAR. Points represent observed allele frequencies at each depth. Source data are provided as a Source Data File.

Genes with dual sensory/reproductive roles display signatures of positive selection

To test if the same genes found under strong selection between Shallow and Deep lineages have been subject to positive diversifying selection in other Orbicella lineages, we aligned the genomes of O. faveolata and its closest relatives, O. franksi⁵⁶ and O. annularis, to compile all orthologous protein-coding-sequences (CDSs) within ±10 kb of the previously identified SNP outlier loci. We then estimated pairwise (O. annularis vs. O. franksi and O. faveolata vs. O. franksi/O. annularis) non-synonymous (d_N) to synonymous (d_S) substitution rates⁵⁷ (ω = d_N/d_S) for those CDSs (Fig. 6). We successfully compared 78 CDSs between O. annularis and O. franksi, finding 14 genes [11 functionally characterized proteins (Supplementary Data 5)] with increased d_N/d_S ratios (ω > 0.5), out of which five are under positive diversifying selection (ω > 1) (Fig. 6A). Likewise, we compared 77 CDSs between O. faveolata and O. franksi and found 18 genes [11 functionally characterized proteins (Supplementary Data 6)] with increased d_N/d_S ratios (ω > 0.5), and five of these CDSs displaying evidence of positive diversifying selection (ω > 1) (Fig. 6B). Finally, we compared 76 CDSs between O. faveolata and O. annularis, with 20 genes [15 functionally characterized proteins (Supplementary Data 7)] with increased d_N/d_S ratios (ω > 0.5), with 7 of these CDSs under positive diversifying selection (ω > 1) (Fig. 6C).

Fig. 6: Signals of positive selection in proteins across Orbicella species.

Pairwise estimate of the nonsynonymous over synonymous fixation rate ratio ω (d_N/d_S) of orthologous protein sequences between Orbicella species found within ±10 Kb windows spanning outlier loci. A Orbicella annularis and O. franksi. B O. faveolata and O. franksi. C O. faveolata and O. annularis. This graph shows ω values among alignment windows with at least three non-synonymous substitutions as implemented in PSGfinder. This measures the extent to which natural selection modulates protein sequence evolution. Black solid (ω > 1) and gray dashed (ω > 0.5) lines indicate thresholds above which positive selection might be acting. Source data are provided as a Source Data File.

Among the highest-ranked CDSs for the O. annularis and O. franksi comparison, we found transferases involved in cell structure and communication across animal phyla⁵⁸ (e.g., carbohydrate sulfotransferase (ω = 3.15) and mannosyltransferase (ω = 1.00)), along with two genes (ω = 1.08–1.88) coding for a stress transcription factor (mafK)⁵⁹, and an uncharacterized protein (ω = 1.26).

For the O. faveolata vs. O. franksi comparison, the highest-ranked CDS (ω = 3.56) is a domain-containing protein zinc finger (BTB/POZ domain-containing protein KCTD7). This gene encodes a member of the potassium channel tetramerization protein family, evolutionarily conserved from Drosophila to mammals, involved in the modulation of downstream signaling pathways such as glutamate homeostasis and neural excitability^60,61. Another gene under selection (ω = 1.57) was a transmembrane protein (toll-like receptor 6) responsible for innate immune response to pathogens and inflammatory processes in O. faveolata^62,63. Immune-related genes can be considered hotspots of positive selection in taxa as distantly related as birds and mammals due to their central role in fighting off pathogens, making them repeated targets of selection⁶⁴. The last gene with ω > 1 (protein white-like) also belongs to a transmembrane protein family of ATP-dependent pumps involved in the uptake of guanine and tryptophan needed for visual pigments and neurotransmitters in Drosophila melanogaster, which affect sexual selection via altered courtship male behavior⁶⁵.

Interestingly, the highest-ranked CDS (ω = 1.78) between O. faveolata and O. annularis was also the BTB/POZ domain-containing protein KCTD7, highlighting the conserved role of glutamate and ion signaling across animal phyla. The second- and third highest-ranked CDSs, as in the O. annularis and O. franksi comparison, were mannosyltransferase (ω = 1.57) and toll-like receptor 6 (ω = 1.56), followed by an uncharacterized protein (ω = 1.25), and two transmembrane proteins (tetraspanin-17-like (ω = 1.15) and protein white-like (ω = 1.14)). Lastly, the last gene under selection was granulin (ω = 1.13), a glycoprotein.

To further test for signals of positive diversifying selection, we used those CDSs with increased d_N/d_S ratios (ω > 0.5) between species pairs to conduct four additional positive selection analyses (aBSREL, BUSTED, MEME, and FEL)⁶⁶ based on sequence variation within a phylogenetic tree using the HyPhy⁶⁷ (Hypothesis testing using Phylogenies) software package. Three of these tests (MEME, aBSREL, and FEL) revealed 58 codon sites evolving under positive selection across the Orbicella phylogenetic tree inferred from those same CDSs (Supplementary Data 5–7). The codons correspond to 10 unique characterized proteins, plus an uncharacterized one. These proteins are (i) a transcription factor regulating gene expression (mafK) during stress⁵⁹, (ii) a serine-rich domain (rtoA) mediating cell cycle and cellular differentiation in the amoeba Dictyostelium⁶⁸, (iii) a kinase cascade inhibitor (Sprouty) in mice⁶⁹, (iv) a calcium-modulated sperm protein (abnormal spindle-like microcephaly-associated protein homolog-DD_CABYR_SP17) that aids in locomotion and also acts as a docking site during fertilization (e.g., acrosome reaction) in mammals⁷⁰, (v) a rhodopsin-like (Allatostatin-A) in the GPCR family with neuropeptide regulatory functions across invertebrate reproductive cycles⁷¹, (vi) a glycoprotein (granulin) part of mammalian spermatogenesis^72,73, (vii–viii) phospholipases involved in the glycosylphosphatidylinositol (GPI)-anchor pathway, such as lipid remodeling in mice⁷⁴ (post-GPI factor 3-like and GPI mannosyltransferase 4-like), (ix) a transmembrane protein (79-like) involved in intracellular calcium trafficking and exocytosis regulation⁵⁸, and (x) a cell surface receptor protein (PHD finger protein rhinoceros-like) involved in the tyrosine kinase signaling pathway⁵⁸. These results lend further support to the role of positive selection acting not only on the proteins that drive adaptive divergence between Shallow and Deep lineages of O. faveolata, but also among Orbicella species throughout the recent evolutionary history of the genus.

Discussion

Similar to many other marine taxa⁸, speciation between depth-segregated lineages is common across the coral tree of life, particularly in the photic zone and shallow waters where tropical reefs occur (Fig. 1). This pattern reflects both limited knowledge of deeper-water taxa and the well-documented ecological segregation of coral reef species along light gradients (e.g., into mesophotic depths). To explore this further, we investigated a recent divergence within a common Caribbean reef builder (Orbicella faveolata), where lineages that diverged approximately 212,000 years ago segregate across a narrow depth range (<20 m). This pattern is consistent with ecological specialization driven by positive, diversifying natural selection^75,76. Genome differentiation between Shallow and Deep lineages occurs mostly at loci encoding GPCRs that respond to environmental sensing, control cytoplasmatic levels of Ca²⁺ flux at cellular channels, and affect signaling pathways linked to coral reproductive cycles (Fig. 4). This pattern is echoed at macroevolutionary scales across cnidarian species in which neuropeptide (e.g., glutamate), light (e.g., solar insolation and moonlight) and temperature variations (e.g., diel-light cycles) regulate reproductive physiology and phenology^{77,78,79,80,81}.

Coral reproductive mode, divergence and dispersal capabilities

Corals rely on their larval stages for dispersal of sexually produced individuals, unlike asexual propagation through fragmentation or budding. Larval competency shapes patterns of genetic differentiation (e.g., isolation by distance) across geographic scales. Reproductive timing, which is often linked to traits such as brooding, spawning, hermaphroditism, or gonochorism, drives temporal segregation. This timing is influenced by environmental cues and reinforced by the chemical signals of congeners, helping synchronize reproduction with favorable conditions⁸².

Broadcast spawning corals typically display short and discrete spawning seasons, with gamete release synchronized to an astonishing degree within species, sometimes within a few minutes each year^{53,54,83,84,85}, with successful planulae (i.e., typical mature and planktotrophic cnidarian larva) dispersing long distances. In contrast, brooding species exhibit frequent periods of planulae release throughout reproductive seasons, with rapid larval settlement and thus a limited dispersal range^86,87,88. Depth-associated divergence has been reported across geographies for both reproductive modes. While differences in dispersal may explain some cases, our understanding of coral reproductive biology remains incomplete. Notably, corals, either brooders or broadcasters, use the same environmental cues tied to natural cycles to time reproduction. Interestingly, species with either reproductive mode alter their reproductive timing under light and temperature manipulation experiments^{24,27,52,77,84}, hinting at a common mechanism for fine-tuning reproductive activity via environmental sensing⁸⁹.

Sensory cues and the induction of coral reproduction

Recently, gene knockout/expression studies have unequivocally linked the biochemical cascades triggering gamete release in cnidarians to the upregulation of rhodopsin-like proteins responsible for transducing peptide, light, and temperature stimuli into cellular responses^24,25,26,79. Ultimately, the timing of reproductive activity (e.g., spawning) determines whether populations can interbreed, allowing (or not) cross-breeding between adapted species to different depths and environmental regimes (e.g., light and temperature)^{13,18,83,90,91,92,93,94,95}. Although the evolutionary implications of breaks in reproductive timing are expected to be stronger for broadcast spawners than brooders, because broadcast-spawned gametes remain viable for a few hours, while brooders can collect sperm over many days and release planulae either seasonally or year-round⁸⁶.

Given our findings, and based on the evidence that light is implicated not only in adaptations to depth within the Orbicella species complex, but also involved in the differential timing of O. annularis and O. fransksi spawning at different depths within the same day, we hypothesize that speciation in corals adapted to different depths and light levels proceeds mainly by changes in the expression of genes responding to different wavelengths of light across depths. Specifically, these changes could occur in rhodopsin-like GPCRs that activate light-sensing and Ca²⁺ mediated transduction cascades conducive to spawning, and thus likely under strong natural selection for reproductive synchrony^24,25,26.

Light as a driver of diversity on shallow coral reefs

Shallow tropical broadcast spawning coral species are structured along small spatial scales despite possessing larvae capable of long-distance dispersal among and within reefs^86,96. Depth-related divergence in corals with long dispersal occurs through a combination of multiple processes throughout their life histories. Coral planulae detect changes in light wavelengths^97,98,99, showing preferences for substrate color during settlement¹⁰⁰, and are more likely to settle in areas where crustose coralline algae are adapted to light environments similar to those of their coral parents¹⁰¹. Together, these cues act as a depth gauge, creating habitat choice patterns between environments with suitable conditions (e.g., irradiance or temperature) that create strong ecological partitioning from an early life stage^98,101,102. Moreover, the long interval from juvenile to adult stages typical of many broadcast-spawning corals further lends itself to long pre-reproductive selection and the extirpation of mismatched habitat-genotype individuals via immigrant inviability⁴⁴. Lastly, corals at different depths will perceive light-spawning cues differently due to the attenuation of long wavelengths of light by water, which can drive asynchronous spawning, thus giving rise to temporal reproductive isolation via allochocrony¹³.

Light is central to the metabolism of mutualistic corals because photosynthetic intracellular algae can provide more than 95% of their energetic needs, and the way corals harvest light may change across depths as the quantity and spectra of light decrease across depths^21,103. Therefore, corals have evolved a rich morphological and physiological repertoire to cope with strong gradients in light availability and intensity owing to their sessile nature^104,105. Both corallite shape and colony architecture respond to light intensity to optimize autotrophy, enabling corals with a wider morphological response to occupy a broader depth range^106,107. Higher polyp density found in shallow colonies, such as in O. faveolata Shallow (Fig. 2E and Supplementary Fig. 2F), acts as a self-shading mechanism to deal with excess UV light, whereas colonies in deep environments have fewer polyps to increase light capture, thereby enhancing photosynthesis^107,108. The higher density of polyps in the Shallow lineage, which overlaps at mid-depths with that of the Deep lineage (Supplementary Fig. 2F), is consistent with findings where multiple morphologies at the center of light gradients produce similar photosynthetic outputs¹⁰⁸, an outcome mediated by corals’ wide phenotypic plasticity¹⁰⁴.

The role of GPCRs in coral sensory ecology

Our combined analyses of genome-wide differentiation and positive selection tests identified signatures of genomic divergence between depth-segregated lineages in cellular membrane-bound receptors that relay environmental signals that regulate reproductive timing and likely facilitate habitat segregation. We identified genomic regions enriched for neurotransmitters involved in chemo (e.g., Allatostatin-A and dopamine) and photoreception (e.g., glutamate and calcium channels) in the rhodopsin-like GPCR molecules central to transduction signaling cascades mediating coral spawning (Fig. 4 and Supplementary Data 3). Specifically, the biochemical cascades triggering gamete release in cnidarians operate via the upregulation of rhodopsin-like proteins responsible for the reception of different wavelengths of light^{24,25,26,90,91,92}. This process is triggered by differential light spectra, which excite light-absorbing pigments (rhodopsin-like GPCR proteins), initiating glutamate signaling pathways that modulate the transfer of Ca²⁺ ions across the cell membrane^24,89. This in turn activates Ca²⁺ modulated (CAMs) and kinase C proteins (KMC) that act as transcriptional factors²⁴. Regulatory changes in intracellular activity affect multiple cellular pathways (e.g., cell cycle, focal adhesion, cell migration, cytoskeleton reorganization, and muscle fiber contraction) that synchronize gamete^{24,25,26,89,90,91}, likely acting as a temporal reproductive barrier in corals adapted to different light regimes (e.g., depth)¹³.

Given the significant body of experimental evidence supporting light as the primary sensory cue for coral spawning, we hypothesize that differential timing of spawning driven by different light perceptions across depths is fine-tuned by expression changes in rhodopsin-like genes, causing corals exposed to different photic environments to perceive spawning cues differently due to light attenuation. The precise regulation of coral spawning (day and hour) is regulated by light/dark cycles associated with twilight and moonrise (lunar cycles) detected by light-sensing proteins, such as rhodopsin and cryptochromes^{25,79,80,91,109}. Individual rhodopsin genes can evolve differing wavelength sensitivities along light intensity gradients^80,110, registering distinct spectral signatures across depths, where a minor change in peak sensitivity shifts color perception¹¹¹ (e.g., from red to blue), as it naturally occurs during light absorption by water. Under this scenario, larvae with different rhodopsin genes, or differentially expressed sets of genes, would settle around their optimal spectral peak, where they could readily respond to light cues closer to their spectral sensitivity as compared to those found at suboptimal light fields¹¹⁰ (i.e., far from the maximum rhodopsin sensitivity). Over time, individuals better-tuned for a given wavelength are selected for¹⁰², and in turn, when sexually mature, these corals will rely on the same photoreception mechanism used for settlement to sense light cues and synchronize spawning during evening twilight around moon-phase^{25,77,110,112} (Fig. 7).

Fig. 7: Proposed scenario for ecological adaptation to different light regimes across depths in Orbicella lineages via opsin evolution.

Alternate opsin genes with distinct wavelength sensitivity (i.e., red and blue) regulate larvae (small blobs around spawning colonies) settlement preferences, causing larvae with increased sensitivity to blue light to settle in deeper waters, as opposed to shallower waters where larvae carrying a red-shifted sensitivity gene settle instead. Over time, matched genotype-light regime larvae/colonies outperform corals less well-tuned to the prevalent light conditions. In turn, these sexually matured colonies perceive light-mediated reproductive cues differently due to light attenuation in the water, leading to different spawning times that would hinder gene flow between depth-segregated lineages (modified from Sweeney et al.¹¹⁰. Reproduced/adapted with permission from Journal of Experimental Biology).

Deep sea Anthozoa are known to lack opsins⁷⁸. Opsin-gene knockout suppresses spawning in a jellyfish (Clytia hemisphaerica) by disrupting maturation-inducing hormones in gonad ectoderm cells⁷⁹. Similarly, colonies of the same coral species exposed to different light wavelength sources, or changes in light regimes, shifted their spawning time resulting in out of synchrony spawning^77,113,114. Additionally, Ca²⁺ is as a mediator in light responsivity of corals, where altered cytoplasmatic Ca²⁺ levels via pharmacological means delays broadcast spawning in corals like O. franksi⁸⁹. In this regard, rhodopsin-like GPCR genes expressed early in coral larvae and thought to be involved in finding suitable settlement sites, but that also regulate spawning behavior in adults, can be a trait subject to divergent selection that also contributes to non-random mating (also known as magic traits^4,5) by creating habitat-specificity as well as controlling reproductive phenology across light environments. In consequence, light-mediated differential time of spawning via light-sensitive proteins may serve as a major driver of reproductive isolation in depth-segregated coral lineages within the photic zone.

In fact, the role of GPCR rhodopsin with pleiotropic effects, or affecting multiple traits (e.g., a magic trait), is well-established in fish adapted to different wavelengths across multiple species and geographic locales^115,116,117. A single amino acid substitution in a opsin gene in the Baltic herring has evolved more than 20 times independently in other species adapted to red-shifted light environments¹¹⁵. Such a pattern of parallel divergence could occur independently in corals as they colonize waters with different optical properties, favoring rhodopsin divergence that fuels speciation¹¹⁵. Our phylogenetic analysis revealed Shallow and Deep lineages to be more closely related to each other than other O. faveolata populations across the Caribbean, suggesting that these lineage divergences may be independently repeated at other locations. Testing this hypothesis will require a higher sampling effort across localities with varying water optical properties spanning a broader geographical range.

Other factors involved in depth-segregation of lineages

Still, other aspects may also be at play in segregating coral lineages by depth. For instance, if the timing of spawning does not fully isolate these corals and viable hybrids are produced, those admixed individuals may be unfit and die soon after settlement or unable to spawn in synchrony with conspecifics. Our estimates of selection across depths indicate that narrow geographical clines (Fig. 5) along steep environmental gradients can arise from immigrant inviability^44,118 (i.e., heterozygous disadvantage) or strong habitat selection¹¹⁹ promoting adaptive divergence between locally adapted phenotypes¹²⁰. Similarly, differential temperature regimes across depths may influence the onset of the gametogenesis causing desynchronization of reproductive cycles leading to temporal reproductive isolation across depths^12,121.

Furthermore, increased d_N/d_S ratios (ω > 0.5) in multiple codons revealed accelerated rates of protein evolution consistent with adaptive divergence among Orbicella species. Several protein domains involved in transcription (mafK⁵⁹), cell differentiation (rtoA⁶⁸) and signaling pathways (Sprouty⁶⁵, post-GPI factor 3-like⁷⁴), sperm-egg recognition proteins (abnormal spindle-like microcephaly-associated protein homolog-DD_CABYR_SP17⁷⁰), sperm maturation (granulin^72,73), and a rhodopsin-like GPCR receptor (Allatostatin-A⁷¹) support a scenario of rapid divergence. Tellingly, genes that mediate sexual reproduction, such as gamete recognition proteins, are among the fastest-evolving molecules involved in barriers to fertilization¹²². Lab fertilization trials among Orbicella species showed gametes of O. faveolata to be largely incompatible with those of the other two species, while in at least one locality, O. annularis sperm failed to fertilize O. franksi eggs¹⁸. The evolution of gamete incompatibilities can arise and reinforce patterns of temporal reproductive isolation, as in the case of O. annularis and O. franksi that spawn 2 h apart, or as well as in the depth-segregated lineages of O. faveolata described here, but not yet tested. All the above alternatives thus may be working synergistically on driving adaptive divergence at different spatial and temporal scales without being mutually exclusive.

In conclusion, coral lineages adapted to different depths experience differential selection across environmental gradients, where differences in light spectra can generate a differential perception of light-mediated spawning cues, secondarily regulated, or reinforced through neurotransmitter release, eventually producing a reproductive isolating mechanism via allochronic spawning. This mechanism would allow sympatric and ecologically segregated corals to evolve reproductive isolation via dual effect genes (i.e., magic traits⁴) involved in phototransduction signals that cue reproduction. Our findings shed light on the pervasive patterns of depth divergence across shallow coral taxa worldwide and offer a plausible and testable hypothesis for the evolution of divergent light-dependent coral lineages in the photic zone.

Methods

Genome-wide phylogenomic reconstruction

To illustrate pervasive patterns of depth differentiation across hard (Scleractinea) and soft (Malacalcyonacea and Scleralcyonacea) coral taxa worldwide, we reproduced an available phylogenetic tree for the class Anthozoa (Phylum: Cnidaria) based on 1729 markers captured with a targeted-enrichment approach, where a subset of loci are isolated and sequenced, to recover phylogenetically informative loci from 234 anthozoans, with representatives from all orders and most families¹²³. We then leveraged a recent literature review²⁹ on cryptic genetic divergence in corals, searched, compiled, and cross-referenced with original publications, for reported cases of such divergence directly or indirectly related to depth gradients within the photic zone (<150 m). Subsequently, we chose each coral genera/species case from around the world, categorized by reproductive mode (broadcast-spawner or brooder) as this influences dispersal capabilities, counted documented cases per ocean basin, and reported the type of genetic marker used, along with other morphological or reproductive kinds of reproductive isolation in at least 15 genera of both hard (Scleractinea) and soft (Malacalcyonacea and Scleralcyonacea) corals (Fig. 1, and Supplementary Data 1). World map in Fig. 1a was generated using the R package maps v 3.4.3 (https://github.com/adeckmyn/maps).

To infer SNP-based phylogenetic relationship among colonies from Puerto Rico, we first leveraged whole-genome data available⁵⁶ for the three Orbicella species (O. faveolata, O. franksi, and O. annularis) from this location, one individual per species, plus two additional individuals for O. faveolata from each of three additional locations (Panama, Mexico, and Florida, US). We also sequenced de novo the genomes of their most closely living related species, Cyphastrea serailia and C. microphtalma¹²⁴. We then selected an equal number of previously identified Shallow (n = 11) and Deep (n = 11) lineage samples, along with the whole-genome Orbicella and Cyphastrea samples, and conducted SNP calling based on a mapping against the largest O. faveolata scaffold (NW_018148670.1 with 4,771,691 bp) in the reference genome. Subsequently, we built a Maximum Likelihood (ML) species tree in RaxML-NG¹²⁵, using the GTR model of sequence evolution. Node support was evaluated with a maximum of 1000 bootstrapping replicates (autoMRE, cutoff = 0.01), and FigTtree¹²⁶ (v1.4.4) (available at http://tree.bio.ed.ac.uk/software/figtree/) was used for tree visualization and editing.

Sample collection and sequencing

Samples were collected in accordance with local regulations under permit number O-VS-PVS15-SJ-00667-22042014. We collected tissue samples from 103 Orbicella faveolata colonies larger than 20 cm in diameter from 5 to 19 m depth at Media Luna Reef (17°56.08′ N; 67°02.91′ W), La Parguera, Puerto Rico, following the reef’s contour across a light (~600 µmol m⁻² s⁻¹) gradient, with notable slope drops between 12–18 m and 18–19 m. We selected colonies at least 10 m apart to avoid sampling clonemates. We particularly avoided collecting any colonies deeper than 10 m that might have resembled O. franksi’s typical, white-bumped morphology. Samples were preserved in 95% ethanol and subsequently flash-frozen for DNA storage. High-purity gDNA extractions via a modified CTAB extraction protocol¹²⁷ were sent to Cornell Sequencing facilities for Genotyping By Sequencing (GBS), libraries were digested with the restriction enzyme apeKI, and sample-specific barcodes with no size-selection steps. The amplification and paired-end sequencing (64 bp) of libraries was performed on a single lane of the Illumina HiSeq 2500 platform that resulted in over 447 million reads with an average of 4.4 million reads per sample library.

Genotyping variants

Raw FASTQ files were mapped to the O. faveolata reference genome⁵⁶, which had a total length of 509.9 Mb (N50 Length: 509.9 Mb 2170 scaffolds; min length 0.062 Mb | N90 Length: 12,074 scaffolds; min length 4 Kb) assembled into 16 linkage groups with a total map length of 2048.5 cM (1 marker per 2.05 cM on average) with some other unplaced scaffolds¹²⁸. DNA for this genome comes from a sperm sample free of microalgae (aposymbiotic) from an individual of O. faveolata sampled from (24.812697° N; Florida 80.66925° W) at 5–10 m¹²³. Approximately 27% of the quality-filtered GBS sequences mapped to unique regions of the reference genome. We followed the dDocent¹²⁹ pipeline for adapter removal, quality trimming, assembly of loci, read mapping, and variant calling. In addition, we removed all indels and retained biallelic SNPs with a Phred score > Q30 and a minimum genotype sequencing depth of 5× in the variant call format (VCF) file. The abovementioned and further filtering steps for each kind of analysis were all done using VCFTools¹³⁰.

Genomic characterization

To explore genome-wide divergence patterns, we retained SNPs that were genotyped in at least 80% of individuals and had a minimum mean coverage of 20×, and removed samples that were missing >50% data, had minor allele frequencies (MAF) below 1%, were out of Hardy-Weinberg Equilibrium (HWE) (p value 0.01). We removed SNPs identified as putatively under selection (see below) and kept only SNPs at least 1000 bp apart to reduce linkage disequilibrium. We searched for potentially genetically identical colonies (i.e., clones) through the clonecorrect function in the R package Poppr^131,132. No clones were identified by this methodology.

Initially, we examined genome-wide variation with DAPC using the R package adegeneT^133,134. We estimated the number of clusters within O. faveolata (from 1 to 10) using find.clusters in adegeneT and selected the optimal number of groups using a Bayesian Information Criterion approach. To avoid overfitting, we used the optim.a.score function to determine the number of principal components (PCs) axes to be retained. Once the optimal number of PCs was determined, a second DAPC analysis was conducted using this value¹³⁵. We then used a Bayesian clustering method as implemented in Admixture¹³⁶ 1.3.0 to infer the number of genetic clusters (from 1 to 10) and potential admixture, with 10 replicates for each value of K. The most likely value of K was determined through the lowest cross-validation error¹³⁷. Finally, we estimated the population fixation index (F_ST) among the inferred clusters via genet.dist in the R package HierFstat^138,139.

Morphological characterization

Underwater pictures of each sampled colony were taken in situ prior to tissue sampling. Images were processed in ImageJ¹⁴⁰. Polyp density (No. polyps/1.5 cm²) was estimated by setting a cm-to-pixel scale, projecting a grid onto the image with the scale correlating to the scale set, and selecting four random squares where polyps were clearly visible. Polyp mean density was checked for normality (Shapiro-Wilk test) and heteroscedasticity through a Breusch-Pagan test after fitting a regression model. The mean polyp density was also compared among sampling depths, and lineages (Shallow/Deep) found in downstream genomic analyses, via a non-parametric Kruskal–Wallis test and a post-hoc Nemenyi’s multiple comparisons test. All statistical analyses were performed in R version 4.1.1.

Demographic history

To infer the demographic history from the allele frequency spectrum (AFS) data, we used the software GADMA¹⁴¹ (Genetic Algorithm for Demographic Model Analysis) using Moments¹⁴² Python library. We calculated the folded two-dimensional site frequency spectrum (2D-SFS) from a similar filtered SNPs dataset used for population structure analyses, except for the MAF and HWE filters so as to not distort the SFS. To transform the filtered VCF file to Moments format, we used the vcf2dadi.pl script (https://raw.githubusercontent.com/z0on/2bRAD_GATK/master/vcf2dadi.pl). To capture information from most SNPs and to compensate for missing data, we down-projected the SFS to 26 (Deep) and 54 (Shallow) alleles. To account for variability in the SFS estimation, we used a nonparametric bootstrapping by resampling the SNP file generated from vcf2dadi.pl five times using dadiBoot.pl (available https://github.com/z0on/2bRAD_GATK/blob/master/dadiBoot.pl). Demographic curves were scaled by a per-generation mutation rate of 1.38 × 10⁻⁹ per base per generation and a generation time of 10 and 35 years⁵⁶. To corroborate the final output from GADMA, we followed Prada and Hellberg’s¹⁴³ approach. Briefly, 107 demographic models were tested in Moments (https://github.com/z0on/AFS-analysis-with-moments), in which the models for two diverging populations can include up to three historical epochs, during which the effective population size can vary from other epochs. Parameters for genetic exchange can also vary among epochs and may be symmetrical or asymmetrical. To rank the different models according to the log-likelihoods and number of parameters, we estimated the evidence ratio from the best model. We then inferred the Akaike information criterion scores, differences between each model against the best model, the relative likelihood of each model given the data, the model probability, and the evidence ratio in favor of each model.

Signatures of selection

To search for candidate loci putatively under diversifying natural selection, the dataset was filtered for SNPs with a minor allele frequency > 0.01, removing SNPs below a threshold of 20% missing data in each previously identified Shallow and Deep lineage. We performed outlier loci tests by first calculating the per-locus F_ST between lineages in VCFTools and identified empirical outliers as those occurring in the top 0.01% of values genome-wide. Next, loci were screened for statistical outliers using BayeScan¹⁴⁴ (v.2.1) and the R package PCAdapt¹⁴⁵, a Bayesian and principal component analysis (PCA) approach, respectively. For BayeScan, the default settings were used with a relaxed q value of 0.5, and for PCAdapt, the q value was set to 0.05. SNP loci were considered outliers only if identified by all three methods. Outlier SNPs were compared against annotated gene regions (within ±10 kb) of the O. faveolata genome to identify putative functional effects. Additionally, we conducted a rank-based GO enrichment analysis that uses a two-tailed Mann–Whitney U test, as implemented in GO_MWU¹⁴⁶. Overall GO term significance was assessed based on a module designed to extract GO terms that best represent independent groups of significant GO terms, utilizing an adjusted p value cutoff from random permutations (for details see https://github.com/z0on/GO_MWU/blob/master/GO_MWU.R).

Cline fitting and selection strength

To infer the strength of selection along the depth gradient, we first estimated allele frequencies for outlier and non-outlier loci and plotted them against depth and linear distance on the reef. Subsequently, to tease out the spatial scale of selection across the depth axis, we fit our outlier loci dataset to classical geographic cline models implemented in the HZAR¹⁴⁷ R package to estimate the center and width of the cline. Using the width of these outlier loci we estimated the strength of selection acting on each locus. To this end, we first calculated the average dispersal distance per generation (σ) by applying Drw₁w₂^1/2, where D is the Linkage Disequilibrium (LD) between the two markers, w₁ and w₂ refer to the cline width for each marker, and r is the recombination rate⁷⁶. Pairwise LD among outlier loci was inferred via the pair.ia function in the R package Poppr^131,132. We used a recombination rate between 0.01 and 0.02 %, based on the reported average marker interval of 2.03 cM for the available O. faveolata genetic linkage map¹²⁸. Finally, the selection coefficient was calculated under two models. The first model assumes a step-like change in the environment (ecotone), in which dispersal distance is similar to or larger than the cline width^75,148. This model uses the equation w = σ/√s, in which s is the selection coefficient. In the second model, the shape of the cline is maintained by selection against heterozygous (heterozygous disadvantage)^43,141. This model uses the equation w = σ/√8/s.

Synonymous vs. non-synonymous substitutions

To further understand the role of natural selection in the genes related to divergence across depth in O. faveolata lineages, and to detect regions of protein-coding genes evolving under positive divergent selection in broadcast spawning corals, we aligned whole-genome data⁵⁶ for the three Orbicella species (O. faveolata, O. franksi, and O. annularis), one individual for each species, plus two additional individuals for O. faveolata from different locations (Mexico and Panama), and sequence de novo the genomes of their closest relatives, Cyphastrea serailia, and C. microphtalma. After mapping and SNP calling, we compiled all aligned nucleotide sequences of orthologous protein-coding sequences (CDSs) within ± 10 kb of the previously identified SNP outlier loci, along with those of the other species. We used vcf2fasta.py to extract FASTA alignments from the VCF file of the five species based on the annotation (GFF) and reference genome of O. faveolata. To search for genes under positive selection, we used the PSGfinder¹⁴⁹ pipeline to estimate pairwise non-synonymous(d_N) and synonymous (d_S) substitution rates⁵⁷ (ω = d_N/d_S) of these candidate regions between O. faveolata and O. franksi, and between O. annularis and O. franksi pairs to estimate ω based on alignment windows. The size of alignment windows for comparisons was determined by the presence of at least three non-synonymous substitutions. The estimation of ω values was obtained using the method “yn00”⁵⁷, including its assumptions, as implemented in PAML version 4.8¹⁵⁰. Windows with an ω > 1 were considered a strong indicative of a gene displaying a signal of positive selection.

We further explore signals of positive selection for those CDSs with an ω > 0.5 in the O. faveolata and O. franksi species comparison. We analyzed CDS alignments using a series of tools from the HyPhy⁶⁷ platform that map sequence variation onto a gene tree to characterize patterns of d_N and d_S nucleotide variation, implemented on the datamonkey web portal⁶⁶ (www.datamonkey.org). Specifically, we ran four tests to detect selection at the phylogeny branch, entire gene, and codon site levels. We first used aBSREL¹⁵¹ (adaptive Branch-Site Random Effects Likelihood) to identify positive selection on individual branches in each gene tree. We then used BUSTED¹⁵² (Branch-site Unrestricted Statistical Test for Episodic Diversification) to identify gene-wide evidence of episodic positive selection. Finally, we ran MEME¹⁵³ (Mixed Effects Model of Evolution) to detect sites evolving under positive selection under a proportion of branches, and FEL¹⁵⁴ (Fixed Effects Likelihood) to infer d_N and d_S substitution rates on a per-site basis for a coding alignment and its corresponding phylogeny.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.