Introduction

Mating signal divergence has long been recognized as a critical step leading to reproductive isolation between diverging populations and ultimately, speciation. Although numerous studies have provided insight into how evolutionary forces promote divergence (see reviews by Ptacek, 2000; Panhuis et al., 2001; Schluter, 2001; Ritchie, 2007), far fewer studies have focused on the genetic changes associated with mating signal divergence, and the potential for phenotypic and genetic associations between suites of signalling traits to shape such divergence (Orr, 2001; Coyne and Orr, 2004). Understanding the genetics of speciation resulting from divergence in mating signals requires identifying the key changes underlying differentiation and reproductive isolation, the order in which such changes occur, the way they interact and determining the evolutionary forces that promote their spread (Butlin and Ritchie, 2009).

A growing number of studies suggest that attractive male traits that contribute to mating signal divergence have an underlying genetic basis that can respond quickly to sexual selection by way of female choice (Beukeboom and van den Assem, 2001; Brooks and Endler, 2001; Brooks, 2002; Lindholm and Breden, 2002; Cooperman et al., 2006; Qvarnström and Bailey, 2009). For example, the genetic basis of traits that confer premating behavioural isolation can be the result of a few genes with major effects (Orr, 1992; Blows and Higgie, 2003; Chenoweth and Blows, 2003; Ortiz-Barrientos and Noor, 2005) or Y-linked genes in linkage disequilibrium with X-linked genes for female mating preferences (Lindholm and Breden, 2002; Qvarnström and Bailey, 2009). When sexual selection operates to promote mating signal divergence, such as through divergence in female mating preferences, preferred male traits should exhibit positive relationships in their expression (Kodric-Brown and Brown, 1984; Johnstone, 1995; Candolin, 2007) even if they are very different in kind (for example, morphological vs behavioural). Such patterns of phenotypic and, potentially, genetic covariation between different components of a mating signal may contribute to the relative ease with which they can diverge, and provide an important pathway in the speciation process.

The importance of selection leading to associations between mating signal traits that promote divergence between species has been relatively unexplored, especially with regard to the underlying genetic changes associated with such divergence. One approach to the study of genetic changes associated with species differences in mating signals has been that of examining the genetics of interspecific hybrids between closely related taxa (Beukeboom and van den Assem, 2001; Williams et al., 2001). Comparing the distribution of phenotypic components of a mating signal between parental species and their hybrids provides insights into the underlying genetic control of species differences in the parental taxa (Civetta and Singh, 1999; Beukeboom and van den Assem, 2001) and the degree to which mating signal components change in concert between species.

Mollies (genus Poecilia, subgenus Mollienesia) are an ideal group in which to examine the genetic basis of species differences in mating signals and the potential for sexual selection to promote phenotypic correlations between signalling traits for several reasons. First, mollies are divided into two major evolutionary lineages, sailfins and shortfins, which vary dramatically in both morphological and behavioural features of their mating signals (Ptacek and Breden, 1998). Sailfin species are characterized by a sexual dimorphism in which males possess a greatly enlarged dorsal fin that is erected and presented to the female in a courtship display used to elicit female cooperation during internal fertilization (Farr, 1989). Shortfin species show a very different mating system. No sexual dimorphism exists in dorsal fin size, and males of most species do not perform courtship displays (Farr, 1989). Mating consists of forced insemination attempts termed gonopodial thrusts. Males swim alongside a female and attempt to insert the gonopodium (modified anal fin that serves as an intromittent organ) into the female's gonopore for sperm transfer, thus, circumventing female cooperation during copulation (Farr, 1989).

Several morphological characters in male sailfin mollies are likely targets of sexual selection. The dorsal fin is used as a signal in both intra- and intersexual interactions (Farr et al., 1986; Ptacek and Travis, 1996), and the exaggerated size of the dorsal fin in larger males is thought to enhance the visibility of the courtship display behaviour and make a male appear larger in size (MacLaren et al., 2004; Kozak et al., 2008). During a courtship display, males often erect the gonopodium and lower it approximately perpendicular to the body, in addition to expanding the dorsal fin (Rosen and Tucker, 1961), and such displays are also used in aggressive encounters between males (Baird, 1968; Travis, 1994). These morphological traits vary allometrically with male body size; larger males have disproportionately longer and higher dorsal fins and disproportionately shorter gonopodia (Ptacek, 1998, 2002; Hankison et al., 2006). As a result of the signalling function of these morphological features of male sailfin mollies, it is possible that positive phenotypic covariation in behaviour and morphology may have arisen as a result of female mating preferences, and such correlated changes in these signalling traits may have shaped species differences between sailfin and shortfin mollies in their mating systems.

To investigate genetic changes associated with species differences in mollies, we used an interspecific crossing scheme that compared male Y-chromosome lines across multiple generations (parental, F1 and two directions of backcross) to determine the underlying genetic architecture of species differences in morphology. In a companion study (Loveless et al., 2009), we examined the patterns of inheritance of these same generations for two male mating behaviours, courtship display and gonopodial thrusting rates, and found a strong Y-linked component to the inheritance of courtship display behaviour.

The goals of the study we report here were threefold. First, we tested the hypothesis that species differences between sailfin and shortfin mollies are greatest in morphological traits associated with mating signals. This hypothesis would be supported if morphological traits that contribute most to parental species differences include those associated with size and shape of the dorsal fin (used in courtship displays) and the gonopodium (used in gonopodial thrusting). Second, we used our breeding design to examine the genetic basis of those morphological traits found to contribute to species differences. We tested the hypotheses that species differences in morphology are inherited in a similar (Y-linked) manner to that of courtship display behaviour (Loveless et al., 2009) or have a different (autosomal) genetic basis than that for mating behaviours. Comparisons of the segregation variance between parental, F1, and backcross generations within and between Y-chromosome lines allowed us to distinguish between Y-linked and autosomal modes of inheritance, and further, additive and additive plus dominance contributions to species differences in morphological traits. Finally, we tested for phenotypic associations between morphological traits (dorsal fin size and gonopodium length) and mating behaviour rates (courtship displays and gonopodial thrusts). Covariation between morphological and behavioural phenotypes supports the hypothesis that sexual selection has the opportunity to promote genetic covariation (linkage disequilibrium) between these different components of the mating signal in sailfin mollies, potentially leading to rapid divergence and speciation of the sailfin molly lineage (Ptacek and Breden, 1998).

Materials and methods

Collection of fish

The fish representing the parental species and used as sires and dams to create the interspecific F1 hybrid crosses were wild caught from Campeche, Mexico (GPS coordinates N 19°14.230′ W 90°50.110′) in May 2001 and 2002. Males and females of both Poecilia velifera and P. mexicana were collected from this site.

Breeding design

The breeding design began with reciprocal crosses using P. velifera (sailfin) and P. mexicana (shortfin) males and females to create the F1 generation of males expressing the same autosomal ratio (50% sailfin, 50% shortfin), but different Y chromosomes. We focused on creating Y-chromosome lines because, in mollies, as well as other poeciliid fishes, genes encoding secondary sexual traits are typically sex linked and more often Y linked, with regions containing these genes suppressing recombination with the X chromosome (Farr, 1983; Zimmerer and Kallman, 1989; Brooks and Endler, 2001; Lindholm and Breden, 2002; Ptacek, 2002). The backcross generations were created by crossing F1 males from each of the Y-chromosome lines (sailfin and shortfin) to each of the two female parental species (P. velifera and P. mexicana). This backcrossing scheme yields males having varying autosomal ratios (∼25 or 75% sailfin autosomes) while keeping the Y-chromosome constant (sailfin Y or shortfin Y).

Sires for each of the generations were chosen haphazardly from stock populations (parentals) or F1 families (backcross generations), with the only restriction being that the parentals or F1 sires from the sailfin Y-chromosome line had a standard length ⩾45 mm. We used this size restriction to ensure that all males carrying the sailfin Y chromosome were of the size class where males perform courtship displays (Hankison and Ptacek, 2007). Three different P. mexicana males (67, 60 and 72 mm) served as sires to create the shortfin Y chromosome, MF1 generation. From these MF1 males, seven sires (35–55 mm) were selected to create the backcross generations that would carry the shortfin Y chromosome. Each MF1 sire was mated with both a P. velifera female and a P. mexicana female to create the backcrosses with varying autosomal ratios. Two different P. velifera males (72 and 75 mm) were used to create the sailfin Y chromosome, VF1 generation. From these VF1 males, six sires (48 mm to 60 mm) were selected to create the backcross generations that would carry the sailfin Y chromosome. Each VF1 sire was mated with both a P. velifera female and a P. mexicana female to make the backcrosses with varying autosomal ratios.

Morphological measures

To determine trait values for morphological characters, euthanized or anaesthetized (buffered 0:50% MS-222) live males were placed on a dissection mat with their caudal fin and dorsal fin fully spread and pinned in place with insect pins; the gonopodium was pinned away from the body as well. A picture was then taken of the left side of the fish using a digital camera (Sony DSC-F707, Sony Electronics, Inc., San Diego, CA, USA) at 2560 × 1920 resolution. Live fish were revived and returned to their individual housing tanks. The program NIH ImageJ (version 1.6, National Institute of Mental Health, Bethesda, MD, USA) was used to measure 14 linear characteristics (Figure 1) of males from each genotype. These included measurements of dorsal fin and caudal fin area, which were determined by tracing the outline of these fins from the digital photograph and using the program's estimate of area.

Figure 1
figure 1

Schematic showing the 14 linear measures taken from each male. SL is standard length; LDF is length of dorsal fin; LFFR is length of first dorsal fin ray; LLFR is length of last dorsal fin ray; DFA is dorsal fin area; PDD is pre-dorsal distance; PAD is pre-anal distance; LG is length of gonopodium; DMB is depth at mid-body; DCP is depth of caudal peduncle; LCF is length of caudal fin; HCF is height of caudal fin; CFA is caudal fin area; DFR # is dorsal fin ray count.

Full size image

All morphological values obtained from the linear and area measures were corrected for size using techniques from Mosimann and James (1979). Specifically, the measurements for each of the 12 linear measures, excluding standard length and dorsal fin ray number, were first log transformed. The log-transformed data were then summed and divided by the total number of traits, 12, using the equation:

This resulted in a log-size calculation for each individual fish. The calculated log-size value was then subtracted from each trait value for all fish, resulting in a size-corrected value for all traits across all individuals measured. All 601 males were then partitioned into groups on the basis of their genotype (P. velifera (V)=50, P. mexicana (M)=55; VF1=57, MF1=76; VBCM=95, VBCV=87, MBCM=101, MBCV=80), and finally, each size-corrected trait was regressed against the log of standard length to ensure that all size effects had been successfully removed. These size-adjusted values for each morphological trait were used in further statistical analyses.

Behaviour profiles

Males of all species of sailfin mollies perform stereotyped mating behaviours: courtship displays and gonopodial thrusts (Farr et al., 1986; Ptacek and Travis, 1996; Hankison and Ptacek, 2007). Shortfin species perform predominantly gonopodial thrusts (Farr, 1989). To compare the mating behaviour profiles of hybrid males of different genotypes to that of parental species of males, sexually mature males (complete fusion of the gonopodium) from each hybrid genotype were tested twice, in direct contact trials (10 min observation period) with a single, non-receptive female; once each with an unfamiliar female from both of the parental species. The results of the two trials were pooled (after determining that there was no difference in mating behaviour rates in response to the two species of females) and the average rate of each mating behaviour was used for all analyses testing for associations between morphological and behavioural traits (see Loveless et al., 2009, for a complete description of methods for behavioural trials).

In an attempt to control for non-normality and account for zeros in the data (some males failed to perform a particular behaviour during the 10 min observation period), we square root transformed all behaviour count data. All statistical analyses involving behaviour rates were performed on square root transformed variables.

Statistical analysis

Principal components analysis

To determine which subset of morphological traits contribute most to differences between the parental species and different hybrid classes, we used a form of discriminate analysis based on principal component analysis (PCA). The PCA was first performed on a pooled data set including males of both parental species, P. velifera and P. mexicana. Assuming that the largest source of variation in the data set was species difference, the traits with large correlation coefficients, that is, loaded heavily, in the PCA were assumed to be important in separating the two parental species. The advantage of this approach to discriminant function analysis is that the covariation among the traits was considered. The analysis was repeated on pooled data from both groups of F1s, and finally on the four groups of backcrosses. Traits that loaded significantly in the PCA of the hybrid classes that were similar to those loading significantly in the parental PCA provided evidence that variation between the two parental species was recovered in the reciprocal F1 and backcross hybrids, suggesting autosomal inheritance rather than strict Y linkage for species-specific morphological trait distinctions.

Genetic analysis of genotype means

To examine the genetic basis of species differences in morphology, we selected the traits that contributed most to species differences on the basis of parental PCA. These selected traits were then subjected to a joint scaling test, a type of multiple regression technique (as summarized in Mather and Jinks (1982) and Lynch and Walsh (1997)) that compares phenotypic means between parental and hybrid lines. Although originally developed for genetic analysis of divergent inbred lines, the method can be used for the analysis of interfertile wild populations, provided that close relatives are not mated (Hard et al., 1992). Briefly, the technique fits the following multiple regression model to the observed line means,

where zi is the trait mean in the ith line, μ is the mean of all line means, a is the additive genetic effect, d is the effect of dominance and ɛi is the sampling error associated with the ith line. Also, M is a matrix of coefficients specifying predicted line means under the additive and additive-plus-dominance models.

We used joint scaling to test for the effects of additivity and dominance and their contribution to divergence of parental lines. If the differences between the species are primarily due to additive effects, then the means of the F1 trait values should be intermediate to the two parental species and the means of the backcross lines should be intermediate to the F1 trait values and to the parent species sharing the most autosomes. However, if dominance effects contribute to species differences, F1 and backcross trait values should resemble one parental species more than the other, with this tendency being stronger for F1 than backcross lines.

A goodness-of-fit χ2 test was used to determine how well each trait fit to a given model. If the goodness-of-fit test results in rejection of the additive model, it indicates that the pattern of inheritance seen in the observed values is likely due to a more complex model involving dominance and/or epistasis (however, we could not test for epistasis due to the absence of an F2 generation). The goodness-of-fit test was also applied to the additive-plus-dominance model, testing for significance of the addition of the dominance term by subtracting the two goodness-of-fit χ2 values from each other. If the difference was not significant (P>0.05), then the dominance term did not significantly add to the model fit, and the additive model provided a sufficient explanation for difference in the traits among the genotypes.

Associations between morphology and behaviour

The traits used in the regression analyses of morphology (length of dorsal fin, length of first dorsal fin ray, number of dorsal fin rays, depth at mid-body and length of gonopodium) on mating behaviour (courtship display rate and gonopodial thrust rate) were chosen because they are likely to be the targets of sexual selection. Variation in these traits can result in an increase or decrease in lateral projection area (the sum of body area and fin areas that encompass the lateral view of a male presented to a female during a courtship display), which has been shown to be an important target of sexual selection by female choice (Rosenthal and Evans, 1998; Karino and Matsunaga, 2002; MacLaren et al., 2004; Kozak et al., 2008). Variation in the length of the gonopodium has also been shown to exist between populations and species of poeciliid fishes (Kelly et al., 2000; Jennions and Kelly, 2002; Hankison et al., 2006), and females of one species preferred males with longer gonopodia (Langerhans et al., 2005).

Two morphological features, the dorsal fin and the gonopodium, are most directly associated with courtship displays or gonopodial thrusts, respectively. To be sure that any associations between morphology and behaviour were independent of Y-linked species differences in behaviour rates (Loveless et al., 2009), we first regressed each mating behaviour rate on the size-corrected values for dorsal fin and gonopodium length of males from both Y lines combined, and then, separately for each Y-chromosome line. Because the wide range of values and skewed distributions (many zero behaviour scores) did not lend themselves to traditional regression techniques due to the lack of normality in distribution of the residuals, as a second approach, we divided the morphological traits into high and low values with respect to the mean value across all genotypes and divided the behaviour rate with respect to ‘performers’ and ‘non-performers.’ We used Pearson's χ2 contingency analysis to determine if there were differences in the proportions between the two groups, that is, if the high/low morphological trait values were significantly related to the high/low behaviour rate values.

Results

Species differences in morphology

We found considerable variation among the eight genotypes in the morphological characters measured (see Supplementary Tables 1 and 2 for family and genotype size-adjusted means, respectively). However, the parental species and their hybrids differed in predictable patterns between generations, with F1 genotypes being intermediate to parentals, and backcross hybrids recovering the full range of variance between parentals, especially in size and shape of the dorsal fin (Figure 2, see also Supplementary Figure 1 for photos).

Figure 2
figure 2

Results of the principal components analysis (PCA) for the parental species and hybrid generations. (a) Parental species, P. mexicana (shortfin) and P. velifera (sailfin), (b) F1 generations, MF1 and VF1; (c) backcross generations, MBCM, MBCV, VBCM and VBCV.

Full size image

The two parental species differed the most along PC 1 (explaining 46% of the variance), which loaded most heavily with features of the dorsal fin, being longer and taller in the sailfin species than in the shortfin species (Table 1, Figure 2a). Considerable overlap occurred along PC 2, but the two parental species differed slightly along this axis (explaining an additional 16% of variance) based primarily on the shortfin species having taller caudal fins and longer gonopodia (Table 1).

Table 1 Correlation of morphological traits used in the principal components analysis calculated to distinguish between the groups of genotypes (parentals, F1s and backcrosses)
Full size table

The F1 hybrids were intermediate to the two parental species along PC 1 (explaining 44% of the variance) and did not differ with respect to the direction of the cross (Figure 2b), suggesting little evidence of Y-linked effects on the inheritance of species differences in dorsal fin shape. The backcross genotypes recaptured more of the original variation along PC 1 (explaining 53% of the variance) between the species (Figure 2c), again suggesting that autosomal genes with additive effects (overlap occurs with both parental species) contributed most to variation among genotypes in dorsal fin shape rather than Y-linked effects between paternal species lines.

Genetic analysis of genotypic means

We chose to perform the joint scaling test on those traits that loaded most heavily in the principal components analysis for PC1 based on the two parental species. These morphological traits contributed most to species differences between sailfin and shortfin mollies. Five of these morphological differences (dorsal fin characters: length of dorsal fin, length of first dorsal fin ray, number of dorsal fin rays and depth of mid-body, as well as length of the gonopodium) are likely to be influenced by sexual selection (Rosenthal and Evans, 1998; Kelly et al., 2000; Jennions and Kelly, 2002; Karino and Matsunaga, 2002; MacLaren et al., 2004; Hankison et al., 2006; Kozak et al., 2008). Four morphological differences (caudal fin characters: height and length of the caudal fin, as well as body shape characteristics: depth of caudal peduncle and pre-anal distance) are likely to be shaped by natural selection (Webb, 1982, 1984; Endler, 1995; Ghalambor et al., 2003; Langerhans et al., 2003; Hankison et al., 2006).

Joint scaling uses least-squares regression to fit the best line to all classes of phenotypic means. The dotted lines in Figure 3 join observed parental means and represent an a priori expectation of additive genetic effects: if sailfins and shortfins have diverged primarily in genes with additive effects, then character means for all hybrid classes should fall along this line. If the species have also diverged in alleles with dominance effects, then character means for hybrid crosses should all be displaced above or below the line; the displacement for F1 hybrids should be double that for backcrosses. Epistasis causes hybrid line means to deviate significantly from expectations of additivity or dominance, but can only be explicitly tested with inclusion of an F2 hybrid generation (Mather and Jinks, 1982). When epistasis terms are absent from the regression model, they are implicitly included in the error term along with sampling error.

Figure 3
figure 3

Observed character means and s.es. for the nine traits (a–i) measured in the parental species and three hybrid generations. A priori expectations of additive genetic effects are represented by the dotted lines drawn between parental means. If sailfins and shortfins have diverged primarily in genes with additive effects, then character means for all hybrid crosses should fall along this line. Note that sample sizes differ among genotypes, so the error bars do not describe variance of the traits within a hybrid class; see Supplementary Table 2 for variances.

Full size image

A model of no effect (that is, zi=μ+ɛi) was soundly rejected in all cases (P<0.01), providing clear evidence of genetic divergence in all nine morphological traits (see Supplementary Table 3 for model estimates of μ and a for each trait). Little evidence of dominance effects was found, although the deviation of F1 hybrids from the line of additivity for length of the first dorsal fin ray (Figure 3b) suggests dominance of shortfin alleles for this trait. Additive genetic effects were best seen for dorsal fin ray number (Figure 3c) suggesting that additive genetic variance is greatest for this character, which contributes most to the primary morphological difference in dorsal fin size between sailfin and shortfin molly species. Deviations from additivity and dominance in the remaining traits (Figures 3a, d–i) suggest epistatic inheritance, but large sampling errors for line means for some traits and the lack of an F2 generation preclude a definitive conclusion of epistasis.

Goodness-of-fit statistics showed that an additive model is best suited to explain the variation seen among the genotypic classes for each of these morphological traits; the difference between the test statistic from the additive model and the additive-plus-dominance model was less than 0.62 (0.00001–0.6173) for all nine morphological traits examined. In no case was the difference between the two models significant (P>0.05 for all nine traits), indicating a good fit for the additive model and no improvement of fit with the addition of dominance to the model. Despite the large amount of variation in most of the traits that cannot be explained by the additive model (Table 2), addition of the dominance term did not help to explain this additional variation. It is possible that the unexplained variation is due to epistatic interactions in all of the traits except dorsal fin ray number, which appears to be a purely additive trait. Without an F2 generation, we cannot test a model of epistasis. Environmental effects may contribute to the high residual variance in these traits as well.

Table 2 Percentage of total variance explained by the additive model when fit to the means of the nine morphological traits included
Full size table

Associations between morphology and behaviour

There were distinct differences among the genotypes with respect to their behavioural profiles, such that those individuals with a higher proportion of sailfin autosomes (PV, MBCV and VBCV) had higher courtship display rates and lower rates of gonopodial thrusting (Loveless et al., 2009). Of the five morphological traits included in the regression analyses on courtship display rates, only three of them (length of dorsal fin, depth at mid-body and gonopodium length) explained over 15% of the variance observed (Table 3). None of the traits included in the regression analyses on gonopodial thrusting rates explained more than 3% of the variance observed (Table 3), suggesting a much weaker effect of species differences in morphology on rates of this mating behaviour.

Table 3 Linear regressions between five morphological traits (LDF, LFFR, DFR #, DMB, LG) and two male mating behaviours (display rate, thrust rate) according to the model: square root (behaviour rate)=slope (size-adjusted trait)+c
Full size table

The two morphological traits predicted to influence courtship display rates, dorsal fin length and gonopodium length, had opposite effects. The length of the dorsal fin was significantly positively associated with courtship display rates (Figure 4a) and this association was the same regardless of Y-chromosome contribution (Figures 4b and c). In contrast, gonopodium length was significantly negatively associated with courtship display rate (Figure 5a) and this association did not differ between Y-chromosome lines (Figures 5b and c). Length of the gonopodium did, however, explain more of the variation in courtship display rate among males from the sailfin Y-chromosome line (∼21%) than among males from the shortfin Y-chromosome line (∼10%).

Figure 4
figure 4

Regression of courtship display rate on size-adjusted dorsal fin length (a) Both Y-chromosome lines combined; (b) shortfin Y-chromosome line; (c) sailfin Y-chromosome line.

Full size image
Figure 5
figure 5

Regression of courtship display rate on size-adjusted gonopodium length (a) Both Y-chromosome lines combined; (b) shortfin Y-chromosome line; (c) sailfin Y-chromosome line.

Full size image

Morphological traits had much less of an influence on rates of gonopodial thrusting (Table 3). Length of the dorsal fin was significantly negatively associated with gonopodial thrust rate (Figure 6a) but explained less than 3% of the variation among genotypes. This negative effect appeared to be the result of a weak influence of dorsal fin length on thrust rates for males from the sailfin Y-chromosome line, but not the shortfin Y-chromosome line (Figures 6b and c). Gonopodium length was significantly positively associated with gonopodial thrust rate (Figure 7a), but again, explained less than 2% of the variation among genotypes. This positive effect appeared to be the result of a weak influence of gonopodium length on thrust rates for males from the sailfin Y-chromosome line, but not the shortfin Y-chromosome line (Figures 7b and c).

Figure 6
figure 6

Regression of gonopodial thrust rate on size-adjusted dorsal fin length. (a) Both Y-chromosome lines combined (b) shortfin Y-chromosome line; (c) sailfin Y-chromosome line.

Full size image
Figure 7
figure 7

Regression of gonopodial thrust rate on size-adjusted gonopodium length. (a) Both Y-chromosome lines combined; (b) shortfin Y-chromosome line; (c) sailfin Y-chromosome line.

Full size image

We used 2 × 2 contingency analyses to test the influence of dorsal fin length and gonopodium length on the propensity of males to be ‘displayers’ or ‘thrusters.’ For each of the morphological characters, we divided males into long (>mean) or short (⩽mean). We further divided males into two categories on the basis of behavioural rates: ‘displayers’ (performed one or more courtship displays) versus ‘non-displayers’ (performed no courtship displays) or ‘thrusters’ (performed one or more gonopodial thrusts) versus ‘non-thrusters’ (performed no gonopodial thrusts). On the basis of these contingencies, it was evident that males with long dorsal fins were almost twice as likely to be displayers (χ2=62.81, P<0.0001; Figure 8a). The opposite pattern was seen for the length of the gonopodium. Males with short gonopodia were nearly twice as likely to be displayers, than those with long gonopodia (χ2=50.40, P<0.0001; Figure 8b).

Figure 8
figure 8

Frequency plots showing the percentage of displayers and non-displayers on the basis of the size class of the morphological trait. (a) Proportion of displayers on the basis of the relative length of the dorsal fin; (b) proportion of displayers on the basis of the relative length of the gonopodium.

Full size image

Although to a much weaker extent, length of the dorsal fin and length of the gonopodium did influence the likelihood that a male was a thruster as well. Males with short dorsal fins were about 10% more likely to thrust than males with long dorsal fins (χ2=15.05, P<0.0001; Figure 9a). Males with a long gonopodium were about 15% more likely to thrust than males with a short gonopodium (χ2=12.43, P=0.0004; Figure 9b).

Figure 9
figure 9

Frequency plots showing the percentage of thrusters and non-thrusters on the basis of size class of the morphological trait. (a) Proportion of thrusters on the basis of the relative length of the dorsal fin; (b) proportion of thrusters on the basis of the relative length of the gonopodium.

Full size image

Discussion

Inheritance of species differences in morphology

Mating signals are often complex multivariate phenotypes, in which different components of the signal convey information regarding species identity, mate quality and readiness to mate (Ryan and Rand, 1993; Johnstone, 1995). In mollies, mating signals are composed of both behavioural features (for example, courtship displays) and morphological features (for example, enlarged dorsal fins) that enhance the attractiveness of the signal to potential mates (MacLaren et al., 2004; Kozak et al., 2008). Sailfin mollies differ from shortfin mollies in morphological traits associated with mating signals (dorsal fin size and gonopodium length) and potentially, swimming performance (caudal fin size). Comparing the segregation variance in morphological traits that contribute to species differences between parental and hybrid generations allows us to distinguish between single locus and polygenic modes of inheritance for these characters (Mather and Jinks, 1982). The use of paternal species Y-chromosome lines further allows us to distinguish between Y-linked and autosomal modes of inheritance. For instance, if dorsal fin shape was inherited in a strongly Y-linked manner, as has been described for courtship display behaviour (Loveless et al., 2009), we would predict that F1 and backcross generations with sailfin sires would show dorsal fin shape distributions that were similar to those of P. velifera males. On the basis of the comparisons between parental and hybrid distributions of morphological traits that contribute most to species differences (PC 1, Figure 2), we conclude that the pattern of inheritance of species-specific morphological traits, particularly dorsal fin shape, appears to be one of autosomal polygenic inheritance.

Species-specific traits analysed using joint scaling techniques fit best to an additive model of inheritance. The additive model best explained the inheritance of dorsal fin ray number, in which more than 90% of the total variance was explained by additivity. Indeed, this trait is used as the primary morphological feature in identifying different species of mollies, with little overlap between species in the number of dorsal fin rays (Miller, 1983). The additive model explained no more than 50% of the total variance for all of the additional traits, and less than 10% of the total variance for pre-anal distance, length of the gonopodium, depth at mid-body and height of the caudal fin. However, traits associated with dorsal fin shape (length (LDF) and height (LFFR)) and caudal fin shape (depth at caudal peduncle (DCP) and length (LCF)) showed a higher percentage of additive genetic variance than other morphological traits. These fins are likely targets of sexual selection (for example, MacLaren et al., 2004; Kozak et al., 2008) and natural selection (for example, Webb, 1984; Ghalambor et al., 2003; Hankison et al., 2006), and our joint scaling results suggest that sufficient additive genetic variation exists for these traits to evolve in response to selective forces (Schluter, 2001).

Even though the additive model did not explain a majority of the variance seen in most of the traits, the addition of the dominance term did not significantly improve the model's fit, thus, it is unlikely that dominance had a strong effect in the divergence of sailfin from shortfin mollies in these morphological features. The lack of dominance is not necessarily surprising, given that dominance and epistasis tend to have larger roles in life history traits, as opposed to morphological traits (Roff and Emerson, 2006). However, it is likely that there are additional factors involved in the inheritance of the morphological traits examined in this study that were not captured by the models tested. Epistasis and environmental influences likely contribute to the large amount of residual variance observed in many of these morphological traits.

Often line cross-analyses are based on very few replicate lines, though each may contain many individuals. Consequently, a different set of parental lines could give quite different estimates (Hatfield, 1997). Unfortunately, our analyses suffered from this common design limitation, only two to six families were raised for each type of hybrid genotype (see Supplementary Information Table 1). Given that time to maturity for many of the hybrid males included in our study takes more than 18 months, we were forced to limit the number of families in each of the genotypes. Increasing family number would likely improve our estimates of additive effects or dominance contributions to these morphological traits.

Associations between mating behaviours and morphology

Females of many poeciliid species prefer large, courting males (Ryan and Keddy-Hector, 1992), and mollies are no exception (Ptacek and Travis, 1997; Kozak et al., 2008). Female mating preferences for multiple components of a mating signal set the stage for correlated responses in these traits as a result of sexual selection promoting associations between them (Cooperman et al., 2006; Candolin, 2007). We found positive associations between size-adjusted length of the dorsal fin, dorsal fin ray number, and size-adjusted depth of a male at mid-body and courtship display rates. This was particularly true for males with relatively long dorsal fins, which were almost twice as likely to display as males with relatively short dorsal fins. An increased dorsal fin size contributes to an increased lateral projection area of a male, and an increase in the rate of courtship displays allows males to draw attention to their large lateral projection area, a known target of female mating preferences in a variety of poeciliid fishes including mollies (Rosenthal and Evans, 1998; Karino and Matsunaga, 2002; MacLaren et al., 2004; Kozak et al., 2008).

Phenotypic associations between morphological traits that increase apparent male size and increased courtship display rates may be the result of linkage disequilibrium between behaviour genes and body or fin size genes. For example, in the swordtail, Xiphophorus nigrensis, Zimmerer and Kallman (1989) found that males that were larger in size displayed to females more frequently than small males, and through breeding experiments, they demonstrated a relationship between the inheritance of male size and courtship behaviour in this species. Male body size in Xiphophorus is inherited as a single locus, Y-linked trait and known large male size at maturity alleles at the Y-chromosome linked P locus were postulated to be linked to genes for courtship behaviour on the Y chromosome as well (Zimmerer and Kallman, 1989). Similar Y-linked effects on courtship display rates have been found in guppies (P. reticulata, Farr, 1983) and in the sailfin mollies, P. latipinna (Ptacek, 2002) and P. velifera (Loveless et al., 2009). Although we did not measure genetic correlations between morphology and behaviour in this study, the positive phenotypic association between dorsal fin shape and courtship display rate is suggestive of the potential for linkage disequilibrium between genes that encode these traits to arise as a result of sexual selection favoring large, courting males.

We also found a negative relationship between size-adjusted length of the gonopodium and the likelihood of a male displaying to a female. This result may reflect the primary difference in the mating systems between sailfin and shortfin mollies. Shortfin males rely on forced insemination and have longer gonopodia. This relationship may have evolved to compensate for the lack of female cooperation in their mating system (Farr, 1989). Rosen and Tucker (1961) found that in most species of poeciliid fish where males do not rely on female cooperation in mating, gonopodium length was, on average, longer than for species where male courtship and female cooperation have evolved. They suggested that, as a result, the increase in the length of the gonopodium allows males to better visualize the position of a female's gonopore in relation to his own gonopodium, thus, giving him better control and manoeuvrability when the female shifts position before insemination actually occurs. In those species possessing shorter gonopodia, Rosen and Tucker (1961) found that males often used visual displays that would result in the female holding a position and allowing insemination to occur, so males from species that perform courtship displays do not need to manoeuvre once they are positioned to inseminate the female. Thus, they hypothesize the switch to a mating system of cooperation relaxed selection for a long gonopodium. Our results for males of P. velifera and their hybrids may reflect this trade-off in gonopodium length and its influence on a mating repertoire of primarily courting versus one of primarily thrusting.

Overall, morphological traits contributed little to variation among genotypes in the rates of gonopodial thrusting. There was a general trend of increased length of the gonopodium associated with an increase in gonopodial thrusting rate, but the amount of variation explained by the regression was less than 3%. Interestingly, this relationship was driven primarily by the sailfin Y-chromosome line, with the shortfin Y-chromosome line showing no relationship between gonopodium length and thrusting rate. This result may reflect the behavioural trade-off in sailfin mollies, in which males with short gonopodia rely more on courtship displays than gonopodial thrusting as their primary mating strategy.

Finally, there was also a negative relationship between the size-adjusted length of the dorsal fin and the rate of gonopodial thrusting. Again, this relationship was driven primarily by the significant association of these traits in the sailfin Y-chromosome line, but not in the shortfin Y-chromosome line. Phenotypically, this result suggests that males with smaller lateral projection areas are more likely to switch from courting to thrusting. Genetically, if larger males (that also possess larger dorsal fins) are more likely to inherit Y-linked alleles for courtship displays, as has been shown for X. nigrensis (Zimmerer and Kallman, 1989) and in sailfin mollies (Ptacek, 2002; Loveless et al., 2009), then a positive phenotypic association between these traits would result. A similar positive association between relative size of the dorsal fin and courtship display rate within P. velifera has been argued to explain the alternative mating strategies exhibited by large and small males in this species, where small males perform only gonopodial thrusts while larger males use thrusts and courtship displays (Hankison and Ptacek, 2007).

Implications for speciation in sailfin mollies

The divergence of the sailfin molly lineage was associated with a switch in the mating system from one of male–male competition with forced insemination to one of female cooperation in mating and mate choice in response to courtship displays (Farr, 1989; Ptacek and Breden, 1998). Results of our breeding design demonstrate a genetic basis for both morphological and behavioural components of the courtship–mating signal, providing an avenue for rapid divergence as a result of sexual selection through female choice. Female mating preferences for large, courting males have been demonstrated in both sailfin and shortfin molly species (Ptacek, 1998; MacLaren et al., 2004; MacLaren and Rowland, 2006; Kozak et al., 2008) and such preferences may have led to rapid divergence in Y-linked alleles for courtship during speciation of sailfin mollies. Additive, autosomal genes contributing to larger dorsal fin size were likely to have increased in frequency as well, due to their contributions to increasing apparent male size (MacLaren et al., 2004; MacLaren and Rowland, 2006; Kozak et al., 2008). A strict test of these hypotheses would involve demonstrating genetic correlations between Y-linked courtship gene(s) and autosomal genes for dorsal fin shape and should be the focus of future studies. The patterns of phenotypic associations between morphology and behaviour demonstrated in this study do, however, argue that sexual selection favoring multiple components of a mating signal has the potential to promote correlated changes leading to rapid divergence in mating signals and subsequent speciation.