Speciation: The Origin of New Species

Under the commonly used ‘Biological Species Concept' (Mayr 1942), the formation of new species involves the evolution of reproductive barriers to the production of viable offspring either before (pre-zygotic barriers) or soon after (post-zygotic barriers) mating. Thus, new species form when individuals from diverging populations no longer recognize one another as potential mates, or opportunities for mating become limited by differences in habitat use or reproductive schedules. In some cases, these pre-zygotic isolating mechanisms fail to prevent inter-breeding among individuals from separate populations. In these cases, viable hybrids may form, or the consequences of a successful mating attempt may end in failure, either due to the production of inviable zygotes or sterile, non-reproductive offspring. These diverse pre- and post-zygotic barriers are of great importance to speciation biologists because they determine how reproductively-isolated populations are from one another, which indicates how far along the often continuous process of speciation that populations are. For example, reproductive isolation is weak in the early stages of speciation, but changes to strong or complete in later stages of speciation (Figure 2). One or more of the many types of isolating mechanisms may play a role in the evolution of species along a continuum (Figure 2). But how and why might reproductive barriers to genetic exchange evolve?

A major area of debate among speciation biologists is the geographic context in which it occurs (Figure 3). Ernst Mayr emphatically defended his view that speciation was most likely when populations became geographically isolated from one another, such that evolution within isolated populations would lead to enough differences among them that speciation would be an eventual outcome. "The ... evolution of isolating mechanisms as a by-product of the steady genetic divergence is inevitable" (Mayr 1963, p. 581). The central idea here is that when populations are geographically separated, they will diverge from one another, both in the way they look and genetically. These changes might occur by natural selection or by random chance (i.e., genetic drift), and in both cases result in reproductive isolation. This view of speciation of geographically isolated populations — termed allopatric speciation — is still widely held among speciation biologists as playing a major role in the evolution of biodiversity (e.g., Price 2007).However, speciation might also occur in overlapping populations that are not geographically isolated (i.e., sympatric speciation, Via 2001). The problem here is how do populations that are living in the same area, and exchanging genes, diverge from one another? Many biologists think this will be extremely difficult (Coyne & Orr 2004), but there are a few compelling examples where populations in different habitats are subject to contrasting patterns of natural selection (i.e., divergent selection) and overcome gene flow to diverge into different species. This could occur, for example, if insects adapted to living on different plants within the same geographic region (Feder et al. 1988). It will be interesting to see how many new examples emerge now that the idea of sympatric speciation is becoming less controversial. Another scenario for speciation in the face of gene flow, albeit at levels that are lower than during sympatric speciation, is ‘parapatric speciation'. Parapatric speciation refers to populations that are situated in geographic proximity to one another, usually with abutting but non-overlapping ranges. Here, a small proportion of each population are in actual contact with one another, and thus considered in sympatry, whereas the majority of individuals reside far enough apart that frequent encounters with one another are rare (Figure 3). There are putative examples of parapatric speciation in salamanders (Niemiller et al. 2008) and walking-stick insects (Nosil et al. 2002), but the phenomenon has received less attention that allopatric or sympatric speciation (Coyne & Orr 2004).

Biologists have long been fascinated with — and sought to explain — the origin and maintenance of biological diversity within and among species. Natural selection is generally recognized as a central mechanism of evolutionary change within species. Thus, natural selection plays a major role in generating the array of phenotypic and genetic diversity observed in nature. But to what extent is selection also responsible for the formation of new species (i.e., speciation)? To what extent do phenotypic and species diversity arise via the same processes, as proposed by Darwin?

Recent years have seen renewed efforts to address these questions. For example, populations living in different ecological environments (e.g., desert versus forest habitats) might undergo divergent and adaptive evolutionary change via divergent natural selection. These same evolutionary changes can also result in the populations evolving into separate species. For example, adaptation to different environments might cause differences between populations in the way individuals tend to look, smell, and behave. In turn, these differences might cause individuals from different populations to avoid mating with one another, or hybrids exhibit reduced fitness if mating occurs. Thus, the populations cease exchanging genes, thereby diverging into separate species because of the adaptive changes that occurred via natural selection. This is a simple description of the ‘ecological speciation' hypothesis (Rundle & Nosil 2005, Schluter 2009).

More specifically, ecological speciation is defined as the process by which barriers to gene flow evolve between populations as a result of ecologically-based divergent selection between environments. This process makes some simple predictions. For example, ecologically-divergent pairs of populations should exhibit greater reproductive isolation than ecologically-similar pairs of populations of similar age (Funk 1998). Figure 4 illustrates an example that supports this prediction. Other predictions are that traits involved in divergent adaptation will also cause reproductive isolation, and that levels of gene flow in nature will decrease as ecological differences between populations increase. All these predictions have now seen support, and outstanding questions concern the genetic bases of ecological speciation and the reasons why the process varies in how far it proceeds (Rundle & Nosil 2005).

A view that is becoming increasingly popular is that sexual selection, or selection related to variation in reproductive success, plays a role in speciation (Panhuis et al. 2001, Ritchie 2007). This model suggests that differential patterns of trait variation related to reproductive success within populations contribute to the reproductive isolation among populations. A compelling example is related to the explosive radiation of cichlid fishes in the African Rift Lakes, where populations with overlapping distributions are diverging as a function of the differential preference of male color in mate selection (Seehausen et al. 2008).

A current debate is whether sexual selection can lead to speciation in the absence of ecological divergence (van Doorn et al. 2009). Indeed, compelling examples that implicate an important role of sexual selection leading to new species sometimes also involve the evolution of different signals used in mate-selection among populations in different ecological contexts, such as light environment (Seehausen et al. 2008, Maan & Seehausen 2010). Here, signals used in mate-selection become adapted to new ecological environments where the transmission of these traits is more perceptible or audible in a new habitat.

Some models of speciation do not include a role for selection of any sort, but rather invoke a key role for chance events. Such models include speciation by polyploidization, genetic drift in stable populations, and founder-events/population bottlenecks. Although such models have a long history in speciation research — for example in relation to speciation of small populations which colonize new islands — clear empirical support for such models in nature or even in laboratory experiments is relatively limited (Coyne & Orr 2004)

Another mechanism of speciation that involves chance events is speciation by polyploidization. Polyploidy, or the presence of three or more complete sets of chromosomes, has been documented in a wide variety of taxa. Particularly prevalent in plants, between 47 and 70% of all angiosperms are polyploid (Ramsey & Schemske 1998). Because polyploidy can lead to hybrid infertility, it is viewed as a mechanism that can rapidly lead to the formation of new species, potentially without selection for the divergence of other characters.

A lack of strong examples for speciation by genetic drift, yet evidence for ecologically-similar species pairs (Price 2007), has led to the development of a powerful alternative mechanism to ecological speciation. This alternative is ‘mutation-order speciation', defined as the evolution of reproductive isolation by the fixation of different advantageous mutations in separate populations experiencing similar selection pressures (Mani & Clarke 1990, Schluter 2009). In essence, different populations find different genetic solutions to the same selective problem. In turn, the different genetic solutions (i.e., mutations) are incompatible with one another, causing reproductive isolation (Schluter 2001, 2009, Price 2007). Whereas different alleles are favored between two populations under ecological speciation, the same alleles would be favored in both populations under mutation-order speciation (i.e., ‘uniform selection'). Divergence occurs anyway because, by chance, the populations do not acquire the same mutations or fix them in the same order. Divergence is therefore stochastic but the process involves selection, and thus is distinct from genetic drift. Selection can be ecologically based under mutation-order speciation, but ecology does not favor divergence as such, and an association between ecological divergence and reproductive isolation is not expected. How might mutation-order speciation arise? Sexual selection might cause mutation-order speciation if reproductive isolation evolves by the fixation of alternative advantageous mutations — for example those which increase individual attractiveness — in different populations living in similar ecological environments. For a summary of these models, see Table 1.

Divergent natural selection: Selection that acts in contrasting directions between two populations, usually with reference to ecological differences between their environments (e.g., large body size confers high survival in one environment and low survival in the other), or the special case where selection favors opposite extremes of a trait within a single population (i.e., disruptive selection).

Ecological speciation: A speciation process in which divergent natural selection drives the evolution of reproductive incompatibility (i.e., isolation) between taxa.

Mutation-order speciation: A speciation process in which different and incompatible mutations (alleles) fix in separate populations that are experiencing similar selective regimes.

Dobzhansky-Muller Incompatibility: Hybrid dysfunction arising from negative interactions (epistasis) between alleles at two or more loci: an allelic substitution at a locus causes no reduction in fitness on its own genetic background, but leads to reduced fitness when placed on the alternative background.

Haldane's Rule: The observation that if only one sex of hybrid offspring suffers intrinsic sterility or inviability, it is the heterogametic sex (see Coyne & Orr 2004 for a review of the causes of this pattern).

Genomic Island: A region of the genome where differentiation between populations is stronger than expected in the absence of divergent selection (stronger than occurs via purely neutral processes such as genetic drift alone).

Natural selection: Differential survival of classes of entities (such as alleles) which differ in some characteristic(s).

Sexual selection: Differential reproductive success of classes of entities (such as alleles) which differ in some characteristic(s).

Reproductive Isolation: Genetically-based differences between populations which reduce or prevent genetic exchange between them (i.e., reproductive barriers).