Since early in the 20th century there has been debate over whether polyploidisation and hybridisation are destructive or creative forces in evolution; this remains one of the outstanding questions that ‘we still don’t know about polyploidy’ (Soltis et al., 2010) and hybridisation. In animals, the occurrence of both polyploids and hybrids has historically been discounted, largely due to the influence of a few high-profile individuals. Muller (1925) wrote a paper entitled ‘Why polyploidy is rarer in animals than in plants’, which is still frequently cited as the justification for not considering polyploidy as a driving force in animal evolution. Mayr’s (1942) view on species concepts rejected the idea that hybrids were anything but reproductive accidents, and the concept of hybrid speciation did not fit with the definitions that he and others proposed at that time. Even in plants, although Stebbins (1940) spent the early part of his career documenting the widespread importance of polyploidy and noted the ability of polyploid lineages to expand into new environments, he later concluded that polyploids were evolutionary dead ends because selection would not be efficient in duplicated genomes (Stebbins, 1971).

Nevertheless, there has also been much emphasis on the alternative view that polyploidisation and hybridisation can promote diversification and speciation, by creating new combinations of genotypes that could increase the adaptive potential compared to the progenitor species (for example, reviewed in Comai, 2005; Mable et al., 2011). This could be due to either heterosis and/or masking of harmful recessives in the case of hybrids or changes in genomic structure or gene regulatory pathways resulting from the process of genome duplication, particularly when followed by differential loss or suppression of the expression of duplicated gene copies. Based on her pioneering studies of mobile genetic elements, McClintock (1984) included both hybridisation and polyploidisation in her list of unanticipated types of ‘genome shock’ that could result in dramatic genomic restructuring and set new contexts for gene expression. Although genome shock could result in increased regulatory flexibility, it could also destabilise genomes in hybrids of the same ploidy levels (homoploid hybrids) due to imbalances in gene expression that could cause sterility or mortality. Stebbins (1940) noted that sterility in diploid hybrids could be ‘rescued’ by duplicating the genome (that is, allopolyploidisation), predicting that polyploidy should be associated with hybrid speciation.

Although the frequency of autopolyploids (polyploids arising from whole genome duplication within a single species) is increasingly being found to be higher than originally predicted (for example, Parisod et al., 2010), the close association of hybridisation and polyploidy means that one barrier to understanding how each of these might contribute to adaptive potential has been disentangling whether genome doubling or introgression represents the greater genome shock. Some of the early genomic approaches to assessing the stability of newly created polyploids suggested that hybridisation might induce more substantial genomic rearrangements than genome duplication (for example, Hegarty et al., 2006), but the two processes have not always been separated (for example, Gaeta et al., 2007). Rapid expansion in genomic technologies provides the exciting potential to examine genome-wide patterns of genetic variation and introgression in relation to gene expression changes under a range of different environmental conditions. The greatest insights would come from comparing homoploid hybrids, allopolyploids and autopolyploids created from the same parental species, and interpreting patterns with respect to physiological, behavioural and fitness implications under varying environmental selection pressures.

The purpose of this special issue is to increase awareness of the potential role of polyploidy and hybridisation in adaptation and invasiveness by highlighting examples of classic case studies, methodological challenges to extending population genetics and phylogenetic approaches to polyploids and hybrids, experimental approaches to investigating the physiological implications of polyploidy, as well as the latest advances in understanding genomic and transcriptomic dynamics in relation to whole-genome duplication. The issue starts with a mini review by Andreas Madlung, emphasising the advantages of integrated approaches to understanding both the ecological and genomic consequences of polyploidy and hybridisation. The rest of the issue is based on research papers resulting from talks presented at the International Conference on Polyploidy, Hybridisation and Biodiversity, held in Pruhonice near Prague, 7−10 May 2012.

The first research paper summarises a classic case of invasion: the Japanese knotweed (genus Fallopia), a complicated complex of hybrids and polyploids that have so far been studied primarily with traditional cytogenetic, molecular and morphological approaches (Bailey). Although it has not been established whether polyploidy, hybridisation or both have increased the potential for invasiveness, the complex has shown repeated patterns of invasion on multiple continents. What is less clear is the long-term potential for adaptation, as the complex expands mostly clonally. The second example of a recent invasion is in the Mimulus gutatus-luteus complex in the United Kingdom. Vallejo-Marin and Lye investigate distribution patterns of an allohexaploid complex and its parental species, and question how a highly clonal hybrid could be such a successful invader. A challenge in such studies is applying allele-frequency-based population genetics models (for example, to analyse microsatellite data) designed for diploids, owing to the difficulty of establishing dosage of alleles. However, they find that the hybrids maintain substantial levels of genetic variation despite clonality and that the tetraploid parental taxon is now rare, suggesting that ongoing origins are not occurring. Distinguishing polymorphisms due to single or repeated hybrid origins from introgression with parental taxa after speciation remains a substantial challenge. Münzbergová et al. investigate the population genetic structure of Aster amellus in relation to ploidy. They find no evidence for gene flow among diploids and hexaploids at any spatial scale considered, confirming the species status of the allohexaploids and suggesting that current introgression is limited by strong reproductive barriers.

Although confirmation of hybrid origins can be conclusive when parental taxa are highly diverged from one another, it is more problematic to exclude the possibility of autopolyploidy when parental taxa are closely related. Population genetic models often require assumptions related to whether there is polysomic (most often associated with autopolyploidy) or disomic inheritance (expected to become fixed most rapidly in allopolyploids arising from genetically divergent parents), which can be difficult to establish. Meirmans and Van Tienderen explore the consequences of falsely assuming full tetrasomic inheritance for interpretation of population genetics divergence patterns. Encouragingly, they find that, whereas this can lead to biases when inheritance is fully disomic, only a small amount of allelic exchange among subgenomes is sufficient to reduce this bias.

Also problematic methodologically is resolving phylogenetic trees in the face of hybridisation and polyploidy. Ferrer et al. attempt to resolve relationships among a species complex (genus Hieracium) that is known to include reproduction through apomixis, polyploidy and extensive hybridisation, in terms of both origins of taxa and later introgression. They find that this leads to incongruency among phylogenetic resolution using different markers, but suggest that combined approaches can help to unravel complex complexes.

Despite the common view that polyploidy should provide greater genetic flexibility and thus confer a competitive advantage over diploid progenitors, there remains a paucity of studies that have experimentally tested possible mechanisms or explicitly compared the alternative that polyploids might be at a disadvantage under some environmental conditions. Using polyploid snails (Potamopyrgus antipodarum) as a case study, Neiman et al. explore the hypothesis that polyploidy might represent a phenotypic cost under conditions where dietary phosphorus is limiting. As polyploidy in this group and others is tied to asexuality, they propose that the cost of replicating a larger genome might affect competitive ability relative to sexual diploids. Coate et al. take a transciptomic approach to uncover mechanisms that could explain observations that allopolyploidy is associated with increased photosynthetic capacity and increased stress tolerance, using plants in the genus Glycine as a model. They provide a detailed analysis of the specific responses of allopolyploids compared to their diploid progenitors under limiting and excess light conditions and discuss specific pathways predicted to be involved in conferring enhanced responses. Repeating such experiments with replicated homoploid and allopolyploid hybrids and their progenitors would be intriguing to establish whether it is ploidy or genomic combination that drives the noted differences.

The final set of papers focuses on recent advances in ‘omic’ technologies for understanding the dynamic nature of polyploid genomes. Carvalho et al. provide a de novo transcriptome assembly of a hexaploid within another classic allopolyploid complex that has been involved in multiple invasions: saltmarsh-adapted plants in the genus Spartina. This assembly will provide a valuable research tool for future studies investigating adaptation in hybrids and polyploids occurring under different environmental conditions. Yoo et al. extend previous and pioneering work from the Wendel lab investigating the evolutionary dynamics of polyploid cotton (for example, Flagel and Wendel, 2010). In the current study they examine whole-genome expression patterns in order to consider the importance of different types of gene expression biases that are expected to occur in allopolyploids (see News and Commentary by Buggs (2012). Contrary to a model based purely on genome shock, they find that changes in patterns of expression increase more in allopolyploids than in the initial diploid hybridisation event and continue to accumulate through evolutionary time. Finally, Estep et al. investigate whether polyploidy is associated with rapid expansion of transposable elements in panicoid grasses and conclude that the process is more stochastic than has been proposed; there is no evidence among the species investigated that polyploid taxa have accumulated higher frequencies of transposable elements than diploids.

The limited set of examples presented in this special issue demonstrate that making generalised predictions about the response of polyploids under changing environmental conditions may be unrealistic. It also remains largely unresolved whether polyploidy, hybridisation or both explain the potential for invasiveness and adaptation to varying environmental conditions. What is required is the integration of ecology, evolution and genomics, in both plants and animals: recent developments in genomic technologies mean that this is now a realistic and tractable goal.