Main

Population genetic theory is central to understanding the dynamics of cancer evolution. Cancers are clonally evolving populations of somatic cells shaped by the processes of mutation, migration, genetic drift and selection. Applied to human cancers, theoretical and empirical population genetic analyses have provided deep insights into distinct modes of cancer cell evolution, clonal diversity, selection and adaptability1,2,3,4,5,6,7. Many reviews have examined cancer through the lens of population genetics8,9,10,11 and recent work applying evolutionary and ecological theory to transmissible cancers has touched on these topics12,13,14. Here, we explore these issues to provide a broad population genetic framework for probing the unusual phenomena of transmissible cancers.

Transmissible cancers are malignant somatic cell clones that spread between individuals via physical transfer of living cancer cells15,16 (Fig. 1a). Such tumours evolve across a longer “tape of life”17 than other cancers, extending well past the life span of any individual host. They therefore exemplify cancer evolution over long timescales, similar in some respects to cancer cell cultures maintained in laboratories. Recent sequencing studies provide empirical data to explore the effects of asexuality on the genomes of these long-lived cancer lineages and to examine predictions based on the population genetics of asexual populations18,19,20,21,22,23,24.

Fig. 1: Transmissible cancers as long-term evolution experiments.
figure 1

a, Clonal transmission of cancer (in this example, canine transmissible venereal tumour, CTVT) between unrelated individual hosts via the physical transfer of living cancer cells. b, Cancers are made up of genetically heterogeneous populations of cells. Coloured circles represent distinct cancer cell populations within the tumour. Population bottlenecks occur during transmission of cancer cells to new hosts. New cancers are typically founded by only a small fraction of the cells present in the donor tumour (the founding population). During the time interval, Δt, the founding population that passes through the bottleneck replicates and expands. The effect of bottlenecks on clonal diversity is shown for single-cell, tight and wide bottlenecks, as represented by the size of the gap between the grey bars. Severe bottlenecks purge most or all of the diversity, whereas less stringent bottlenecks preserve more diversity. c, In many evolution experiments, including the LTEE with E. coli, populations are propagated by serial transfer of a proportion of the previously grown population into fresh medium (top panel). The bottleneck at each transfer reduces diversity, but the number of cells transferred is large enough that many beneficial mutations survive and eventually become fixed in the population. Aliquots of the evolving populations are collected periodically and stored (typically frozen) for future analyses. Single-cell bottlenecks are used in so-called mutation accumulation experiments (bottom panel). By plating cells and picking individual colonies, one imposes single-cell bottlenecks. These severe bottlenecks repeatedly purge genetic variation and thereby prevent adaptation by natural selection, which requires that variation.

Fig. 2: Gross appearance and histology of transmissible cancers.
figure 2

a, Gross appearance of CTVT in a female dog (image by M. Meyer). b, DFT1-affected Tasmanian devil with facial tumours characteristic of devil facial tumour disease (DFTD) (image by A. Kreiss). c,d, Representative images of haematoxylin and eosin (H&E) stained histological sections of a DFT1 (c) and DFT2 (d) tumour. Scale bar, 50 μm. Images provided by D. Hayes, Animal Health Laboratory, Department of Primary Industries, Parks, Water and Environment, Tasmania. e, Haemolymph extraction from a warty venus clam to identify BTN cells. Image provided by A. L. Bruzos and S. Díaz. f, H&E stained histological section of gills from a common cockle infiltrated with BTN cells. Scale bar, 100 μm. Image provided by A. L. Bruzos.

We begin this Review with a brief introduction to cancers as clonal organisms and an overview of the known transmissible cancers. We then consider key population genetic processes in the context of clonal evolution and how the parameters that govern these processes might influence transmissible cancer evolution. In particular, we concentrate on the process of genome decay that may result from the switch to clonal inheritance. Throughout the Review, we compare transmissible and non-transmissible cancer data with predictions from population genetic theory for clonal diploids as well as findings from experimental evolution and genome-scale studies of asexual organisms. Of special note is the iconic long-term evolution experiment (LTEE) of Lenski and colleagues, wherein 12 populations of the bacterium Escherichia coli were founded from a common ancestral strain in 1988 and have since evolved for 75,000 generations25,26,27. Indeed, it is tempting to think of transmissible cancer lineages as naturally occurring LTEEs.

However, before we discuss the parallels, it is important to recognize the substantial ways in which transmissible cancer populations differ from microbial populations used in experimental evolution. Experimental microorganisms are usually haploid and they evolve in controlled environments. Evolution experiments also typically begin with a model organism that is unicellular and largely asexual, whereas the ‘natural experiment’ of a transmissible cancer is one where an initially sexual population becomes clonal and transitions from multicellularity to unicellularity. Conventional (non-transmissible) cancers might then also be considered short-term experiments involving these same transitions. Transmissible cancers are not restricted to a fixed number of hosts and lineages (unlike the 12 LTEE populations) and therefore they have the potential to diversify across a much larger number of transmission lines. Additionally, these cancers are inoculated into a continuously changing ‘medium’ at each transmission, with resources and immune pressures varying in each new host. Despite these differences, experimental and other studies of asexually evolving lineages can provide insights relevant to the population dynamics and fitness landscapes of cancer cells28,29,30,31.

Cancers as clonal organisms

Most cancers originate from the abnormal clonal outgrowth of a single somatic cell32,33. The classic portrait of cancer evolution depicts a process akin to Darwinian evolution, with stochastic mutational events introducing somatic genetic variation into an expanding population of asexual cancer cells33,34. This framework can also encompass heritable traits acquired through epigenetic changes35,36. Cancer cell clones that acquire mutations conferring a replicative or survival advantage, often called ‘drivers’, are selected and drive successive waves of expansion of increasingly disordered clones and tumour progression. Here, clone refers to a group of cells that share a common genotype owing to descent from a common ancestor. So-called ‘passenger’ mutations, which may be neutral or even harmful to the cancer lineage, can nonetheless multiply by hitchhiking on the genome of expanding clones carrying driver mutations. Somatic mutations accumulate inexorably over time even in normal cells during the renewal and maintenance of healthy tissue, before any malignant transformation. In fact, recent studies have demonstrated that physiologically normal tissues often harbour many potentially competing clones5,37,38.

More than 60 years ago, Julian Huxley proposed that cancer cells might be viewed as new asexual species arising through spontaneous mutation and thus represent an accelerated instance of speciation39,40,41. New individuals arise from somatic cells that are genetically identical to ancestral individuals at all genomic sites except those affected by somatic mutations. Asexual reproduction of neoplastic cells also implies the absence of meiotic recombination and, as a consequence, deviation from Hardy–Weinberg equilibrium and other theoretical expectations that presume sexual reproduction42.

Asexual reproduction is an uncommon mode of reproduction for eukaryotes. Most multicellular asexual organisms are of recent origin and derive from sexual ancestors43. Strictly asexual animal species are especially rare and they are expected to have short evolutionary lifespans. Population genetic theory makes several bleak predictions regarding genome evolution in strictly asexual lineages44,45. In particular, strict clonality is predicted to cause the irreversible accumulation of deleterious mutations, leading to genome decay, especially in lineages with small or moderate effective population sizes typical of most animal species46,47. Moreover, the absence of recombination should lead to inefficient selection48, while limited genetic diversity in clonal populations reduces their evolutionary potential. However, important exceptions exist and some asexual animals, such as bdelloid rotifers (with hundreds of asexual species), are thought to have survived without sex for at least 40 million years49,50.

Transmissible cancers

Cancers frequently acquire properties that cause cells to leave the primary tumour site and establish new tumours elsewhere in the body. This final step in tumour progression—metastasis—is an ominous feature of most malignant cancers. By contrast, transmissible cancers can move between different individual hosts via the transfer of living cancer cells. We might think of this transmission as an otherwise unprecedented next step in the invasion–metastasis cascade. These rare, contagious ‘inter-individual metastases’ have overcome strong physical and immunological barriers by acquiring adaptations both to allow the transmission of cells between allogeneic hosts and to evade the immune response12,13. To survive, these clones must also achieve a genome structure compatible with long-term continuity.

Compared to cancers that are restricted to an individual host, the apparent rarity of transmissible cancers suggests that their occurrence is infrequent and improbable. Alternatively, perhaps they arise more frequently than realized but are short-lived or remain undetected51,52,53. However, once established in a population, such cancers can propagate quickly and persist for thousands of years19. Ten naturally occurring transmissible cancer lineages have been described to date: one in domestic dogs54,55, two independent lineages in Tasmanian devils56,57,58 and several widespread independent lineages in marine bivalves23,24,59,60,61 (Table 1). Each of these asexual lineages originated in founders whose somatic cells acquired changes that drove carcinogenesis and these lineages continue to exist long after the death of the individuals that gave rise to them. Box 1 briefly describes the natural history of these transmissible cancers.

Table 1 Known transmissible cancer lineages

Population genetics: core processes and effects

Populations of transmissible cancer cells undergo the same core population genetic processes that govern the evolution of all asexual populations: mutation, selection and drift. We begin the Review with a discussion of these core processes. We then delve into how these processes give rise to more complex and specific phenomena in asexual populations, such as Muller’s ratchet, mutational meltdown and the fate of transposable elements (TEs), and we consider recent findings especially relevant to transmissible cancers.

Mutation and recombination

Evolution requires the generation of heritable variation. In sexually reproducing organisms, this variation is produced by both mutation and recombination (sensu segregation and reassortment). Even in some asexually reproducing organisms (including most bacterial species in nature), horizontal gene transfer (HGT) leads to some inter-organismal recombination. In conventional cancers, lineages are strictly asexual. Many types of mutation can play a role in cancers: point substitutions, insertions, deletions, chromosomal rearrangements and copy-number alterations, including even the duplication or deletion of whole chromosomes. Epigenetic changes that are stably inherited over cell generations could also be viewed as a kind of mutation in the context of cancer cell populations. Genomic instability is considered a hallmark of non-transmissible cancer62 and late-stage cancers in particular often become increasingly unstable with rapid and disruptive genomic rearrangement via chromothripsis (massive, clustered rearrangement), kataegis (localized hypermutation), chromoplexy (large chains of rearrangements affecting multiple chromosomes) and other mutational processes63,64,65. All else being equal, the supply of new mutations increases as a cancer cell lineage proliferates within an individual and, in the case of transmissible cancers, as the cells spread to new hosts, thereby fuelling inter- and intra-tumour heterogeneity11.

Selection

Selective pressures produce changes in genotype frequencies within a population based on differences in relative fitness. Mutations can fall along a continuum of selective effects from lethal to deleterious to selectively neutral to advantageous. Selection has especially strong effects on genome evolution in asexual organisms: each beneficial mutation that sweeps to fixation eliminates pre-existing diversity because the entire genome is one linkage group. However, most new mutations with fitness effects are deleterious66,67.

Selective pressures change over time and are dependent on environmental context. Early cancer cell populations may expand exponentially but eventually they become constrained and cells must compete for limited space and resources, leading to slower growth68,69. Neutral drift, negative selection and positive selection can all influence the evolution of tumour genomes. While positive selection drives tumour progression, negative selection can remove deleterious mutations and potent neo-antigens70. A growing, but sometimes contentious, body of literature supports neutral evolution as an important process shaping tumour genomes71,72,73,74,75.

Given the long-term potential for mutations to accrue in transmissible cancers, these lineages may be particularly sensitive to mutational burdens and require negative selection to maintain stability. Indeed, there is evidence that negative selection acts on CTVT mitochondrial genomes to maintain their functionality76. One recent study of the CTVT genome suggested that negative selection is limited to genes that encode essential functions19. Moreover, there was no detectable excess of non-synonymous relative to synonymous mutations, implying little ongoing positive selection for new variants in CTVT. Collectively, these findings suggest that weak negative selection and genetic drift are the main evolutionary forces acting on CTVT genomes, and perhaps other transmissible cancers, over long periods19.

The observation of weak negative selection in cancer could be explained in several ways, as highlighted by refs. 70,77. In any somatic lineage, many genes are likely to be irrelevant to fitness78, especially in long-lived cancers under selection for increased survival and proliferation. Also, having two copies of a gene typically buffers the impact of new mutations, probably masking their phenotypic effects70. Furthermore, high mutation rates and clonal growth in the absence of recombination allows deleterious passenger mutations to hitchhike with beneficial drivers, provided the fitness effect of the driver is greater than that of the passenger. Weakly deleterious mutations can thus become fixed in a cancer cell population by hitchhiking alongside drivers. Genome-wide linkage prevents deleterious mutations from being separated from other variants, thereby leading to their progressive accumulation and impeding the spread of beneficial mutations79. In conventional cancers, tumour structure and population size can also influence genetic drift70. However, the extent to which these factors affect selection in transmissible cancers requires further investigation.

Drift

Drift is the stochastic process that describes temporal fluctuations in the frequency of alleles or genotypes in a population due to random birth–death events: by chance, one parent or lineage leaves more surviving offspring than another and its genetic contribution therefore expands. Drift generally has a greater impact in determining the fate of genetic variants in small populations than in large ones80. However, it should be noted that new mutations, even in very large populations, are invariably rare when they first appear and therefore they are strongly affected by drift. Even beneficial mutations are often lost by random drift before they can become established in a population25,81,82.

Transmission bottlenecks, particularly in asexual populations, can promote genetic drift; neutral and even deleterious alleles can rapidly fix and beneficial mutations can be lost, rendering selection inefficient. For example, when microbial populations are experimentally subjected to repeated extreme bottlenecks during serial transfer (Fig. 1b,c), evolution is dominated by random mutation and drift (ref. 83 and references therein). Similarly, during periodic transmission between hosts, only one or a few cells from the donor typically seed a tumour in a new host15,59 (Fig. 1b,c) and the transmitted cells may be a random set of those that are potentially transmissible. An important difference between the microbial experiments and transmissible cancers is that as the number of infected hosts increases, the effects of drift are reduced at the ‘species’ level (across the set of transmission lines). However, it is usually not feasible to follow transmission lines in these cancers directly and over short time intervals.

For positive selection to be consequential within a single transmission line, it must be sufficiently strong to overcome the variation-purging effect of bottlenecks. Indeed, there are likely to be strong selective barriers during transmission and thus bottlenecks do not rule out the possibility of an important role for positive selection during transmissible cancer evolution. Strong positive selection might also come into play during tumour growth, when mutation and clonal expansion occur within hosts and between transmissions. Looking at the whole population and across all transmission lines, selection dynamics will also be influenced by changes in the effective population size84. Finally, the absence of positive selection does not necessarily imply the preponderance of neutral evolution, as additional processes come into play, including those described below.

Muller’s ratchet

In clonal organisms, the progressive accumulation of deleterious mutations can lead to a decline in fitness (Fig. 3)46,47. Without new beneficial mutations, the maximum accessible fitness in an asexual population is determined by the individuals with the fewest and least deleterious mutations. In small populations, this ‘least-loaded’ class of individuals is readily lost by drift85 and, without recombination, it cannot be reconstituted. Repeated losses of the least-loaded class can be envisioned as successive clicks of a hypothetical ratchet, called Muller’s ratchet46,47. The rate at which the ratchet advances depends on the interplay of population size, mutation rate and the strength of selection86. In the case of transmissible cancers, the rate at which the ratchet advances will depend on the size of the infected host population. Therefore, when infection and transmission are extensive, the ratchet is slowed at the level of the whole population of parasitic tumours.

Fig. 3: Accumulation of deleterious mutations during Muller’s ratchet.
figure 3

Horizontal bars represent genomes of different clones labelled A–F in a population. Red crosses indicate deleterious mutations. The most-fit clone is indicated by the dashed box. Time runs from left to right. In small populations, the most-fit clone can readily be lost by random genetic drift, which advances Muller’s ratchet. Clonal diversity also tends to decline over time and, because new mutations are more often deleterious than beneficial, the ratchet continues to advance. At higher mutation rates (bottom panel), new deleterious mutations arise more often and Muller’s ratchet advances more quickly.

Furthermore, a feedback loop can arise in which an increasing load of deleterious mutations causes a decline in population size, which leads to a faster-advancing ratchet and more rapid population decline87. In ever-smaller populations, allele frequency changes are increasingly driven by drift, and deleterious mutations without effective selection can cause extinction. This synergy between random genetic drift and mutation accumulation is called a mutational meltdown87,88.

Mitochondrial meltdown

Similar considerations apply to the normally non-recombining and clonally inherited mitochondrial genome89,90. Mitochondria are ancient haploid asexual lineages and they accumulate mutations during somatic growth. Cancer cell lineages are particularly susceptible to mitochondrial DNA erosion91,92, which raises the question of how long-lived transmissible cancers maintain functional mitochondrial genomes13,14.

While the CTVT nuclear genome is clonal93, the CTVT mitochondrial genomes are polyclonal and derive from horizontal transfer of mitochondrial DNA (mtDNA) from intermediate host dog genomes to CTVT76,94,95 (Fig. 4a). The original CTVT mtDNA haplotype, which was present in the founder dog that spawned this cancer lineage, has been replaced and is not found in any extant tumour76.

Fig. 4: Horizontal transfer and selection of mtDNA.
figure 4

a, CTVT mitochondrial genomes (mtDNAs) are polyclonal and have been acquired by occasional horizontal transfer of mtDNAs from host cells. The CTVT mitochondrial genome (red) is replaced by the host mitochondrial genome (black), owing to either genetic drift or selection, before the CTVT cells are transmitted to a new host (blue). b, Selection can act on mitochondrial genomes both within and between cells. When cell lineages are physically separated, mitochondria compete against one another within cells. This within-cell competition can lead to the accumulation of ‘selfish’ mtDNAs carrying mutations that confer a replicative advantage but are also deleterious to the cell. c, Competition between cells, by contrast, facilitates selection against such ‘selfish’ mtDNAs.

The replacement of CTVT mitochondrial haplotypes that have been degraded by deleterious mutations, via replacement with ‘rejuvenated’ host haplotypes, could provide a selective advantage to the lineage96. However, the possibility that CTVT acquired host mtDNA via neutral processes cannot be ruled out, nor can the possibility of ‘selfish’ mitochondrial genotypes with a replication or survival advantage be excluded (Fig. 4b,c). A recent example of this last scenario is the report of a single naturally occurring canine mtDNA haplotype that has repeatedly invaded CTVT cells22. An insertion in the mtDNA control region is thought to provide this haplotype with a replicative advantage, driving it to fixation in multiple CTVT lineages22.

In addition, there is evidence for mtDNA recombination activity in transmissible cancers. Multiple complex mtDNA recombination events have been reported in CTVT76. Recombination might be triggered by DNA damage and the resulting cellular responses, which could play a role in mtDNA repair and support long-term survival76. Recombinant mtDNA haplotypes have been reported in MtrBTN2-affected Mytilus chilensis individuals involving an MtrBTN2 Mytilus trossulus-derived sequence and host M. chilensis-derived sequence, and more recently in BTN2-affected M. trossulus from the Sea of Japan (Table 1)60,97. These instances of recombination, along with a recent report of mitochondrial movement between MtrBTN2 sublineages24, imply that horizontal transfer of mtDNA also occurs in bivalve cancers60,97. Horizontal transfer of mtDNA has not yet been reported in DFTD20,57. Whether dynamic mitochondrial populations are a common feature of transmissible cancers and whether the recombinant mitotypes have selective advantages, remain questions for future studies.

Counteracting the ratchet

Compensatory mechanisms can counteract deleterious mutations in some asexual lineages. Loss of heterozygosity (LOH) can occur via gene-conversion processes or hemizygous deletions. LOH may provide advantages similar to genetic segregation, potentially slowing Muller’s ratchet98 and facilitating the spread of beneficial mutations99. Long-tract LOH events were recently proposed to reduce the load of deleterious mutations in asexual human-infective trypanosomes100. Widespread LOH has also been reported in CTVT93, although it is not known whether these changes arose as a way to eliminate deleterious mutations. Notably, gains of heterozygosity, rather than LOH, were recently reported in MtrBTN2 (ref. 24).

LOH events can also reduce fitness if they expose existing deleterious recessive alleles through loss of complementation101,102,103. In some asexual species, such as the crustacean, Daphnia pulex, it has been suggested that LOH contributes more to genomic degeneration than mutation accumulation104,105. The relative importance of LOH events and mutation accumulation is further complicated because certain kinds of epistasis can slow the advance of Muller’s ratchet106, while even weak purifying selection can stop it entirely107.

Another means of counteracting the ratchet is by DNA uptake and introgression. Modelling has shown that HGT in prokaryotes can mitigate the effects of Muller’s ratchet, even if on average more deleterious mutations are introduced than are removed108. Introgression of paternal DNA from sympatric sexual species was reported in Poecilia formosa, a clonal fish109. HGT has also been seen in bdelloid rotifers110,111, where it could compensate for the absence of recombination. Whether this process plays an important role in the biology of transmissible cancers remains to be seen. Changes in ploidy may also provide ways of counteracting Muller’s ratchet13,14,112.

Meselson effect

The ‘Meselson effect’ is another potentially important consequence of mutation that follows from long-term asexual reproduction in diploids45,49. Without recombination, heterozygosity should tend to increase indefinitely because alleles at the same locus can diverge over time113. This effect was first reported at the genome level in the typically asexual diploid pathogen Trypanosoma brucei gambiense100; more recently it was also reported in a parthenogenetic animal, the orbatid mite114. However, other homogenizing genome processes, such as gene conversion, can oppose this effect and limit the extent of allelic divergence104,115. For example, a high rate of mitotic recombination relative to mutation appears to have prevented the Meselson effect in some asexual lineages of Daphnia104.

Transmissible cancer genomes have not yet been examined for the Meselson effect but recent studies provide a window into the divergent mutational landscapes of these cancers. The CTVT genome has remained largely diploid and stable despite marked aneuploidy and widespread copy-number changes, rearrangements and retrotransposon insertions93. Importantly, a recent study has shown that the massive mutational burden observed in CTVT reflects the lineage’s age, rather than an intrinsically high point-mutation rate19. Likewise, DFT1 is a relatively stable cancer lineage in terms of genome integrity20,57,116. However, it was recently shown that DFT1 cell lines have increased frequency of copy-number variants compared to DFT1 primary tumours and it was suggested that artificial culture conditions may select for genetically unstable DFT1 clones20. The HeLa tumour cell line, established more than 70 years ago, might be similar in this regard. HeLa cells are relatively stable in terms of point-mutation variation, with few new point mutations seen after early passaging117. These studies suggest that extensive genomic instability can go hand in hand with limited single-nucleotide variants during cancer evolution.

Transposable element burden

The fate of TEs in asexual populations is uncertain. TEs are mobile DNA elements capable of autonomous self-replication, including both ‘copy-and-paste’ and ‘cut-and-paste’ behaviours. While TE insertions may occasionally be beneficial, most are deleterious and reduce the host’s fitness118. A recent study of the dynamics of insertion sequence (IS) elements in the E. coli LTEE found that increased IS activity initially promoted adaptation in some lineages, but this activity became detrimental over time across most or all populations119. Recombination, segregation and genetic exchange in sexual organisms can improve the efficacy of selection and facilitate the elimination of deleterious insertions. In asexual lineages arising from sexual ancestors, active TEs could, in principle, accumulate indefinitely and the resulting burden might even lead to extinction120,121. However, selection might favour mutations that silence these elements and clonal propagation has been predicted to result in populations that are free of active elements122,123. It seems likely, then, that only those asexual lineages with effective mechanisms for excising or suppressing TEs could persist over long evolutionary timescales. The near absence of retrotransposons in bdelloid rotifers has been offered as evidence of their ancient asexual status120. This argument presumes that the loss of active transposons allowed bdelloids to avoid the extinction that might otherwise have followed the transition to clonality120.

In some conventional cancer genomes, evidence suggests that the mechanisms that normally suppress mobile DNA elements are compromised, which allows these elements to become more active, and that insertions of retroelements can cause oncogenic mutations124. Somatic retrotransposition is an ongoing process in the CTVT genome even after thousands of years of asexual evolution93. Also, a LINE-1 retrotransposon insertion upstream of c-myc, diagnostic of CTVT125, has been suggested to be an early driver in the evolution of the transmissible cancer phenotype93,126. Steamer, an LTR retrotransposon, is highly expressed and massively amplified in the transmissible cancer genomes of soft-shell clams and it has been suggested that this element had a role in the evolution of the disseminated neoplasia126,127. The tumour-specific retrotransposon landscape of DFTD remains to be characterized.

Mutators

Mutators are genetic variants that cause an increased genome-wide mutation rate, thus producing more deleterious, neutral and beneficial mutations each generation than do non-mutator genotypes. Mutators occur when mutations disrupt some aspect of DNA repair or replication pathways. Mutators often produce a strong bias toward specific mutation types, depending on the underlying causal mutation. For example, a frameshift mutation in the mutT gene in one of the E. coli LTEE populations caused a >100-fold increase in the overall point-mutation rate that was dominated by A:T to C:G transversions128,129. Later mutations in the mutY gene partially compensated for the overall increase in the mutation rate while also introducing an additional bias toward C:G to A:T transversions128,129.

In general, mutations are more likely to be deleterious than beneficial. Nonetheless, mutators can evolve in asexual lineages because they generate rare beneficial mutations at a higher per-capita rate than repair-proficient genotypes and, owing to the absence of recombination, they are linked to the beneficial mutations to which they give rise130. In this way, asexuality facilitates the evolution of elevated mutation rates by coupling mutators to any advantageous mutations that they generate131. Mutators can hitchhike with beneficial mutations even in large populations, where the increased mutation rate may lead to little or no increase in the rate of fitness gains132. Mutators typically emerge in response to either new environments, such as in the LTEE, or rapidly shifting conditions, such as when a pathogen faces changing host defences and therapies133. Such conditions allow mutators to become established because new selective pressures create more opportunities for beneficial mutations. Despite their potential to accelerate adaptive evolution, mutators also generate higher loads of deleterious mutations. Mutators thus accelerate genetic decay in small populations, where the random processes of mutation and drift overwhelm the capacity of natural selection to retain well-adapted genotypes, such that fitness tends to decline (discussed in ref. 134).

In conventional cancers, the emergence of mutators appears to be an early step in progression and associated with specific tumour types135,136. Chromosome and microsatellite instability are often cited as examples of hypermutator phenotypes observed in tumours with impaired DNA mismatch repair137. A highly context-specific mutational motif (C to T mutations within GTCCA pentanucleotides) was reported in the CTVT genome as well as hyperactivity of an endogenous mutational process in some tumour sublineages; it is possible that these mutational pulses are caused by the emergence of mutator alleles19,138. However, as discussed earlier, genomic stability is probably required for long-term survival of transmissible cancer lineages and selection against destabilizing mutators is likely to become particularly important in long-lived lineages. Indeed, there is some evidence that the process driving the GTCCA motif ceased to operate ~1,000 years ago, which suggests that it could have been generated by a transient mutator phenotype19. As noted in the context of the LTEE, antimutators (mutations that compensate for mutators) can also evolve that reduce the mutation rate and thus the rate of increase in the load of deleterious mutations129. Whether antimutators might explain the abrupt cessation of the process responsible for this specific motif in CTVT has yet to be determined.

Linkage and interference

Because tumours evolve asexually, the entire genome functions as a single linkage unit. The fate of any mutation depends not only on its own fitness effect but also on the genetic background in which it occurs. Thus, mutations that happen to occur in a background that has advantageous alleles can reach high frequencies by hitchhiking with them139 and this process should be especially prevalent in clonal organisms140. In particular, the spread of deleterious mutations linked to beneficial mutations becomes more probable in asexual populations, potentially contributing to genome decay141.

Advantageous mutations that arise in different lineages within the same asexual population cannot recombine into a single genetic background. Instead, clones with beneficial mutations compete not only against their progenitors but also against one another, which interferes with the spread and substitution of beneficial mutations82. In fact, clones carrying beneficial mutations can be driven extinct by this clonal interference, despite reaching substantial frequencies before eventually being overtaken. This process of clonal interference has been shown to impose a ‘speed limit’ on the rate of adaptive evolution132. While more prevalent in asexual organisms, this effect can sometimes be observed even in sexually reproducing populations142. In general, clonal interference occurs more often in large populations and with high mutation rates because those features lead to more beneficial mutations that must compete with one another132,143.

Theory predicts that linkage between genetic loci reduces the efficiency of selection, and linkage is tightest and most prevalent in asexual organisms. In general terms, linkage couples the targets of selection—whether positive selection for beneficial mutations or negative selection against deleterious ones—with any linked variants that have the opposite effect, such that selection at one site can interfere with the response to selection at linked sites144. These so-called ‘Hill–Robertson’ effects can take a number of forms. For example, weakly beneficial alleles linked to highly detrimental alleles can be driven to extinction via background selection, while weakly deleterious alleles linked to strongly beneficial alleles can reach fixation. In asexual genomes, the fates of mutant alleles are especially dependent on the genetic backgrounds in which they arise. A study of human cancer genomes found that Hill–Robertson effects, and not relaxed selection, lead to a large burden of deleterious mutations in many cancer types4. In light of this evidence, a possible explanation for the high mutational burden seen in CTVT19 is that background-linked selection has dominated the evolutionary process in this transmissible cancer.

Fitness and adaptation

The bottom line in population genetics is fitness, that is the propensity of a specific genotype to leave descendants under a particular set of conditions. Fitness gains require mutations that confer some benefit to the individuals with that genotype within a population in its present environment. The rate of adaptation then depends on the rate at which beneficial mutations arise, avoid extinction (especially while they are rare) and eventually become fixed in the population by natural selection. However, adaptive paths are not necessarily paved only by beneficial mutations, as deleterious mutations can serve as ‘stepping stones’ across fitness valleys, thereby also promoting adaptive evolution145.

Adaptation depends on mutation effects, rates and interactions. Experiments with microorganisms show that adaptation often arises as a result of relatively few beneficial mutations with large benefits, rather than a large number of mutations with small effects26,134,146,147. Similarly, new traits required for conventional cancer progression are often acquired by driver mutations in a few key genes148. However, empirical and theoretical analyses of E. coli populations in the LTEE show that fitness can continue to increase for at least 60,000 generations, even in a constant environment27,149. Thus, even small fitness gains can become important in large populations over long timescales.

From this evolutionary perspective, populations of cancer cells can be expected to undergo progressive adaptation until further beneficial mutations are exhausted, a stable equilibrium is reached or the host dies. Even non-transmissible cancers are dynamically adapting lineages that evolve within the complex and changing, albeit mortal, ecosystems of their hosts. Transmissible cancers face the same challenges as their conventional counterparts, along with the added complexity of moving between host individuals and adapting to changing microenvironments12. Changes in the frequency of alleles that appear to confer a selective advantage to DFT1-affected Tasmanian devils have recently been reported150. The DFT1 genomic regions exhibiting these signatures of selection contain genes associated with cancer risk and immune functions, supporting an adaptive explanation based on the emergence of immune-modulated resistance150. During the long-term evolution of cancers, there is also a tension between adaptive evolution and reducing genetic load, with load reduction becoming relatively more important as the cancer lineage becomes better adapted to its environment. It is interesting to note the apparent genital tropism of CTVT and facial tropism of DFTD and to consider whether this specificity is explained by the transmission route alone15, by synchrony between the cell-of-origin and a permissive host microenvironment51,151 or by some barrier to adaptive evolution that limits the opportunity for shifts in tropism.

A question of vital interest in cancer biology concerns the dynamics of fitness: do cancer lineages eventually reach a fitness peak? This question is of particular importance for transmissible cancers because they are potentially immortal. CTVT seems to have optimized its adaptation to the transmissible cancer niche early in its history19. The recent finding of weak negative selection in CTVT is consistent with the action of Muller’s ratchet and it suggests that its fitness may even be declining with time. Mechanisms that could limit adaptive genome evolution in these lineages include reductions in the number and effect-size of beneficial mutations as they become better adapted to their niche, leading to diminishing fitness returns. Furthermore, transmissible cancer clones are geographically dispersed, while adaptation may occur mostly within local populations. Without ongoing migration, selective sweeps would fix beneficial mutations only within the local populations and thus would not produce detectable genomic signatures of adaptive evolution.

Future directions for research

Transmissible cancers present a fascinating opportunity to study the genomic consequences of asexual evolution in natural populations and to test population genetic predictions. Furthermore, these exceptional tumours provide a chance to look for parallel phenotypic and genomic evolution across different transmissible cancer lineages, across independently occurring lineages within the same host species and across tumour sublineages arising from the same clone18,152. Studying such parallelism has proved to be a powerful way of identifying driver mutations, not only in the E. coli LTEE but also in natural infections including most recently SARS-CoV-2 (refs. 26,153,154,155). Experimental studies in yeast have found parallel chromosomal abnormalities in response to strong selective pressures, similar to changes often observed during tumour progression156,157,158. A recent study highlights the value of this approach for transmissible cancers, where genome architectures, mutational processes and putative driver mutations were found to be strikingly similar in the independently arising DFT1 and DFT2 lineages18. Identifying genomic signatures of adaptive evolution is an important challenge for future research and one that will help us understand the evolution of transmissible cancers more generally.

Those asexual lineages that suffer severe effects from a lack of recombination should go extinct quickly. By contrast, long-lived asexual lineages, including transmissible cancers, presumably represent the subset of asexual lineages that face lesser problems. Future studies should address the genetic mechanisms and population dynamics that allow these lineages to persist. For example, there has been no systematic study to date that examines the mechanism underlying the LOH events in CTVT and whether it might be advantageous in reducing the mutational load and thereby allowing long-term survival of this lineage. A comprehensive analysis of the frequency, size, distribution and gene content of LOH events across time in transmissible cancers, or even in immortalized cell lines, would therefore be worthwhile. Additionally, it is not known whether and to what extent transmissible cancers maintain strict clonality throughout their history. Remarkably, spontaneous recombination involving fusion of genetically distinct tumour cells has been reported to provide a mechanism for parasexual recombination159. Whether a similar process might take place in transmissible cancer genomes and whether recombination that involves exogenous DNA (besides mitochondria), host or otherwise, ever occurs in these lineages requires further study.

Describing the long-term dynamics of adaptation by natural selection is a question of fundamental interest for clonal populations27. Transmissible cancers present an opportunity to observe directly the rich and dynamic population genetic processes that characterize adaptation to new environments. In CTVT, the cycle of infection takes about 6 months; if this has been the case for 6,000 years, then there have been roughly 12,000 transmission events along each transmission line, which would provide considerable opportunities for adaptation. While CTVT sequencing studies have generated data over only a relatively short window in this lineage’s long history, the recent emergence, rapid identification and heterochronous sampling of DFTD means that the rate of molecular evolution can be observed and analysed almost in real time. Moreover, by following the fate of sublineages with different mutations, it may be possible to infer some features of the underlying distribution of fitness effects.

Another topic that remains to be explored in the context of transmissible cancers is spatial structure and population subdivision. At any given time, a single tumour can harbour multiple genotypes, leading to competition and, in the case of beneficial mutations, clonal interference141. The spatial distribution of these competing clones and the geometry of tumour growth are crucial factors that affect the fate of clones29. In transmissible cancers, these processes can play out not only within tumours but also on a global scale. Spatial diversification has been observed in DFT1, CTVT and MtrBTN2 lineages and sublineages continue to coexist even when introduced to the same host populations following secondary contact19,20,24. The contribution of migration to population genetic processes in transmissible cancer lineages has yet to be thoroughly studied.

Finally, mathematical approaches may need to be developed or refined to infer the parameters characterizing the population genetics of transmissible cancer progression. One key issue concerns the predominance of stochastic versus deterministic processes in the long-term evolution of these cancers19. Assessing the parameters necessary to understand the roles of random drift and natural selection in transmissible cancers will benefit greatly from further studies.