Introduction

A long-standing debate in anthropology concerns our own origins as anatomically modern humans (Homo sapiens sapiens). The qualifying term ‘anatomically modern’ is used to identify our early ancestors that were physically much the same as living humans, but also to distinguish them from earlier hominins that could be called ‘human’ at some level based on features such as an increased brain size relative to body size and the possession of a material culture (including stone tools). The fossil record of the past 2 Myr shows modern humans evolving from earlier humans, often referred to as ‘archaic humans’, a broad group that includes the species H. heidelbergensis as well as the Neandertals of Europe and the Middle East. What is less clear is the evolutionary relationship of modern humans to the various archaic human populations, as well as to earlier ancestors. Did modern humans evolve via anagenesis from a single archaic species across the Old World, or did they first arise in Africa? If the latter, then did modern populations expanding out of Africa replace the archaic human populations that lived outside of Africa, or did they interbreed with them? Were the Neandertals a separate species from modern humans and, if so, did any hybridization take place?

These and other questions fall under what has been termed the ‘modern human origins debate’. Although this debate is often focused on the fossil and archeological records, studies of genetic variation have become increasingly important as a source of insight. Much of the work in this area has consisted of detailed analyses of patterns of genetic variation in living human populations. The strategy here is based on the realization that whatever our species' evolutionary past, it has left visible signatures on our genome. Expectations of current genetic variation under different evolutionary scenarios are compared with observed genetic variation in our species in order to test various origin models. In addition, the last decade has also seen an increase in the analysis of ancient DNA, such that mitochondrial and nuclear DNA sequences are now available for the Neandertals, an archaic human group. The purpose of this review is to highlight some of the major findings of genetic analysis (using both living and ancient DNA) and their use (and misuse) in the modern human origins debate. The focus here is primarily on recent findings and the status of the debate as I perceive it.

The fossil record of evolution in the genus Homo

In order to understand the contributions of genetic research to the modern human origins debate, it is first necessary to provide a brief review of the fossil record for human evolution over the past 2 Myr. Only a brief review is given here; more detail is available in many current texts on human evolution (e.g., Conroy, 2005; Stringer and Andrews, 2005). Molecular evidence suggests that the hominin and African ape lines diverged about 6–7 Myr ago. The fossil record of the first possible bipeds dates back over 6 Myr ago in Africa. By 4.2 Myr ago, there is definite evidence of bipedal hominins in Africa (Australopithecus anamensis). These early hominins walked upright (at least on the ground), had ape-sized brains and larger protruding faces and teeth. The species H. erectus appeared in Africa 1.8 Myr ago and is characterized by modern limb proportions, increased brain size, reduction in the size of the teeth and developments in stone tool technology. Until this point in time, hominin evolution had taken place exclusively in Africa, but populations of H. erectus dispersed to Eastern Europe and Southeast Asia about 1.7 Myr ago (note: some anthropologists refer to the initial African population as the species H. ergaster and reserve the name H. erectus for the Southeast Asian populations). Some populations of Southeast Asian H. erectus may have survived until 27 000–54 000 years ago, and perhaps are related to the newly named dwarf species H. floresiensis (although others consider the type specimen to be a pathological modern human).

The descendants of H. ergaster/H. erectus have often been referred to broadly as archaic humans, a label that bridges the gap between early humans (H. ergaster/H. erectus) and modern humans (H. sapiens sapiens). The archaic humans have, on average, a brain size approaching that of modern humans, but with a lower and differently shaped skull and larger face and brow ridges compared with modern humans. Although it had been common for some time to refer to the archaics as archaic H. sapiens, there is growing sentiment to classify these forms into two (or more) species. A number of these specimens are classified as H. heidelbergensis, a species that lived in parts of Africa, Europe and possibly Asia from about 800 000–200 000 years ago. A somewhat different archaic form, the Neandertals, lived in Europe and the Middle East from about 130 000–28 000 years ago, and has a number of unique craniofacial traits that distinguish them from H. heidelbergensis and H. sapiens. Classification of the Neandertals has always been a contentious issue, with some anthropologists proposing that they should be classified as a subspecies of H. sapiens (H. sapiens neanderthalensis) and others arguing that they be considered a separate species (H. neanderthalensis).

Models of modern human origins

The core question in the modern human origins debate is what is the evolutionary relationship between archaic and modern humans? It has been common to read many articles and books that describe the debate in terms of two opposing camps—the ‘out of Africa’ model and the multiregional evolution model. In reality, this dichotomy is a bit of an oversimplification, because there is overlap between some variants of these models, and the two models are often misrepresented. Adding to possible confusion is the fact that there are two separate but related questions regarding modern human origins (Relethford, 2001b, 2007). The first of these questions concerns the time and place of the transition from archaic to modern humans. Did modern human anatomy emerge first in Africa followed by dispersal across the rest of the Old World, or did modern humans instead emerge because of the mixing of different evolutionary changes taking place in different parts of the Old World? The second question concerns the evolutionary relationship between archaic and modern forms. Were they separate species that arose through cladogenesis with little if any hybridization, or were they part of a single evolutionary lineage (anagenesis)? A variety of models, some only subtly different from others, have been generated through different answers to these questions. Some models share answers to one question, but differ on the other. Consequently, there is often disagreement over what models are supported by fossil and genetic observations. Thus, it is useful to examine briefly a few of the basic models.

The African replacement model is one form of what is referred to in general terms as an out of Africa model. According to the African replacement model, anatomically modern humans arose as a new species (H. sapiens) in Africa between 150 000 and 200 000 years ago. By 100 000 years ago or so, populations of this new species began expanding throughout the Old World, replacing preexisting archaic human species outside of Africa (such as the Neandertals of Europe). Under this model, there was virtually no genetic input from these archaic populations. All living humans thus can trace all of their ancestry 200 000 years ago to Africa.

The multiregional evolution model presents a different explanation for the origin of modern humans. The multiregional model was first developed to explain how some traits, such as increased cranial capacity and reduction of the face, could evolve across the Old World while at the same time other traits could retain regional distinctiveness over time, such as the high prevalence of shovel-shaped incisors in past and present Asian populations. The seeming conflict between similarity between populations and regional continuity over time was explained by a balance of gene flow, selection and genetic drift (Wolpoff et al., 1984; Relethford, 2001b). According to some proponents of the multiregional model, there was no single time or place associated with the origin of modern humans. Indeed, some have argued that the distinction between archaics and moderns is arbitrary and difficult to define. Instead, this view of multiregional evolution posits that the anatomic and genetic changes leading to modern humans took place piecemeal across the Old World, and modern humans eventually resulted from the regional coalescence of these changes because of gene flow between populations. All of these changes took place within a single evolutionary lineage. Contrary to some representations of the multiregional model, it does not claim that the appearance of modern humans was due to independent or parallel evolution in different parts of the Old World (Wolpoff et al., 2000; Relethford, 2001b).

Although the modern human origins debate is frequently discussed in terms of the more extreme views regarding out of Africa and multiregional models described above, in reality a number of anthropologists have argued for models that combine an initial African origin of modern humans with varying degrees of gene flow taking place between modern humans dispersing out of Africa and preexisting human populations outside of Africa. Elsewhere (Relethford, 2001b, 2007) I have labeled such models ‘primary African origin’ models, characterized by the hypothesis that modern human anatomy did emerge first in Africa (in common with the African replacement model), but that there was some degree of genetic mixture with preexisting archaic populations outside of Africa (in common with the multiregional model). One example of a primary African origin model is Smith's ‘assimilation model’, a variant of the multiregional model that allows for an initial African origin (Smith et al., 2005).

The fossil evidence

It may seem strange to have a section about the fossil evidence in a paper focusing on the genetic evidence written in a genetics journal. Although brief, this section is vital to any understanding and evaluation of the genetic evidence for modern human origins. Both the fossil and genetic evidence must be considered in any assessment of the debate, as both sorts of data inform each other, and any proposed answers must be compatible with both. Arguments over which sort of data is ‘best’ have no place in the debate. Each type of data has its own advantages and disadvantages, and therefore both must be examined.

One of the more obvious inferences that can be made from the fossil record concerns the question of the time and place for the emergence of modern humans. Where and when in the fossil record do we see the first appearance of modern humans? According to a strict multiregional view that there was a gradual transition between archaic and modern anatomy, it will be difficult if not impossible to pinpoint a single point of origin. On the other hand, both the African replacement model and the assimilation model predict that anatomically modern humans appeared first in Africa. Complicating the debate is disagreement over anatomical definitions of modernity and over geologic dates. For many years, there was suggestive evidence of an early appearance of modern humans in Africa as early as 130 000 years ago, although the anatomy and dating was debated. More recently, the picture seems clearer, with evidence of early moderns in Africa at 160 000 years ago, classified by the discoverers as H. sapiens idaltu (White et al., 2003). In addition, modern humans from the Omo site in Ethiopia have recently been redated to 195 000 years ago (McDougall et al., 2005). Compared with these early dates, the first appearance of modern humans outside of Africa is later in time, with dates around 92 000 years ago in the Middle East, 60 000–40 000 years ago in Australia and 40 000–30 000 years ago in Europe. Given the fossil record as currently exists, it seems clear that the prediction of modern humans appearing first in Africa is supported and the type of regional coalescence predicted by some variants of multiregional evolution is less likely. As noted later in this paper, an initial African origin is also compatible with the genetic evidence. This does not mean, however, that the evidence necessarily rules in favor of the African replacement model. Instead, the debate has shifted for many to one of replacement versus admixture. In other words, what happened to the archaic populations that lived outside of Africa when they encountered modern populations from Africa? Did they become extinct, or was there some genetic contact? If there was gene flow, then was it the consequence of gene flow between local populations and/or long-range dispersal within a species, or hybridization between species, or perhaps different forms of admixture in different parts of the Old World?

There is suggestive fossil evidence for genetic mixture of archaics and moderns. Some anatomical traits are persistent over time, tending to occur more often in certain geographic populations both past and present. This regional continuity is frequently cited as evidence for a genetic contribution from archaic populations (Wolpoff et al., 1984; Smith et al., 2005), although others have argued that it might represent the retention of traits from a shared ancestor of archaics and moderns. Other fossil evidence supporting genetic contact includes the discovery of a 4-year old anatomically modern child from Portugal that dates back 25 000 years ago that has some Neandertal traits suggesting partial Neandertal ancestry (Duarte et al., 1999), although others have argued that these traits are not of Neandertals (Tattersall and Schwartz, 1999).

The genetic evidence: gene trees

The rapid development of molecular genetics and the emergence of coalescent theory in population genetics have provided valuable tools for the construction and analysis of genetic genealogies known as gene/haplotype trees. Coalescent theory makes use of the fact that genetic drift over time will result in the extinction of lineages, which in turn means that when looking backward from the present-day generation, any sample of DNA markers will coalesce to a common ancestor. The application of analytic methods based on coalescent theory means that, given a sample of genetic markers, it is possible to identify characteristics regarding that sample's most recent common ancestor (MRCA), specifically the time back to the MRCA and the geographic place that the MRCA lived. If the sample of individuals includes adequate representation from across all of humanity, then these inferences tell us about the MRCA of our species. It is important to keep in mind that such methods work back only as far as the MRCA. Because of the coalescent process, all genetic variation coalesces to the MRCA and thus no information is available for population history before this individual (Templeton, 2005).

Early work on gene trees focused on mitochondrial DNA (mtDNA), inherited strictly from one's mother. The maternal haploid inheritance of mtDNA means that recombination is not a problem and one can reconstruct the gene tree. The pioneering application to the modern human origins debate was analysis of mtDNA by Cann et al. (1987), where they found evidence that the MRCA lived in Africa roughly 200 000 years ago. The idea of a common female ancestor of humanity led, perhaps inevitably, to this ancestor being given the name ‘Eve’. Although there were some methodological concerns with their analyses, later analyses confirmed the recent African origin of humanity's most recent common mitochondrial ancestor (Relethford, 2001b).

Both the location (Africa) and the date (200 000 years ago) of our common mtDNA ancestor have been argued as support for the African replacement model of modern human origins. The date was considered too recent to be compatible with the multiregional model, which argued for a common ancestor dating back to Africa close to 2 Myr ago. Although the observation of a recent mtDNA ancestor is certainly compatible with the African replacement model, compatibility does not automatically imply proof (Relethford, 2001b; Templeton, 2007). Compatibility with a given model only constitutes proof of a model when the observed data are incompatible with all other models. In this regard, it is important to keep in mind that any gene tree tells us only about the MRCA for that particular DNA marker. Different DNA markers will have different MRCAs, a reason why the ancestral female detected from the mitochondrial analyses is often referred to by the qualifying name ‘mitochondrial Eve’. She tells us about our common mitochondrial ancestor, and nothing about other parts of our genome. Because of difference in mutation rates and genetic drift, different DNA markers may have different evolutionary histories. As such, evolutionary inferences based on one DNA marker do not necessarily apply to others, and multiple loci must be examined in order to begin to build a picture of a population's history (Templeton, 2005). In addition, potential problems such as recurrent mutation make an exclusive focus on mtDNA alone problematic. More loci are needed.

In addition to mtDNA, gene trees have now been constructed using Y-chromosome DNA (thus sampling paternal ancestry) and a number of nuclear autosomal DNA regions that show little or no recombination. The results vary, with some showing recent African ancestry and some showing more ancient African ancestry. Most, but not all, gene trees show an African root (Takahata et al., 2001). In recent years, more effort has been made to look at the question of modern human origins using multiple-locus comparisons. The most comprehensive of these analyses has been performed by Templeton (2005, 2007) who examined 25 DNA regions: mtDNA, Y chromosome DNA, 11 X-linked markers and 12 autosomal markers using a 6-Myr-old date for the human–chimpanzee divergence for calibration. Using a method known as nested-clade phylogeographic analysis, Templeton found that 15 of these markers showed evidence of geographic expansion. The estimated ages of range expansion vary significantly across these markers and do not fit a model of a single expansion, but instead cluster into three groups: (1) an expansion out of Africa 1.9 Myr ago (95% CI=0.99–3.10 Myr), (2) an expansion out of Africa 650 000 years ago (95% CI=390 000–970 000 years ago) and (3) an expansion out of Africa 130 000 years ago (95% CI=9600–169 000 years ago).

The three out-of-Africa expansions detected in Templeton's multilocus analysis correlate temporally in an interesting way with the fossil record (Relethford, 2007). The earliest range expansion corresponds to the initial appearance and dispersal out of Africa of H. erectus. The second range expansion corresponds to a rapid increase in cranial capacity that took place about 700 000 years ago and overlaps with the appearance of H. heidelbergensis. The third and most recent out-of-Africa expansion corresponds to dates suggested for the dispersal of anatomically modern humans. Given the large confidence intervals typical of coalescent analysis, this correspondence should be taken as suggestive and not conclusive, but the apparent congruence of the fossil and genetic records is interesting and deserves continued attention, particularly as data on more low-recombination DNA regions become available.

The genetic evidence for three out-of-Africa expansions, each possibly associated with events in the fossil record corresponding to the appearance of a new species, might be taken as support for a broader picture of speciation within the genus Homo, of which the origin of anatomically modern humans is but the latest speciation event. However, although the most recent expansion (169 000 years ago) is similar to those often reported in the literature in support of an African replacement model, Templeton argues that the genetic evidence of earlier expansions rejects the hypothesis of a total replacement. Gene trees based on living humans will only pick up evidence from the ancestors of living humans, and if we are all descended from a single recent (<200 000 years) expansion out of Africa, then there should be no evidence of any earlier expansions. In other words, speciation and complete replacement should erase any genetic evidence for earlier speciation (Templeton, 2005, 2007). If confirmed, evidence of multiple expansions might point to a model of human evolution where the transition between one paleospecies and the next occurred through anagenesis. The first expansion, associated with H. erectus, established hominins outside of Africa for the first time. Subsequent expansions may have involved some level of gene flow between new populations and preexisting non-African populations. Of course, the level of gene flow is likely to have varied from time to time and from place to place, and it seems likely that some populations were replaced, but unlikely that this occurred to all populations.

Not everyone agrees with these interpretations. For example, Satta and Takahata (2002) note that although Templeton's results might reflect an African origin with subsequent interbreeding, they might also reflect population structuring within Africa with some populations being more isolated from Eurasia, which could account for high haplotype diversity within Africa as well as the large proportion of gene trees that have African roots. I do not view this possibility as necessarily being incompatible with multiple African expansions and subsequent interbreeding. It is possible that a variety of different models could fit observed patterns of genetic variation.

The genetic evidence: regional differences in genetic diversity

Not all living human populations show the same average level of genetic variability, and these differences in present-day diversity can provide us with inferences about our evolutionary history. DNA markers typically show higher levels of genetic diversity (heterozygosity and nucleotide diversity) in sub-Saharan African populations. This observation has been made for mtDNA (Cann et al., 1987), nuclear microsatellite DNA (Relethford and Jorde, 1999) and Alu insertion markers (Watkins et al., 2001). The same observation has been made on measures of variation from phenotypic traits; within-group variances are highest in sub-Saharan African populations for both craniometric measures (Relethford and Harpending, 1994; Manica et al., 2007) and skin color (Relethford, 2000).

Why would one geographic region consistently show higher levels of genetic and phenotypic diversity? One possibility is greater time depth for the accumulation of mutations. The longer a population has been in existence, the greater the number of mutations that will accumulate. Under an African origin model, mutations would accumulate longer in Africa, as any populations dispersing out of Africa would likely be small, and the subsequent founder effect would effectively ‘reset’ the accumulation of mutations in the non-African populations. Thus, a model of an initial African origin followed by dispersals out of Africa at a later point in time would generate the regional differences in genetic and phenotypic diversity that we see today. If correct, the observation of higher African diversity supports the other genetic (and fossil) evidence for an African origin for modern humans, but does not distinguish between an African origin with replacement and an African origin with admixture outside of Africa except to say that if there was any admixture it was not of sufficient magnitude to erase the genetic signature of an African origin.

Furthermore, the fact that the model of accumulated mutations is compatible with the observed genetic data does not mean that it is correct if there are other reasonable interpretations that are also compatible. In the case of genetic diversity, another possible explanation is regional differences in population size, because expected diversity is proportionally related to effective population size. Smaller populations experience more genetic drift and are therefore lower levels of diversity. If the long-term effective population size of Africa were larger throughout most of recent human evolution, then diversity would be greater in Africa than elsewhere, again consistent with our observations of present-day variation. Analyses of craniometric data and microsatellite DNA support this hypothesis (Relethford and Harpending, 1994; Relethford and Jorde, 1999). A larger African population is also consistent with archeological and ecological inferences (Relethford, 2001b; Eller et al., 2004). If higher levels of genetic diversity in sub-Saharan Africa are due to a larger long-term effective population size, then the observation of higher diversity does not provide any resolution about the modern human origins debate. All of the models proposed to date can easily accommodate a larger African population. In this case, genetic data may be telling us more about the demographic, rather than phylogenetic, history of our species.

The genetic evidence: a cline in genetic diversity

Inferences about modern human origins can also be made from the observation that in addition to sub-Saharan Africa having the highest levels of genetic diversity, there is also a geographic pattern in regional diversity. Specifically, genetic diversity outside of Africa tends to be a subset of the diversity within Africa (Tishkoff et al., 1996; Watkins et al., 2001). In addition, global analyses of microsatellite DNA markers (Prugnolle et al., 2005; Ramachandran et al., 2005) and craniometric traits (Manica et al., 2007) have shown clear patterns of genetic and phenotypic diversity decreasing as the distance from East Africa increases.

This geographic cline in diversity is consistent with an African origin and subsequent dispersal of modern humans out of Africa. The nature of this dispersal is still being debated. For example, Ramachandran et al. (2005) argue that the diversity cline is consistent with a model of serial founding effects out of Africa, whereas Liu et al. (2006) suggest that a model of colonization events with gene flow is more appropriate. Further work also needs to consider the suggestion discussed above for larger African population sizes and, more generally, alternative demographic scenarios that might apply. It is possible that a number of different models can all fit the diversity cline equally well. Furthermore, the fundamental issue of the fate of archaic populations outside of Africa remains. At present, the diversity cline might reflect a genetic signature of an expansion out of Africa, but in my view does not resolve the debate over replacement versus assimilation. Although many analyses have used the diversity cline in support of a replacement model, Eswaran et al. (2005) have provided an expansion model that incorporates assimilation from archaic populations.

The genetic evidence: the estimated number of human ancestors

The relationship between genetic diversity and population size has meant that it is possible to estimate the effective population size of a species. When applied to data from living human populations, such estimates typically suggest a long-term average of 10 000 individuals of reproductive age (Relethford, 2001b). Because long-term population size reflects a harmonic mean over time, this figure further suggests that humans have recently expanded from a relatively small number of ancestors, a view that fit with many conceptions of an African replacement model. The reasoning here is that such a small number of individuals could not have been spread out throughout the Old World and still be connected via gene flow, making any sort of multiregional evolution unlikely (Harpending et al., 1993). A small species effective size is thus considered more likely explained in terms of an expansion from a small initial population in Africa. The situation is not that simple, however, as the genetic estimates of population size are effective population sizes that can differ greatly from actual census size and the latter is not a proxy for the former. Effective size can be considerably less than census size. Eller et al. (2004) show that a model of extinction and recolonization of local populations with reasonable parameters could result in a long-term census size of several hundred thousand individuals and still produce an effective population size of about 10 000. In another example, Eswaran et al. (2005) have shown how a wave-of-advance model could also explain low estimates of effective species size even given a larger census size. Of course, the compatibility of a model and data does not necessarily prove the model, but it does show that the initial view of the small species effective size supporting an African replacement is also not necessarily correct.

The genetic evidence: ancient DNA and the fate of the Neandertals

The debate over replacement versus assimilation is nowhere more intense than in continuing discussions of the evolutionary status and ultimate fate of the Neandertals. Questions regarding whether Neandertals should be considered a subspecies of H. sapiens or a separate species ultimately boil down to the question of how much of our ancestry derives from them. Even if they were a biologically separate species, the question of possible genetic ancestry remains because modern humans and Neandertals could have been allotaxa that could hybridize (Smith et al., 2005). What genetic evidence exists for some potential Neandertal ancestry in modern humans?

Although the question of Neandertal ancestry has long been addressed using the fossil record, the past decade has also seen the rapid development of Neandertal genomics as methods of ancient DNA analysis have been applied to Neandertal fossils. The first example of this approach was the pioneering work of Krings et al. (1997) extracting a 378-bp sequence of mtDNA from the original Neandertal specimen. Subsequent analyses of ancient DNA have shown that the results from this study were not a fluke, and short mtDNA sequences have now been extracted from 11 other European Neandertal fossils (Green et al., 2006). As a group, the Neandertal mtDNA sequences are more similar to each other than to living humans. Although it was thought initially that Neandertal mtDNA exhibited relatively low levels of variation (Krings et al., 2000), the accumulation of more specimens suggests their mtDNA diversity might have been greater (Caramelli et al., 2006; Orlando et al., 2006).

The differences between Neandertal and living human mtDNA have often been interpreted as support for the view that Neandertals were a separate species that diverged about half a million years ago. This conclusion has not been universally accepted. Although the number of sequence differences between Neandertals and living humans is greater than found among living humans today, this difference is still within the range seen between chimpanzee subspecies, consistent with the view of some that Neandertals were a separate subspecies rather than a separate species (Relethford, 2001b).

Regional comparisons of mtDNA have also been used to support the claim that Neandertals were a separate species with little if any hybridization. It has been argued that if there was a genetic contribution of European Neandertals to living Europeans, then the mtDNA of Neandertals should be more similar to mtDNA from living Europeans than to mtDNA from living humans in other geographic regions. Instead, Neandertal mtDNA is genetically equidistant from living humans across the world, an observation taken as evidence that Neandertals were a separate species (Krings et al., 1997). The accuracy of the underlying assumption of this argument, however, depends on rates of gene flow and the age of the Neandertal specimens, and it is possible for Neandertals to have contributed more genetically to living Europeans, but over time this regional difference could have diminished because of interregional gene flow (Relethford, 2001a).

The major finding from Neandertal mtDNA that supports the hypothesis that the Neandertals were not part of our ancestry is the fact that no Neandertal mtDNA sequences have been found among living humans. In addition, studies to date of ancient DNA from anatomically modern humans show genetic differences from Neandertal mtDNA (Caramelli et al., 2003; Serre et al., 2004). This noticeable genetic difference can be explained in two ways. First, it is possible that there are no surviving Neandertal mtDNA sequences in our species because the Neandertals became extinct and did not contribute to our ancestry. A second possibility is that the Neandertals were part of our ancestry but their specific mitochondrial haplotypes were lost over time due to genetic drift. The question of Neandertal ancestry has now become a question of the probability of haplotype survival, and as such is dependent on the parameters used to determine these probabilities, including population sizes, rates of gene flow and duration of contact between Neandertals and modern humans in Europe. Studies to date have suggested that the total possible amount of Neandertal ancestry for living humans was not very large, but it has not yet been possible to distinguish between a model of some limited Neandertal ancestry and a model of no Neandertal ancestry (Currat and Excoffier, 2004; Serre et al., 2004).

Although samples of Neandertal mtDNA are accumulating, there are obvious limitations on exactly how much we can learn from mtDNA, which essentially represents but a single locus. One of the most exciting developments in recent months has been technological advances that have allowed the sequencing of nuclear DNA. Two different methods were applied to a Neandertal fossil from Vindija Cave, Croatia that dates back to 38 000 years ago. One study (Noonan et al., 2006) sequenced 62 500 base pairs and the other study (Green et al., 2006) sequenced over one million base pairs. These initial studies confirm the genetic distinctiveness of Neandertals relative to living humans and suggest that the lineages diverged about 500 000–700 000 years ago. The question of gene flow and ancestry remains uncertain at present. Noonan et al. (2006) looked for derived alleles at low frequencies in living Europeans that matched the derived alleles found in the Neandertal sequence in order to estimate the amount of Neandertal ancestry in living humans. Their maximum-likelihood estimate was 0% suggesting complete extinction of the Neandertals, but had a wide enough confidence interval (0–20%) so that some limited admixture could not be ruled out. The analysis of Green et al. (2006) used a different approach by comparing the DNA sequences of the Neandertal specimen, living humans and chimpanzees, with chimpanzees representing the ancestral state of different single nucleotide polymorphisms (SNPs) and living humans representing the derived state. Derived alleles found in both living humans and the Neandertal specimen represent alleles that existed in the last common ancestor of modern humans and Neandertals. They found that the Neandertal sequence had the derived allele in over 30% of the SNPs, a figure they argue is too large to be compatible with a simple model where the lineages leading to Neandertals and modern humans diverged and then remained completely isolated, thus arguing for some subsequent gene flow between them.

In addition to the need for larger samples, the problem of possible contamination from modern humans in ancient DNA analysis needs to be considered. For example, Wall and Kim (2007) investigated differences between the Green et al. and Noonan et al. studies, and suggested a possibility that these differences may have been due to errors in the former. Thus, results from nuclear DNA analyses should be considered tentative and suggestive rather than conclusive. The nuclear DNA data at present do not convincingly argue for or against admixture between Neandertals and modern humans. Further analysis, combined with the possibility of a complete Neandertal genome in the near future (Dalton, 2006) may provide more definitive answers regarding possible Neandertal admixture.

The evidence from ancient DNA, both mitochondrial and nuclear, must be considered alongside the fossil evidence regarding the fate of the Neandertals. As a group, the Neandertals are extinct and have been perhaps as long as 28 000 years. The real question is the nature of their disappearance. Although the idea of replacement by modern humans, presumably better adapted biologically and/or culturally, has been a popular conclusion, closer examination of the fossil evidence, combined with genetic analysis, suggests that the situation may not be that straight forward. The earliest modern humans in Europe show the presence (though at reduced frequency) of unique Neandertal traits, a pattern that is not expected under a model of complete replacement (Smith et al., 2005). These traits become less common over time, and are often absent in living Europeans, suggesting that over time the Neandertals become extinct through ‘swamping’ genetically of larger population of modern humans moving into Europe (Relethford, 2001b; Smith et al., 2005). Under this model, the Neandertal gene pool was assimilated rather than replaced. Therefore, the overall ancestral contribution of Neandertals to living modern humans may be very small, a suggestion consistent with analyses of both mtDNA (Serre et al., 2004), as well as patterns of linkage disequilibrium in living humans (Plagnol and Wall, 2006). A continuing challenge is to develop methods capable of distinguishing between a model of very low Neandertal ancestry and a model of no Neandertal ancestry.

The genetic evidence: adaptive genetic introgression

Both the fossil and genetic evidence support an initial origin of anatomically modern humans in Africa 200 000 years ago followed by dispersal across the Old World. The primary debate at this point is the extent to which earlier archaic humans living outside of Africa contributed to our species' ancestry. The discussion this far has been on the total amount of archaic ancestry as inferred from neutral loci and traits, and thus reflects the genetic impact of genetic mixture. At least in the case of the Neandertals of Europe, DNA analysis suggests that this mixture was very low and possibly zero. Consequently, it is tempting to suggest that the debate over very low mixture or no mixture is of little import because even if present in small amounts, such mixture would have had little evolutionary impact (Currat and Excoffier, 2004). Although this is certainly the case for neutral loci, the situation could be rather different when considering the additional impact of natural selection.

Genetic introgression occurs when an allele is introduced from one group (a species or subspecies, for example) into another at a low initial frequency. Introgression can have a significant impact if the newly introduced allele is favored through natural selection. Hawks and Cochran (2006) review examples of such adaptive introgression in plants and animals and make the case that the genetic introgression of ‘archaic’ alleles into the gene pool of an expanding modern human population can explain why some analyses (both on fossils and genetic data) show evidence of ancient mixture and others do not. It is possible that an expanding modern human population demographically and genetically ‘swamped’ the contributions from archaic populations outside of Africa for the most part, but some alleles persisted because of natural selection. Hawks and Cochran (2006) argue that adaptive introgression might have had a major impact on the evolution of modern humans, and outline some possible avenues for future research to test this hypothesis.

One possible example has been suggested by Evans et al. (2006), who proposed that the microcephalin (MCPH1) gene is an example of adaptive introgression in our species from an archaic human source. They found that haplogroup D of this locus has a high frequency in living humans (70%) but a fairly recent coalescent date of roughly 37 000 years ago. Their analysis further shows support for the following scenario. Two lineages of the genus Homo diverged roughly 1.1 Myr ago, and the lineage leading to H. sapiens experienced fixation of the non-D haplogroup and the D haplogroup became fixed in the other lineage. At about 37 000 years ago, the D haplogroup was introduced through an interbreeding event, and the frequency of this introgressed allele increased rapidly because of positive selection, perhaps relating to changes in brain size or function. Evans et al. argue that the only other demographic scenario that could explain the genetic history of MCPH1 gene, the complete mixture of the two initially diverged lineages, should result in similar genetic histories across the genome, which does not appear to be the case.

There are several problems with the suggestion that the MCPH1 gene is an example of adaptive introgression. Studies have found no association between the normal variants of the gene and brain size (Woods et al., 2006) or intelligence test scores (Mekel-Bobrov et al., 2007). In addition, Currat et al. (2006) have suggested that the patterns of variation in MCPH1 can instead be explained by a demographic model of African origin and expansion rather than adaptive introgression. Although the MCPH1 gene may not prove to be a good example, adaptive introgression is still a viable explanation for suggestions of archaic influence on the modern genome (Hawks and Cochran, 2006) and is a worthwhile area for further research. What will be needed in such research is demonstration that selection accounts for patterns of diversity, rather than a neutral model, and demonstration of a selective advantage for the introgressed allele.

Conclusions

Although the modern human origins debate continues, there are signs of agreement on some general points, and it is clear that genetic data, from both living populations and from ancient DNA, have played a major role. Both geneticists and paleoanthropologists have increasingly accepted an initial African origin of modern humans. At present, the fossil evidence points to an earlier appearance of modernity in Africa than elsewhere, and a variety of genetic analyses also support an initial African origin, although this pattern is not as clear because of the strong possibility of a larger long-term effective population size in Africa and its subsequent effects on patterns of genetic variation. Although debate continues, there is also growing realization that while Africa may have been the primary source of our ancestry, it might not be the only one (Templeton, 2005, 2007). Elsewhere I have suggested that modern human ancestry can be described as mostly, but not exclusively, out of Africa (Relethford, 2001b). Much recent research has been directed at the quantification of how much non-African archaic ancestry exists. Some recent syntheses of the fossil record and genetic evidence make a strong case for the assimilation of non-African archaic genes into the modern human gene pool (Smith et al., 2005). Although the analysis of ancient mtDNA has suggested that there was little genetic input in Europe (the Neandertals), it is possible that some genes might reflect greater archaic ancestry because of adaptive introgression. These populations became extinct, but may have left some genetic legacy behind. Continued analysis of the human genome for evidence of recent selection and the possibility of a Neandertal genome, combined with continued improvements in the fossil record, promise to make the coming decades an exciting time for modern human origins research.