A parsimonious neutral model suggests Neanderthal replacement was determined by migration and random species drift

Most hypotheses in the heated debate about the Neanderthals’ replacement by modern humans highlight the role of environmental pressures or attribute the Neanderthals’ demise to competition with modern humans, who occupied the same ecological niche. The latter assume that modern humans benefited from some selective advantage over Neanderthals, which led to the their extinction. Here we show that a scenario of migration and selectively neutral species drift predicts the Neanderthals’ replacement. Our model offers a parsimonious alternative to those that invoke external factors or selective advantage, and represents a null hypothesis for assessing such alternatives. For a wide range of parameters, this hypothesis cannot be rejected. Moreover, we suggest that although selection and environmental factors may or may not have played a role in the inter-species dynamics of Neanderthals and modern humans, the eventual replacement of the Neanderthals was determined by the repeated migration of modern humans from Africa into Eurasia.

I am not an expert in paleontology but am familiar with the mathematical model. In general, I found the subject of this manuscript fascinating and thought provoking and for these reasons it could be a great contribution for Nature Communication. On the other hand, I did not find the analysis of the robustness of the results and their comparison with the data sufficiently clear.
I have the impression that most of these problems are caused the presentation style rather than the content itself, and the whole manuscript would benefit immensely from a thorough reorganization of the manuscript, mostly in the model and results sections.
In particular, while the discussion of the model itself is quite clear to me, what I miss in the Model section are (schematic) estimates of realistic parameter ranges of the model. What degree of asymmetry in migration do the authors estimate, and why? What are reasonable number and size of bands? This is discussed in the Results section but it would be more appropriate to have this information here. Moreover, much later in the paper, it appears that the basic timestep of the paper is 25 years. Also this information should appear here and should be supported by some estimate.
Also the results suffer from a similar lack of organization. I do not understand the reason to study the average time that a band changes hand. Why not study directly the distribution of fixation times and its fluctuations? After all, it seems to me that the main quantities of interest here are the fixation time and the fixation probability. It would be also very helpful to separate the numerical results from the model from and the comparison with archaeological evidences in different subsections.
For all these reasons, I recommend a revision of this manuscript.

Reviewer #2 (Remarks to the Author):
The major claim of this paper is that modern humans could have replaced neanderthals without having any selective advantage. Although this possibility has been mentioned before, as the authors note, this is the first time that it has been developed and explored through simulation, showing that it is plausible across a wide-range of reasonable values of key parameters. The results will be of wide interest and will certainly influence thinking in the field. The work is undoubtedly convincing and the authors themselves point that the obvious development is to give it a spatial dimension. Like them I think the result is unlikely to be qualitatively different. The simulations are clearly presented and reproduc ible. The authors show a good knowledge of the relevant archaeological and genetic literature and are right to point to the importance of the recent finding of modern human introgression in a 100,000 year old individual from the altai even if that is a long way from Europe. I recommend publication.

Reviewer #3 (Remarks to the Author):
The manuscript submitted by O.Kolodny and M. W. Feldman proposes a scenario of migration and selectively neutral species drift to explain the replacement of Neanderthals by modern humans in Western Eurasia. The authors claim this scenario to be more parsimonious than the scenarios based on environmental causes or selective disadvantage usually entertained to explain the demise of Neanderthals. The scenario is said to be in line with archeological evidence and is tested via a mathematical model showing that under a wide range of conditions, repeated migrations and selectively neutral species drift are sufficient to explain the replacement within a length of time compatible with the archeological data.
Mathematical modeling cannot pretend to represent the reality of such a complex process, however its premises should be reasonably in line with the empirical evidence, which in my view is not the case. To start, one can question whether human populations able to develop social networks, large size coalitions and, at times, organized collective violence can simply be modeled like selectively neutral alleles randomly varying at a genetic locus. Such modeling might be operational with species of frogs migrating into a piece of jungle, but I doubt it can be used with human societies of hunter-gatherers driven by group identity and territory control. Even if one accepts the authors' methodology, the proposed model still fails to match the empirical evidence in two important aspects and its output is therefore questionable. These aspects relate to the spatial distribution of Neanderthals and modern humans, and to the notion of time of overlap between groups in a given area, which is critical in the model.
In the model, small bands of individuals can migrate from one deme to another. Deme 1 is Europe and the Levant and the Deme 2 is Africa. These groups can die and be replaced by other bands in a random way, either by individuals originating from the same deme or by migrants arriving from the other deme during the most recent time step. Under the conditions of unidirectional and constant migration, for most values of carrying capacities, the groups of migrants replace all the indigenous groups in Deme 1 after a certain period of coexistence without needing to display any selective advantage. To my understanding, the model neglects the spatial distribution of the groups within Deme 1 and migrants can freely migrate from anywhere in Africa (Deme 2) to anywhere in the Levant/Europe (Deme 1) where the groups coexist as long as it is necessary for the complete establishment of the migrants everywhere. To say the least, this picture is far from that provided by the archeological reality. Spat ial distribution and border effects might have played an important role in the replacement process.
The authors refer to various dating available in the literature to assume "10,000 to 15,000 years during which both Moderns and Neanderthals coexisted in the Levant and Europe, including a few thousand years in western Europe and including regional overlap and even recurring replacement of one species by another in particular dwelling sites". Without entering the debates surrounding many published dates and the problems arising from the identification of biological groups based on archeological evidence, the fact is that the times of overlap mentioned in the paper refer to continental coexistence between the two biological groups rather than to any proven case of local coexistence.
In Southwestern Asia, the often-entertained notion of coexistence of Neanderthals and modern humans for a long period of time is rather misleading. Modern humans are indeed documented in Israel as early as ca 120 ka ago and Neanderthals might have been present nearby until ca 50 ka BP. However, until 50 ka BP, not a single modern human is documented in western Eurasia north of the Galilee region. Meanwhile, the Southernmost found Neanderthals ever come from the Mount Carmel just south of Haifa. Therefore, between 120 ka BP and 50 ka BP, the overlap zone between the domain of early modern humans of African origin and that of western Eurasian Neanderthals is only documented in the Levant on a band of territory of 20 or 30 miles. Furt her discoveries might increase this distance in the future. However, at the continental scale, rather than a large area open to potential migration and admixture, we might well be dealing with a rather narrow border zone that fluctuated north to south through time.
In the middle latitudes of Eurasia, there is no evidence for the occurrence of the two biological groups in the same region for any significant period of time. North of the Levant, the first undisputed directly dated modern human comes from Ust -Ishim in Western Siberia (ca 45 ka cal BP) and is followed by an individual from Romania ca 41.4 ka cal BP ka. Further west, published evidence from UK and Italy ca 42-43 ka BP is rather dubious (Zilhao et al 2015; White et al. 2012). The latest directly dated Neanderthals were found in The Grotte du Renne and Saint-Césaire (center and western France) ca 41.5 ka cal BP. During this time period (40-45 ka cal BP), human groups produced various lithic assemblages representing local technical traditions. These assemblages are rather well defined and often taken as proxies for some kind of ethno-cultural entities. Some were likely made by late Neanderthals, others, particularly in central and eastern Europe, by modern migrants. Most of these assemblages are documented for several millennia within delimited territories covering hundreds of thousands of km2. Although evidence for hybridization demonstrates that contact between Neanderthal and modern bands took place, hybridization events might have been restricted to the border zones and/or to periods of expansion of modern groups swamping local Neanderthals. This picture seems at odd with the large number of "changes of hand" (including changes from locals to immigrants) for each territory necessary over long periods of time to support the null hypothesis proposed by the authors (up to 1500 changes in Fig 3A). To date there is not a single site in Europe where multiple replacements involving the two biological groups are observed. When lithic assemblages assigned to the two groups are identified in one site, the archeological evidence consistently supports a systematic replacement of the Neanderthals by modern humans with no return of the former.
Marginal remarks: 1-The authors seem to take for granted that at t he time of the replacement, European Neanderthal populations were demographically weaker than modern immigrants. This is far from being demonstrated (e.g. Dogandžić & McPherron, 2013;Kuhlwilm M, et al., 2016;Richter, 2016).
2-"We find it important to address one major line of argument in this context, which suggests that Moderns had a cognitive and cultural advantage, potentially in the form of symbolic thought or language, over Neanderthals (9, 44-49, 72, 85). To date, genetic and cranio-morphological comparisons between the species have not produced any unequivocal evidence that would support this argument". What about Prüfer et al. (2014) or McCoy et al (2017)? 3-What, in the view of the authors, triggered the migration of modern humans out of "Africa" (including the south of the Levant and the Arabian Peninsula)? Is it only demography? If this is true, isn't it reasonable to assume that this large African population experienced a higher rate of cultural innovation and was submitted to stronger se lection pressure toward more cognitive complexity? May 2017 Detailed response to reviewers' comments We sincerely appreciate the close reading and thorough assessment of the manuscript by all three reviewers. We have found the reviewers' comments constructive, and believe our revised version addresses their concerns comprehensively. Below we respond to these comments in detail and refer to respective changes made in the manuscript, including the addition of a supplementary section in which we describe a new model of species' replacement and its results. This model includes a spatial component, addressing the major concerns raised by reviewer #3. Reviewers' comments: Reviewer #1 (Remarks to the Author): Manuscript "Random drift with a determined outcome...." by Kolodny and Feldman proposes a neutral population model to explain the extinction of Neanderthals. The main conclusion of the manuscript is that one can not rule out a scenario in which Moderns and Neanderthals competed neutrally on the basis of available observations. The authors further argue that the neutral model should be taken as a null model to assess the performance of alternative models. I am not an expert in paleontology but am familiar with the mathematical model. In general, I found the subject of this manuscript fascinating and thought provoking and for these reasons it could be a great contribution for Nature Communication.
Thank you! On the other hand, I did not find the analysis of the robustness of the results and their comparison with the data sufficiently clear. I have the impression that most of these problems are caused the presentation style rather than the content itself, and the whole manuscript would benefit immensely from a thorough reorganization of the manuscript, mostly in the model and results sections. We have made changes following the suggestions below in order to address this concern, and feel that the manuscript is greatly improved. See details below. Thank you! In particular, while the discussion of the model itself is quite clear to me, what I miss in the Model section are (schematic) estimates of realistic parameter ranges of the model. What degree of asymmetry in migration do the authors estimate, and why? What are reasonable number and size of bands? This is discussed in the Results section but it would be more appropriate to have this information here. Moreover, much later in the paper, it appears that the basic timestep of the paper is 25 years. Also this information should appear here and should be supported by some estimate.
This and the following comment have been addressed by reorganizing the manuscript: the results section which included the explanation of the hypothesis rejection test has been moved to the Model section, with some added clarifications.
In particular, we have highlighted which parameters' values need to be assumed in order to derive the model's numerical results and which need to be assumed later, in order to compare the results to the archaeological record. Because these parameter assumptions are highly debated, it was important for us to avoid "baking" these assumptions into the model itself, and the model is structured such that few assumptions need to be made in the implementation of the model and the derivation of numerical results from it, relegating the arguable assumptions to the post--simulation application of the hypothesis test. This allows the reader to consider our model in light of the specific parameter estimates from the literature that he/she finds most reliable. Your comment has helped us realize that this needed further clarification, which we have now carried out. The Model section now includes a presentation of the range of parameters estimated from the literature for band numbers and sizes; these are also called upon in a later result section to demonstrate the hypothesis test based on the provided figure.
The order of figures 2 and 3 has been switched, to reflect their relative importance to the argument we make and to improve readability. We have added references in the model section where we present our justification for focusing on unidirectional migration, and now note explicitly that bi--directional migration (both symmetric and a--symmetric) is provided for the sake of completeness, although only few researchers in the field perceive this possibility as realistic. We are agnostic regarding which scenario is more realistic, but it seems that the absence of migration of Neanderthals into Africa is one of the few points of near--agreement in the literature regarding the Neanderthal--Modern dynamics. The figure of 25 years refers to the length of a generation, which is the unit (not the parameter value) that we use; in the context in which it is now presented, we believe this would be clearer. Also the results suffer from a similar lack of organization. I do not understand the reason to study the average time that a band changes hand. Why not study directly the distribution of fixation times and its fluctuations? After all, it seems to me that the main quantities of interest here are the fixation time and the fixation probability. The explanation regarding the interpretation of figure 3 (now figure 2B) was unclear, and has been changed significantly, addressing this concern.
We study fixation times directly as the reviewer proposes ( Figure 1A), as well as species co--habitation time until replacement ( Figure 2B), which is similar but more relevant, in a sense: it measures the analogue of the fixation time, but starts the measurement not from the beginning of the simulation but from the beginning of the period in which the two species' presence is likely to be detected in Europe. The units in which this time period is presented in Figure 2B are the number of changes--of--hand--per--band--territory, because this unit is more independent of literature--derived assumptions (as discussed above) than a translation of this numerical result to years. It is also easier to interpret in realistic terms than using the term "model time step", thus supporting an easier interpretation of figure 2B which presents the results of the hypothesis test for each point in the space of parameter value combinations. It would be also very helpful to separate the numerical results from the model from and the comparison with archaeological evidences in different subsections.
We have left these results in a common section, but split their discussion into separate paragraphs and changed the wording to highlight clearly which result is a numerical result and which is a comparison to the literature. This section has also been shortened significantly, so it is much more readable and orderly. Reviewer #2 (Remarks to the Author): The major claim of this paper is that modern humans could have replaced neanderthals without having any selective advantage. Although this possibility has been mentioned before, as the authors note, this is the first time that it has been developed and explored through simulation, showing that it is plausible across a wide--range of reasonable values of key parameters. The results will be of wide interest and will certainly influence thinking in the field. The work is undoubtedly convincing and the authors themselves point that the obvious development is to give it a spatial dimension. Like them I think the result is unlikely to be qualitatively different. Mathematical modeling cannot pretend to represent the reality of such a complex process, however its premises should be reasonably in line with the empirical evidence, which in my view is not the case. We sincerely appreciate the reviewer's careful reading of our paper and the detailed comments. Addressing them has significantly improved our manuscript: they have highlighted a number of aspects that required explicit discussion or clarifications, which we have now added in the main text and in two additional supplementary sections. We agree with the reviewer on many points, and his comments helped us find and remedy problems in the text. To address the reviewer's primary concern -that the lack of a spatial component in the model may lead to significant deviation from what should be our realistic expectations regarding the species' replacement dynamics -we have implemented a new model that includes a spatial component. A full treatment of the spatial complexity of the dynamics, as is discussed in the main text and supplementary section C, requires complex modeling and a large number of strong assumptions, making the model less general and more sensitive to specific details about which there is no agreement among researchers of the field. Such treatment of the topic is an important future avenue of research, well beyond the scope of our current study. In line with the goal of the current study, we have constructed the simplest model that would qualitatively capture the effect of a spatial component in the inter--species dynamics of interest, such that it would be tractable, intuitive, and would require a minimal number of assumptions. This model's results differ significantly from those of our primary model in the number of inter--species "change of hands" of each territory, reducing it significantly. This addresses the reviewer's second major concern. These are discussed in detail below and in the added supplementary section, section D.
To start, one can question whether human populations able to develop social networks, large size coalitions and, at times, organized collective violence can simply be modeled like selectively neutral alleles randomly varying at a genetic locus. Such modeling might be operational with species of frogs migrating into a piece of jungle, but I doubt it can be used with human societies of hunter--gatherers driven by group identity and territory control.
• Competition (including inter--group violence and inter--group competition) between bands, based on band identity and control of a resource (territory or other) is completely in line with our model's assumptions; the Moran model we use doesn't assume that these interactions do not occur; it merely assumes that the outcome of each competition (i.e. which of the competing bands will die out) is independent of the species identity of that band. This is now made explicit in the text, lines 125--135. We have also added a discussion of the topic and related issues in supplementary section C, subsection 4 (see also below).
• The parameter value used for the mean rate of band replacement should be one that takes into account the mean effect of such inter--band competition. The rates that are cited in the Results section 1.2, based on the rate inferences in ( (Soltis, Boyd, & Richerson, 1995)), are derived from anthropological accounts of small traditional groups with individualized group identities that conduct warfare and have replaced one another in recent times. As discussed, these are far from representative of the Neanderthals and Moderns 40kya, but are the best available, as far as we know. • Ecological modeling with regard to human behavior and human dynamics has been done extensively and successfully in the past (e.g. (Banks et al., 2006(Banks et al., , 2008Belovsky, 1988;Gilpin, Feldman, & Aoki, 2016;Winterhalder & Smith, 2000)). We suggest it is a productive approach, as long as it is applied carefully. In particular, we suggest it is a useful approach as a null model of expected population dynamics.
• It has been proposed that species' differences in the ability/tendency to create large social networks and coalitions may have played a role in the Neanderthals' replacement (e.g. (Gat, 1999)), but there is no unambiguous evidence in support of this suggestion. Moreover, we have no means of knowing what the coalition and violence dynamics were, and it is not clear how such behavior would bias the process (whether they would shorten or elongate it, for example). Although such dynamics may have transpired, and the exploration of such possible dynamics is interesting, it is a scenario that is based on more assumptions than our model, and we believe it is reasonable to avoid these when constructing a parsimonious null model. We now discuss this topic in Supplementary section C.4. • We think that most people's intuition would be to assume that coalitions of this sort would tend to be of bands of one species against bands, or coalitions of bands, of the other species. Historical accounts of modern ethnic groups that expand to new regions generally do not support this intuition: when European colonialists spread through the Americas, for example, they fought amongst them repeatedly, created coalitions with native groups against other native groups, against other Europeans, and against opposing mixed coalitions (see, (Matthew & Oudijk, 2014;Restall, 2004); obviously, these are within the same species, but demonstrate coalitions that do not necessarily follow lines of ethnic relatedness). Similar dynamics were common in the colonial era in Africa (e.g. (Vandervort, 1998)). This suggests that even if coalitions are considered, there is no a--priori reason to believe that they were uniquely within one species and against the other; thus, there are no grounds to assume, in a null model, that coalitions would produce a skew in the expected dynamics towards shorter or longer replacement times compared to those predicted by our model. Even if one accepts the authors' methodology, the proposed model still fails to match the empirical evidence in two important aspects and its output is therefore questionable. These aspects relate to the spatial distribution of Neanderthals and modern humans, and to the notion of time of overlap between groups in a given area, which is critical in the model. We agree that these are important factors, but suggest that it is valuable to explore a null model that does not include spatial distribution. Spatial structure is discussed in the manuscript and in supplementary section C, and is mentioned explicitly as an important avenue for future exploration. We do not think such explicit incorporation would yield qualitatively different results from those of our current model, and have now added a simple model of species replacement that includes a spatial component, largely confirming this prediction (see supplementary section D). Incorporation of spatial distribution is, as the reviewer implies, tied into time of overlap as well, and calls for a different type of modeling, such as that of diffusion waves or of a shifting front of interspecies interaction as in the model we now added. This is part of the reason that we address an overall period of overlap in the entirety of Europe or Eurasia in our current (main text) model. Using regional estimates of time of overlap is incompatible with a model that does not consider regional dynamics explicitly. This is addressed in more detail below, and we have now incorporated a more explicit discussion of this point in supplementary section A.2. In addition, to address this major concern, we have now added a second model of species replacement that includes a spatial component. It is highly simplified, as a full treatment of spatial dynamics is well beyond the scope of the current study, but it serves to demonstrate that our qualitative findings hold in a spatially explicit setting, to address some specific concerns raised in the comments below, and to highlight some of the main differences between a spatially explicit model and the non--spatial null model. This is discussed in detail below and in the new supplementary section D. In the model, small bands of individuals can migrate from one deme to another. Deme 1 is Europe and the Levant and the Deme 2 is Africa. These groups can die and be replaced by other bands in a random way, either by individuals originating from the same deme or by migrants arriving from the other deme during the most recent time step. Under the conditions of unidirectional and constant migration, for most values of carrying capacities, the groups of migrants replace all the indigenous groups in Deme 1 after a certain period of coexistence without needing to display any selective advantage. To my understanding, the model neglects the spatial distribution of the groups within Deme 1 and migrants can freely migrate from anywhere in Africa (Deme 2) to anywhere in the Levant/Europe (Deme 1) where the groups coexist as long as it is necessary for the complete establishment of the migrants everywhere. This is correct. Notably, either species can go extinct in deme 1; in fact, Moderns typically establish in Europe but then go extinct multiple times in each simulation, before the establishment event of the lineage that eventually increases in frequency and reaches fixation. To say the least, this picture is far from that provided by the archeological reality. Agreed. This is an obvious simplification: any population model that assumes within-deme panmixia is necessarily a simplification of reality. This simplification is common to the vast majority of evolutionary, genetic, and population dynamics' models. Spatial distribution and border effects might have played an important role in the replacement process. We agree, but it is important to explore the dynamics of a simple null model that does not consider these realistic complications. The model that we have now added to address these concerns demonstrates some prominent differences and similarities between spatial and non--spatial models of the replacement dynamics (see below). The authors refer to various dating available in the literature to assume "10,000 to 15,000 years during which both Moderns and Neanderthals coexisted in the Levant and Europe, including a few thousand years in western Europe and including regional overlap and even recurring replacement of one species by another in particular dwelling sites". Without entering the debates surrounding many published dates and the problems arising from the identification of biological groups based on archeological evidence, the fact is that the times of overlap mentioned in the paper refer to continental coexistence between the two biological groups rather than to any proven case of local coexistence. Agreed; this was our intent, given the model's assumption of within--deme panmixia. This is now noted in supplementary section A.2. In Southwestern Asia, the often--entertained notion of coexistence of Neanderthals and modern humans for a long period of time is rather misleading. Modern humans are indeed documented in Israel as early as ca 120 ka ago and Neanderthals might have been present nearby until ca 50 ka BP. However, until 50 ka BP, not a single modern human is documented in western Eurasia north of the Galilee region. Meanwhile, the Southernmost found Neanderthals ever come from the Mount Carmel just south of Haifa. Therefore, between 120 ka BP and 50 ka BP, the overlap zone between the domain of early modern humans of African origin and that of western Eurasian Neanderthals is only documented in the Levant on a band of territory of 20 or 30 miles. Further discoveries might increase this distance in the future. However, at the continental scale, rather than a large area open to potential migration and admixture, we might well be dealing with a rather narrow border zone that fluctuated north to south through time. True; although given the sparseness of the record in this regions from these periods, it would not be surprising if this border zone will increase significantly in the future. To some, the finding of Modern introgression in Altai Neanderthals is suggestive of a