Tree species traits affect which natural enemies drive the Janzen-Connell effect in a temperate forest


A prominent tree species coexistence mechanism suggests host-specific natural enemies inhibit seedling recruitment at high conspecific density (negative conspecific density dependence). Natural-enemy-mediated conspecific density dependence affects numerous tree populations, but its strength varies substantially among species. Understanding how conspecific density dependence varies with species’ traits and influences the dynamics of whole communities remains a challenge. Using a three-year manipulative community-scale experiment in a temperate forest, we show that plant-associated fungi, and to a lesser extent insect herbivores, reduce seedling recruitment and survival at high adult conspecific density. Plant-associated fungi are primarily responsible for reducing seedling recruitment near conspecific adults in ectomycorrhizal and shade-tolerant species. Insects, in contrast, primarily inhibit seedling recruitment of shade-intolerant species near conspecific adults. Our results suggest that natural enemies drive conspecific density dependence in this temperate forest and that which natural enemies are responsible depends on the mycorrhizal association and shade tolerance of tree species.


Natural enemies (e.g. fungal pathogens, insects, and mammalian herbivores) have long been postulated to play crucial roles in determining the diversity of plant communities1,2,3,4. The Janzen-Connell hypothesis3,4 provides one mechanism, suggesting that host-specific natural enemies are concentrated where their host plant species occur at high densities (e.g. close to conspecific maternal adults), suppressing recruitment and survival of their host plants (negative conspecific density dependence)3,4,5,6. Such self-inhibition by conspecifics puts rare species at an advantage and facilitates plant species coexistence and diversity. Several empirical studies have reported that seedling recruitment and survival decrease with the density of conspecific neighbors, but are relatively unaffected by heterospecifics7,8,9. There is also increasing experimental evidence that insect herbivores and, pathogenic fungi in particular, are responsible for mediating the conspecific density dependence observed in several plant populations5,10,11,12,13,14,15. Despite growing consensus that natural enemies regulate plant population dynamics, however, the extent to which this process contributes to diversity at the community scale remains poorly known (but see refs. 16,17).

The strength of conspecific density dependence varies considerably among species16,18,19, which is one factor complicating assessments of its contribution to plant community diversity. This variation may be correlated with plant species’ traits20,21,22, which potentially provides a basis for extrapolation to the community-scale. There is some evidence that slow-growing, shade-tolerant tree species are less affected by conspecific neighbors than shade-intolerant species19,20,21,23, potentially because shade-tolerance is associated with greater investment in defense or storage, and hence lower susceptibility to natural enemies24,25. Additionally, types of mycorrhizal association (e.g. arbuscular mycorrhizal, AM or ectomycorrhizal, EM) differ in their effectiveness in protecting plant roots from antagonists26,27, which could translate into variation in susceptibility of hosts to natural enemies and hence density dependence. For example, a continental-scale study reported that EM seedlings respond positively to the density of conspecific trees, while AM seedlings respond negatively28. However, the relative importance of different traits in modifying the strength of natural-enemy-mediated density dependence in intact communities remains largely untested.

The strength of conspecific density dependence may also vary among ecosystems. The Janzen-Connell hypothesis originally postulated that natural enemy-induced conspecific density dependence should be more negative in tropical than in temperate systems1,3,4. Nevertheless, the relationship between latitude and the strength of conspecific density dependence remains disputed29,30,31,32,33. Inferences based on non-manipulative surveys are sensitive to analytical methods, with both marked weakening of conspecific density dependence with latitude31,32 and no relationship reported using the same datasets33. A meta-analysis of manipulative experiments of individual plant populations reported no consistent relationship between conspecific density dependence and latitude30. The strength of conspecific density dependence varies greatly among experiments and species so community-wide experiments have proved useful tools in determining the importance of density dependence for diversity in tropical forests16,17. However, community-scale experimental manipulations of natural enemies, equivalent to those performed in tropical forests (e.g. refs. 16,17), have not been attempted in less species-rich, temperate forests.

We report on a field experiment in an old-growth temperate forest in northeast China to test for conspecific density dependence at the community scale, to identify the groups of natural enemies that may be responsible and to examine whether variation in the strength of density dependence among plant species is associated with two important plant traits: shade tolerance and type of mycorrhizal association. We test two hypotheses: (1) Plant-associated fungi and insect herbivores are critical drivers of conspecific density dependence and hence seedling diversity and composition in this temperate forest, and (2) Species varying in mycorrhizal association and shade-tolerance differ in their sensitivity to enemy-mediated density dependence. Specifically, based on the literature we predict that AM trees are more sensitive to fungal-mediated conspecific density-dependence than EM trees28, and that conspecific density dependence mediated by both plant-associated fungi and insects are greater for shade-intolerant than shade-tolerant species19,20,23. We manipulated the access of three groups of natural enemies, large herbivores (using fences), insect herbivores (using insecticide) and plant-associated fungi (using fungicide), to seedlings in 180 1 m2 quadrats (Supplementary Fig. 1). We followed seedling recruitment and survival in these quadrats from 2015 to 2017. We focus on the earliest life stages of trees because they are most strongly affected by natural enemies16,34,35. We quantify conspecific and heterospecific adult densities by summing the inverse-distance weighted basal area of all trees >5 cm diameter at breast height (DBH) within a 20 m radius of each quadrat. We use the 20 m radius because intraspecific interactions are often undetectable beyond this distance36,37. Our results indicate that plant-associated fungi have a negative effect on seedling recruitment and survival at high conspecific adult density. Insect herbivores also inhibit seedling recruitment near conspecific adults, but they do not affect seedling survival. Additionally, which groups of natural enemies impact seedling recruitment near conspecific adults depends on two important tree species traits (i.e. mycorrhizal association and shade-tolerance). Plant-associated fungi have a stronger negative effect on seedling recruitment for EM-associated and shade-tolerant species than AM-associated and shade-intolerant species. In contrast, seedling recruitment is more strongly inhibited by insect herbivores near conspecific adults for shade-intolerant species than shade-tolerant species.


Community-wide conspecific density dependence

Across the 180 quadrats, 3929 tree seedlings recruited from 16 species (Supplementary Table 1) in the censuses from 2015 to 2017. The overall mortality rate through the growing seasons was 29.4%.

Seedling survival decreased with conspecific adult densities, although not significantly (binomial generalized linear mixed-effects model [GLMM]: z = −1.53, P = 0.13). Protection from fungi increased both recruitment and survival of seedlings when conspecific adult densities were high (Fungicide × conspecific adult density interactions, recruitment: Poisson GLMM, z = 4.87, P< 0.001; survival: binomial GLMM, z = 3.18, P= 0.001; Fig. 1). Suppression of insects also increased seedling recruitment at high conspecific adult densities (Poisson GLMM: Insecticide × conspecific adult density interaction, z = 2.19, P= 0.029), but did not affect seedling survival appreciably. The effect of large vertebrates was not estimated to be large in any of the models (Recruitment: Poisson GLMM, z = −0.23, P = 818; Survival: binomial GLMM, z = −0.22, P = 0.823; Fig. 1), so was not considered further in the analyses.

Fig. 1: Effects of natural enemies and neighborhoods on seedling recruitment and survival.

a, c Parameter estimates from Generalized Linear Mixed Effects Models (GLMM) fitted to seedling recruitment (n = 1019) and seedling survival (n = 3929) data from all species in the seedling assemblage. The models estimated the effects of pesticide application, conspecific adult density (A.con) and the interactions between them. Heterospecific adult density (A.het) and exclosure treatment were included as covariates in both models. The survival model also included terms for densities of conspecific and heterospecific seedlings (S.con and S.het) and the effect of seedling height (Height). Solid points indicate parameter estimates that are significantly different (P < 0.05) from zero (dashed lines) and bars indicate 95% confidence intervals. b, d The relationships between neighboring conspecific adult densities and seedling recruitment and seedling survival under each pesticide treatment. Lines and shaded polygons indicate the GLMM predictions and their 95% confidence intervals. Dots and bars represent the mean and SE of the observed values, which were calculated by adding model residuals to the predicted values. We averaged the observed values within bins to facilitate visualization (with bin width increasing as the conspecific crowding index decreases). Source data are provided as a Source Data file.

Mycorrhizal association and conspecific density dependence

A total of 1926 individuals (49% of all seedlings) recorded were from five species of EM trees, most of which (93% of EM seedlings) were shade tolerant. The remaining seedlings belonged to 11 AM species, which included both shade-tolerant (seven species, 757 individuals) and shade-intolerant (four species, 1246 individuals) species. The influence of natural enemies on seedling recruitment and survival near adult conspecifics differed between EM and AM tree species. The relationship between conspecific adult density and recruitment of EM species was significantly more positive in the fungicide treatment than in either the control or insecticide treatments (Poisson GLMM: Fungicide × conspecific adult density interaction, z = 2.89, P = 0.004; Fig. 2). Survival of EM species decreased as adult conspecific density increased (binomial GLMM: z = −2.63, P = 0.008), but fungicide did not appreciably ameliorate that effect (binomial GLMM: z = 1.23, P = 0.220; Fig. 3). In contrast to EM species, the relationship between conspecific adult density and recruitment of AM species was significantly less affected by fungicide treatment (Poisson GLMM: significant AM × Fungicide × conspecific adult density interaction, z = −2.22, P= 0.027). Additionally, survival of AM species was significantly less dependent on conspecific adult density (binomial GLMM: AM × conspecific adult density interaction, z = 2.54, P = 0.011). The effects of insecticide treatment on recruitment and survival did not differ between mycorrhizal association types (Fig. 3).

Fig. 2: Effects of natural enemies, conspecific adult density, and plant species’ mycorrhizal association type or shade tolerance status on seedling recruitment.

a, d Parameter estimates from Generalized Linear Mixed Effects Models (GLMM) fitted to estimate the effects of pesticide application, conspecific adult density (A.con), mycorrhizal association, or shade tolerance (shade-intolerance coefficient indicated as and the two- and three-way interactions. Solid points indicate parameter estimates that are significantly different (P < 0.05) from zero (dashed lines) and bars indicate 95% confidence intervals. b, c The relationships between conspecific neighboring adult densities and seedling recruitment under each treatment for EM and AM species, respectively. e, f The relationships between conspecific neighboring adult densities and seedling recruitment under each treatment for shade-tolerant and shade-intolerant species. Lines and shaded polygons indicate the GLMM predictions and their 95% confidence intervals. Dots and bars represent the mean and SE of the observed values, which were calculated by adding model residuals to the predicted values. We averaged the observed values within bins to facilitate visualization (with bin width increasing as the conspecific crowding index decreases). Source data are provided as a Source Data file.

Fig. 3: Effects of natural enemies, conspecific adult density, and plant species’ mycorrhizal association type or shade tolerance status on seedling survival.

a, d Parameter estimates from Generalized Linear Mixed Effects Models (GLMM) fitted to estimate the effects on survival of pesticide application, conspecific adult density (A.con), mycorrhizal association, or shade tolerance (shade-intolerance coefficient indicated as and the two- and three-way interactions. Solid points indicate parameter estimates that are significantly different (P < 0.05) from zero (dashed lines) and bars indicate 95% confidence intervals. b, c The relationships between neighboring conspecific adult densities and seedling survival under each treatment for EM and AM respectively. e, f The relationships between neighboring conspecific adult densities and seedling survival under each treatment for shade-tolerant and shade-intolerant species. Lines and shaded polygons indicate the GLMM predictions and their 95% confidence intervals. Dots and bars represent the mean and SE of the observed values, which were calculated by adding model residuals to the predicted values. We averaged the observed values within bins to facilitate visualization (with bin width increasing as the conspecific crowding index decreases). Source data are provided as a Source Data file.

Shade tolerance and conspecific density dependence

Ten shade-tolerant species (2540 individuals) accounted for 65% of all seedlings. Recruitment of shade tolerant species increased with adult conspecific density, and this positive relationship strengthened significantly when fungicide was applied (Poisson GLMM: z = 2.52, P = 0.012; Fig. 2). In contrast, recruitment of shade intolerant species was slightly negatively associated with adult conspecific density and was unaffected by fungicide addition (Fig. 2). These differences between shade tolerant and intolerant species in terms of the effects of conspecific adult density and fungicide addition on recruitment were statistically significant (Poisson GLMMs: Shade-intolerance × conspecific adult density interaction, z = −4.31, P < 0.001; Shade-intolerance × Fungicide interaction, z = −2.84, P = 0.005). Nevertheless, the interactive effect of adult conspecific density and fungicide on seedling recruitment was similar for shade tolerant and shade intolerant species (i.e. the three-way interaction was non-significant). Insecticide, on the other hand, increased recruitment of shade-intolerant seedlings near conspecific adults significantly more than recruitment of shade tolerant species (Poisson GLMM: Shade-intolerance × Insecticide × conspecific adult density interaction, z = 2.52, P= 0.012). Shade tolerance did not influence the response of seedling survival to either conspecific adult density, biocides or their interactions (Fig. 3).

Seedling diversity and composition

Despite the effects of pesticide application on recruitment and survival, seedling diversity (Shannon’s diversity index) was unaffected by suppressing natural enemies (linear mixed-effects models of Shannon diversity index: Fungicide: t336 = 0.98, P = 0.330; Insecticide: t336 = 0.16, P = 0.875). Spraying fungicide, however, did influence species composition, significantly decreasing the dissimilarity between seedlings and surrounding adult trees (linear mixed-effects model: t336 = −3.78, P < 0.001; Fig. 4). Thus, it appears that the fungi lead to recruitment of species that are dissimilar to nearby adult trees, even if diversity at the plot scale is not affected. Insects do not appear to affect the species composition of recruits in this forest.

Fig. 4: Effects of natural enemies on seedling diversity and composition.

Suppression of plant-associated fungi and insect herbivores on a Shannon diversity of seedlings (represented by the effective number of species), and b the abundance-weighted Bray-Curtis dissimilarity between the species composition of seedlings and surrounding adults. We pooled the data from two censuses for each treatment (n = 120). Each point shows an observed value for seedling diversity or dissimilarity in each quadrat. The boxes show the first and third quartiles, the lines within the boxes indicate the medium, the whiskers outside the boxes are the smallest and largest values within 1.5× interquartile range. Asterisks at the top of boxes indicate significant pairwise difference (P = 0.05 level) among treatments, which were estimated with linear mixed-effects models. Source data are provided as a Source Data file.


The deleterious effects of natural enemies on seedling recruitment and survival, assessed at the community scale, increase with the local density of conspecific adults. While this pattern is consistent with the Janzen-Connell hypothesis, the relationship between adult density and seedling survival is weak in plots unprotected from natural enemies. Suppressing natural enemies does not reduce tree species diversity, which is consistent with the lack of dependence of seedling survival on local conspecific seedling density, regardless of biocide treatment. Instead, fungicide application lead the species composition of seedlings to more closely resemble that of the surrounding adults, which could result from fungi increasing mortality of seedlings near conspecific adults. Thus, our results suggest that plant-associated fungi suppress the recruitment of tree species dominating the nearby canopy, which will increase turnover of tree species and could increase diversity.

While the evidence that natural enemies reduce seedling performance at high conspecific density in our system is in line with previous pesticide application experiments in field conditions in tropical forests15,16,17, the strength of conspecific density dependence observed in this temperate forest appears much weaker. Additionally, in contrast to previous community-wide manipulations of natural enemy communities (e.g. refs. 16,17), we found no evidence that attack by either plant-associated fungi or insect herbivores is related to the density of conspecific seeds and seedlings, or that it increase seedling diversity. Instead, our results suggest that density of adult conspecifics is a better predictor of natural enemy impacts on recruitment and survival of seedlings. These natural enemy impacts appear to be important, even though the net effect of conspecific density is only weakly negative in plots when natural enemies are unmanipulated. Both recruitment and survival go from negatively to positively related to conspecific adult density when fungi and, to a lesser extent, insects are excluded. As a result, seedling communities are less similar to adults than when fungi are suppressed. Several field and shade-house experiments involving small numbers of tree species have indicated that performance is inhibited near conspecific adults as a result of plant-associated fungi5,11,15,38,39 and insect herbivores15,40. Our results extend the experimental evidence that seedling performance is suppressed by adult conspecifics to a community scale.

While the presence of conspecific density dependence is well-established in plant communities, recent studies have observed that the strength of density dependence varies widely across species with contrasting traits20,21. Our analyses identified that recruitment of EM-associated and shade-tolerant species is more strongly inhibited by adult conspecifics, although the correlation between mycorrhizal association and shade-tolerance (most EM seedlings are shade tolerant), hinders any definite distinction between the effects of the two traits. Suppression of plant-associated fungi increase seedling recruitment of EM species in areas of high conspecific adult density. While there is little difference between EM and AM species in the effects of adults on seedling recruitment, fungi do not appear to be responsible for conspecific density dependence in AM species. Interestingly, previous studies have suggested that EM seedlings should preferentially recruit near conspecific adults28,41,42, in contrast to our results. One possible explanation for this disagreement is that we concentrated on the earliest stages in the trees’ lives, before mycorrhizal associations are generally established. At this stage, the mycorrhizal colonization rate is much lower (Supplementary Fig. 2) compared to that of mature trees43. Adult conspecifics provide a source of EM fungi that facilitate nutrient uptake and may protect seedlings from antagonistic fungi27,28. The benefits of adult proximity for older saplings able to establish mycorrhizal associations may outweigh the costs from heightened exposure to fungal pathogens, especially under lowlight, understorey conditions. Pre-mycorrhizal seedlings will only incur the costs of proximity to adult conspecifics, however, perhaps generating the relationships we observed.

Species’ shade-tolerance is also related to the strength of conspecific density dependence in our analyses, as well as the identity of the natural enemies associated with density dependence. In control plots, the increase in recruitment in areas of high conspecific adult density probably just reflects greater seed input, given that dispersal limitation is strong in our forest44 and leads to an overall positive relationship between conspecific adult density and seed numbers (Supplementary Fig. 3). Importantly, seed input is much higher near conspecific adults for shade-tolerant than shade-intolerant species (Poisson GLMM: Shade-intolerance × conspecific adult density, z = −2.25, P = 0.024. Supplementary Fig. 3). Fungicide addition strengthens the relationship, suggesting that fungi reduce seedling recruitment of shade-tolerant species near adult conspecifics. In contrast, seedling recruitment of shade intolerant species is not concentrated near adult conspecifics, perhaps reflecting greater dispersal ability and a need for these species to disperse into high light areas as well as the fact that the canopy is mostly dominated by shade-tolerant species (Supplementary Fig. 4). Interestingly, insecticide addition increases recruitment of shade intolerant species near conspecific adults, whereas fungicides have no effect. These results suggest that shade tolerance predicts the type of natural enemies that plant species are susceptible to. Shade-tolerant species tend to be most often attacked by aggressive, necrotrophic fungal pathogens, whereas shade intolerant species are often infected by less aggressive biotrophic pathogens45. Necrotrophs may kill large numbers of susceptible plants and their distributions can be highly spatially structured46, making them prime candidates for mediating plant-soil feedbacks close to adult conspecifics. Shade-intolerant species generally have lower wood density and smaller seed size21,47,48, traits associated with lower investment in defense against insect attack than shade-tolerant species24,25. Previous studies have highlighted that shade-tolerance is associated with the extent that species experience conspecific density dependence19,20,21,49, but we believe that this is the first time that shade-tolerance has been linked to the class of natural enemies mediating conspecific density dependence. We caution that these relationships are phenomenological and because suites of traits tend to be inter-correlated, life-history traits other than shade-tolerance might be truly driving them. However, because different natural enemies vary in their capacity to exert density dependent control of plant populations, for example due to variation in their mobility, host-specificity and virulence, the identities of natural enemies may explain variation of density dependence among tree species50,51. Investigating how the drivers of conspecific density dependence vary with life-history traits could provide a fruitful avenue for future research.

Although much of the evidence that natural enemies cause conspecific density dependence in tree seedlings has come from tropical forests5,15,16,30, there are a growing number of studies in temperate systems that also provide evidence for such effects11,23,52,53. While the strength of conspecific density dependence observed in this temperate forest is weaker than that observed in tropical studies (e.g. refs. 8,16,17,19), our results suggest that the causes of density dependence may be consistent across latitudes. Several studies in tropical forests have suggested that while insects influence seedling recruitment and might mediate density dependence in some species15,40, plant-associated fungi play a bigger role in mediating conspecific density dependence at the community scale16,17. Similarly, although insects contribute to conspecific density dependence in our experiment, suppressing fungi has a greater effect on seedling recruitment and survival than insecticide application at the community scale. While fences are effective at excluding large herbivores such as wild boars and roe deer (Supplementary Fig. 5) from our plots, our results do not support a role of large mammalian herbivores in density dependent control of plant populations, in agreement with previous studies54,55. The low host-specificity of many large herbivores56 may explain why they do not appear to contribute to density dependence in our experiment. Alternatively, the low number of replicate exclosure plots (three pairs of fenced and unfenced plots) may limit our ability to detect impacts of large herbivores on plant communities. Further, better replicated experiments that exclude large herbivores may be necessary to determine their role in shaping plant communities in these ecosystems. Although further work is necessary to determine the identity and host-specificity of insect herbivores and plant-associated fungi, perhaps using molecular techniques (e.g. refs. 57,58), on the basis of the evidence presented here, insects and plant-associated fungi appear to play a greater role in mediating conspecific density dependence than large herbivores.

Overall, our field experiment provides evidence of a community-wide Janzen-Connell effect in this temperate forest. Comparison of these results to those from similar experiments in tropical forests16,17 suggests that causes of conspecific density dependence (e.g. plant-associated fungi and insect herbivores) may be consistent across latitudes. Additionally, our results indicate that the strength of density dependence largely depends on the association between the types of natural enemies and the mycorrhizal association type or shade tolerance of tree species. Although natural-enemy-mediated density dependence does not translate into increased seedling diversity, our results suggest that plant-associated fungi play a critical role in shaping species composition and turnover in our temperate forest. Notably, our study highlights the need to consider multiple types of natural enemies and traits of plant species to improve our understanding of the mechanisms of conspecific density dependence and species coexistence.


Study site

This study was conducted within the Changbai Mountain National Nature Reserve in northeast China (42°23’ N, 128°05’ E, ~ 800 m a.s.l). The climate is characterized by an annual mean temperature of 2.8 °C (−13.7–19.6 °C) and average annual precipitation of ~700 mm59, mostly as rain from June to September and snow otherwise, with the ground covered by snow from November to April. The vegetation is dominated by broad-leaf and mixed forest over slightly acidic (mean pH = 5.45) dark brown forest soil. Dominant tree species include Tilia amurensis, Fraxinus mandschurica, Quercus mongolica, and Pinus koraiensis.

Experimental design

We established three 55 m × 50 m blocks, separated by at least 200 m, within an old growth temperate forest in September 2014 (experimental layout in Supplementary Fig. 1). The central area (35 m × 30 m) of each block was split into two equally sized plots, with one plot randomly allocated to a large herbivore exclosure treatment and enclosed with a 1-m high fence constructed of 4–5 steel wires vertically spaced by 0.2 m. Within each plot, we demarcated 50 1 × 1 m2 seedling quadrats on a grid, with quadrats separated by ~2 m and avoiding large roots. We randomly selected thirty of these quadrats for use in the experiment described here. These 30 quadrats were divided equally and randomly among three pesticide treatments: fungicide, insecticide, and control.

To confirm that our fences were effective at excluding large herbivores, we installed infrared cameras (Bestguarder SG-990V, Shenzhen, China) at three locations within and outside each fence between September 2015 and September 2017. In addition, in each census we examined all quadrats for evidence of large mammals (e.g. footprints or feces).

We initiated pesticide treatments in July 2015 and continued until October 2017. Plants were protected from true fungi and oomycetes through a combination of two fungicides, Amistar (Syngenta Ltd, active ingredient Azoxystrobin) and Ridomil Gold MZ 68 (Syngenta Ltd, active ingredients metalaxyl and mancozeb). Both fungicides have low toxicity to non-target organisms and have been reported to have little effect on mycorrhizal colonization60,61. Insects were deterred by applying a broad spectrum, systemic insecticide, Engeo (Syngenta Ltd, active ingredient Thiamethoxam). We applied pesticides biweekly except during the winter. Pesticide concentrations followed the manufacturer’s guidelines (Fungicide: 0.005 g of Amistar + 0.25 g of Ridomil Gold; Insecticide: 0.0025 ml of Engeo, each dissolved in 50 ml of water). Control quadrats were sprayed with an equal volume of water.

We conducted censuses of woody plants in June 2015, 2016, and 2017. All woody plants <1 cm DBH (Diameter at Breast Height) were tagged, mapped and identified to species in the 180 quadrats. In September of each year, we checked the status (survival/dead) of each existing seedling. We focused on survival status in the first few months after recruitment (seed-to-seedling transition), because seedlings are most vulnerable to natural enemies during this stage16,62. We calculated recruitment as the number of new seedlings >1 cm tall in each quadrat in every census. Including tiny seedlings would allow us to indirectly measure the effects of natural enemies on seedling survival during the seed to seedling transition. We did not evaluate the effect of seed abundance on recruitment because the majority of tree species recruit from seed bank rather than seed rain, and the time germinating from seeds varies from months to years across species. In June 2017, we identified all trees (DBH > 5 cm) within each block and recorded their species identity, DBH and distance to each quadrat within 20 m.

Statistical analysis

Analyses of seedling recruitment and survival only considered tree seedlings recruited after pesticide applications were initiated. We used the crowding index (i.e. A.con and A.het) to represent the conspecific and heterospecific adult densities, which were calculated by summing the inverse-distance weighted basal areas of all conspecific and heterospecific adults within a 20-m radius of each quadrat8,63. The range of the conspecific crowding index within the 20 m radius of each quadrat was from 0 to 0.42. We also calculated the conspecific (S.con) and heterospecific seedling densities (S.het) as the number of seedlings within each quadrat (S.con: 0–27 seedlings m−2; S.het: 0–44 seedlings m−2).

We used generalized linear mixed-effects models (GLMM) to evaluate the effects of enemy types and neighboring plants on seedling recruitment and survival. All continuous explanatory variables were log-transformed and then standardized (i.e. by subtracting the mean and dividing by the standard deviation) to facilitate the direct comparison of the relative importance of each variable64. We included pesticide treatment, exclosure treatment and conspecific adult density (i.e. A.con) and the two- and three-way interactions among them as fixed factors in the full model for recruitment. Heterospecific adult density (A.het) was included as a covariate in the recruitment model. For survival, we added covariates describing seedling neighborhoods (conspecific seedling density-S.con and heterospecific seedling density-S.het) in addition to all the covariates included in the recruitment model. We did not include seedling densities in recruitment analyses because the number of recruits were confounded with the seedling densities. Seedling height was also included as a covariate to account for the potential positive relationship between survival and seedling size65. To evaluate the contribution of two- or three-way interactions in explaining seedling recruitment and survival, the full models considering all interactions were compared with all possible reduced models from which one or more interactions were removed using likelihood ratio tests (χ2 with degree of freedom of the differences in the number of parameters66). The models assumed Poisson and binomial error distributions for recruitment and survival respectively, and we confirmed that model residuals were not overdispersed. To account for spatial and temporal autocorrelation between closely located plots and censuses in the data, we included block and census as categorical fixed effects14. We also considered quadrat and species identity as random intercepts and allowed the effect of census to vary among quadrats as a random effect. We did not include random slopes for different species because preliminary analyses indicated that allowing slopes to vary among species led to less parsimonious models (higher AIC) than intercept-only models, and the coefficients did not change qualitatively.

To explore correlates of interspecific variation in covariate effects, we categorized each tree species by shade tolerance (shade-tolerant or shade-intolerant) and mycorrhizal association (arbuscular mycorrhizal-AM or ectomycorrhizal-EM; Supplementary Table 1). We considered three-way interactions between mycorrhizal association or shade tolerance, pesticide treatment, and conspecific adult density as fixed effects. We did not consider S.con and S.het in these models as preliminary analyses suggested their effects on seedling survival were negligible.

Two species (T. amurensis and F. mandschurica) accounted for 74% of all seedlings. To evaluate the influence of these two species on the community-wide and correlates of interspecific variation results, we conducted a sensitivity analysis. We randomly removed 25, 50 and 75% of individuals of T. amurensis and F. mandschurica from the community data sets (while including all individuals of the other species) and refitted the models, repeating this procedure 999 times. We compared parameter estimates from these simulations to the models including all seedlings. Despite their numerical dominance of the seedling assemblage, our results were insensitive to removing random fractions of individuals of these two species (Supplementary Figs. 611).

Finally, we examined whether suppression of plant-associated fungi and insect herbivores altered species diversity and composition in the seedling assemblage. We estimated the diversity of seedlings in each plot and census using Shannon’s diversity index (H)67. Turnover in species composition between conspecific adult neighbors within 20 m of each quadrat and the seedlings within the quadrat was quantified using the Bray-Curtis index68. This metric quantifies the effects of biocide treatments on the compositional development of the seedling community along the trajectory towards the surrounding adults. We used linear mixed-effects models with pesticide treatment as a fixed effect. Census was also included as a covariate to account for the temporal autocorrelation. We included random intercepts for plots.

All the data analyses were conducted using R v 3.4.269 using the package “lme4”70.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The datasets analyzed in this study are archived on Figshare ( The source data underlying Figs. 14, Supplementary Figs. 24 and Supplementary Figs. 611 are provided as a Source Data file.

Code availability

The code supporting the results are archived on Figshare (


  1. 1.

    Gillett, J. B. Pest pressure, an underestimated factor in evolution. Syst. Assoc. Publ. 4, 37–46 (1962).

  2. 2.

    Mordecai, E. A. Pathogen impacts on plant communities: unifying theory, concepts, and empirical work. Ecol. Monogr. 81, 429–441 (2011).

  3. 3.

    Janzen, D. Herbivores and the number of tree species in tropical forests. Am. Nat. 104, 501–528 (1970).

  4. 4.

    Connell, J. H. On the role of natural enemies in preventing competitive exclusion in some marine animals and in rain forest trees. Dyn. Popul. 298, 298–312 (1971).

  5. 5.

    Mangan, S. A. et al. Negative plant-soil feedback predicts tree-species relative abundance in a tropical forest. Nature 466, 752–755 (2010).

  6. 6.

    Terborgh, J. Enemies maintain hyperdiverse tropical forests. Am. Nat. 179, 303–314 (2012).

  7. 7.

    Connell, J., Tracey, J. & Webb, L. Compensatory recruitment, growth, and mortality as factors maintaining rain-forest tree diversity. Ecol. Monogr. 54, 141–164 (1984).

  8. 8.

    Comita, L. S. & Hubbell, S. P. Local neighborhood and species’ shade tolerance influence survival in a diverse seedling bank. Ecology 90, 328–334 (2009).

  9. 9.

    Inman-Narahari, F. et al. Density-dependent seedling mortality varies with light availability and species abundance in wet and dry Hawaiian forests. J. Ecol. 104, 773–780 (2016).

  10. 10.

    Augspurger, C. & Kitajima, K. Experimental studies of seedling recruitment from contrasting seed distributions. Ecology 73, 1270–1284 (1992).

  11. 11.

    Packer, A. & Clay, K. Soil pathogens and spatial patterns of seedling mortality in a temperate tree. Nature 404, 278–281 (2000).

  12. 12.

    Hood, L. A., Swaine, M. D. & Mason, P. A. The influence of spatial patterns of damping-off disease and arbuscular mycorrhizal colonization on tree seedling establishment in Ghanaian tropical forest soil. J. Ecol. 92, 816–823 (2004).

  13. 13.

    Bagchi, R. et al. Testing the Janzen-Connell mechanism: pathogens cause overcompensating density dependence in a tropical tree. Ecol. Lett. 13, 1262–1269 (2010).

  14. 14.

    Liu, X. et al. Experimental evidence for a phylogenetic Janzen-Connell effect in a subtropical forest. Ecol. Lett. 15, 111–118 (2012).

  15. 15.

    Fricke, E. C., Tewksbury, J. J. & Rogers, H. S. Multiple natural enemies cause distance-dependent mortality at the seed-to-seedling transition. Ecol. Lett. 17, 593–598 (2014).

  16. 16.

    Bagchi, R. et al. Pathogens and insect herbivores drive rainforest plant diversity and composition. Nature 506, 85–88 (2014).

  17. 17.

    Krishnadas, M., Bagchi, R., Sridhara, S. & Comita, L. S. Weaker plant-enemy interactions decrease tree seedling diversity with edge-effects in a fragmented tropical forest. Nat. Commun. 9, 4523 (2018).

  18. 18.

    Comita, L. S., Muller-Landau, H. C., Aguilar, S. & Hubbell, S. P. Asymmetric density dependence shapes species abundances in a tropical tree community. Science 329, 330–332 (2010).

  19. 19.

    Zhu, Y. et al. Density-dependent survival varies with species life-history strategy in a tropical forest. Ecol. Lett. 21, 506–515 (2018).

  20. 20.

    Kobe, R. K. & Vriesendorp, C. F. Conspecific density dependence in seedlings varies with species shade tolerance in a wet tropical forest. Ecol. Lett. 14, 503–510 (2011).

  21. 21.

    Lebrija-Trejos, E., Reich, P. B., Hernandez, A. & Wright, S. J. Species with greater seed mass are more tolerant of conspecific neighbours: a key driver of early survival and future abundances in a tropical forest. Ecol. Lett. 19, 1071–1080 (2016).

  22. 22.

    Liu, Y. & He, F. Incorporating the disease triangle framework for testing the effect of soil-borne pathogens on tree species diversity. Funct. Ecol. 33, 1211–1222 (2019).

  23. 23.

    McCarthy-Neumann, S. & Ibanez, I. Plant-soil feedback links negative distance dependence and light gradient partitioning during seedling establishment. Ecology 94, 780–786 (2013).

  24. 24.

    Coley, P. D. & Barone, J. A. Herbivory and plant defenses in tropical forests. Annu. Rev. Ecol. Syst. 27, 305–335 (1996).

  25. 25.

    Fine, P. V. A. et al. The growth-defense trade-off and habitat specialization by plants in Amazonian forests. Ecology 87, S150–S162 (2006).

  26. 26.

    Marx, D. H. Ectomycorrhizae as biological deterrents to pathogenic root infections. Annu. Rev. Phytopathol. 10, 429–454 (1972).

  27. 27.

    Laliberte, E., Lambers, H., Burgess, T. I. & Joseph Wright, S. Phosphorus limitation, soil-borne pathogens and the coexistence of plant species in hyperdiverse forests and shrublands. New Phytol. 206, 507–521 (2015).

  28. 28.

    Bennett, J. A. et al. Plant-soil feedbacks and mycorrhizal type influence temperate forest population dynamics. Science 355, 181–184 (2017).

  29. 29.

    HilleRisLambers, J., Clark, J. S. & Beckage, B. Density-dependent mortality and the latitudinal gradient in species diversity. Nature 417, 732–735 (2002).

  30. 30.

    Comita, L. S. et al. Testing predictions of the Janzen-Connell hypothesis: a meta-analysis of experimental evidence for distance- and density-dependent seed and seedling survival. J. Ecol. 102, 845–856 (2014).

  31. 31.

    LaManna, J. A. et al. Plant diversity increases with the strength of negative density dependence at the global scale. Science 356, 1389–1392 (2017).

  32. 32.

    LaManna, J. A. et al. Response to Comment on ‘Plant diversity increases with the strength of negative density dependence at the global scale’. Science 360, aar5245 (2018).

  33. 33.

    Chisholm, R. A. & Fung, T. Comment on ‘Plant diversity increases with the strength of negative density dependence at the global scale’. Science 360, aar4685 (2018).

  34. 34.

    Bever, J. D., Mangan, S. A. & Alexander, H. M. Maintenance of plant species diversity by pathogens. Annu. Rev. Ecol. Evol. Syst. 46, 305–325 (2015).

  35. 35.

    Green, P. T., Harms, K. E. & Connell, J. H. Nonrandom, diversifying processes are disproportionately strong in the smallest size classes of a tropical forest. Proc. Natl Acad. Sci. USA 111, 18649–18654 (2014).

  36. 36.

    Wang, X. et al. Species associations in an old-growth temperate forest in north-eastern China. J. Ecol. 98, 674–686 (2010).

  37. 37.

    Murphy, S. J., Wiegand, T. & Comita, L. S. Distance-dependent seedling mortality and long-term spacing dynamics in a neotropical forest community. Ecol. Lett. 20, 1469–1478 (2017).

  38. 38.

    Liu, Y., Fang, S., Chesson, P. & He, F. The effect of soil-borne pathogens depends on the abundance of host tree species. Nat. Commun. 6, 10017 (2015).

  39. 39.

    Liang, M. et al. Adult trees cause density-dependent mortality in conspecific seedlings by regulating the frequency of pathogenic soil fungi. Ecol. Lett. 19, 1448–1456 (2016).

  40. 40.

    Swamy, V. & Terborgh, J. W. Distance-responsive natural enemies strongly influence seedling establishment patterns of multiple species in an Amazonian rain forest. J. Ecol. 98, 1096–1107 (2010).

  41. 41.

    McGuire, K. L. Common ectomycorrhizal networks may maintain monodominance in a tropical rain forest. Ecology 88, 567–574 (2007).

  42. 42.

    Dickie, I. A., Koele, N., Blum, J. D., Gleason, J. D. & McGlone, M. S. Mycorrhizas in changing ecosystems. Botany 92, 149–160 (2014).

  43. 43.

    Zhou, M. et al. Nitrogen deposition and decreased precipitation does not change total nitrogen uptake in a temperate forest. Sci. Total Environ. 651, 32–41 (2019).

  44. 44.

    Li, B., Hao, Z., Bin, Y., Zhang, J. & Wang, M. Seed rain dynamics reveals strong dispersal limitation, different reproductive strategies and responses to climate in a temperate forest in northeast China. J. Veg. Sci. 23, 271–279 (2012).

  45. 45.

    Garcia-Guzman, G. & Heil, M. Life histories of hosts and pathogens predict patterns in tropical fungal plant diseases. New Phytol. 201, 1106–1120 (2014).

  46. 46.

    Jarosz, A. & Davelos, A. Effects of disease in wild plant-populations and the evolution of pathogen aggressiveness. New Phytol. 129, 371–387 (1995).

  47. 47.

    Kitajima, K. Relative importance of photosynthetic traits and allocation patterns as correlates of seedling shade tolerance of 13 tropical trees. Oecologia 98, 419–428 (1994).

  48. 48.

    Poorter, L. et al. The importance of wood traits and hydraulic conductance for the performance and life history strategies of 42 rainforest tree species. New Phytol. 185, 481–492 (2010).

  49. 49.

    McCarthy-Neumann, S. & Kobe, R. K. Tolerance of soil pathogens co-varies with shade tolerance across species of tropical tree seedlings. Ecology 89, 1883–1892 (2008).

  50. 50.

    Adler, F. R. & Muller-Landau, H. C. When do localized natural enemies increase species richness? Ecol. Lett. 8, 438–447 (2005).

  51. 51.

    Beckman, N. G., Neuhauser, C. & Muller-Landau, H. C. The interacting effects of clumped seed dispersal and distance- and density-dependent mortality on seedling recruitment patterns. J. Ecol. 100, 862–873 (2012).

  52. 52.

    HilleRisLambers, J. & Clark, J. S. Effects of dispersal, shrubs, and density-dependent mortality on seed and seedling distributions in temperate forests. Can. J. For. Res. 33, 783–795 (2003).

  53. 53.

    Petermann, J. S., Fergus, A. J. F., Turnbull, L. A. & Schmid, B. Janzen-Connell effects are widespread and strong enough to maintain diversity in grasslands. Ecology 89, 2399–2406 (2008).

  54. 54.

    Paine, C. E. T. & Beck, H. Seed predation by neotropical rain forest mammals increases diversity in seedling recruitment. Ecology 88, 3076–3087 (2007).

  55. 55.

    Theimer, T. C., Gehring, C. A., Green, P. T. & Connell, J. H. Terrestrial vertebrates alter seedling composition and richness but not diversity in an Australian tropical rain forest. Ecology 92, 1637–1647 (2011).

  56. 56.

    Hammond, D. S. & Brown, V. K. Dynamics of Tropical Communities 51–78 (Blackwell Science, 1998).

  57. 57.

    Sarmiento, C. et al. Soilborne fungi have host affinity and host-specific effects on seed germination and survival in a lowland tropical forest. Proc. Natl Acad. Sci. USA 114, 11458–11463 (2017).

  58. 58.

    Forrister, D. L., Endara, M.-J., Younkin, G. C., Coley, P. D. & Kursar, T. A. Herbivores as drivers of negative density dependence in tropical forest saplings. Science 363, 1213–1216 (2019).

  59. 59.

    Yang, H., Li, D., Wang, B. & Han, J. Distribution patterns of dominant tree species on northern slope of Changbai Mountain. Res. For. Ecosyst. 5, 1–14 (1985).

  60. 60.

    Miles, E. The Effects Of Fungicides On Mycorrhizae Colonization Of Tropical Tree Seedlings. BSc thesis, Univ. of Glasgow, UK (The Scottish Agricultural College, 2008).

  61. 61.

    Maron, J. L., Marler, M., Klironomos, J. N. & Cleveland, C. C. Soil fungal pathogens and the relationship between plant diversity and productivity. Ecol. Lett. 14, 36–41 (2011).

  62. 62.

    Harms, K. E., Wright, S. J., Calderón, O., Hernández, A. & Herre, E. A. Pervasive density-dependent recruitment enhances seedling diversity in a tropical forest. Nature 404, 493–495 (2000).

  63. 63.

    Canham, C. D., LePage, P. T. & Coates, K. D. A neighborhood analysis of canopy tree competition: effects of shading versus crowding. Can. J. For. Res. 34, 778–787 (2004).

  64. 64.

    Gelman, A. & Hill, J. Data Analysis Using Regression And Multilevel/hierarchical Models (Cambridge University Press, 2006).

  65. 65.

    Johnson, D. J., Condit, R., Hubbell, S. P. & Comita, L. S. Abiotic niche partitioning and negative density dependence drive tree seedling survival in a tropical forest. Proc. Biol. Sci. 284, 20172210 (2017).

  66. 66.

    Tabachnick, B. G. & Fidell, L. S. Using Multivariate Statistics (Allyn and Bacon, 2001).

  67. 67.

    Shannon, C. E. & Weaver, W. The Mathematical Theory Of Communications (University of Illinois Press, 1949).

  68. 68.

    Berger, W. & Parker, F. Diversity of planktonic foraminifera in deep-sea sediments. Science 168, 1345–1347 (1970).

  69. 69.

    R Core Team. R: A language and environment for statistical computing. in R Foundation for Statistical Computing (2017).

  70. 70.

    Bates, D., Maechler, M., Bolker, B. & Walker, S. lme4: Linear mixed-effects models using Eigen and S4 (2013).

Download references


We thank S.J. Wright, M.S. Luskin, D.J. Johnson, S.J. Davies, and V. Milici for valuable comments. We also thank B. Wu, J. Hu, Z. Mao, S. Fang, and members of Wang lab for field assistance. Portions of this work benefited from the 2016, 2017, and 2018 ForestGEO workshops attended by S.Jia. This work was financially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant XDB31030000), the National Natural Science Foundation of China (Grant 31722010), the Key Research Program of Frontier Sciences, Chinese Academy of Sciences (Grant ZDBS-LY-DQC019), the LiaoNing Revitalization Talents Program (Grant XLYC1807039), and the K. C. Wong Education Foundation.

Author information

X.W. and S.J. conceived the idea and designed the research. S.J., X.W., Z.Y., F.L., J.Y., G.L., and Z.H. collected the data. S.J. and R.B. conducted the data analyses and wrote the first draft. All authors contributed to the final manuscript.

Correspondence to Xugao Wang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Communications thanks Xubing Liu and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Source data

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jia, S., Wang, X., Yuan, Z. et al. Tree species traits affect which natural enemies drive the Janzen-Connell effect in a temperate forest. Nat Commun 11, 286 (2020).

Download citation


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.