Introduction

Soils in the northern permafrost region contain ~1300 ± 300 Pg of soil organic carbon (SOC)1,2,3, more than one-third of the global total SOC4, formed under cold climates with limited decomposition. Carbon release from this large pool due to warming-induced thawing and acceleration of microbial decomposition may act as an important positive feedback to climate change, but the magnitude and timing of this carbon release remains uncertain3,5,6,7. Model estimates of changes in permafrost extent and carbon stocks vary widely for the recent decades8 as well as for future scenarios9,10, partly because different representations of soil thermodynamics lead to different soil temperature responses to air temperature changes. For the ground surface, the energy budget is composed of net radiation, sensible heat, latent heat from evapotranspiration, and heat flux into and out of the ground. On the other hand, below the surface, thermal dynamics is dominated by heat conduction11 driven by surface soil temperature variations, and also affected by latent heat released or absorbed during soil water phase changes which can be important for wet soils in permafrost regions12.

The heat conduction in the soil, assuming a uniform and constant medium, can be described by the one-dimensional Fourier’s Law (Eqs. [1] and (2) in “Methods” section)13,14, in which thermal diffusivity, D, is the key parameter that describes the rate at which soil temperature changes given a temperature gradient. The value of D is determined by soil composition (minerals, air, water/ice, organic matter) and soil structure11. Among the different soil constituents, thermal diffusivity of soil organic matter (SOM) is an order of magnitude smaller than that of typical soil minerals, and slightly smaller than water11. Structurally, a higher organic matter content in the soil increases soil porosity15, which also decreases soil thermal conductivity and D especially when the soil pores are filled with air. As a result, SOM acts as an insulator and the presence of SOM cools the soil during summer, while its warming effect during winter is less important due to the insulating snow cover16,17. The summer-time cooling effect of SOM in turn restrains microbial decomposition5 and favours the accumulation of SOC. Given the high SOC storage in permafrost soils and the high vulnerability and sensitivity of these carbon stocks to projected warming6,18, it is important to assess the quantitative impact of SOC on D and the potential acceleration of soil warming due to the loss of SOC and its insulating effect.

Considering a harmonic forcing as the upper boundary condition (e.g., the seasonal cycle of surface soil temperature), the solution of the heat conduction equation (Eq. [4]) gives an exponential dampening of the temperature amplitude with increasing depth, with its e-folding depth reflecting the D value13,14 (Eq. [5]). Accordingly, D can be derived from temperature amplitudes concurrently observed at two soil depths. This equation has been used to calculate D and its influence factors in in situ studies19,20,21, but these studies were limited to a few sites. A large-scale application of this equation has been applied by ref. 22 to study the snow insulation effect between air and soil temperature at 20 cm depth, but did not touch the research problem of heat conduction in sub-surface soils below 20 cm.

Here we infer D from two large data sets of depth-specific soil temperature measurements, the Russian Historical Soil Temperature Data23 (hereafter RHST) and the International Polar Year Thermal State of Permafrost24 (hereafter IPY), at 274 locations that spread throughout Russia and North America including permafrost and non-permafrost regions (Supplementary Fig. 1). A key requirement for Eqs. (4) and (5) to be valid for inferring D using real-world soil temperature profiles is that non-conductive processes, like the latent heat associated with soil freezing, have minor impacts on the heat budget of soils at seasonal-cycle time-scale. Therefore, we examined the shape of temperature profiles at each site and selected only the sites where monthly temperature oscillations conform to sinusoidal functions, that is, to the solution (Eq. [4]) of heat conduction (see detailed data processing and discussion in Methods). After this data-filtering, 184 sites were retained. The calculated D value using Eq. [5] can approximately represent the thermal diffusivity for the bulk soil in the corresponding depth interval at those selected sites. Then, we searched for emerging relationships between the observation-derived D values and a suite of soil properties extracted from broad-scale soil maps, in order to identify the dominant factors that control D and subsequently the propagation of temperature waves from ground surface to deeper soils. We find that SOC is the strongest predictor to explain the variability of D among sites. Using the same method to calculate D from CMIP5 model outputs, we find that the different parameterization of D in these models dominates their large-scale performance in simulating today’s permafrost extent. Further with a land surface model, we show that the thermal insulation of SOC increases modelled present-day permafrost carbon stock (0–3 m depth) by 39% ( + 230 Pg C) and permafrost area by 33% (+3.4 million km2), increasing agreement with broad-scale observational data. These findings demonstrate the need to explicitly represent the coupling between SOC and soil thermal dynamics in models that are utilized to project future permafrost changes.

Results

Organic matter content controls soil thermal diffusivity

We find a significant negative correlation between soil thermal diffusivity (log-transformed) and soil organic carbon density (Fig. 1). This empirical result confirms that soils with more organic materials have a lower thermal diffusivity. Among all the tested soil characteristics including soil texture (fractions of sand, silt and clay)25, bulk density25, organic carbon content per unit dry weight25 (g kg−1), and soil volumetric moisture content26,27 (Supplementary Figs. 2 and 3), the SOC density shown in Fig. 1 has the strongest explanatory power for D variations among the different site-depths. Considering the interrelation between these soil properties, e.g., SOC density is negatively correlated with bulk density (Supplementary Fig. 4), we conducted partial correlation analysis to test if the D-SOC relationship persists after removing the effect of other variables. The partial correlation between D and SOC density remains significant after controlling the other properties separately or combined (Supplementary Table 1). This suggests that the impact of SOC on D is a composite result of soil composition and structure alterations such that its correlation with D cannot be fully accounted for by the variability in the texture- and porosity-related properties.

Fig. 1
figure 1

Relationship between soil thermal diffusivity and soil organic carbon (SOC). Thermal diffusivities (D) are derived from depth-specific measurements of monthly soil temperature from two data sets, RHST23 (circles) and IPY24 (crosses), at 184 sites (Supplementary Fig. 1), grouped into four depth intervals above 3 m (see Methods). The vertical error bars indicate 25th–75th percentiles among the available years for each site-depth. SOC densities (kg C m−3) are derived from two databases, WISE25 (blue) and NCSCD28 (red), integrated into the same four depth intervals. The blue and red solid lines indicate the linear regression lines of log10 (D) vs. SOC from WISE (y = −0.012 × −0.35) and from NCSCD (y = −0.016 × −0.30), respectively, with 95% confidence intervals shown in the dashed lines

Full size image

Uncertainties in the results shown in Fig. 1 and Supplementary Fig. 2 include a spatial-scale mismatch: the local soil temperature observations are not accompanied by collocated records of soil texture, moisture and organic carbon; we therefore resort to the global (WISE25) and regional (NCSCD28) databases for these soil properties, which are not fully representative for site-level observations. In addition, even at broad-scale, soil inventories bear substantial uncertainties29,30. The two databases used here differ in total carbon stock by about 20% for their overlapping areas (Supplementary Fig. 5ab, 690 Pg C in WISE vs. 825 Pg C in NCSCD for 0–2 m depth). Both databases derive their spatial distributions by linking field data of soil profiles to global/regional soil classification maps, their accuracy depending on both the accuracy of the up-scaling maps and the representativeness of samples within each soil class. Thus, the high landscape complexity in the northern permafrost zone and limited soil profiles available from Siberia25,31 add to uncertainties in this region. To partly reduce this uncertainty, we excluded the site-depth data whose relative difference in SOC density between the two databases is higher than 25% (see Methods and Supplementary Fig. 5c).

Apart from the predominant role of SOC, soil moisture also affects D, especially in dry soils where an increased soil water act as bridges to improve thermal contacts between soil particles11. A laboratory experiment32 showed that for sandy soils, D increases rapidly with moisture content when the water content is less than 0.1–0.2 m3 m−3, and then remains stable or slightly decreases; while for cohesive soils of silty and clayey texture, the variation of D with moisture is smaller. To investigate if soil moisture can explain the spatial variability of D, we used two data sets, the satellite-based surface soil moisture product (ESA-CCI-SM26) and the depth-specific soil moisture simulated by a land surface model (ERA-Interim/Land27). Neither of them shows significant correlation with D, possibly because the annual mean moisture contents at the sites are mostly higher than 0.2 m3 m−3, a saturation level sufficient to bridge air gaps between soil grains (see Supplementary Fig. 3 and Supplementary Discussion). Apart from total moisture content, its phase composition also affects D, since ice has a much higher D than water (1.1 vs. 0.14 × 10−6 m2 s−1)11. Soils in the northern permafrost region often contain abundant ground ice33 that can exceed soil porosity, including segregated ice and ice wedges34. These occurrences of excess ice beyond soil pores modify soil thermal properties and add to the heterogeneity of permafrost soils35, contributing to the scatter in observations shown in Fig. 1 and Supplementary Figs. 2 and 3. Considering a potential confounding effect of ground ice on the correlation between SOC and D shown in Fig. 1 (as a deeper depth could feature both lower SOC density and higher ice content, as well as a higher D), we calculated the regression for the top depth interval only (<0.4 m) (Supplementary Fig. 6), which is generally above the permanently frozen layer and less affected by large ice volumes. The similar results as Fig. 1 indicate that the observed strong correlation between SOC and D is unlikely confounded by ice content.

D is a strong predictor for simulated permafrost extent

Thermal diffusivity (D) is a key feature in the soil thermal regime. Its parameterization in land surface models may fundamentally impact their broad-scale patterns of soil temperature and permafrost distribution. Yet, this process has been given little attention. To test this, we analyzed the outputs of the historical experiment from a set of Earth System Models (ESMs) that participated in the fifth phase of the Climate Model Inter-comparison Project (CMIP5)36. As the simulated climate differs among the models, the total area of grid cells with mean annual air temperature (MAAT) less than 0 °C (AMAAT<0) ranges from 22 to 32 million km2, with a standard deviation of 2.7 million km2 (Fig. 2a). As expected, simulated near-surface permafrost area (Ap, defined as modelled active layer thickness less than 3 m (ref. 37), see Methods) is significantly correlated with AMAAT<0, with a slope slightly larger than 1, which might be related to the continuity of permafrost (namely, in models with a warm bias, permafrost area is underestimated more strongly, as they miss the sub-grid discontinuous permafrost in the southern grid cells). However, the difference in MAAT among the models can only partly explain the much wider variation in Ap across the models, with a standard variation of 7.5 million km2 (Fig. 2a). The ratio between the two areas (Ap/AMAAT<0) is determined by both air-soil interface energy exchanges affected by vegetation and snow covers, and sub-surface heat transfer in the soil column affected by soil thermal properties. Here it appears to be the latter that dominates Ap/AMAAT<0 in the CMIP5 models, since 69% of the variation in Ap/AMAAT<0 can be explained simply by differences in simulated D between models (Fig. 2b). Note that the D values for each model are diagnosed from their outputted monthly soil temperatures after excluding the pixels where non-conductive processes are not negligible (the same treatment as the observations, see Methods).

Fig. 2
figure 2

Relationship between permafrost area and thermal diffusivity in the CMIP5 models. a Modelled area of regions with surface mean annual air temperature (MAAT) less than 0 °C (AMAAT<0) and area of near-surface permafrost (Ap, see “Methods” section) during the period 1961–1990, from the historical runs of CMIP5 models. Models from the same institute are shown in the same color. The error bars indicate 25th–75th percentiles among the 30 years. The black star indicates observations derived from CRU TS v3.24 climate dataset65 and the IPA (International Permafrost Association) continuous and discontinuous permafrost extent33. b Area ratio between near-surface permafrost and regions with MAAT ≤ 0 °C (Ap/AMAAT<0) during the period 1961–1990, against median thermal diffusivity from each CMIP5 model at the same site-years as the observations, with the horizontal error bars indicating 25th–75th percentiles. The solid grey line in (b) represents the linear regression line between Ap/AMAAT<0 and log10(D) for CMIP5 models (y = −0.55 × + 0.33)

Full size image

To further analyze whether simulated snow depth also controls Ap/AMAAT<0, we calculated the effective snow depth22 (the mean snow depth for each month weighted by its duration, see “Methods” section) for each model. In contrast to the strong correlation between Ap/AMAAT<0 and D (Fig. 2b and Supplementary Fig. 7a), effective snow depth is not significantly correlated with Ap/AMAAT<0 among the CMIP5 models (Supplementary Fig. 7b). This is probably because of the following: (1) the insulation of snow occurs during winter, thus its effect on the maximum thaw depth is much smaller compared to the summer-time insulation by SOM, although the winter soil warming can be carried over to summer in regions with thick snow cover38; and (2) the CMIP5 models did not represent snow thermodynamics well (among these models, only CCSM4, MIROC and NorESM families used an explicit multilayer snow scheme39).

Interestingly, in Fig. 2b, the observation lies near the linear regression line of Ap/AMAAT<0 against D from the CMIP5 models, implying that the models generally have a reasonable structure for the soil thermal regime (most of the models adopt multilayer finite difference heat conduction8), but individual models have large biases of Ap/AMAAT<0 related to their biases of D. This means that improving the parameterization of soil thermal properties should be a priority in order to capture permafrost distribution in these models. Note that most of the land models of CMIP5 ESMs did not consider the impact of SOM on soil physical properties37; nonetheless, modelled D differs greatly among the models (Supplementary Fig. 8), possibly due to varied parameters for soil properties, different soil texture maps, and effects of different hydrological regimes.

Thermal effect of SOC enhances soil carbon accumulation

The reduction of soil thermal diffusivity by the presence of organic matter (Fig. 1) leads to a stronger dampening of the seasonal-cycle amplitude along the soil depth, and thus cools the deeper soils during summer, thins the active layer thickness (ALT), and reduces microbial respiration of SOC during the thawing season. To investigate the magnitude of this SOM effect on soil temperature and SOC accumulation, we conducted two simulations of the ORCHIDEE-MICT40 land surface model, with (hereafter yesSOM) and without (hereafter noSOM) the impacts of SOM on soil thermo- and hydrodynamics. The SOM insulation effect in the model was implemented via SOM-dependent soil physical parameters including thermal conductivity, heat capacity, porosity and available water capacity, with detailed descriptions in ref. 40. The inclusion of SOM effect allows the model to reproduce the relationship between SOC density and D, and to lower D values which are more consistent with observations than the results of noSOM (Supplementary Fig. 9). Peak summer-time monthly mean soil temperature decreases by about 4 °C on average at 0.8 and 1.6 m depths for the sites in permafrost region in yesSOM compared to noSOM (Supplementary Fig. 10). Consequently, ALT decreases by 0.6 m averagely across the region due to the presence of SOM (Fig. 3a, b), and the permafrost area (ALT < 3 m) increases from 10.4 in noSOM to 13.8 million km2 in yesSOM, closer to the empirical estimate of 15.3 million km2 for the continuous and discontinuous categories33.

Fig. 3
figure 3

Impact of organic matter on modelled active layer thickness (ALT) and soil organic carbon (SOC). a, b Modelled mean ALT during 1990–2010 from yesSOM (a) and the difference between yesSOM and noSOM (b). The color-filled circles on (a) indicate observations from the CALM network. c, d Modelled mean SOC during 1990–2010 from yesSOM (c), and its relative difference against noSOM (d). The red and magenta lines indicate boundaries of northern permafrost according to the IPA map33

Full size image

For SOC, modelled total carbon stock for 0–3 m depth in the northern permafrost zones (22 million km2, according to IPA33 including all permafrost categories) increases by 39%, from 590 Pg C in noSOM to 820 Pg C in yesSOM (Fig. 3e, f and Supplementary Fig. 11), which is still less than the estimate of 1030 Pg C in NCSCD database1 over the same region. This discrepancy may in part be attributed to peat formation not being included in the current model40. The accumulation of SOC depends on both carbon input rate controlled by vegetation productivity, and carbon decomposition rate regulated by many abiotic (e.g., temperature, moisture, oxygen) and biotic factors41. Modelled gross and net primary productivity (GPP and NPP) is generally higher in yesSOM (Supplementary Fig. 12), since a higher water holding capacity due to the existence of SOM42 can alleviate plant water stress during growing season. The relative increase of total primary productivities for the permafrost region is 8% (9.5 vs. 8.8 Pg C per year for GPP; 5.3 vs. 4.9 Pg C per year for NPP); thus, the 39% increase of SOC in yesSOM is mainly driven by the cooler summer-time soil temperature that slows down soil respiration in the model (Supplementary Discussion).

Discussion

The key finding of this study is the dominant (spatial) impact of soil organic matter on soil thermal diffusivity (D) supported by empirical evidence, which is shown in model simulations to affect the broad-scale soil thermodynamics and in turn affects the soil organic carbon (SOC) accumulation. Thermal mediation of soil organic layer is known to stabilize the underlying permafrost in relatively warm regions where the climate alone cannot sustain the persistence of permafrost43. Here we provide a quantitative empirical estimate for the insulation effect of SOC through its impact on D. We recognize that, apart from heat conduction controlled by D, the latent heat associated with soil water/ice phase changes could be substantial in the soil energy budget, especially on the border of permafrost zones (Supplementary Fig. 1), which complicates the permafrost stability. This latent heat could partly explain the slower warming rate of the mean annual permafrost temperature in discontinuous zones during the past decades24,44,45. However, this latent heat effect delays but does not prevent the eventual permafrost thawing45. Under the long-term climate warming, as permafrost might greatly retreat poleward39, heat conduction would finally dominate, inferred from the cold sites outside the permafrost regions today (Supplementary Fig. 1); and the SOC-D relationship suggests a significant impact of changes in SOC on soil temperature in the former permafrost regions.

The implication of such SOC-D feedback for future permafrost dynamics, therefore, depends on the direction and magnitude of soil carbon changes in the future. Although enzyme kinetics theory and short-term observations5,46 suggest an initially increasing SOC decomposition rate with warming, large uncertainties still remain in the long-term response of soil carbon to climate change, due to uncertainties in the rate of SOC formation (including changes of plant productivity, soil-aggregate formation, etc.)4,47 and in microbial physiological changes/adaptation to warming48. A recent compilation of field warming studies49 showed diverging changes in SOC stock of top 10 cm depth after warming, with statistically neutral change across all studies. These studies are, however, limited by a generally shallow depths of warming treatment, which missed the response of subsoils50; as well as a paucity of measurements for SOC stock changes at deeper soil horizons6, especially for the formerly frozen soils. Laboratory incubation experiments, though, have demonstrated a high lability of permafrost soils under unfrozen conditions51,52. On the other hand, modeling studies mostly projected carbon losses in permafrost soils in the long run, albeit varying rates and magnitudes3,53. The protection of permafrost by SOC is also manifested in fire disturbances, where the combustion of surface organic layer led to increased soil temperature and deeper thaw depth54,55. Thus, the projected increase in fire activities in the northern high latitudes56 might also be an important driver of permafrost thaw through the impact of SOC on soil thermal properties.

Apart from the dominant role of SOC in D, which controls the temperature propagation below ground (in the case of minor latent heat induced by freezing/thawing), the aboveground features including vegetation and snow cover also impact the vulnerability of permafrost to a rising air temperature by controlling heat exchanges between air and land surface57. Besides, degradation of ice wedges may lead to land subsidence and thus thermokarst formation, which is estimated to affect 20% of the northern permafrost region58. Such abrupt changes in the landscape strongly alter micro-topography and thus local hydrological and thermal regimes59. Meanwhile, the decadal trajectories after initial thaw are closely linked to accumulation of organic matter (itself related to vegetation succession) on the thermokarst troughs, which helps to stabilize ice wedges and prevent further thaw60. This mechanism, however, might be perturbed under future warming and intensified fire activities61. Such complex interactions call for models that quantitatively integrates thermo-hydrological processes, their heterogeneity at small spatial scales, vegetation dynamics and soil carbon cycle, thermal impacts of organic matter, and fires, in order to improve our understanding and prediction of the overall vulnerability of permafrost.

The emergent slope of permafrost area versus D from CMIP5 models (Fig. 2b) and the empirical relationship of D versus SOC (Fig. 1) suggest a non-negligible sensitivity of permafrost area to potential SOC changes (a loss of ca. 0.2 million km−2 per 1 kg C m−3 reduction of SOC on average). Without the SOC-D coupling, CMIP5 models have projected a linear relationship between air temperature change and permafrost area change39. The recent estimate62 for the sensitivity of permafrost area to warming was also based on the assumption of an unchanged present-day MAAT-permafrost relationship for future scenarios. In contrast, the results shown here implies a non-linear trajectory of permafrost area changes in response to warming, highlighting a critical need to explicitly represent the coupling between SOC and soil thermal dynamics in models that are utilized to project the future of permafrost.

Methods

Calculation of D from soil temperature measurements

On the basis of Fourier’s law, the one-dimensional heat conduction can be described by:

$$c\frac{{\partial T}}{{\partial t}} - \frac{\partial }{{\partial z}}\left( {\lambda \frac{{\partial T}}{{\partial z}}} \right) = 0,$$
(1)

where T is temperature at time t and depth z; λ is thermal conductivity (J m−1 s−1 K−1); c is volumetric heat capacity (J m−3 K−1).

If λ and c are constants, Eq. [1] can be re-written as:

$$\frac{{\partial T}}{{\partial t}} - D\frac{{\partial ^2T}}{{\partial z^2}} = 0,$$
(2)

where D is soil thermal diffusivity (m2 s−1), equalling to the quotient of λ and c.

To solve Eq. [2], we consider the case that the upper boundary condition is given by a sine-wave temperature oscillation, which represents the diurnal or seasonal cycle of surface soil temperature driven by air temperature fluctuations:

$$T\left( {0,t} \right) = \overline {T_0} + A_0{\mathrm{sin}}(\omega t + \varphi ),$$
(3)

where A0 is amplitude of the temperature oscillation at the surface, equaling to half of the difference between warmest and coldest values; ω is its angular frequency, equalling to \(\frac{{2\pi }}{{86400 \times 365}}\) if we consider the seasonal cycle in a year; φ is its phase; \(\overline {T_0}\) is the mean surface temperature in a wave period.

In the solution of Eq. [2], after a sufficient long time, the term that describes the transient disturbance since t = 0 becomes negligible (i.e., the effect of the initial temperature function is lost)13. Thus, the solution of Eq. [2] for the periodic steady-state is as follows13,14:

$$T\left( {z,t} \right) = \overline {T_0} + A_0e^{ - \sqrt {\frac{\omega }{{2D}}} z}\sin \left( {\omega t + \varphi - \sqrt {\frac{\omega }{{2D}}} z} \right).$$
(4)

Equation (4) infers an exponential dampening of the temperature amplitude with increasing depth19,63:

$$\frac{{A_{z1}}}{{A_{z2}}} = e^{(z2 - z1)\sqrt {\frac{\omega }{{2D}}} }$$
(5)

where A is amplitude of the temperature wave at depth z1 or z2.

Therefore, D can be calculated from observed temperature amplitudes at two soil depths during the same period.

Equation (4) is an idealized mathematical model to describe the real-world soil thermal regime, its applicability requiring several necessary conditions: first, the heat transfer in the soil is dominated by conduction, so that convection, radiation, and particularly the latent heat release/absorption associated with phase transitions are of minor importance at the seasonal-cycle time-scale; second, the soil thermal properties are relatively constant over time and uniform along the considered depth interval; and third, soil temperature at the upper boundary follows a sinusoidal function. If the real soil thermodynamics at a site significantly violates any of the above conditions, it is unlikely that the soil temperature function T(z,t) would take the simple sinusoidal form as Eq. [4]. We therefore assume that, if the measured soil monthly temperatures at two adjacent depths both conform to sine-waves (with a small error), the soil thermodynamics in this interval can be approximated by Eq. [2], and the calculated D using Eq. [5] is a valid value that describes the thermal diffusivity for the bulk soil in this depth interval at the site.

In this study, we used two data sets of in-situ soil temperature measurements to calculate D: the Russian Historical Soil Temperature Data23 (RHST), and the International Polar Year Thermal State of Permafrost24 (IPY). RHST provides monthly soil temperature records at 263 sites since the 1880s until 1990, but the majority of data exist after 1960. Here we used the last ten years’ data, from 1981 to 1990. RHST contains measurements at multiple depths for near surface (0.02–0.2 m) and deeper (mostly at 0.2, 0.4, 0.8, 1.2, 1.6, 2.4, and 3.2 m) soils, but the near surface measurements were taken in bare areas whereas the deeper soil measurements were taken under natural cover of grasses and snow64; so we only used the data below 0.2 m in our analysis. The IPY soil temperatures were measured at various depths dependent on the site, mostly for the years 2007–2013. The daily observations in IPY dataset were aggregated to monthly means.

For both data sets, we conducted the following data processing:

First, only the sites with available monthly temperature records for at least two nearby depths (depth difference less than 1 m) and for at least 12 consecutive months were used (in total 274 sites, 219 from RHST and 55 from IPY, Supplementary Fig. 1).

Second, to determine whether the monthly soil temperatures evolve as a sinusoidal cycle, we first fitted the 12 months to a sine curve as Eq. [3] using non-linear least squares regression, and then calculated the sum of square error (SSE) after normalizing by amplitude such that the SSE for different depths are comparable:

$${\rm{SSE}} = \mathop {\sum }\limits_{i = 1}^{12} (\frac{{T_i - \hat T_i}}{{\hat A}})^2,$$
(6)

where Ti is the temperature for month i; \(\hat T_i\) is the predicted temperature with the fitted sine curve; \(\hat A\) is the amplitude of the fitted curve.

To derive an SSE threshold, we assume that the fluctuations of monthly air temperature, according to the CRU dataset65, generally approximate sine-waves, and calculated their SSE values for the northern hemisphere (Supplementary Fig. 13). The 75th percentile of the SSE values for air temperatures, 0.34, was chosen to filter the soil temperature data: the observation for a site-depth-year is removed if its SSE is higher than 0.34. After this filtering, 170 sites from RHST and 15 sites from IPY remained. Note that the retained sites do not necessarily fulfil the SSE threshold for all years or depths; those depth-years that exceed the threshold were also removed before the calculation of D.

Most of the high SSE situations are associated with the so-called zero curtain effect, namely, the soil temperature stays near 0 °C during extended periods of freezing/thawing, maintained by the release/absorption of latent heat and by other non-conductive processes including vapor transport and evaporation/condensation66. Supplementary Fig. 14 presents four examples from the IPY dataset to illustrate the disparity or similarity between soil temperature observations and the theoretical sine-waves. Whenever prominent zero curtain occurs, the SSE value will be much higher than 0.34 (Supplementary Fig. 14ab). Note that the presence of zero curtain effects, although of smaller impact on the season cycle after the data screening by SSE threshold, may lead to an underestimation in the derived D values.

In terms of the spatial distribution of removed sites, the sites located in warmer permafrost regions (discontinuous, sporadic and isolated permafrost zones) have a higher removal percentage (52%, 41 sites out of 79) than those in the continuous permafrost zone (35%, 28 out of 80) and in non-permafrost regions (17%, 20 out of 115) (Supplementary Fig. 1). This could be plausibly explained by the near-0 °C of the mean annual permafrost temperature in the relatively warmer permafrost regions, where a partial freezing/thawing at upper permafrost and inter-annual climate variability could induce significant amounts of latent heat24. A comparison of MAAT of retained vs. removed sites also shows significant differences in both permafrost regions (including continuous and discontinuous categories) and non-permafrost region, with the removed sites having a closer-to-zero MAAT than retained sites (Supplementary Fig. 15). We also compared the ground ice abundance (extracted from the IPA map33, which gives three levels of abundance) to test whether the filtering would preferentially remove the sites with high ground ice contents. As shown in Supplementary Fig. 16, no such difference was found between removed and retained sites. No significant difference in the surface soil moisture (extracted from ESA-CCI-SM v02.2 product26) was found between the removed and retained sites (not shown).

To further test if applying the SSE threshold can effectively filter out the cases where the phase change-induced latent heat cannot be neglected at monthly time-scales, we conducted analyses based on simulations of the ORCHIDEE-MICT land surface model40 (see the last part of Methods).

Third, for the retained site-depth-year data after step ii), we calculated the D value for each depth according to Eq. [5], using the observed temperature amplitude of this layer and of the adjacent upper layer, only if the two depths are nearby (<1 m). Since we try to correlate D with soil properties extracted from the global/regional soil databases which have fixed/limited layers (see below), the derived D values for different depths were integrated into four depth intervals: 0–0.4, 0.4–0.8, 0.8–2, and 2–3 m, taking the median D if multiple depths are available within each interval. Finally, if multiple years of D are available for a site-depth, their median value is used for the regression analysis against soil properties, while their 25th–75th percentiles are shown in Fig. 1.

Soil databases

In order to identify the dominant influential factors on D, and considering that the soil temperature data sets do not report collocated measurements of soil properties at the Russian meteorological stations in the RHST dataset or the boreholes in the IPY dataset, we seek to extract soil information for these sites from the spatially explicit global/regional soil databases. To our knowledge, there are four such soil databases that are recent and/or widely used: HWSD67; WISE30sec25 (hereafter WISE); SoilGrids250m68 (hereafter SoilGrids); and NCSCD28. Among them, WISE is developed based on HWSD, but includes updates on the mapping unit composition, adopts a consistent taxotransfer procedure, and provides information down to 2 m25. We therefore used WISE instead of HWSD in this study. The SoilGrids product, unlike the other three databases that rely on soil classification maps to derive spatially explicit information, predicts soil properties using statistical methods from a list of covariates (climate, vegetation, topography, lithological units, etc), and thus can generate high-resolution maps (250 m) dependent on the resolution of input covariates68. However, in a pre-processing, we found that SoilGrids gives a much higher SOC stock (in kg C m−2) than other databases in most grid cells in the northern hemisphere, and gives a global total SOC storage of 2757 Pg C for 0–1 m and 4570 Pg C for 0–2 m (calculated using their 250 m resolution map). This total SOC storage is almost twice the estimate by the majority of previous studies (see a short review in ref30.). Therefore, we did not use SoilGrids in this study.

Compared to WISE, NCSCD provides SOC stock down to 3 m, and is based on more soil samples in permafrost region than WISE25. However, NCSCD is a regional database for the northern permafrost zones only, and does not provide other properties like soil texture and bulk density. We therefore used both databases for SOC and used WISE for other soil properties. At their highest resolution (0.012° for NCSCD and 30 arc second for WISE), we took the grid cells whose center is nearest to the coordinates of each observation site. The two databases provide different depth intervals (4 intervals down to 3 m in NCSCD, and 7 intervals down to 2 m in WISE), and we integrated their original layers into the same four depth intervals as D values mentioned above (0–0.4, 0.4–0.8, 0.8–2, and 2–3 m), using depth-weighted averages. WISE does not provide information below 2 m, so its variables for the 4th depth interval are non-valid value in all the regression analyses.

Using the large-scale soil databases to indicate local-scale observations introduces uncertainties. To reduce this uncertainty, we compared the SOC density for the available site-depths from the two databases (Supplementary Fig. 5c), and excluded those with a relative difference (calculated as \(\left| {\frac{{{\rm{SOC}}_{{\rm{NCSCD}}} - {\rm{SOC}}_{{\rm{WISE}}}}}{{{\rm{SOC}}_{{\rm{NCSCD}}} + {\rm{SOC}}_{{\rm{WISE}}}}}} \right|\)) larger than 25%. This filtering process removes 105 site-depth SOC values, leaving 100 pairs of D-SOCNCSCD and 278 pairs of D-SOCWISE as shown in Fig. 1, spanning 184 sites (170 sites from RHST and 14 sites from IPY, Supplementary Fig. 1 red). For consistency, the excluded site-depths were also removed from the analysis of the other soil properties as shown in Supplementary Fig. 2.

A linear relationship was found between log-transformed D and SOC density (Fig. 1); we therefore used log10(D) in all the correlation analyses. A log-transformation of D also improves its normality. To explore the potential multivariate relationships, we conducted a stepwise regression analysis using SOC density (from WISE and NCSCD respectively) and all the other soil properties as predictive variables. In the case of SOC from NCSCD, none of the other variables are significant in the stepwise regression to explain D variations, namely, the addition of other properties does not increase the explanatory power compared to the log10(D)-SOCNCSCD regression model. In the case of SOC from WISE, only silt fraction is significant, but R2 only marginally increases from 0.35 to 0.37, and the slope of SOCWISE in the stepwise regression equation (−0.010) is only slightly different from that in the single-variable regression (−0.012).

Treatment of CMIP5 models

We used all CMIP5 ESMs that provide access to their historical simulation (r1i1p1) on the website https://esgf-node.ipsl.upmc.fr/search/cmip5-ipsl/ (accessed in July 2017) and that provide surface air temperature and vertically-resolved soil temperatures (in total 27 models). To calculate active layer thickness (ALT) of each grid cell, we firstly calculated the maximum monthly soil temperature (Ts,max) for each soil level, then took the deepest depth where Ts,max was at or above 0 °C. If the depth of 0 °C was between two soil levels, a linear interpolation was used to locate the depth. Permafrost is defined to be present in a grid cell if ALT is less than 3 m or the deepest model soil level, following ref37. which gives an approximation of near-surface permafrost.

For the simulated thermal diffusivity, the same procedure as the calculation of D for the sites was carried out using the outputted monthly soil temperatures by each model for all land grid cells and soil layers. Specifically, the same SSE threshold of 0.34 was adopted to exclude the pixel-depth-years whose monthly soil temperature oscillations do not conform to sine-waves. Since the CMIP5 models have different (or a lack of) parameterizations for the soil freezing process37, their SSE values differ (Supplementary Fig. 17). In general, the models that do not consider the latent heat associated with water phase change (CMCC, IPSL, and MPI families) produce sine wave-shaped monthly soil temperatures and thus have smaller SSE values (Supplementary Fig. 17). Applying the SSE threshold before calculating D values for each model could reduce the confounding effect of their different parameterizations for the freezing-induces latent heat.

The CMIP5 models have different vertical soil layers (Supplementary Fig. 18), and here we only calculated the D values for the upper 3 m, and integrated them into the same four depth intervals as the observations. Then, to compare with the observation-derived D values at the RHST and IPY sites, the same locations (nearest grid cell center to the site coordinates) were selected for each model, thus deriving the results shown in Fig. 2b and Supplementary Fig. 7.

For the simulated snow depth, in order to better describe its insulation effect, we calculated the effective snow depth (Seff) defined as the mean snow depth weighted by its duration, following ref. 22:

$$S_{{\rm{eff}}} = \frac{{\mathop {\sum }\nolimits_{i = 1}^6 S_iw_i}}{{\mathop {\sum }\nolimits_{i = 1}^6 w_i}},$$
(7)

where i (1–6) represents the winter months from October to March; wi is a weighting factor that equals to 6 for October and decreases to 1 for March; Si is the mean snow depth for the month i. In this way, an earlier accumulation of snow, with its longer time period to insulate the soil during the whole snow season, has a larger weighting and yields a higher Seff than the case of a linearly accumulating snowpack. Note that this calculation did not account for the compaction/densification-induced increase in snow thermal conductivity that weakens its insulating effect during the late snow season, as demonstrated by the observed smaller temperature gradient (soil temperature at 20 cm depth minus air temperature) under a certain snow depth during the late snow season than that during the early season under the same snow depth40.

Only 21 models out of the 27 models provided outputs for snow depth (or snow mass) at the time we accessed the CMIP5 repository. For the models that only provided snow mass but not depth (the MPI and HadGEM families), a snow density of 250 kg m−3 was used for conversion39.

Simulations of the ORCHIDEE-MICT land surface model

We incorporated the impacts of organic carbon on soil thermal and hydrological properties in the ORCHIDEE-MICT land surface model, assuming that soil physical properties are weighted averages of mineral soil and pure organic soil. Detailed descriptions can be found in ref. 40. One modification compared to ref. 40. is that the soil thermal conductivity is calculated as geometric average of mineral and organic soils (instead of arithmetic average in ref. 40), which is more consistent with the empirical linear relationship between log10(D) and SOC density (Fig. 1). We conducted two simulations for the northern hemisphere (>30°N) at 2° spatial resolution forced by CRUNCEP v8 climate: yesSOM, in which an observation-based SOC map was used in the thermal and hydrological modules to derive soil properties; and noSOM, in which the thermal and hydrological modules see zero SOC, i.e., no impacts of SOC on soil properties. For yesSOM, the observational SOC map was constructed from NCSCD28 for permafrost regions and from WISE25 for non-permafrost regions, using linear vertical interpolation to convert their original soil horizons into ORCHIDEE-MICT vertical layers40. Here we did not use the prognostically modelled SOC by the carbon module of ORCHIDEE-MICT, in order to exclude bias of the carbon cycle module for the analysis of SOM effect. For both simulations, spin-ups were first conducted forced by looped 1901–1920 climate and pre-industrial CO2 level of 286 ppm, covering 200 years of the full model and 20,000 years of the soil carbon sub-model to reach equilibrium for the pre-industrial period. Afterwards, transient runs for 1861–2010 were conducted forced by historical climate and rising CO2 concentrations.

We also made use of the simulations to test the validity of the SSE threshold, and to compare the temperature amplitude-derived diffusivity (D) against the real diffusivity as used in the one-dimensional Fourier’s equation to solve for soil temperatures in the model. ORCHIDEE-MICT includes a soil-freezing parameterization, in which the latent heat of fusion is incorporated into an apparent heat capacity, so that Eq. (1) writes 69:

$$c_{{\rm{app}}}\frac{{\partial T}}{{\partial t}} - \frac{\partial }{{\partial z}}\left( {\lambda \frac{{\partial T}}{{\partial z}}} \right) = 0,$$
$${\mathrm{in}}\;{\mathrm{which}}\;c_{app} = c - \rho L\frac{{\Delta \theta }}{{\Delta T}},$$
(8)

where ρ is ice density (kg m−3); L is latent heat of fusion (J kg−1); ∆T equals to 2 K, corresponding to the freezing window between −1 °C and 1 °C in the model; ∆θ equals to the total water content (m3 m−3) for the layer, providing the available amount to freeze/thaw in the freezing window.

Then, thermal diffusivities:

$$D_{{\rm{with}} - {\rm{fusion}}} = \frac{\lambda }{{c_{{\rm{app}}}}}$$
$$D_{{\rm{no}} - {\rm{fusion}}} = \frac{\lambda }{c}$$
(9)

can be calculated for each time-step and averaged over the year to compare with the amplitude-derived diffusivity (D). Detailed parameterizations of λ and c in ORCHIDEE-MICT are given in ref. 40, which generally follows the empirical equations of ref. 70 to account for the impact of soil moisture on thermal conductivity λ.

As shown in Supplementary Fig. 19a, the relative difference of annual mean Dwith-fusion and Dno-fusion (\(1 - \frac{{D_{{\rm{with}} - {\rm{fusion}}}}}{{D_{{\rm{no}} - {\rm{fusion}}}}}\), which ranges from 0 to 1) are generally small (less than 30%) for those locations where modelled monthly soil temperatures conform to sine-waves (SSE < 0.34). This suggests that applying the SSE threshold can effectively exclude the cases in which the latent heat significantly changes the apparent thermal diffusivity. Note that some site-depths have high SSE values (and thus were excluded) even though they are less affected by soil freezing/thawing (Supplementary Fig. 19a), probably because the other conditions for the validity of Eq. (4) are not met, e.g., the diffusivity varies too much over the year or along the considered depth interval, or the surface soil temperature itself does not conform to a sine-wave (due to, e.g., a non-sine shaped air temperature, or a strong winter insulation of snow causing asymmetric effects on the seasonal-cycle).

Supplementary Fig. 19b compares the amplitude-derived diffusivity (D) with Dwith-fusion, which shows a good correlation between them (R2 = 0.85), although D is generally smaller than Dwith-fusion when their values are low. This suggests that the amplitude-derived diffusivities (after excluding high SSE values) are reasonable indicators for the internal parameterization of soil thermal properties in land surface models.