In recent years, an evaluation technique for Earth System Models (ESMs) has arisen—emergent constraints (ECs)—which rely on strong statistical relationships between aspects of current climate and future change across an ESM ensemble. Combining the EC relationship with observations could reduce uncertainty surrounding future change. Here, we articulate a framework to assess ECs, and provide indicators whereby a proposed EC may move from a strong statistical relationship to confirmation. The primary indicators are verified mechanisms and out-of-sample testing. Confirmed ECs have the potential to improve ESMs by focusing attention on the variables most relevant to climate projections. Looking forward, there may be undiscovered ECs for extremes and teleconnections, and ECs may help identify climate system tipping points.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Additional information

Journal peer review information: Nature Climate Change thanks Benjamin Sanderson and Tapio Schneider for their contribution to the peer review of this work.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


  1. 1.

    IPCC Climate Change 2013: The Physical Science Basis. (eds Stocker, T. F. et al) (Cambridge Univ. Press, 2013).

  2. 2.

    Taylor, K. E., Stouffer, R. J. & Meehl, G. A. An overview of CMIP5 and the experiment design. B. Am. Meteorol. Soc. 93, 485–498 (2012).

  3. 3.

    Hall, A. & Manabe, S. The role of water vapor feedback in unperturbed climate variability and global warming. J. Climate 12, 2327–2346 (1999).

  4. 4.

    Soden, B. J., Wetherald, R. T., Stenchikov, G. L. & Robock, A. Global cooling after the eruption of Mount Pinatubo: a test of climate feedback by water vapor. Science 296, 727–730 (2002).

  5. 5.

    Rind, D., Healy, R., Parkinson, C. & Martinson, D. The pole of sea-ice in 2x CO2 climate model sensitivity part 1: the total influence of sea-ice thickness and extent. J. Climate 8, 449–463 (1995).

  6. 6.

    Rind, D., Healy, R., Parkinson, C. & Martinson, D. The role of sea ice in 2xCO(2) climate model sensitivity part 2: hemispheric dependencies. Geophys. Res. Lett. 24, 1491–1494 (1997).

  7. 7.

    Ivanova, D. P., Gleckler, P. J., Taylor, K. E., Durack, P. J. & Marvel, K. D. Moving beyond the total sea ice extent in gauging model biases. J. Climate 29, 8965–8987 (2016).

  8. 8.

    Parkinson, C. L., Vinnikov, K. Y. & Cavalieri, D. J. Evaluation of the simulation of the annual cycle of Arctic and Antarctic sea ice coverages by 11 major global climate models. J. Geophys. Res. Oceans 111, 14 (2006).

  9. 9.

    Gleckler, P. J., Taylor, K. E. & Doutriaux, C. Performance metrics for climate models. J. Geophys. Res. Atm. 113, 20 (2008).

  10. 10.

    Hall, A. & Qu, X. Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett. 33, L03502 (2006).

  11. 11.

    Cox, P. M. et al. Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability. Nature 494, 341–344 (2013).

  12. 12.

    Cox, P. M., Betts, R. A., Jones, C. D., Spall, S. A. & Totterdell, I. J. Acceleration of global warming due to carbon-cycle feedbacks in a coupled climate model. Nature 408, 184–187 (2000).

  13. 13.

    Cox, P. M. et al. Amazonian forest dieback under climate-carbon cycle projections for the 21st century. Theor. Appl. Climatol. 78, 137–156 (2004).

  14. 14.

    Caldwell, P. M. et al. Statistical significance of climate sensitivity predictors obtained by data mining. Geophys. Res. Lett. 41, 1803–1808 (2014). This paper demonstrates that statistically significant, but physically meaningless, emergent relationships can be found in ESM ensembles, illustrating an important potential pitfall of the EC technique.

  15. 15.

    Kubo, R. The fluctuation-dissipation theorem. Rep. Prog. Phys 20, 255–284 (1966).

  16. 16.

    Lorenz, E. N. Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130–141 (1963).

  17. 17.

    Kirk-Davidoff, D. B. On the diagnosis of climate sensitivity using observations of fluctuations. Atmos. Chem. Phys. 9, 813–822 (2009).

  18. 18.

    Majda, A. J., Abramov, R. & Gershgorin, B. High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability. Proc. Natl Acad. Sci. USA 107, 581–586 (2010).

  19. 19.

    Leith, C. E. Climate response and fluctuation dissipation. J. Atmos. Sci. 32, 2022–2026 (1975). The first suggestion to relate climate sensitivity to climate variability through the Fluctuation–Dissipation theorem.

  20. 20.

    Cox, P. M., Huntingford, C. & Williamson, M. S. Emergent constraint on equilibrium climate sensitivity from global temperature variability. Nature 553, 319–322 (2018). Emergent constraint on ECS from global temperature variability.

  21. 21.

    Wenzel, S., Cox, P. M., Eyring, V. & Friedlingstein, P. Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models. J. Geophys. Res. Biogeosci. 119, 794–807 (2014).

  22. 22.

    Williamson, M. S., Cox, P. M. & Nijsse, F. J. M. M. Theoretical foundation of emergent constraints: relationships between climate sensitivity and global temperature variability in conceptual models. Preprint at https://arxiv.org/pdf/1810.12765.pdf (2018).

  23. 23.

    Tian, B. J. Spread of model climate sensitivity linked to double-Intertropical Convergence Zone bias. Geophys. Res. Lett. 42, 4133–4141 (2015).

  24. 24.

    Gordon, N. D. & Klein, S. A. Low-cloud optical depth feedback in climate models. J. Geophys. Res. Atm. 119, 6052–6065 (2014). This is the earliest demonstration of an emergent constraint for the cloud optical-depth feedback.

  25. 25.

    Terai, C. R., Klein, S. A. & Zelinka, M. D. Constraining the low-cloud optical depth feedback at middle and high latitudes using satellite observations. J. Geophys. Res. Atm. 121, 9696–9716 (2016).

  26. 26.

    McCoy, D. T., Hartmann, D. L. & Grosvenor, D. P. Observed Southern Ocean cloud properties and shortwave reflection. Part II: phase changes and low cloud feedback. J. Climate 27, 8858–8868 (2014).

  27. 27.

    Senior, C. A. & Mitchell, J. F. B. Carbon-dioxide and climate: the impact of cloud parameterization. J. Climate 6, 393–418 (1993).

  28. 28.

    Tselioudis, G., Rossow, W. B. & Rind, D. Global patterns of cloud optical-thickness variation with temperature. J. Climate 5, 1484–1497 (1992).

  29. 29.

    Qu, X. & Hall, A. What controls the strength of snow-albedo feedback? J. Climate 20, 3971–3981 (2007). This paper documented the overwhelming similarities within ESMs between the seasonal cycle and future climate change versions of snow-albedo feedback, moving the snow-albedo feedback EC along in the confirmation process.

  30. 30.

    Qu, X. & Hall, A. On the persistent spread in snow-albedo feedback. Clim. Dynam. 42, 69–81 (2014).

  31. 31.

    Sanderson, B. M., Knutti, R. & Caldwell, P. A representative democracy to reduce interdependency in a multimodel ensemble. J. Climate 28, 5171–5194 (2015).

  32. 32.

    Annan, J. D. & Hargreaves, J. C. Reliability of the CMIP3 ensemble. Geophys. Res. Lett. 37, L02703 (2010).

  33. 33.

    Knutti, R., Masson, D. & Gettelman, A. Climate model genealogy: Generation CMIP5 and how we got there. Geophys. Res. Lett. 40, 1194–1199 (2013).

  34. 34.

    Pennell, C. & Reichler, T. On the effective number of climate models. J. Climate 24, 2358–2367 (2011).

  35. 35.

    Kamae, Y. et al. Lower-tropospheric mixing as a constraint on cloud feedback in a multiparameter multiphysics ensemble. J. Climate 29, 6259–6275 (2016).

  36. 36.

    Wagman, B. M. & Jackson, C. S. A test of emergent constraints on cloud feedback and climate sensitivity using a calibrated single-model ensemble. J. Climate 31, 7515–7532 (2018).

  37. 37.

    Caldwell, P. M., Zelinka, M. D. & Klein, S. A. Evaluating emergent constraints on equilibrium climate sensitivity. J. Climate 31, 3921–3942 (2018). This paper performed comparative analysis of the multiple ECs for climate sensitivity and offered techniques to assess the independence and confirm ECs for climate sensitivity.

  38. 38.

    Ceppi, P., Hartmann, D. L. & Webb, M. J. Mechanisms of the negative shortwave cloud feedback in middle to high latitudes. J. Climate 29, 139–157 (2016). This paper performed verification of the microphysical mechanism underlying the cloud optical-depth feedback, moving the cloud optical-depth feedback EC along in the confirmation process.

  39. 39.

    Adam, O., Schneider, T., Brient, F. & Bischoff, T. Relation of the double-ITCZ bias to the atmospheric energy budget in climate models. Geophys. Res. Lett. 43, 7670–7677 (2016).

  40. 40.

    Stroeve, J., Holland, M. M., Meier, W., Scambos, T. & Serreze, M. Arctic sea ice decline: faster than forecast. Geophys. Res. Lett. 34, L029703 (2007).

  41. 41.

    Boe, J. L., Hall, A. & Qu, X. September sea-ice cover in the Arctic Ocean projected to vanish by 2100. Nat. Geosci. 2, 341–343 (2009).

  42. 42.

    Mahlstein, I. & Knutti, R. September Arctic sea ice predicted to disappear near 2 degrees C global warming above present. J. Geophys. Res. Atm. 117, 11 (2012).

  43. 43.

    Collins, M. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) Ch. 12, 1029–1136 (Cambridge Univ. Press, 2013).

  44. 44.

    IPCC: Summary for Policymakers. In Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) 1–29 (Cambridge Univ. Press, 2013).

  45. 45.

    Bowman, K. W., Cressie, N., Qu, X. & Hall, A. A hierarchical statistical framework for emergent constraints: application to snow-albedo feedback. Geophys. Res. Lett. 45, L080082 (2018).

  46. 46.

    DeAngelis, A. M., Qu, X., Zelinka, M. D. & Hall, A. An observational radiative constraint on hydrologic cycle intensification. Nature 528, 249–253 (2015).

  47. 47.

    Thackeray, C. W., Qu, X. & Hall, A. Why do models produce spread in snow albedo feedback? Geophys. Res. Lett. 45, 6223–6231 (2018). An examination of how parameterization choices within ESMs lead to different magnitudes for snow-albedo feedback, a crucial step for model improvement in this feedback process.

  48. 48.

    McWilliams, J. C. Irreducible imprecision in atmospheric and oceanic simulations. Proc. Natl Acad. Sci. USA 104, 8709–8713 (2007).

  49. 49.

    Simpson, I. R. & Polvani, L. M. Revisiting the relationship between jet position, forced response, and annular mode variability in the southern midlatitudes. Geophys. Res. Lett. 43, 2896–2903 (2016).

  50. 50.

    Kidston, J. & Gerber, E. P. Intermodel variability of the poleward shift of the austral jet stream in the CMIP3 integrations linked to biases in 20th century climatology. Geophy. Res. Lett. 37, L042873 (2010).

  51. 51.

    Li, G., Xie, S. P., He, C. & Chen, Z. S. Western Pacific emergent constraint lowers projected increase in Indian summer monsoon rainfall. Nat. Clim. Change 7, 708–712 (2017).

  52. 52.

    Dakos, V. et al. Slowing down as an early warning signal for abrupt climate change. Proc. Natl Acad. Sci. USA 105, 14308–14312 (2008).

  53. 53.

    Lucarini, V. & Sarno, S. A statistical mechanical approach for the computation of the climatic response to general forcings. Nonlinear Proc. Geoph. 18, 7–28 (2011).

  54. 54.

    Thompson, J. M. T. & Sieber, J. Climate tipping as a noisy bifurcation: a predictive technique. IMA J. Appl. Math. 76, 27–46 (2011).

  55. 55.

    Scheffer, M. et al. Early-warning signals for critical transitions. Nature 461, 53–59 (2009).

  56. 56.

    Drijfhout, S. et al. Catalogue of abrupt shifts in Intergovernmental Panel on Climate Change climate models. Proc. Natl Acad. Sci. USA 112, E5777–E5786 (2015).

  57. 57.

    Boulton, C. A., Good, P. & Lenton, T. M. Early warning signals of simulated Amazon rainforest dieback. Theor. Ecol. 6, 373–384 (2013).

  58. 58.

    Brient, F. et al. Shallowness of tropical low clouds as a predictor of climate models’ response to warming. Clim. Dynam. 47, 433–449 (2016).

  59. 59.

    Brient, F. & Schneider, T. Constraints on climate sensitivity from space-based measurements of low-cloud reflection. J. Climate 29, 5821–5835 (2016).

  60. 60.

    Zhai, C. X., Jiang, J. H. & Su, H. Long-term cloud change imprinted in seasonal cloud variation: more evidence of high climate sensitivity. Geophys. Res. Lett. 42, 8729–8737 (2015).

  61. 61.

    Trenberth, K. E. & Fasullo, J. T. Simulation of present-day and twenty-first-century energy budgets of the southern cceans. J. Climate 23, 440–454 (2010).

  62. 62.

    Fasullo, J. T. & Trenberth, K. E. A less cloudy future: the role of subtropical subsidence in climate sensitivity. Science 338, 792–794 (2012).

  63. 63.

    Su, H. et al. Weakening and strengthening structures in the Hadley Circulation change under global warming and implications for cloud response and climate sensitivity. J. Geophys. Res. Atm. 119, 5787–5805 (2014).

  64. 64.

    Huber, M., Mahlstein, I., Wild, M., Fasullo, J. & Knutti, R. Constraints on climate sensitivity from radiation patterns in climate models. J. Climate 24, 1034–1052 (2011).

  65. 65.

    Tett, S. F. B., Rowlands, D. J., Mineter, M. J. & Cartis, C. Can top-of-atmosphere radiation measurements constrain climate predictions? Part II: climate sensitivity. J. Climate 26, 9367–9383 (2013).

  66. 66.

    Knutti, R., Meehl, G. A., Allen, M. R. & Stainforth, D. A. Constraining climate sensitivity from the seasonal cycle in surface temperature. J. Climate 19, 4224–4233 (2006).

  67. 67.

    Lutsko, N. J. & Takahashi, K. What can the internal variability of CMIP5 models tell us about their climate sensitivity? J. Climate 31, 5051–5069 (2018).

  68. 68.

    Sherwood, S. C., Bony, S. & Dufresne, J. L. Spread in model climate sensitivity traced to atmospheric convective mixing. Nature 505, 37–42 (2014).

  69. 69.

    Lipat, B. R., Tselioudis, G., Grise, K. M. & Polvani, L. M. CMIP5 models’ shortwave cloud radiative response and climate sensitivity linked to the climatological Hadley cell extent. Geophys. Res. Lett. 44, 5739–5748 (2017).

  70. 70.

    Volodin, E. M. Relation between temperature sensitivity to doubled carbon dioxide and the distribution of clouds in current climate models. Izv. Atmos. Ocean. Phys. 44, 288–299 (2008).

  71. 71.

    Siler, N., Po-Chedley, S. & Bretherton, C. S. Variability in modelled cloud feedback tied to differences in the climatological spatial pattern of clouds. Clim. Dynam. 50, 1209–1220 (2018).

  72. 72.

    Clement, A. C., Burgman, R. & Norris, J. R. Observational and model evidence for positive low-level cloud feedback. Science 325, 460–464 (2009).

  73. 73.

    Qu, X., Hall, A., Klein, S. A. & DeAngelis, A. M. Positive tropical marine low-cloud cover feedback inferred from cloud-controlling factors. Geophys. Res. Lett. 42, 7767–7775 (2015).

  74. 74.

    O’Gorman, P. A. Sensitivity of tropical precipitation extremes to climate change. Nat. Geosci. 5, 697–700 (2012). Emergent constraint on changing hydrologic extremes.

  75. 75.

    Lin, Y. L. et al. Causes of model dry and warm bias over central US and impact on climate projections. Nat. Commun. 8, 881 (2017).

  76. 76.

    Bowman, K. W. et al. Evaluation of ACCMIP outgoing longwave radiation from tropospheric ozone using TES satellite observations. Atmos. Chem. Phys. 13, 4057–4072 (2013).

  77. 77.

    Bracegirdle, T. J. & Stephenson, D. B. On the robustness of emergent constraints used in multimodel climate change projections of Arctic warming. J. Climate 26, 669–678 (2013).

  78. 78.

    Chadburn, S. E. et al. An observation-based constraint on permafrost loss as a function of global warming. Nat. Clim. Change 7, 340–344 (2017). Emergent constraint based on spatial rather than temporal variability.

  79. 79.

    Wenzel, S., Cox, P. M., Eyring, V. & Friedlingstein, P. Projected land photosynthesis constrained by changes in the seasonal cycle of atmospheric CO2. Nature 538, 499–501 (2016).

  80. 80.

    Kwiatkowski, L. et al. Emergent constraints on projections of declining primary production in the tropical oceans. Nat. Clim. Change 7, 355–358 (2017).

Download references


A.H. is supported by the National Science Foundation under Grant No. 1543268, and the U.S. Department of Energy Regional and Global Climate Modelling Program contract B618798:2. P.C. is supported by the European Research Council (ERC) ECCLES project (agreement number 742472) and the EU Horizon2020 CRESCENDO project (agreement number 641816). C.H. is supported by the NERC CEH National Capability Fund. S.K. is supported by the Regional and Global Climate Modelling Program of the Office of Science of the U.S. Department of Energy under the auspices of the U.S. DOE by LLNL under contract DE-AC52-07NA27344. This paper benefited from discussions at the Aspen Global Change Institute (AGCI) in 2017, during a model evaluation workshop that was attended by A.H., P.C. and S.K.

Author information


  1. Department of Atmospheric and Oceanic Sciences, University of California — Los Angeles, Los Angeles, CA, USA

    • Alex Hall
  2. College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK

    • Peter Cox
  3. Centre for Ecology and Hydrology, Wallingford, UK

    • Chris Huntingford
  4. PCMDI, Lawrence Livermore National Laboratory, Livermore, CA, USA

    • Stephen Klein


  1. Search for Alex Hall in:

  2. Search for Peter Cox in:

  3. Search for Chris Huntingford in:

  4. Search for Stephen Klein in:


A.H. drafted large portions of the paper, informed by discussions with C.H., P.C., and S.K., and an earlier manuscript drafted mainly by C.H. C.H., P.C. and S.K. each also drafted pieces of the paper. AH revised the paper in response to reviewer comments, after gathering feedback from C.H., P.C., and S.K. C.H. managed the references throughout the drafting process.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Alex Hall.

About this article

Publication history




Issue Date