Structural differences among models account for much of the uncertainty in projected climate changes, at least until the mid-twenty-first century. Recent observations encompass too limited a range of climate variability to provide a robust test of the ability to simulate climate changes. Past climate changes provide a unique opportunity for out-of-sample evaluation of model performance. Palaeo-evaluation has shown that the large-scale changes seen in twenty-first-century projections, including enhanced land–sea temperature contrast, latitudinal amplification, changes in temperature seasonality and scaling of precipitation with temperature, are likely to be realistic. Although models generally simulate changes in large-scale circulation sufficiently well to shift regional climates in the right direction, they often do not predict the correct magnitude of these changes. Differences in performance are only weakly related to modern-day biases or climate sensitivity, and more sophisticated models are not better at simulating climate changes. Although models correctly capture the broad patterns of climate change, improvements are required to produce reliable regional projections.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Taylor, K. E., Stouffer, R. J. & Meehl, G. A. An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 93, 485–498 (2012).
Flato, G. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) 741–866 (IPCC, Cambridge Univ. Press, 2013).
Kirtman, B. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) 953–1028 (IPCC, Cambridge Univ. Press, 2013).
Collins, M. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) 1029–1136 (IPCC, Cambridge Univ. Press, 2013).
Hawkins, E. & Sutton, R. The potential to narrow uncertainty in regional climate predictions. Bull. Am. Meteorol. Soc. 90, 1095–1107 (2009).
Hawkins, E. & Sutton, R. The potential to narrow uncertainty in projections of regional precipitation change. Clim. Dynam. 37, 407–418 (2011).
Braconnot, P. et al. Evaluation of climate models using palaeoclimatic data. Nature Clim. Change 2, 417–424 (2012).
MARGO Project Members Constraints on the magnitude and patterns of ocean cooling at the Last Glacial Maximum. Nature Geosci. 2, 127–132 (2009).
Bartlein, P. J. et al. Pollen-based continental climate reconstructions at 6 and 21 ka: A global synthesis. Clim. Dynam. 37, 775–802 (2011).
Joussaume, S. et al. Monsoon changes for 6000 years ago: Results of 18 simulations from the Paleoclimate Modelling Intercomparison Project (PMIP). Geophys. Res. Lett. 26, 859–862 (1999).
Pinot, S. et al. Tropical palaeoclimates at the Last Glacial Maximum: Comparison of Paleoclimate Modelling Intercomparison Project (PMIP) simulations and paleodata. Clim. Dynam. 15, 857–874 (1999).
Braconnot, P. et al. Results of PMIP2 coupled simulations of the mid-Holocene and Last Glacial Maximum—Part 1: Experiments and large-scale features. Clim. Past 3, 261–277 (2007).
Harrison, S. P. et al. Model benchmarking with glacial and mid-Holocene climates. Clim. Dynam. 43, 671–688 (2014).
Joshi, M. M., Lambert, F. H. & Webb, M. J. An explanation for the difference between twentieth and twenty-first century land–sea warming ratio in climate models. Clim. Dynam. 41, 1853–1869 (2013).
Pithan, F. & Mauritsen, T. Arctic amplification dominated by temperature feedbacks in contemporary climate models. Nature Geosci. 7, 181–184 (2014).
Izumi, K., Bartlein, P. J. & Harrison, S. P. Consistent behaviour of the climate system in response to past and future forcing. Geophys. Res. Lett. 40, 1–7 (2013).
Hargreaves, J. C. & Annan, J. D. Can we trust climate models? WIREs Clim. Change 5, 435–440 (2014).
Schmidt, G. A. et al. Using paleo-climate comparisons to constrain future projections in CMIP5. Clim. Past 10, 221–250 (2014).
Lunt, D. J. et al. A multi-model assessment of last interglacial temperatures. Clim. Past 9, 699–717 (2013).
Hill, D. J. et al. Evaluating the dominant components of warming in Pliocene climate simulations. Clim. Past 10, 79–90 (2014).
Salzmann, U. et al. Challenges in quantifying Pliocene terrestrial warming revealed by data–model discord. Nature Clim. Change 3, 969–974 (2013).
Miller, G. H., Lehman, S. J., Refsnider, K. A., Southon, J. R. & Zhong, Y. Unprecedented recent summer warmth in Arctic Canada. Geophys. Res. Lett. 40, 5745–5751 (2013).
Izumi, K., Bartlein, P. J. & Harrison, S. P. Energy-balance mechanisms underlying consistent large-scale temperature responses in warm and cold climates. Clim. Dynam. 44, 3111–3127 (2015).
Allan, R. P. Examination of relationships between clear-sky longwave radiation and aspects of the atmospheric hydrological cycle in climate models, reanalysis, and observations. J. Climate 22, 3127–3145 (2009).
Trenberth, K. E. & Shea, D. J. Relationships between precipitation and surface temperature. Geophys. Res. Lett. 32, L14703 (2005).
Lau, W. K. M., Wu, H-T. & Kim, K-M. A canonical response of precipitation characteristics to global warming from CMIP5 models. Geophys. Res. Lett. 40, 3163–3169 (2013).
Li, G., Harrison, S. P., Bartlein, P. J., Izumi, K. & Prentice, I. C. Precipitation scaling with temperature in warm and cold climates: An analysis of CMIP5 simulations. Geophs. Res. Lett. 40, 4018–4024 (2013).
Kutzbach, J. E. & Street-Perrott, F. A. Milankovitch forcing of fluctuations in the level of tropical lakes from 18 to 0 kyr BP. Nature 317, 130–134 (1985).
Wohlfahrt, J., Harrison, S. P. & Braconnot, P. Synergistic feedbacks between ocean and vegetation on mid- and high-latitude climates during the mid-Holocene. Clim. Dynam. 22, 223–238 (2004).
Marzin, C. & Braconnot, P. The role of the ocean feedback on Asian and African monsoon variations at 6 kyr and 9.5 kyr BP. C. R. Geosci. 341, 643–655 (2009).
Wang, T., Wang, H. J. & Jiang, D. B. Mid-Holocene East Asian summer climate as simulated by the PMIP2 models. Palaeogeogr. Palaeoclimatol. Palaeoecol. 288, 93–102 (2010).
Zhao, Y. & Harrison, S. P. Mid-Holocene monsoons: A multi-model analysis of the inter-hemispheric differences in the responses to orbital forcing and ocean feedbacks. Clim. Dynam. 39, 1457–1487 (2012).
Jiang, D., Lang, X., Tian, Z. & Ju, L. Mid-Holocene East Asian summer monsoon strengthening: Insights from Paleoclimate Modeling Intercomparison Project (PMIP) simulations. Palaeogeogr. Palaeoclimatol. Palaeoecol. 369, 422–429 (2013).
Jiang, D., Tian, Z. & Land, X. Mid-Holocene net precipitation changes over China: Model–data comparison. Quat. Sci. Rev. 82, 104–120 (2013).
Perez-Sanz, A., Li, G., Gonzalez, P. & Harrison, S. P. Evaluation of seasonal climates of northern Africa and the Mediterranean in the CMIP5 simulations. Clim. Past 10, 551–568 (2014).
Prado, L. F., Wainer, I. & Chiessi, C. M. Mid-Holocene PMIP3/CMIP5 model results: Intercomparison for the South American monsoon system. Holocene 21, 1915–1920 (2013).
Kutzbach, J. E., Bonan, G. B., Foley, J. A. & Harrison, S. P. Vegetation and soils feedbacks on the response of the African monsoon to orbital forcing in the early to middle Holocene. Nature 384, 623–626 (1996).
Claussen, M. & Gayler, V. The greening of the Sahara during the mid-Holocene: Results of an interactive atmosphere–biome model. Glob. Ecol. Biogeogr. Lett. 6, 369–377 (1997).
Broström, A. et al. Land surface feedbacks and palaeomonsoons in northern Africa. Geophys. Res. Lett. 25, 3615–3618 (1998).
Braconnot, P., Joussaume, S., Marti, O. & de Noblet, N. Synergistic feedbacks from ocean and vegetation on the African monsoon response to mid-Holocene insolation. Geophys. Res. Lett. 26, 2481–2484 (1999).
Otto, J., Raddatz, T., Claussen, M., Brovkin, V. & Gayler, V. Separation of atmosphere–ocean–vegetation feedbacks and synergies for mid-Holocene climate. Glob. Biogeochem. Cycles 23, L09701 (2009).
Claussen, M., Bathiany, S., Brovkin, V. & Kleinen, T. Simulated climate–vegetation interaction in semi-arid regions affected by plant diversity. Nature Geosci. 6, 954–958 (2013).
Levis, S., Bonan, G. B. & Bonfils, C. Soil feedback drives the mid-Holocene North African monsoon northward in fully coupled CCSM2 simulations with a dynamic vegetation model. Clim. Dynam. 23, 791–802 (2004).
Wang, Y. et al. Detecting vegetation–precipitation feedbacks in mid-Holocene North Africa from two climate models. Clim. Past 4, 59–67 (2008).
Tian, Z. & Jiang, D. Mid-Holocene ocean and vegetation feedbacks over East Asia. Clim. Past 9, 2153–2171 (2013).
Prentice, I. C., Liang, X., Medlyn, B. & Wang, Y. Reliable, robust and realistic: The three R's of next-generation land-surface modelling. Atmos. Chem. Phys. Discuss. 14, 24811–24861 (2014).
Roehrig, R., Bouniol, D., Guichard, F., Hourdin, F. & Redelsperger, J-L. The present and future of the West African monsoon: A process-oriented assessment of CMIP5 simulations along the AMMA transect. J. Climate 26, 6471–6505 (2013).
Zheng, W. & Braconnot, P. Characterization of model spread in PMIP2 mid-Holocene simulations of the African monsoon. J. Climate 26, 1192–1210 (2013).
Yu, G. & Harrison, S. P. An evaluation of the simulated water balance of Eurasia and northern Africa at 6000 yr BP using lake status data. Clim. Dynam. 12, 723–735 (1996).
Wohlfahrt, J. et al. Evaluation of coupled ocean–atmosphere simulations of Northern Hemisphere extratropical climates in the mid-Holocene. Clim. Dynam. 31, 871–890 (2008).
Mueller, B. & Seneviratne, S. I. Systematic land climate and evapotranspiration biases in CMIP5 simulations. Geophys. Res. Lett. 41, 128–134 (2014).
Mauri, A., Davis, B. A. S., Collins, P. M. & Kaplan, J. O. The influence of atmospheric circulation on the mid-Holocene climate of Europe: A data–model comparison. Clim. Past 10, 1925–1938 (2014).
Masson, V. et al. Mid-Holocene climate in Europe: What can we infer from PMIP model data comparisons? Clim. Dynam. 15, 163–182 (1999).
Brewer, S., Guiot, J. & Torre, F. Mid-Holocene climate change in Europe: A data–model comparison. Clim. Past 3, 499–512 (2007).
Davis, B. A. S. & Brewer, S. Orbital forcing and the role of the latitudinal temperature/insolation gradient. Clim. Dynam. 32, 143–165 (2009).
van Oldenborgh, G. J. et al. Western Europe is warming much faster than expected. Clim. Past 5, 1–12 (2009).
Berger, A., Brandefelt, J. & Nilsson, J. The sensitivity of the Arctic sea ice to orbitally induced insolation changes: A study of the mid-Holocene Paleoclimate Modelling Intercomparison Project 2 and 3 simulations. Clim. Past 9, 969–982 (2013).
Goosse, H., Roche, D. M., Mairesse, A. & Berger, M. Modelling past sea ice changes. Quat. Sci. Rev. 79, 191–206 (2013).
Chavaillaz, Y., Codron, F. & Kageyama, M. Southern westerlies in LGM and future (RCP4.5) climates. Clim. Past 9, 517–524 (2013).
Rojas, M. Sensitivity of Southern Hemisphere circulation to LGM and 4 × CO2 climates. Geophys. Res. Lett. 40, 965–970 (2013).
DiNezio, P. N. & Tierney, J. E. The effect of sea level on glacial Indo-Pacific climate. Nature Geosci. 6, 485–491 (2013).
Hargreaves, J. C., Annan, J. D., Ohgaito, R., Paul, A. & Abe-Ouchi, A. Skill and reliability of climate model ensembles at the Last Glacial Maximum and mid-Holocene. Clim. Past 9, 811–823 (2013).
Gleckler, P. J., Taylor, K. E. & Doutriaux, C, Performance metrics for climate models. J. Geophys. Res. 113, D06104 (2008).
Annan, J. D. & Hargreaves, J. C. Understanding the CMIP3 multimodel ensemble. J. Climate 24, 4529–4538 (2011).
Hessler, I. et al. Implication of methodological uncertainties for mid-Holocene sea surface temperature reconstructions. Clim. Past 10, 2237–2252 (2014).
Kageyama, M. et al. Mid-Holocene and Last Glacial Maximum climate simulations with the IPSL model – Part I: Comparing IPSL CM5A to IPSL CM4. Clim. Dynam. 40, 2447–2468 (2013).
Kageyama, M. et al. Mid-Holocene and Last Glacial Maximum climate simulations with the IPSL model: Part II: Model–data comparisons. Clim. Dynam. 40, 2469–2495 (2013).
Knutti, R. The end of model democracy? Clim. Change 102, 395–404 (2010).
Rauser, F., Gleckler, P. & Marotzke, J. Rethinking the default construction of multi-model climate ensembles. Bull. Am. Meteorol. Soc. http://dx.doi.org/10.1175/BAMS-D-13-00181.1 (2014).
PALAEOSENS project members Making sense of palaeoclimate sensitivity. Nature 491, 683–691 (2012).
Annan, J. D., Hargreaves, J. C., Ohgaito, R., Abe-Ouchi, A. & Emori, S. Efficiently constraining climate sensitivity with ensembles of paleoclimate simulations. SOLA 1, 181–184 (2005).
Crucifix, M. Does the Last Glacial Maximum constrain climate sensitivity? Geophys. Res. Lett. 33, L18701 (2006).
Schneider von Deimling, T., Held, H., Ganolpolski, A. & Rahmstorf, S. Climate sensitivity estimated from ensemble simulations of glacial climate. Clim. Dynam. 27, 149–163 (2006).
Hargreaves, J. C., Annan, J. D., Yoshimori, M. & Abe-Ouchi, A. Can the Last Glacial Maximum constrain climate sensitivity? Geophys. Res. Lett. 39, L24702 (2012).
Tripati, A. K. et al. Modern and glacial tropical snowlines controlled by sea surface temperature and atmospheric mixing. Nature Geosci. 7, 205–209 (2014).
Yoshimori, M., Hargreaves, J. C., Annan, J. D., Yokohata, T. & Abe-Ouchi, A. Dependency of feedbacks on forcing and climate state in physics parameter ensembles. J. Climate 24, 6440–6455 (2011).
Otto-Bliesner, B. L. et al. Last Glacial Maximum and Holocene climate in CCSM3. J. Climate 19, 2526–2544 (2006).
Schmittner, A. et al. Climate sensitivity estimated from temperature reconstructions of the Last Glacial Maximum. Science 334, 1385–1388 (2011).
Claquin, T. et al. Radiative forcing effect of ice-age dust. Clim. Dynam. 20, 193–202 (2003).
Hoelzmann, P. et al. Mid-Holocene land-surface conditions in northern Africa and the Arabian peninsula: A data set for the analysis of biogeophysical feedbacks in the climate system. Glob. Biogeochem. Cycles 12, 35–51 (1998).
Lézine, A-M., Hély, C., Grenier, C., Braconnot, P. & Krinner, G. Sahara and Sahel vulnerability to climate changes, lessons from Holocene hydrological data. Quat. Sci. Rev. 30, 3001–3012 (2011).
Elenga, H. et al. Pollen-based reconstruction for Southern Europe and Africa 18,000 years ago. J. Biogeogr. 27, 621–634 (2000).
Watrin, J., Lézine, A-M. & Hély, C. Plant migration and ecosystems at the time of the “green Sahara”. C. R. Geosci. 341, 656–670 (2009).
Kuper, R. & Kropelin, S. Climate-controlled Holocene occupation in the Sahara: Motor of Africa's evolution. Science 313, 803–807 (2006).
Manning, K. & Timpson, A. The demographic response to Holocene climate change in the Sahara. Quat. Sci. Rev. 101, 28–35 (2014).
Prentice, I. C., Harrison, S. P. & Bartlein, P. J. Global vegetation and terrestrial carbon cycle changes after the last ice age. New Phytol. 189, 988–998 (2011).
Coe, M. T. & Harrison, S. P. The water balance of northern Africa during the mid-Holocene: An evaluation of the 6ka BP PMIP experiments. Clim. Dynam. 19, 155–166 (2002).
Ward, P. J., Aerts, J. C. J. H., de Moel, H. & Renssen, H. Verification of a coupled climate–hydrological model against Holocene palaeohydrological records. Glob. Planet. Change 57, 283–300 (2007).
Haywood, A. M. et al. Pliocene Model Intercomparison Project (PlioMIP): Experimental design and boundary conditions (Experiment 1). Geosci. Model Dev. 3, 227–242 (2010).
Taylor, K. E. Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res. 106, 7183–7192 (2001).
Wilks, D. S. Statistical Methods in the Atmospheric Sciences 3rd edn (International Geophysics Series 100, Academic, 2011).
Wilks, D. S. On “field significance” and the false discovery rate. J. Appl. Meteorol. Climatol. 45, 1181–1189 (2006).
Joussaume, S. & Taylor, K. E. in Proc. 1st Int. AMIP Sci. Conf. 425–430 (WCRP Series Report 92, WMO, 1995).
Gates, W. L. AMIP: The Atmospheric Model Intercomparison Project. Bull. Am. Meteorol. Soc. 73, 1962–1970 (1992).
Abe-Ouchi, A. & Harrison, S. P. Constraining the carbon-cycle feedback using palaeodata: The PalaeoCarbon Modelling Intercomparison Project. Eos 90, 140 (2009).
This paper is a contribution to the ongoing work on the PMIP. We thank all of the modelling groups who have contributed to the CMIP5 archive. We acknowledge financial support from the Centre for Past Climate Change, University of Reading. G.L. was supported by an international postgraduate research scholarship at Macquarie University. P.J.B. and K.I. were supported by the US National Science Foundation paleoclimatology programme. M.K., P.B. and K.I. acknowledge financial support from Labex L-IPSL.
The authors declare no competing financial interests.
About this article
Cite this article
Harrison, S., Bartlein, P., Izumi, K. et al. Evaluation of CMIP5 palaeo-simulations to improve climate projections. Nature Clim Change 5, 735–743 (2015). https://doi.org/10.1038/nclimate2649
The contrasting effects of thermodynamic and dynamic processes on East Asian summer monsoon precipitation during the Last Glacial Maximum: a data-model comparison
Climate Dynamics (2021)
Advances in Atmospheric Sciences (2021)
Scientific Data (2020)
Climate Dynamics (2020)
Climate Dynamics (2020)