Abstract
The Intergovernmental Panel on Climate Change (IPCC) reports provide policy-relevant insights about climate impacts, vulnerabilities and adaptation through a process of peer-reviewed literature assessments underpinned by expert judgement. An iconic output from these assessments is the burning embers diagram, first used in the Third Assessment Report to visualize reasons for concern, which aggregate climate-change-related impacts and risks to various systems and sectors. These burning embers use colour transitions to show changes in the assessed level of risk to humans and ecosystems as a function of global mean temperature. In this Review, we outline the history and evolution of the burning embers and associated reasons for concern framework, focusing on the methodological approaches and advances. While the assessment framework and figure design have been broadly retained over time, refinements in methodology have occurred, including the consideration of different risks, use of confidence statements, more formalized protocols and standardized metrics. Comparison across reports reveals that the risk level at a given temperature has generally increased with each assessment cycle, reflecting accumulating scientific evidence. For future assessments, an explicit, transparent and systematic process of expert elicitation is needed to enhance comparability, quality and credibility of burning embers.
Key points
-
The Intergovernmental Panel on Climate Change (IPCC) has used the reasons for concern framework and burning embers diagrams since 2001 to assess and communicate risks from increasing global mean temperature on human and natural systems.
-
The framework and burning embers diagrams are developed using expert judgement, based on available information about climate impacts.
-
While assessment methods and figure design have been broadly retained across IPCC reports, risk levels at given temperatures have generally increased over time, owing to more comprehensive science.
-
Structured expert-elicitation methods can reduce bias and increase reproducibility by specifying the process for selecting experts, providing external information, eliciting individual and consensus judgements, and facilitating group interaction.
-
Recent IPCC Special Reports introduced formal protocols and standardized metrics to elicit risk thresholds. Despite challenges, these changes contributed to transparency and reliability.
-
Further development and use of standardized and transparent methods to elicit risk thresholds and build burning embers diagrams will continue to increase the robustness and credibility of IPCC assessments.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Lenton, T. M. et al. Climate tipping points — too risky to bet against. Nature 575, 592–595 (2019).
United Nations. United Nations Framework Convention on Climate Change (1992).
Garner, G., Reed, P. & Keller, K. Climate risk management requires explicit representation of societal trade-offs. Clim. Change 134, 713–723 (2016).
Nordhaus, W. D. Managing the Global Commons: The Economics of Climate Change (MIT Press, 1994).
Leemans, R. & Vellinga, P. The scientific motivation of the internationally agreed ‘well below 2 °C’ climate protection target: a historical perspective. Curr. Opin. Environ. Sustain. 26–27, 134–142 (2017).
Fischlin, A., Ji, Z., Vladu, F. & Bisiaux, A. Report on the Structured Expert Dialogue on the 2013–2015 Review of the United Nations Framework Convention on Climate Change (UNFCCC). Final Report FCCC/SB/2015/INF.1 (UNFCCC Secretariat, 2015).
Smith, J. et al. in Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds. McCarthy, J. J., Canziani, O. F., Leary, N. A., Dokken, D. J. & White, K. S.) 913–967 (Cambridge Univ. Press, 2001).
McCarthy J. J., Canziani O., Leary N. A., Dokken D. J. & White K. S. (eds) Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (Cambridge Univ. Press, 2001).
O’Neill, B. C. et al. IPCC reasons for concern regarding climate change risks. Nat. Clim. Change 7, 28–37 (2017).
Matthews, J. B. R. in Global Warming of 1.5°C. An IPCC Special Report on the Impacts of Global Warming of 1.5°C Above Pre-industrial Levels and Related Global Greenhouse Gas Emission Pathways, in the Context of Strengthening the Global Response to the Threat of Climate Change, Sustainable Development, and Efforts to Eradicate Poverty (eds Masson-Delmotte, V. et al.) 539–562 (IPCC, 2018).
Paris Agreement. United Nations Treaty Collection. United Nations https://treaties.un.org/doc/Treaties/2016/02/20160215%2006-03%20PM/Ch_XXVII-7-d.pdf (2016).
Intergovernmental Panel on Climate Change (IPCC). Chapter Outline of the Working Group II Contribution to the IPCC Sixth Assessment Report (AR6). 46th Session of the IPCC (IPCC, 2017).
Mahony, M. Climate change and the geographies of objectivity: the case of the IPCC’s burning embers diagram. Trans. Inst. Br. Geogr. 40, 153–167 (2015).
Oppenheimer, M., Little, C. M. & Cooke, R. M. Expert judgement and uncertainty quantification for climate change. Nat. Clim. Change 6, 445–451 (2016).
Budescu, D. V., Por, H. H. & Broomell, S. B. Effective communication of uncertainty in the IPCC reports. Clim. Change 113, 181–200 (2012).
Swart, R., Bernstein, L., Ha-Duong, M. & Petersen, A. Agreeing to disagree: uncertainty management in assessing climate change, impacts and responses by the IPCC. Clim. Change 92, 1–29 (2009).
Socolow, R. H. High-consequence outcomes and internal disagreements: tell us more, please. Clim. Change 108, 775–790 (2011).
Adler, C. E. & Hirsch Hadorn, G. The IPCC and treatment of uncertainties: topics and sources of dissensus. Wiley Interdiscip. Rev. Clim. Change 5, 663–676 (2014).
Sutton, R. T. ESD Ideas: A simple proposal to improve the contribution of IPCC WGI to the assessment and communication of climate change risks. Earth Syst. Dyn. 9, 1155–1158 (2018).
Yohe, G. & Oppenheimer, M. Evaluation, characterization, and communication of uncertainty by the Intergovernmental Panel on Climate Change — an introductory essay. Clim. Change 108, 629–639 (2011).
Moss, R. H. & Schneider, S. H. in Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC (eds Pachauri, R., Taniguchi, T. & Tanaka, K.) 33–51 (World Meteorological Organization, 2000).
Manning, M. et al. (eds) Describing Scientific Uncertainties in Climate Change to Support Analysis of Risk and of Options. Report of IPCC Workshop 11–13 (IPCC, 2004).
Mastrandrea, M. D. et al. Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties (IPCC, 2010).
Intergovernmental Panel on Climate Change (IPCC). Climate Change and Land: An IPCC Special Report on Climate Change, Desertification, Land Degradation, Sustainable Land Management, Food Security, and Greenhouse Gas Fluxes in Terrestrial Ecosystems (eds Shukla, P. R. et al) (IPCC, 2019).
Intergovernmental Panel on Climate Change (IPCC). in IPCC Special Report on the Ocean and Cryosphere in a Changing Climate (eds Pörtner, H.-O. et al.) (IPCC, 2019).
Jones, J. & Hunter, D. Qualitative research: consensus methods for medical and health services research. BMJ 311, 376–380 (1995).
Peel, A. et al. Use of expert judgement across NICE guidance-making programmes: A review of current processes and suitability of existing tools to support the use of expert elicitation. Appl. Health Econ. Health Policy 16, 819–836 (2018).
Waggoner, J., Carline, J. D. & Durning, S. J. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad. Med. 91, 663–668 (2016).
Butler, A. J., Thomas, M. K. & Pintar, K. D. M. Systematic review of expert elicitation methods as a tool for source attribution of enteric illness. Foodborne Pathog. Dis. 12, 367–382 (2015).
Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111, 7176–7184 (2014).
Aspinall, W. P. & Cooke, R. M. in Quantifying scientific uncertainty from expert judgement elicitation. In Risk and Uncertainty Assessment for Natural Hazards (eds Rougier, J., Sparks, S. & Hill, L. J.) 64-99 (Cambridge Univ. Press, 2011).
Graefe, A. & Armstrong, J. S. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. Int. J. Forecast. 27, 183–195 (2011).
Black, N. et al. Consensus development methods: a review of best practice in creating clinical guidelines. J. Health Serv. Res. Policy 4, 236–248 (1999).
Tol, R. S. J. Equitable cost-benefit analysis of climate change policies. Ecol. Econ. 36, 71–85 (2001).
Tol, R. S. J. Estimates of the damage costs of climate change: part 1: benchmark estimates. Environ. Resour. Econ. 21, 47–73 (2002).
Tol, R. S. J. Estimates of the damage costs of climate change: Part II. Dynamic estimates. Environ. Resour. Econ. 21, 135–160 (2002).
Intergovernmental Panel on Climate Change (IPCC). Climate Change 2001: Synthesis report (Cambridge Univ. Press, 2001).
Caddy, J. F. Limit reference points, traffic lights, and holistic approaches to fisheries management with minimal stock assessment input. Fish. Res. 56, 133–137 (2002).
Wilson, D. C. & Pascoe, S. Chapter 13 Delivering complex scientific advice to multiple stakeholders. Dev. Aquacult. Fish. Sci. 36, 329–353 (2006).
Bystrom, A. B., Naranjo-Madrigal, H. & Wehrtmann, I. S. Recomendaciones de manejo basadas en indicadores para la pesca artesanal con líneas de fondo en Costa Rica, Centroamérica. Rev. Biol. Trop. 65, 475–493 (2017).
Simms, R. A. et al. Development of maternity dashboards across a UK health region; current practice, continuing problems. Eur. J. Obstet. Gynecol. Reprod. Biol. 170, 119–124 (2013).
Donnelly, L. & Harrison, M. Geomorphological and geoforensic interpretation of maps, aerial imagery, conditions of diggability and the colour-coded RAG prioritization system in searches for criminal burials. Geol. Soc. Spec. Publ. 384, 174–194 (2013).
Ruffell, A. & McAllister, S. RAG system for the management forensic and archaeological searches of burial grounds. Int. J. Archaeol. 3, 1–8 (2015).
Mahony, M. & Hulme, M. The colour of risk: An exploration of the IPCC’s “burning embers” diagram. Spont. Gen. 6, 75–89 (2012).
Vellinga, P. & Swart, R. The greenhouse marathon: a proposal for a global strategy. Clim. Change 18, vii–xii (1991).
Schneider, B. Burning worlds of cartography: a critical approach to climate cosmograms of the Anthropocene. Geo Geogr. Env. 3, e00027 (2016).
White, K. S. et al. Technical summary in Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (eds McCarthy, J. J., Canziani, O. F., Leary, N. A. & Dokken, D. J.) 19–73 (Cambridge Univ. Press, 2001).
Mastrandrea, M. D. & Schneider, S. H. Probabilistic integrated assessment of “dangerous” climate change. Science 304, 571–575 (2004).
Gattuso, J. P. et al. Contrasting futures for ocean and society from different anthropogenic CO2 emissions scenarios. Science 349, aac4722 (2015).
Magnan, A. K. et al. Implications of the Paris agreement for the ocean. Nat. Clim. Change 6, 732–735 (2016).
Fischlin, A. Berücksichtigen wir in der Klimapolitik genügend Sicherheitsmargen? Do we have sufficient safety margins in climate policy? GAIA Ecol. Perspect. Sci. Soc. 18, 193–199 (2009).
Schneider, S. H. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 779–810 (Cambridge Univ. Press, 2007).
Smith, J. B. et al. Assessing dangerous climate change through an update of the Intergovernmental Panel on Climate Change (IPCC) “reasons for concern”. Proc. Natl Acad. Sci. USA 106, 4133–4137 (2009).
Intergovernmental Panel on Climate Change (IPCC). in Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Solomon, S. et al.) 1–18 (Cambridge Univ. Press, 2007).
Intergovernmental Panel on Climate Change (IPCC) in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 7–22 (Cambridge Univ. Press, 2007).
Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 843–868 (Cambridge Univ. Press, 2007).
Fischlin, A. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 4 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 211–272 (Cambridge Univ. Press, 2007).
Hennessy, K. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 11 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 507–540 (Cambridge Univ. Press, 2007).
Intergovernmental Panel on Climate Change (IPCC). in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 23–78 (Cambridge Univ. Press, 2007).
Oppenheimer, M. et al. in Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds Field, C. B. et al.) 1039–1099 (Cambridge University Press, 2014).
Intergovernmental Panel on Climate Change (IPCC). Climate Change 2014: Synthesis Report. Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (eds Core Writing Team, Pachauri, R. K. and Meyer, L. A.) (IPCC, 2014).
Field, C. B. et al. Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation: Special Report of the Intergovernmental Panel on Climate Change (Cambridge Univ. Press, 2012).
Intergovernmental Panel on Climate Change (IPCC). in Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation. A Special Report of Working Groups I and II of the Intergovernmental Panel on Climate Change (eds Field, C. B. et al) 1–19 (Cambridge Univ. Press, 2012).
Hoegh-Guldberg, O. et al. in Global Warming of 1.5°C. An IPCC Special Report on the Impacts of Global Warming of 1.5 °C Above Pre-industrial Levels and Related Global Greenhouse Gas Emission Pathways, in the Context of Strengthening the Global Response to the Threat of Climate Change, Sustainable Development, and Efforts to Eradicate Poverty Ch. 3 (eds Masson-Delmotte, V. et al.) 175–311 (IPCC, 2018).
Palazzo, A. et al. Linking regional stakeholder scenarios and shared socioeconomic pathways: quantified West African food and climate futures in a global context. Glob. Environ. Change 45, 227–242 (2017).
Byers, E. et al. Global exposure and vulnerability to multi-sector development and climate change hotspots. Environ. Res. Lett. 13, 055012 (2018).
Hanasaki, N. et al. A global water scarcity assessment under Shared Socio-economic Pathways - Part 2: Water availability and scarcity. Hydrol. Earth Syst. Sci. 17, 2393–2413 (2013).
Hanasaki, N. et al. A global water scarcity assessment under Shared Socio-economic Pathways - Part 1: Water use. Hydrol. Earth Syst. Sci. 17, 2375–2391 (2013).
Oppenheimer, M. et al. in IPCC Special Report on the Ocean and Cryosphere in a Changing Climate Ch. 4 (eds Pörtner, H.-O. et al.) 321–445 (IPCC, 2019).
World Meteorological Organization. United In Science: High-Level Synthesis Report of Latest Climate Science Information Convened by the Science Advisory Group of the UN Climate Action Summit 2019 (World Meteorological Organization, 2019).
Robinson, A., Calov, R. & Ganopolski, A. Multistability and critical thresholds of the Greenland ice sheet. Nat. Clim. Change 2, 429–432 (2012).
Levermann, A. et al. The multimillennial sea-level commitment of global warming. Proc. Natl Acad. Sci. USA 110, 13745–13750 (2013).
Levermann, A. et al. Projecting Antarctic ice discharge using response functions from SeaRISE ice-sheet models. Earth Syst. Dyn. 5, 271–293 (2014).
Golledge, N. R. et al. The multi-millennial Antarctic commitment to future sea-level rise. Nature 526, 421–425 (2015).
Fürst, J. J., Goelzer, H. & Huybrechts, P. Ice-dynamic projections of the Greenland ice sheet in response to atmospheric and oceanic warming. Cryosphere 9, 1039–1062 (2015).
Pattyn, F. et al. The Greenland and Antarctic ice sheets under 1.5 °C global warming. Nat. Clim. Change 8, 1053–1061 (2018).
Pörtner, H.-O. et al. in Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 9 (eds Field, C. B. et al.) 411–484 (Cambridge Univ. Press, 2014).
National Institute for Health and Care Excellence (NICE). Developing NICE Guidelines: The Manual. Process and Methods Guides (NICE, 2014).
National Institute for Health and Care Excellence (NICE). Guide to the Methods of Technology Appraisal 2013 (NICE, 2013).
Stafford, A. C., Bindoff, I. K., Tenni, P. C., Peterson, G. M. & Doran, C. M. A methodological framework for estimating the clinical and economic value of community pharmacists’ clinical interventions using expert opinion. J. Clin. Pharm. Ther. 37, 378–385 (2012).
Khodyakov, D. et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med. Res. Methodol. 11, 174 (2011).
Sinha, I. P., Smyth, R. L. & Williamson, P. R. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 8, e1000393 (2011).
Moher, D., Schulz, K. F., Simera, I. & Altman, D. G. Guidance for developers of health research reporting guidelines. PLoS Med. 7, e1000217 (2010).
Ferri, C. P. et al. Global prevalence of dementia: a Delphi consensus study. Lancet 366, 2112–2117 (2005).
Cresswell, K. M. et al. Global research priorities to better understand the burden of iatrogenic harm in primary care: an international Delphi exercise. PLoS Med. 10, e1001554 (2013).
Hasson, F., Keeney, S. & McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs. 32, 1008–1015 (2000).
Shrader-Frechette, K. Experts in Uncertainty: Opinion and Subjective Probability in Science. Ethics Vol. 103 (ed. Cooke, R. M.) 599–601 (Oxford Univ. Press, 1993).
Ferguson, J. H. The NIH consensus development program: the evolution of guidelines. Int. J. Technol. Assess. Health Care 12, 460–474 (1996).
Dalkey, N. & Helmer, O. An experimental application of the DELPHI method to the use of experts. Manag. Sci. 9, 458–467 (1963).
Delbecq, A. L. & Van de Ven, A. H. A group process model for problem identification and program planning. J. Appl. Behav. Sci. 7, 466–492 (1971).
Dias, L. C., Morton, A. & Quigley, J. in Elicitation: the Science and Art of Structuring Judgement (eds Dias, L. C., Morton, A. & Quigley, J.) 1–14 (Springer, 2017).
James, D. & Warren-Forward, H. Research methods for formal consensus development. Nurse Res. 22, 35–40 (2015).
Clemen, R. T. & Winkler, R. L. Combining probability distributions from experts in risk analysis. Risk Anal. 19, 187–203 (1999).
European Food Safety Authority. Guidance on expert knowledge elicitation in food and feed safety risk assessment. EFSA J. 12, 3734 (2014).
Walker, K. D., Evans, J. S. & MacIntosh, D. Use of expert judgment in exposure assessment. Part I. Characterization of personal exposure to benzene. J. Expo. Sci. Environ. Epidemiol. 11, 308–322 (2001).
McCormack, C. G. et al. Key impacts of climate engineering on biodiversity and ecosystems, with priorities for future research. J. Integr. Environ. Sci. 13, 103–128 (2016).
Griscom, B. W. et al. Natural climate solutions. Proc. Natl Acad. Sci. USA 114, 11645–11650 (2017).
Mach, K. J. et al. Climate as a risk factor for armed conflict. Nature 571, 193–197 (2019).
Bamber, J. L. & Aspinall, W. P. An expert judgement assessment of future sea level rise from the ice sheets. Nat. Clim. Change 3, 424–427 (2013).
Mukherjee, N. et al. The Delphi technique in ecology and biological conservation: Applications and guidelines. Methods Ecol. Evol. 6, 1097–1109 (2015).
Bhave, A. G., Conway, D., Dessai, S. & Stainforth, D. A. Water resource planning under future climate and socioeconomic uncertainty in the Cauvery River Basin in Karnataka, India. Water Resour. Res. 54, 708–728 (2018).
Zommers, Z. & Alverson, K. in Resilience: The Science of Adaptation to Climate Change (eds Zommers, Z. & Alverson, K.). 329–336 (Elsevier, 2018).
Berrang-Ford, L., Pearce, T. & Ford, J. D. Systematic review approaches for climate change adaptation research. Reg. Environ. Change 15, 755–769 (2015).
Zickfeld, K., Morgan, M. G., Frame, D. J. & Keith, D. W. Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proc. Natl Acad. Sci. USA 107, 12451–12456 (2010).
Kriegler, E., Hall, J. W., Held, H., Dawson, R. & Schellnhuber, H. J. Imprecise probability assessment of tipping points in the climate system. Proc. Natl Acad. Sci. USA 106, 5041–5046 (2009).
Bantel, K. A. Comprehensiveness of strategic planning: the importance of heterogeneity of a top team. Psychol. Rep. 73, 35–49 (1993).
Bolger, F. & Wright, G. Improving the Delphi process: lessons from social psychological research. Technol. Forecast. Soc. Change 78, 1500–1513 (2011).
Hurlbert, M. et al. in Climate Change and Land: An IPCC Special Report on Climate Change, Desertification, Land Degradation, Sustainable Land Management, Food Security, and Greenhouse Gas Fluxes in Terrestrial Ecosystems Ch. 7 (eds Shukla, P. R. et al.) 673–800 (IPCC, 2019).
Jairath, N. & Weinstein, J. The Delphi methodology (part two): a useful administrative approach. Can. J. Nurs. Adm. 7, 7–20 (1994).
Bolger, F., Stranieri, A., Wright, G. & Yearwood, J. Does the Delphi process lead to increased accuracy in group-based judgmental forecasts or does it simply induce consensus amongst judgmental forecasters? Technol. Forecast. Soc. Change 78, 1671–1680 (2011).
Singh, G. G. et al. Group elicitations yield more consistent, yet more uncertain experts in understanding risks to ecosystem services in New Zealand bays. PLoS One 12, e0190326 (2017).
Speirs-Bridge, A. et al. Reducing overconfidence in the interval judgments of experts. Risk Anal. 30, 512–523 (2010).
Sperber, D., Mortimer, D., Lorgelly, P. & Berlowitz, D. An expert on every street corner? Methods for eliciting distributions in geographically dispersed opinion pools. Value Health 16, 434–437 (2013).
Adger, W. N., Brown, I. & Surminski, S. Advances in risk assessment for climate change adaptation policy. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 376, 20180106 (2018).
Yohe, G. “Reasons for concern” (about climate change) in the United States. Clim. Change 99, 295–302 (2010).
Warszawski, L. et al. The inter-sectoral impact model intercomparison project (ISI–MIP): Project framework. Proc. Natl Acad. Sci. USA 111, 3228–3232 (2014).
Frieler, K. et al. Assessing the impacts of 1.5°C global warming – simulation protocol of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2b). Geosci. Model Dev. 10, 4321–4345 (2017).
Sharpe, S. Telling the boiling frog what he needs to know: why climate change risks should be plotted as probability over time. Geosci. Commun. 2, 95–100 (2019).
Colson, A. R. & Cooke, R. M. Expert elicitation: using the classical model to validate experts’ judgments. Rev. Environ. Econ. Policy 12, 113–132 (2018).
Cooke, R., Mendel, M. & Thijs, W. Calibration and information in expert resolution; a classical approach. Automatica 24, 87–93 (1988).
Field, C. B. et al. (eds) Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (Cambridge Univ. Press, 2014).
Acknowledgements
The authors thank J. Smith for useful reviews and informed comments on the paper. A.K.M. thanks the French National Research Agency (‘Investments for the Future’ programme, ANR-10-LABX-14-01). All views expressed herein are the responsibility of the authors and do not necessarily reflect their respective institutions, the Intergovernmental Panel on Climate Change or the United Nations.
Author information
Authors and Affiliations
Contributions
Z.Z. and W.T. designed the study. Z.Z., E.L.D., W.T., K.C., M. Howden, K.W., J.P.E. and S.G. developed the improved expert-elicitation procedure for the SRCCL. A.K.M., H.-O.P. and Z.S. contributed to the SROCC burning embers development. P.M. harmonized the burning embers for three RFCs across the IPCC reports and developed figures 2 and 3. M. Howden, P.M. and W.T. led the analysis and design of figure 5. Z.Z.I., A.F. and C.R. contributed substantial additions to the history section. B.C.O. and P.M. helped research and draft the section on changes in risks. S.G., M. Hulbert and Z.Z.I. wrote the section on expert elicitation. Z.Z. led the drafting of the paper. All authors contributed to the editing and revision of the manuscript and helped develop conclusions.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information
Nature Reviews Earth & Environment thanks Kristie Ebi, Michael Mastrandrea and Michael Oppenheimer for their contribution to the peer review of this work.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Related links
The Ember Factory in the Python Package Index: https://pypi.org/project/EmberFactory/
The Ember Factory online application: https://climrisk.org/emberfactory
Supplementary information
Rights and permissions
About this article
Cite this article
Zommers, Z., Marbaix, P., Fischlin, A. et al. Burning embers: towards more transparent and robust climate-change risk assessments. Nat Rev Earth Environ 1, 516–529 (2020). https://doi.org/10.1038/s43017-020-0088-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s43017-020-0088-0
This article is cited by
-
Climatic risks to adaptive capacity
Mitigation and Adaptation Strategies for Global Change (2024)
-
Envisioning a future with climate change
Nature Climate Change (2023)
-
Status of global coastal adaptation
Nature Climate Change (2023)
-
The politics of climate risk assessment
npj Climate Action (2023)
-
Climate change exacerbates snow-water-energy challenges for European ski tourism
Nature Climate Change (2023)