Estimating global mean sea-level rise and its uncertainties by 2100 and 2300 from an expert survey

Sea-level rise projections and knowledge of their uncertainties are vital to make informed mitigation and adaptation decisions. To elicit projections from members of the scientific community regarding future global mean sea-level (GMSL) rise, we repeated a survey originally conducted five years ago. Under Representative Concentration Pathway (RCP) 2.6, 106 experts projected a likely (central 66% probability) GMSL rise of 0.30–0.65 m by 2100, and 0.54–2.15 m by 2300, relative to 1986–2005. Under RCP 8.5, the same experts projected a likely GMSL rise of 0.63–1.32 m by 2100, and 1.67–5.61 m by 2300. Expert projections for 2100 are similar to those from the original survey, although the projection for 2300 has extended tails and is higher than the original survey. Experts give a likelihood of 42% (original survey) and 45% (current survey) that under the high-emissions scenario GMSL rise will exceed the upper bound (0.98 m) of the likely range estimated by the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, which is considered to have an exceedance likelihood of 17%. Responses to open-ended questions suggest that the increases in upper-end estimates and uncertainties arose from recent influential studies about the impact of marine ice cliff instability on the meltwater contribution to GMSL rise from the Antarctic Ice Sheet.


INTRODUCTION
Variability of global mean sea-level (GMSL) projections among studies, even when using the same emissions scenarios, has led to confusion among decision-making communities and the public 1,2 . The Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (AR5) projected GMSL by summing the contributions from physical processes to provide a likely (i.e., central 66% probability; see exegesis by ref. 3 ) estimate of GMSL rise of 0.52-0.98 m in the case of unmitigated growth of emissions (Representative Concentration Pathway [RCP] 8.5) by 2100, relative to 1986-2005 4 . However, substantial uncertainties surrounding the response of the Greenland and Antarctic Ice Sheets to dynamic processes such as marine ice sheet instability (MISI) and marine ice cliff instability (MICI) 5,6 indicate that the physical models used in process-based predictions of GMSL rise are incomplete 7 . Indeed, AR5 noted that additional GMSL rise up to several decimeters was possible due to MISI, but it was not included in the estimate of Antarctic Ice Sheet rapid dynamics due to likelihood of such a contribution being unquantified 4 . The recent IPCC Special Report on the Ocean and Cryosphere in a Changing Climate (SROCC) 8 included MISI (but not MICI). Consequently, estimates from SROCC for 2100 under RCP 8.5 are higher and the likely range larger than in AR5 (0.61-1.10 m). The majority of individual studies published since 2013 for high emission scenarios project GMSL rise >1 m by 2100 1 , including several that suggest the possibility of a rise >2 m 9,10 , in line with projections of semiempirical models [11][12][13][14] .
As an alternative to models, GMSL rise may be projected by surveying experts [15][16][17] . Broadly, two different approaches have been used to extract estimates of the likelihood of future GMSL from experts in the field. Structured expert judgment is a formal method in which a small number of experts are guided in the interpretation of probabilities in a workshop setting before having their responses weighted based on their performance on calibration questions. This method was used to assess the probability distribution of future ice-sheet changes 15,16 . More informal approaches, also known as "broad" elicitations or expert (surveys), ask many experts a small number of questions, aiming for wide participation by minimizing the required time investment for participation. Broad elicitations are appropriate for interdisciplinary problems that involve large uncertainties such as sea-level projection 18 . We previously surveyed experts 17 (hereafter H14) to provide a probabilistic assessment of sea-level rise by 2100 and 2300 under two contrasting temperature scenarios. Now is an appropriate time to repeat the survey of H14 because during the past five years, progress has been made in better understanding several processes expected to influence GMSL rise (e.g., MICI) and in advancing the methodologies for projecting GMSL rise. We retain the same aim as the original study: to survey experts in the scientific community about how much GMSL change they anticipate for two time periods (2000-2100 and 2000-2300) under two temperature scenarios (derived from the upper and lower extremes of the RCP 2.6 and RCP 8.5 scenarios; Fig. 1). We also inquire about the sources of experts' uncertainties when projecting GMSL. The results of this survey (hereafter H20) are compared with the H14 survey results and published literature to assess how and why the sea-level community's projections evolved during the past 5 years and if there is movement toward a 1 Asian School of the Environment, Nanyang Technological University, Singapore 639798, Singapore. 2 19 , where uncertainties accurately reflect current scientific knowledge.

Sea-level experts
The H14 survey identified the 500 individuals who (co-)authored the most publications on the subject of sea level during the period 2009-2013. This cohort of experts published a minimum of six papers (mean of 8.6 with a standard deviation of 3.3 and maximum of 33). In this survey, we retained the definition of an expert as someone who published at least six papers over the period 2014-2018 and 878 individuals met this criterion (mean of 11.5 publications with a standard deviation of 5.7 and maximum of 55). The number of papers published annually on the topic of "sea level" (term appearing in the title or listed as a keyword) more than doubled between 2009 and 2018 from 869 to 2012 ( Fig. 2a; see "Methods"). The subset of these papers containing the term "sea level" in the title almost doubled over the same period from 279 to 527. Notably, there was a spike in the number of papers published in 2013 that we attribute to authors completing publications in time for them to be incorporated in IPCC AR5. According to ref. 20 , the recent annual growth rate for publications  annually where the term "sea level" (and common variants) appears in the title of the paper (blue squares) and/or in the keywords (author defined or KeyWord Plus; red circles). For comparison, we predicted the number of papers published in each year since 2009 using an annual growth rate of 8.4%, which is the recent trend in natural science fields (ref. 20 ; dashed lines). b Number of (co) authors on sea-level publications (gray circles). Box plots represent the median (solid horizontal line), 25th and 75th percentiles (hinges), and the largest value no further than 1.5 times the interquartile range from the hinge (~95% confidence interval for comparing medians; whiskers). B.P. Horton et al. in the natural sciences is 8.4%. Therefore, growth in the number of publications about sea level is similar to wider publication trends in the natural sciences (Fig. 2a). The number of authors on sealevel papers is highly variable among publications, but shows a modest increase through time (Fig. 2b). The mean number of authors by year between 2009 and 2013 was 4.1 to 4.5, while between 2014 and 2018 it ranged from 4.7 to 5.2. We therefore conclude that the increase in the number of individuals who (co-) authored six or more publications about sea level in a 5-year period from H14 to H20 arose principally from a growing number of scientific papers published annually, but also because the average number of (co-)authors on sea-level papers increased by approximately one between 2009 and 2018.

Estimates of GMSL rise by 2100 and 2300
For GMSL projections, the experts provided two uncertainty ranges and a median estimate for RCP 2.6 (which limits warming to 1°C by 2100 and 1.2°C by 2300 relative to 1986-2005), and RCP 8.5 (which limits warming to 4.5°C by 2100 and 12.6°C in 2300 relative to 1986-2005). The inner GMSL projection uncertainty range (17th to 83rd percentiles) defines the central 66% probability range of GMSL, which corresponds to the "likely" ranges of GMSL rise provided in IPCC AR5 3 . The outer range (5th to 95th percentiles) defines the central 90% probability range of GMSL, which corresponds to the "very likely" ranges of GMSL rise provided in IPCC AR5 3 . Figure 3 presents box plots of the survey results for the 5th, 17th, 83rd, and 95th percentiles across all respondents (n = 106) to provide an overview of the variability in the quantitative information provided by the survey. Expert projections also informed probability density functions (PDFs; see Supplementary Information) whereby the expert percentile estimates are assumed to have been derived through a hierarchical structure from a latent distribution that provides the relative likelihood of GMSL under RCP 2.6 and RCP 8.  (Table 1).
Under RCP 8.5, the likely range of GMSL rise is 0.63-1.32 m, the very likely range is 0.45-1.65 m, and the median is 0.93 m by 2100. By 2300, the likely range is 1.67-5.61 m, the very likely range is 0.88-7.83 m, and the median is 3.29 m (Table 1).
We found no relationship between experts' GMSL projections and their self-reported career stage, discipline, or geographic location (Supplementary Information and Supplementary Figs. 1-3), which repeats the findings of H14. This is an important test for expert surveys, because it can reveal differences among the views of experts with different degrees of expertise, experience, or scientific standing.

Expert projections
A recent structured expert judgment 15 , focusing on ice sheet contributions to GMSL presented a likely range of 0.8-1.7 m, and a very likely range of 0.6-2.4 m for GMSL rise in 2100 under a high emission scenario that is roughly comparable to RCP 8.5. There are obvious differences between the ranges derived from the structured expert judgment and our expert survey, particularly in the magnitude of the upper tails of the distributions. The structured expert judgment 15 provided combined ice sheet sea-level rise contributions, which alone show a very likely range of 0.1-1.8 m by 2100 under the high emission scenario. This potentially suggests a difference between GMSL projections from experts with a primary expertise in ice sheets 15 , and GMSL projections from the field more broadly, as in our survey, where there was a relatively equal distribution of respondents with primary expertise in statistical or physical process modeling (32%), data collection or processing (35%), and impacts of sea-level rise (30%). From tiered questions, 13% of respondents in our survey identified ice-sheets as their expertise (Supplementary Information).  Expert projections for 2100 also show differences with peerreviewed literature regarding sea-level projection studies for 2100 under RCP 8.5. Reference 21 summarized recent literature projections of GMSL rise for 2100 and 2300. Across RCPs, median projections from peer-reviewed literature for GMSL rise by 2100 range from as low as 0.4 m under RCP 2.6 (e.g., ref. 22 ) to as high as 1.5 m under RCP 8.5 in simulations allowing for an aggressively unstable Antarctic Ice Sheet 10 . Assessing across studies yields at least a 90% probability of a GMSL rise of 0.2-1.0 m under RCP 2.6, and 0.4-2.4 m by 2100 under RCP 8.5 21 .
The expert projections for 2300 also differ from the small number of studies that presented projections this far into the future (e.g., refs. 9,23,24 ). Unsurprisingly, the difference among scenarios is extremely large by 2300-by which time, the extension of RCP 2.6 is characterized by an atmospheric CO 2 concentration lower than today, whereas the extension of RCP 8.5 causes an atmospheric CO 2 concentration of nearly 2000 ppm 25  The wide span of expert projections, particularly for RCP 8.5 in 2300, is immediately apparent and reflects the deep uncertainty in longer-term, high-emissions projections. Reference 15 , which saw long upper tails in distributions of total ice sheet contributions to GMSL rise, suggests that the long upper tail behavior reflects expert views that large amplitude, nonlinear instabilities could be triggered at higher temperatures. This viewpoint could be feeding into expert predictions of GMSL rise here also. From open-ended questions, we recorded 162 responses regarding the sources of uncertainty under the time period 2000-2100, and 160 responses under time period 2000-2300 ( Supplementary Fig. 4). Under GMSL rise for 2000-2100, ice sheets were the most common source of uncertainty for our experts (81%), followed by model/data limitations (8%), ocean-atmosphere (7%), and anthropogenic uncertainties (4%; see "Methods" for definitions). Under GMSL rise for 2000-2300, ice sheets remained the most common source of uncertainty (76%), followed model/data limitations and oceanatmosphere uncertainty (both at 6%), and anthropogenic uncertainty (5%).
The Antarctic and Greenland Ice Sheets are the largest potential contributors to GMSL rise, being the largest reservoirs of landbased ice on Earth (e.g., ref. 26 ). Experts who identified the Antarctic Ice Sheet as the greatest source of uncertainty account for 23% of responses for 2100 and 21% for 2300. The contribution from the Antarctic Ice Sheet is dominated by ice dynamic processes because ice shelves melt from below or disintegrate from surface meltwater penetrating crevasses, which reduce buttressing of the outlet glaciers flowing onto the ice shelves (e.g., refs. 27,28 ). The uncertainty arises from an imperfect understanding of Antarctic Ice Sheet processes and the internal variability of climate forcing of the ice sheet 29 . In particular, experts identified MISI and MICI as challenging to capture accurately in ice sheet models (e.g., refs. 30,31 ). The Greenland Ice Sheet was identified as the greatest source of uncertainty associated with the ice sheet contribution to GMSL for 18% of responses for 2100 and 13% for 2300. Uncertainty of the sea level contribution from the Greenland Ice Sheet is dominated by surface mass balance changes, with elevation and albedo positive feedbacks that amplify surface melting at the margins (e.g., refs. [32][33][34] ). Experts also attributed uncertainty about the contribution to GMSL rise from ice sheets to the influence of submarine basins, or feedbacks between ice sheets and ocean/ climate system (Supplementary Information).
Evolution of sea-level rise estimates and ranges More than 90 projections of GMSL rise were published since H14 1 . We can assess the influence of these publications on the expectations of the sea-level research community by comparing H20 and H14 because the main target questions and the temperature scenarios are identical ( Fig. 4 and Table 1). For the 2100 projections of GMSL rise under RCP 2.6 and RCP 8.5, the PDFs display similar distributional forms across both survey years. Notably, the tail of the PDF for RCP 2.6 in H14 extends slightly beyond the corresponding tails in the H20 survey, suggesting a slight increase in certainty about the 2100 GMSL projections among experts for a low-emissions scenario. These findings are consistent with the recently published database of GMSL projections from ref. 1 , which illustrated minimal change in the range of projections for 2100 under low-emissions scenarios since 2014.
The differences between the PDFs of H14 and H20 for 2300 projections across survey years are more distinguishable. The H20 results have extended tails and a clear shift upwards in anticipated GMSL rise as compared with H14 (Fig. 4). This is an indication that, although experts appear to be more uncertain about the extent of GMSL rise by 2300, overall projections increased. Similar to the 2100 results, these results appear consistent with recently published literature. Although far more publications focused on projecting GMSL for 2100 than 2300, recent studies suggested ranges of GMSL rise by 2300 that exceed the range provided in IPCC AR5, especially for high-emissions scenarios (e.g., refs. 9,23,24 ). Reference 15 concluded that the uncertainties related to ice sheets (particularly MICI), grew over the last 6 years, and that GMSL rise projections grew accordingly. Such uncertainty is most noticeable in the tails of projections, and on longer time scales (e.g., refs. 5,9 ).
IPCC reports tend to provide intentionally cautious and conservative estimates of GMSL rise 1 , rather than focusing on high-end, physically plausible sea-level rise risk 35 . IPCC AR5 and SROCC projected a likely upper limit of GMSL rise of 0.98 and 1.10 m, respectively, for RCP 8.5 by 2100; there is a corresponding 17% probability that GMSL rise will exceed this estimate 4 . From the PDFs (Fig. 4), we estimate "exceedance probabilities" for H14 and H20, defined as the probability of exceeding the upper bound of the likely range from IPCC AR5 and SROCC. According to the H20 survey PDF, there is a 45% probability that GMSL rise will exceed 0.98 m and a 33% probability that GMSL rise will exceed 1.10 m by 2100 under RCP 8.5. This finding is similar to H14, where the PDF suggests a 42% and 30% probability of exceedance, respectively. The broad elicitations of H14 and H20 suggest expert estimates of GMSL rise in 2100 are often higher than the upper bound of IPCC AR5 and SROCC.
Upper estimates for high emission scenarios Open-ended questions included in the survey regarding specific resources (e.g., academic literature, government reports, etc.) that were most influential to the professional judgments of the experts (Fig. 5) provide insight into the reasons why such a large proportion of experts believed that the IPCC AR5 and SROCC upper bounds for GMSL under RCP 8.5 would be exceeded by 2100. In total, respondents listed 79 references that influenced their estimates, of which 67 were reported for the 2100 scenario. The most cited reference for both 2100 (19%) and 2300 (13%) was ref. 5 , the only ice-sheet model that incorporates MICI. Reference 5 postulates that ice cliffs become unstable at elevations higher than ∼90 m above sea level, facilitating the collapse of ice sheets during warm periods (the Antarctic Ice Sheet is particularly vulnerable to this process). Reference 5 suggests that incorporating MICI improves agreement of their model with reconstructions of GMSL during the Last Interglacial, Pliocene, and Miocene 5,36 . However, it was shown that MICI is not necessarily required, and that better agreement might also be achieved with greater ocean B.P. Horton et al. warming and/or incorporation of meltwater-ocean feedbacks that increase ice loss 37,38 . Both refs. 37,38 feature in the list of cited resources from experts, but due to the brief period of time between publication and the H20 survey, their impact is potentially underrepresented in our results. While the role of the IPCC is prominent in societal and policy debates, we have no clear understanding of its impact on the field of climate science itself, or on other scientific fields. Here, the AR5 was the second most cited source 4 , with 12% for 2100 and 10% in 2300.
The expert projections for 2100 and 2300 generally illustrate that substantial uncertainty still remains in projecting GMSL rise 1 , with a considerable proportion of experts projecting a greater rise by 2100 than the IPCC AR5 (Fig. 6). Our survey suggests that the high upper-end estimates and uncertainties arise from the expected response of the Antarctic Ice Sheet to warming. The multimeter GMSL rise projected by some experts under RCP 8.5 would expose up to hundreds of millions of people to coastal flooding 39 and devastate coastal ecosystems 40 . However, the expert projections also clearly illustrate the potential for evading such large GMSL rise through successful reduction of emissions.

Choice of experts
The choice of experts is a critical element of expert surveys 41,42 . We followed a similar approach taken for the H14 survey to identify sea-level experts. H14 chose the 500 experts who (co-)authored the most papers. H20 selected experts who (co-)authored ≥6 papers, the minimum number of papers published by the top 500 (co-)authors in H14. To objectively select sea-level experts, we used the scientific publication database Web of Science of Clarivate to identify the most active publishers of sea-level papers. On 15 February 2019, we searched for all papers published in peerreviewed journals since (and including) 2014 where the term "sea level" appeared in the title, keywords or "KeyWords Plus" (an algorithm used to review words or phrases that appear in the cited references of an article) to identify scientists who (co-)authored the greatest number of these papers.
We obtained a sample of 878 experts who published at least six papers on "sea level" since 2014. We found e-mail addresses for 817 of these experts and accordingly invited them to participate in the survey on 18 March 2019, using a unique identifier to ensure anonymity and avoid duplicate responses. A total of 458 experts opened the e-mail invitation, and of these 112 completed the survey, which is typical for this type of internet survey (e.g., ref. 43 ). The main reason given for declining to participate was a (perceived) lack of expertise in projecting GMSL rise. We closed the survey on 30 June 2019. We could not analyze six responses from participants because they either left all boxes blank or filled with a question mark. Not all survey respondents completed every percentile box.
Thus, a total of 106 sea-level experts from 817 invites (13%) provided their probabilistic assessment of GMSL rise, given two temperature scenarios derived from the upper and lower extremes of the RCP scenarios 25 (Fig. 1). This compares to 90 from 360 invites (44%) in H14. Survey nonresponse increases the risk of error 44 . There is an international trend toward declining response rates in social surveys (e.g. refs. 44,45 ) that has been attributed to difficulty in contacting individuals, and their reluctance to participate 46 because of the rise of online surveys, mobile phones, information requests, societal changes, and greater awareness of privacy issues 44 .
In the RCP 2.6 scenario, there is warming of~1°C by 2100 (relative to 1986-2005), with high probability that the global temperature stays below the 2015 Paris Agreement, which aims to limit warming to 2 or 1.5°C above preindustrial level. RCP 2.6 is equivalent to RCP 3-PD (where "PD" stands for Peak and Decline) that was used in the H14 survey. In the RCP 8.5 emission scenario, there is warming of 4.5°C by 2100. For the extended RCP 2.6 scenario, a global temperature increase of 0.0 to 1.2°C is projected for the late 23rd century (2281-2300 average) relative to 1986-2005 47 . For the extended RCP 8.5, global temperature increase of 3.0-12.6°C is projected over the same time period 47 .

Likelihood language
In recent years, there have been a number of different definitions of likelihood language, particularly as it pertains to GMSL estimates. Reference 48 defined "likely" to mean at least 66% probability, and "very likely" to mean at least 90% probability. Such definitions are consistent with certain publications that discuss GMSL projections, such as ref. 21 ,   Fig. 6 Box plots of likely (17th to 83rd percentiles) and very likely (5th to 95th percentiles) sea-level rise for RCP 2.6 (blue) and RCP 8.5 (red) greenhouse gas scenarios for 2100 from projections studies: H14 and H20 surveys; IPCC AR54; and SROCC8. We have standardized projection windows for the surveys and the IPCC following ref. 1 . where probabilities of future GMSL are assessed broadly across a large number of studies. However, the sea-level rise projections provided in IPCC AR5 followed a slightly different definition of likelihood language, defining "likely" to mean a central 66% probability range, and "very likely" to be a central 90% probability range 3 . Similarly, the recent SROCC defined "likely range" as the 17-83% probability range, and "very likely range" as the 5-95% probability range 8 . Our use of likelihood language follows the IPCC AR5 and SROCC definitions, consistent with refs. 3,8 .

Box plots and PDFs
The survey results are presented as box plots and PDFs. The PDFs provide easily interpretable summaries of the survey responses while accounting for the variability and dependencies across the individual expert projections (see Supplementary Information). For each of the future emission scenarios, we generated PDFs through Markov Chain Monte Carlo sampling from the posterior distribution for anticipated GMSL. The model for estimating the latent sea-level projection distribution, from which we draw the samples to generate the PDFs, is set up in a hierarchical framework such that the individual expert projections provide information for the latent distribution. The hierarchical modeling approach used to estimate the GMSL PDFs has the further advantage of down-weighting outlying expert projections in the summary distribution. This is due to the smoothing effect of the model assumptions whereby the individual expert percentile estimates were assumed to come from an underlying distribution, the mean of which was assumed to come from the latent sea-level projection distribution. Note, the smoothing effect is less severe than simply summarizing the individual percentiles across experts based on calculating central summary statistics (means or medians). To illustrate the down-weighting effect seen in the PDFs, consider that, 4.8 m is the most extreme expert projection for the upper bound of the likely range (i.e., the 83rd percentile) by 2100 under RCP 8.5. This can be interpreted as the belief that there is a 17% chance of GMSL rise reaching or exceeding 4.8 m by 2100. However, according to the PDF for this scenario, there is a negligible chance of GMSL rising by 4.8 m by 2100. Indeed, according to the PDF, projections of GMSL rise exceeding 2.48 m by 2100 are outside of the range of what is likely under this scenario (e.g., refs. [49][50][51][52] ).

Open-ended questions
We invited the experts to state what their greatest source of uncertainty was for their estimates for 2100 and 2300 under both RCP 2.6 and RCP 8.5 (for the exact phrasing of the questions see Supplementary Information). In addition, we invited the experts to provide specific resource(s) (e.g., academic literature, government reports, etc.) that were most influential to their responses. Responses to these questions were open-ended to elicit diverse opinions from experts without being influenced by the survey instrument. However, open-ended questions have the disadvantage (compared to closeended questions) of having both more non-responses and a need for extensive analysis. We nonetheless decided to use open-ended questions to avoid biases in influencing our respondents' opinion about their sources of uncertainty and resources regarding sea-level rise estimates.
We placed the responses for sources of uncertainty into four main categories: (1) ice-sheet uncertainty, (2) anthropogenic uncertainty, (3) model/data limitations, and (4) ocean-atmosphere uncertainty. Ice-sheet uncertainty refers to the stability of ice sheets and ice-ocean-climate system feedbacks. Anthropogenic uncertainty refers to the uncertainties regarding the level of greenhouse gas emissions in the atmosphere as a result of permafrost thawing, fossil fuels, and potential mitigation strategies implemented by society, or the effect of land water storage from dam impoundment or groundwater withdrawal. Model/data refers to uncertainty about scientific estimations and models or data gaps needed to model these systems accurately. Ocean-atmosphere uncertainty refers to the distribution of heat as a result of ocean circulation. In many cases, experts provided more than one source of uncertainty for their sea-level rise estimates. We reported the number of times a particular source of uncertainty was provided in our open-ended questions, therefore the number of selected uncertainties can (and does) exceed the number of individual experts who responded to the survey.
For the open-ended questions about the specific resources used to support their sea-level rise estimates, most experts provided the author, year, and journal of publication, from which we were able to identify the specific reference. We reported the number of times each article was stated by our respondents. We also received responses that were generic (e.g., articles from a specific author) or a misspelling from the expert. We did not report these cases because we could not identify the specific references to which the respondent was referring.
We received approval to conduct our survey from the NTU Institutional Review Board on 17 January 2019 (IRB-2018-10-048).

DATA AVAILABILITY
All data generated during this study are included in this published article (and its Supplementary Information)

CODE AVAILABILITY
The code for producing the probability density functions (PDF) is available at https:// github.com/ncahill89/H20Survey.