Introduction

Sustainable Development Goals connect policies addressing climate change adaptation with complementary benefits for poverty reduction, economic stability and or public health, and highlight the need of interventions that cut across those priorities, particularly in in low- and middle-income countries (LMICs)1,2. Support for the right interventions, however, has so far been hindered by a typically fragmented understanding of their effectiveness among other factors. To address this gap, our study systematically reviews the evidence (i.e., quantitative studies that are accessible online) on the effectiveness of climate change adaptation interventions in LMICs in light of different outcomes.

Evidence gap maps (EGMs) and systematic reviews can inform policy makers and practitioners by synthesizing evidence3. There is increasing global evidence on the effectiveness of adaptation interventions in LMICs (including small island developing states, or SIDS), particularly in the agricultural and coastal sectors4,5,6. Evidence in the agricultural sector includes case studies, experiments, quasi-experiments, and reviews focusing on interventions that target farmer behaviour, agricultural productivity, and livelihood resilience7,8,9,10,11. In the coastal sector, the evidence-base includes case studies and experiments focusing on interventions such as NbS to prevent economic damages or reduce livelihood vulnerability, among others12,13. Increasingly common among all these studies is the recognition of interactions between climate change interventions and risk reduction on the one hand and development priorities on the other14,15,16.

Despite the growing evidence, an integrative synthesis of adaptation effectiveness does not exist, and information remains scattered across studies and types of interventions and outcomes. Existing syntheses have focused on the state-of-the-art5,17,18,19,20, metrics21, planning22,23, financing24, responses17,25,26, or specific types of interventions and/or outcomes27. Also, syntheses have not tended to distinguish between industrialized and LMICs or sectors and/or focused only on effective interventions28.

Major questions to be addressed concern the relevance, effectiveness, efficiency, impact, sustainability and policy coherence of adaptation interventions, particularly in developing country contexts5,29. This paper aims to address some of these gaps through the following research question: to what extent different types of climate change adaptation interventions in the agricultural and coastal sectors are effective in achieving risk-reduction and development outcomes in LMICs?

In addressing the above question our study contributes to two scholarly debates. First, we contribute to the debate of whether it is more effective that interventions target climate protection (here hazards or exposure), or sustainable development more largely (here social vulnerability and adaptive capacity)5. Second, we contribute to the debate around the effectiveness of different types of interventions, including the distinction between hard (technological and infrastructure-based) versus soft (behavioural or institutional) interventions30 and the growing interest in Nature-based Solutions (NbS) due to their potential cost-effectiveness and multiple benefits across contexts31.

Based on a systematic review of 363 empirical observations published in the scientific literature, we find that adaptation interventions can be effective with regard to both risk reduction and development-related outcomes. That said, we find more evidence of risk reduction outcomes in the coastal sector and of development-related outcomes in the agricultural sector. NbS is the only intervention that has positive effects across all outcome categories in both sectors. Evidence is most clear about the contribution of NbS to reducing immediate risks and impacts from climate change in the coastal sector, and promoting economic benefits in the agricultural sector. In the agricultural sector, evidence is also robust about the positive effects of informational/educational and infrastructural interventions and points to the need to explore more systematically their interactions. Also importantly, the study also reveals that interventions can not only fail to achieve expected effects but also have negative effects. This is particularly the case of social/behavioural interventions in the coastal sector when assessed against development-related outcomes.

Results

Distribution of studies across sectors, geographies, intervention types and outcome categories

The number of studies varied substantially between the two sectors. The agricultural sector had four times the number of studies (84 studies with 266 observations) compared to the coastal sector (19 studies with 97 observations). However, the number of observations (evidence linking an intervention with an outcome) per study was much higher in the coastal sector compared to the agricultural sector (5.1 versus 3.2 observations per study on average, respectively).

In the coastal sector, most authors referred to a specific threat (e.g., storms, see level rise) and did not frame their study as related to development but still contained in formation about development-like outcomes. Alternatively, most of the agricultural sector studies both referred to a specific threat (e.g., droughts, floods) and to development. The studies were geographically dispersed (see Fig. 1). The majority of studies were located in China (18 studies), followed by multi-country studies (17), India (13), Pakistan (6), Bangladesh, Ethiopia and Kenya (each 4). From a sectoral perspective, studies were regionally concentrated: 32% of the studies in the coastal sector were conducted in South Asia, and 11 to 16% each in East Asia and the Pacific, Sub-Saharan Africa, and Latin-America and the Caribbean. In the agricultural sector, studies in South Asia, East Asia and the Pacific and sub-Saharan Africa were much more prominent (each 24 to 27%) compared to those based in Latin America and the Caribbean.

Fig. 1: Geographic representation of studies included in the systematic review.
figure 1

Sources: Country layer obtained from Natural Earth—Free vector and raster map data at 1:10 m, 1:50 m, and 1:110 m scales (naturalearthdata.com). Studies grouped by region based on WDI - The World by Income and Region (worldbank.org). Map elaborated with QGIS 3.4.2.

Full size image

The distribution of intervention types differed by sector and outcome categories (Table 1). Most of the observations (and studies) in the coastal sector were NbS (38 observations, 7 studies) and social/behavioural interventions (30, 4 studies). The studies in the agricultural sector were more evenly distributed and covered all the intervention types. This was expected and was partially a result of the larger number of studies in the agricultural (84) compared to the coastal sector (19).

Table 1 Number of observations by sector, intervention type and outcome category
Full size table

Effectiveness of adaptation interventions by sectors

Overall, there were considerably more positive than negative effects reported across all interventions, outcomes, and sectors. However, positive effects were more frequent in the agricultural sector as compared to the coastal sector, and effects also varied by intervention type (Fig. 2). Figure 2 presents the effect size and direction (positive versus negative) of the seven different adaptation interventions by the two sectors.

Fig. 2: Number of observations and their effect direction across intervention types split by the coastal and agricultural sector.
figure 2

- - - = large significative negative (−3), - - = small/ medium sig. neg. (−2), - = very small sig. neg./ negligible (−1), 0 = neutral/ not sig. (0), + = very small sig. positive/ negligible (1), + + = small/ medium sig. pos. (2), + + + = large sig. pos. (3). a results of reviewed studies in the agricultural sector. b results of reviewed studies in the coastal sector.

Full size image

In the coastal sector (Fig. 2a), NbS were noticeably associated with positive effects (29 positive against 9 neutral/negative observations). Social/behavioural interventions were more associated with neutral or negative effects (17 neutral/negative against 13 positive observations). However, most of the negative/neutral effects came from one single study on the effects of relocation of the population away from coastal areas and small islands32. Technological interventions were also mostly associated with negative effects (3 out of 4 observations). However, this was only reported by one single study.

In the agricultural sector (Fig. 2b), most of the interventions showed positive effects (69% of all observations), although there were also neutral (~19%) and negative effects (~12%). Technological and informational interventions showed mostly positive effects (~75% of all observations in each group of interventions). Alternatively, financial and social/behavioural interventions showed a fair number of negative effects (representing 19% and 22% of all observations within each group, respectively).

Strength of effects by intervention type and outcome category

Table 2 displays the strength of effects based (means) for each pair of intervention type and outcome category, for the two sectors. As shown, we found substantial evidence of effects on both the risk reduction-related (climate hazard and exposure) and development-related (social/economic vulnerability and enabling environment) outcomes in both sectors. That said, evidence is more evenly distributed in the coastal than in the agricultural sector, where most of the evidence informs about development related outcomes (only 23% informs about risk reduction).

Table 2 Strength of effects (means) for each pair of intervention type and outcome category for the coastal (n = 97 obs.) and the agricultural sector (n = 266 obs.)
Full size table

In the coastal sector, evidence suggests that interventions at large tend to be more positive with regard to risk-reduction than with development-related outcomes, but this varies across intervention types. Discounting missing data (i.e., variables for which studies did not have information), there were positive mean effects across at least two of the outcomes for NbS, built infrastructure, informational, and institutional interventions (Table 2). Mean effects were particularly high for institutional (2.0 mean positive effect) and built infrastructure (1.5) interventions, although the number of observations for these interventions was low.

NbS were more positive in reducing the risk of climate-related hazard and exposure (1.2 mean positive effect), than in decreasing social or economic vulnerability (0.8) or contributing to the enabling environment (0.7). Forest cover, for example, has been shown to protect property and human lives across coastal districts in West Bengal33.

Technological financial, and social/behavioural interventions showed negative mean effects for development-related outcomes in the coastal sector (Table 2). Technological interventions showed negative mean effects when they targeted social or economic vulnerability (−2.0 negative mean effect) and the enabling environment (−0.5). Financial interventions showed negative mean effects when they targeted the enabling environment (−0.5). Social/behavioural interventions showed negative mean effects when targeting social or economic vulnerability (−0.7) but positive mean effects when reducing the risk of climate hazard and exposure (0.6).

Social or economic vulnerability concentrated most of the negative effects. Most of these came from a single study32. The study explores the impact of the relocation of people from flood-prone coastal areas and islands on livelihoods using survey data from 130 households in Bangladesh. It finds that relocation has negative effects on living conditions as well as negative effects on risk reduction including food security or access to drinking-water. Conversely, the effects on risk reduction were mostly positive. The evidence on these effects came from multiple studies and a diversity of interventions, including migration, livelihood transformation, and participation in community development. Evidence from 1003 individuals across Nigeria has shown that participation in community development significantly reduces the risk of flood impacts to the household34.

In the agricultural sector, all intervention types with the exception of Technological interventions (due to missing data) had positive mean effects across all outcome categories, although in varying degrees (Table 2). Informational interventions showed considerable positive mean effects across all outcomes. As mentioned above, development-related outcomes (i.e., social or economic vulnerability) were most frequently studied. Indigenous knowledge, for example, has been found to have significant positive effects on the accuracy of accounts of past droughts among farmers in Eastern Africa35.

Built infrastructure and NbS also showed relatively high positive mean effects when targeting decreased social or economic vulnerability. Research in China has shown, for example, that households investing in irrigation infrastructure can obtain higher yields than otherwise in the advent of droughts36; and global meta-analyses have confirmed that intercropping improves agricultural yield stability9.

Social/behavioural and financial interventions that targeted social or economic vulnerability showed the lowest mean effects among all interventions in the agricultural sector. Evidence from 700 livestock farmers from all four major provinces of Pakistan, for example, has shown that those resorting to migration (i.e., in search of water and fodder) as a climate change adaptation strategy tend to have lower milk production and income compared to those who did not migrate37. Similarly, data from 266 municipalities in the state of Bahia, Brazil, has shown that access to markets and credits can lead to significant decreases in agricultural and livestock production38.

Contextual conditions

Importantly enough, interventions do not occur in a vacuum; ideally, their effects need to be “controlled for” a variety of other influential aspects. A considerable amount of the studies reviewed consisted of multivariate regression analysis and highlighted some of those other influential aspects (i.e., covariates). In the coastal sector, the effectiveness of “Nature based” solutions such as mangroves to protect population from floods can hold while controlling for population density, elevation, distance from rivers or warning measures39. In the agricultural sector the effectiveness of “Informational/educational” interventions like extension services in promoting resilience against droughts and floods can hold regardless farmers’ wealth, education, marital status, institutional participation, size of cultivated land, number of farm plots, technological improvements, access to information or savings40. Further research shall delve deeper and more systematically into the study of covariates.

Effectiveness of combinations of interventions

We assessed the contribution of combinations of adaptation interventions on their effectiveness. Independent of the sector, 61% of the reviewed studies applied a single intervention. Specifically, in the coastal sector single interventions were most common (74% of the studies), while only 7% of the studies tested three interventions (Fig. 3). Given the small sample size for the coastal sector (N = 19 studies), the evidence on the effect of combined interventions needs to be taken with caution. Conversely, in the agricultural sector, every third study combined three or more interventions. When three or more interventions were combined, the proportion of significant positive effects was only slightly higher compared to single or double interventions (Fig. 3).

Fig. 3: Effectiveness of combinations of multiple interventions across sectors.
figure 3

positive (blue colour), neutral (yellow colour) and negative (red colour) effects across cases with 1, 2 and 3 or more simultaneous interventions.

Full size image

Discussion

At large, our data show that adaptation interventions can be effective with regard to both risk reduction and development-related outcomes. That said, we found more evidence of risk reduction outcomes in the coastal sector than in the agricultural sector, and vice-versa. Also, in the coastal sector, the evidence points to potential trade-offs between risk reduction and development-related outcomes for some intervention types that will need to be further tested. More robust is the evidence about the positive effects of interventions across outcomes in the agricultural sector, illustrating the fine line existing between climate risk-reduction and development in this sector41,42,43.

NbS showed positive effects across all outcome categories in both sectors. This is consistent with current literature arguing that NbS produce multiple co-benefits, i.e., improve economic, social and environmental outcomes, while also contributing substantially to the reduction of climate-related risks5,44. Previous systematic reviews have shown that nearly two thirds of the NbS studied reduced negative climate impacts45. Yet, 80% of the evidence was derived from developed countries in the Global North. Our findings broaden the evidence base on NbS, specifically with regard to their effectiveness in LMICs. Previous research has also identified a lack of evidence on the effects of NbS on socio-economic and developmental outcomes31,45. Here, we were able to provide some of that evidence, in particular on the positive effects of crop diversification, water conservation and coastal habitat restoration on crop yields, food security and poverty reduction, respectively9,33. These findings also align with the claims that NbS and associated agro-ecological practices can enhance land ecosystem services and ensure sustainable land use systems in the agricultural sector46,47.

In the agricultural sector, informational interventions (e.g., early warning, farmer schools, extension services) were the only intervention with clearly positive effects on both risk reduction and development-related outcomes. This confirms the importance of perceptions and learning in environments impacted by climate change. Informational interventions can not only empower households and communities48, but also allow governments and non-governmental organizations to understand farmer perceptions and how they influence their decision-making and risk-taking49. These interventions also question preconceptions, assumptions, and beliefs about the food system, which can result in paradigm shifts and new opportunities for transformational development50. Also, our results about the positive effects of technological and built-infrastructure interventions (e.g., investments in water infrastructure to improve crop yields, and transitions from dryland to irrigation cultivation) to reduce social or economic vulnerability echoes previous findings about the importance of effective water management for development in the Global South51,52.

A number of key messages for development practitioners, governments and NGOs can be drawn from our results. First, although interventions may have multiple effects, there are still patterns linking types of interventions and outcomes; thus, there is an opportunity for policymakers and practitioners to tailor their interventions to optimize the outcomes those interventions are most effective with. Some dyads of interventions and outcomes showed indeed more robust results than others (i.e., NbS and hazard and exposure reduction in the coastal sector; and informational interventions and social or economic vulnerability in the agricultural sector). Second, policy makers should assess the effects of their interventions in conjunction with other influential factors. As shown in the results, there is a variety of influential factors (covariates in statistical terms) that intervention assessments need to control for. Ultimately, those factors inform about the scope conditions under which the effects of interventions could be maximized. Third, there is also the possibility that negative effects occur53; policy makers should be avid to detect these and make corrections early enough in the implementation process.

As further research, our analysis suggests the need to move beyond the debates on individual intervention effectiveness (e.g., hard versus soft, green versus grey interventions) and focus instead on finding synergies between interventions and combinations of interventions31,54. Implementing multiple interventions in parallel could more effectively target individual and multiple climate risks, reach adaptation outcomes, and create synergies than if they were implemented individually. In our study, combinations of interventions had only slightly more positive effects than single interventions in the agricultural sector. That said, most of those combinations included NbS and technological and social and behavioural interventions37,55,56. Further research is needed to explore whether combinations of certain types of interventions have stronger effects than other combinations or than the interventions alone.

Finally, it is instrumental for enhancing the evidence-base for adaptation policy and programmes, to conduct more primary research closing existing gaps and reduce uncertainties for decision makers. This is particularly the case for the coastal sector, which has relatively little evidence compared to the agricultural sector, and for long-term, development-related outcomes. Moreover, the increasing complexity of development interventions, other particularities such as non-linearity of intervention outcomes or shifting baselines in the context of climate change, as well as the demand for transformational change and adaptive management, are calling for further investments into integrated systems of monitoring, rigorous evaluation and learning capacities29,57. By the same token, intervention effectiveness assessments need to further include individual, household and community qualitative data, to ensure benefits reach those most at risk or in need in the long term.

Methods

Concepts

Following the IPCC reports53, we conceptualize climate risks as the interaction of climate hazards with the vulnerability and exposure of human and natural systems. Interventions of climate change adaptation can reduce and manage these risks as well as contribute to longer term development outcomes. Thus, we classify outcomes into three categories depending on how risk reduction versus development-oriented they are5. The categories include:

  1. 1.

    The reduction of climate hazard and exposure of individuals or communities. An example would be the protection and restoration of mangroves to reduce the risk of coastal flooding.

  2. 2.

    The reduction of social or economic vulnerability of individuals or communities. An example would be irrigation techniques that reduce the vulnerability of farmers to water scarcity. This category also includes increasing adaptive capacity (i.e., reducing future vulnerability). For example, farmers that diversify their livelihoods or learn about new cropping techniques are less vulnerable to future climate impacts.

  3. 3.

    The contribution to the enabling environment, including environmental, socioeconomic or institutional improvements at the system-level. Examples are disaster risk programmes or livelihood programmes.

Also, we follow previous research to classify interventions into seven types5,58 (Table 3).

Table 3 Types of interventions and examples
Full size table

Literature debates

Our analysis contributes to two scholarly debates. First, we contribute to the debate of whether it is more effective that interventions target climate protection (here hazards or exposure), or sustainable development more largely (here social vulnerability and adaptive capacity)5. In the terminology used before the IPCC’s Special Report for Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation (SREX), this debate was often casted into the terms “outcome vulnerability” versus “contextual vulnerability”59,60. While this debate is not new, it remains unresolved and has direct implications for the distribution of development finance through, e.g., the Green Climate Fund61. While some commentators advocate for climate-risk oriented criteria for funding allocation, others, particularly from developing countries, argue for a broader development framing of adaptation. While it is not possible to resolve this debate conceptually, empirical contributions regarding the relative effectiveness of interventions addressing protection versus sustainable development capacity can advance the practical side of this debate.

Complementarily, we contribute to the debate around the effectiveness of different types of interventions. Some commentators have argued that a focus on hard (technological and infrastructure-based) interventions, may pay insufficient attention to governance and social barriers that are better addressed by soft (behavioural or institutional) interventions30. Scholars have, for example, pointed to the lack of clear organizational responsibilities for adaptation at higher levels of governance62, conflicts and trade-offs arising from other sectoral policies63 and complexity and routines of government organizations64, and interplay between private and public responsibilities65 as barriers to adaptation. Thus, assessing adaptation effectiveness requires considering a broad range of adaptation intervention types58.

The debate over hard versus soft interventions has in recent years taken on the role of NbS. There is growing interest in NbS as promising adaptation measures in a range of settings, due to their potential cost-effectiveness and multiple benefits31. Globally, the IUCN has recently developed a Global Standard on NbS66. while NbS for adaptation are particularly emphasized in both the EU Adaptation Strategy67 and the EU Green Deal68. However, whether NbS can facilitate sustainable development better than other infrastructure-based solutions is still debated31,45,69,70.

A debate this study does not contribute directly but deserves also to be mentioned is the top-down bias of effectiveness assessments and the need to further integrate community perceptions, and locally understood social and economic processes that play a critical role in experiences of ‘effective’ interventions71. Our study builds indeed on many studies that have been carried out by outsiders with predefined understandings of effectiveness. This limitation is related to the quantitative nature of the studies reviewed.

Data

This systematic review selected studies that were already included in an evidence gap map (EGM) on adaptation, which is one of the most up-to-date and comprehensive databases on the effectiveness of adaptation interventions in low and middle-income countries (LMICs)5. The EGM followed the systematic map protocol, which followed guidelines set out by the Centre for Evidence-Based Conservation72, and included quantitative or mixed-methods studies and systematic reviews in the analysis. The inclusion criteria for this meta-analysis were adapted from previous research5 following the PICOS standard (Table 4).

Table 4 Inclusion criteria
Full size table

In a previous study5, we systematically searched databases of peer-reviewed literature (Web of Science, Scopus, 3ie database and CEE library) and grey literature from several organisational websites for studies on climate change adaptation in low- and middle-income countries (LMICs) as defined by the Organisation for Economic Co-operation and Development (OECD). All literature that had an English abstract and was written in English, Spanish, French or German was included. This yielded a sample of 13,121 studies. The sample was narrowed down by excluding books, book sections and conference proceedings and screening abstract and tiles following several exclusion criteria. Importantly for our purpose, all studies that did not report on the effectiveness of an adaptation intervention were excluded. This yielded a final set of 463 studies (Fig. 4), which is published as an interactive EGM at the website of the International Initiative for Impact Evaluation (3ie)29.

Fig. 4: PRISMA flow chart.
figure 4

Interventions 1 and 3 refer to NbS and technological interventions.

Full size image

In the previous study5, we categorized studies into four sectors of (i) Water, (ii) Forestry, fishing and agriculture, (iii) Land-use and built environment, and (iv) Society, economy and health. Since we focused only on the coastal and agricultural sectors, we excluded 152 studies from their database that did not match these two sectors. We focused on the agricultural and coastal sectors for several reasons. The agricultural sector, along with the forest sector are most directly related with development in LMICs due to the importance of rural areas and the primary sector for those countries’ economies. The forestry sector is critical for climate change mitigation, but the impact of climate change on forest activities has been less documented than in the agricultural sector. The coastal sector has been pioneering in studies of climate change and additionally allowed us to capture intervention effects in urbanized areas. Also, interventions in the coastal sector have tended to target risk reduction outcomes, so by including those interventions we are able to widen the diversity of outcomes studied (the agricultural sector tends to include development-related outcomes).

We also excluded primary or non-review studies on NbS or technological interventions in the agricultural sector, and studies which did not have sufficient data for coding. Primary studies on NbS or technological interventions in the agricultural sector were excluded due to the disproportionately large number of systematic reviews for these interventions, which we included in the review. This led to a final batch of 103 included studies, 19 and 84 of which belonged to the coastal and agricultural sectors, respectively (Fig. 4; see also Supplementary References).

The included studies were coded into a comprehensive coding matrix, which included information about: author, publication year, country, study design, sector, intervention type, outcome category, and effects direction and size, among other variables (see coding book below). We employed a rigorous qualitative consensus approach73 to ensure the reliability of our coding. This involved clear coding guidelines, regular communication among coders and iterative discussions to reach agreement. The coding included two stages. First, all three coders coded 6 studies collaboratively until agreement reached saturation; all coders coded the same study and discussed their codes, one study after the other, until coders reached a similar understanding of the variables (i.e., until coders had the same codes of the intervention type, outcome, and effects direction and size variables for two studies in a row). Then, the database of studies was split among the coders and each of them coded her/his batch independently. Questions at this stage were nevertheless solved collaboratively. This strategy enabled us to maintain a high level of coding consistency, enhancing the validity of our study’s findings. The database had a hierarchical design: one study could include multiple observations, which were the combination of one intervention and one effect of this intervention in an outcome. Table 5 includes the final number of studies and observations per sector.

Table 5 Number of studies and observations per sector
Full size table

To measure effects, we looked at the direction and size of effects (“Effects direction” and “Effect size” variables), and the statistical significance of the findings. Direction was coded as positive, neutral or negative. Neutral was coded when the effects were not significant, or the author explicitly mentioned that there were no effects. Effect size was coded via an ordinal scale (“small”, “medium”, “high”) whenever the effects direction was positive or negative74. Coding effect sizes required translating the quantitative measures such as means, non-parametric tests, regression coefficients into our ordinal scale. Whenever authors complemented quantitative metrics with qualitative comments about the size we used the latter. In the studies where authors did not qualify effects as being “small”, “medium” or “high” we assumed that the size of the effects was “medium”. The only exception to this rule were observations where the metric value was very small (this was the case for <0.1 beta regression coefficients, mean differences, average treatment effects, and <5% percentage differences between the intervention and control groups).

The data was checked for several issues: (1) typos: although many variables included fixed response options others (e.g., country) did not; (2) incongruencies: in our coding book, the coding of some variables depended on the coding of other variables (for example, if the “Effects direction” variable was coded as “neutral”, then the “Effects size” variable had to be coded as “NA”); and blank cells: “No information” and “Not Applicable” values were coded as “33” and “99”. Thus, the dataset should not contain blank cells. Also, “Effects direction” and “Effects size” were recoded into “Effects direction 7-point” with the following values: large significant negative =−3, small/ medium sig. neg. =−2, very small sig. neg./negligible =−1, neutral/ not sig. =0, very small sig. pos./ negligible =1, small/ medium sig. pos. =2, large sig. pos. =3.

Out of the 103 studies of the database, 11 contained more than 5 observations per study (marked with an asterisk in Supplementary Table 1) and 9 of them contained more than 3 observations about one intervention and outcome type indicating the same effects direction (marked with two asterisks in Supplementary Table 1). Although many observations from single studies may be less generalizable, our unit of analysis was the case and not the study and we weighted all observations equally.

Data limitations

The systematic review faced several methodological limitations that may inform future studies. First, there was the potential for a sample bias in the included studies. The keyword search included only terms in English, and thus excluded potentially relevant articles in other languages. Second, included studies were mostly quantitative studies. Thus, adaptation interventions that are more prone for qualitative evaluations (e.g., informational and institutional interventions or interventions targeting the enabling environment) are likely underrepresented. Also, the lack of qualitative data prevented us from questioning definitions of effectiveness and identifying patterns in the causal mechanisms that connect interventions and outcomes. Although this was not part of our objective, it is an important area of research in the field. Qualitative assessments can not only be particularly effective at capturing the impact of multiple-hazard risks75 but have also been accepted as a necessary source of rich data in integrated environmental assessment by the IPCC, Millennium Ecosystem Assessment regional and national studies76.

Third, there was the potential for publication bias. This review included only 19 studies from the coastal sector, which in turn covered mostly observations about NbS and social/behavioural interventions. This clustering of intervention types in the coastal sector may be due to the low implementation costs of these interventions compared to infrastructure interventions (e.g., building sea walls or transport infrastructure) specifically in and for LMICs. Infrastructure interventions may be underrepresented also because there tend to be fewer ex-post impact evaluations and more ex-ante predictive and modelling studies, the latter providing no indication of effectiveness. More generally, few studies indicated interventions having negative outcomes. This apparently positive-results publication bias is of potentially major concern.

Fourth, a general issue in extracting the data was that some studies do not label interventions as adaptation interventions, making it difficult to identify them in the literature77. This may be specifically the case for technical interventions in the coastal context, as these are often reported in coastal engineering journals without reference to climate change or adaptation. Relatedly, many studies are not explicit about specific climate change threats like droughts, floods, increases in temperature, or climatic variability. This complicated their link to intervention types and outcome categories. While some interventions like financial aid or certain infrastructure solutions may well apply to short term hazards, other interventions like institutional reforms or certain NbS may be more suitable for longer term changes78.

Relatedly, just half of the studies in the agricultural sector and 60% in the coastal sector included multilevel regression analyses, pointing to a variety of influential factors (i.e., covariates) other than the interventions that would need further consideration. Studies in the coastal sector, for example, have shown the importance of controlling the effectiveness of mangroves to protect population for aspects such as population density, elevation, distance from rivers or warning measures39. And, in the agricultural sector, studies have illustrated how resilience against droughts and floods can be explained by “Informational/educational” interventions like extension services, as well as other factors like farmers’ wealth, education, marital status, institutional participation, size of cultivated land, number of farm plots, technological improvements, access to information or savings40.

Last, the review was bound by the methodological challenge of coding a wide diversity of studies. Adaptation is highly heterogeneous involving a range of different actors, activities, scales and sectors and is thus subject to a ‘dependent variable problem’, whereby studies of adaptation effectiveness often measure very different aspects and therefore require very careful definitions of contexts, interventions and outcomes17,79. We partially coped with this challenge by classifying interventions and outcomes into categories and coding effect sizes via an ordinal scale. Given the large heterogeneity of studies and outcomes, however, a more rigorous quantitative meta-analysis was not possible. Lack of information in the reviewed studies prevented the systematic coding of social, economic, institutional and biophysical contexts in which the interventions were implemented as well as potential barriers and limits to adaptation, even though these may considerably influence the effectiveness of adaptation interventions.