A scoping review of research funding for small-scale farmers in water scarce regions

1The Institute for Resources, Environment, and Sustainability, University of British Columbia, Vancouver, British Columbia, Canada. 2School of Public Policy and Global Affairs, University of British Columbia, Vancouver, British Columbia, Canada. 3Policies, Institutions, and Livelihoods Program, International Livestock Research Institute, Nairobi, Kenya. 4The Mediterranean and Tropical Livestock Systems Joint Research Unit SELMET (CIRAD, INRAE, SUPAGRO), Université de Montpellier, Abidjan, Côte d’Ivoire. 5Agriculture and Food, Commonwealth Scientific and Industrial Research Organisation, St. Lucia, Queensland, Australia. 6School for Environment and Sustainability, University of Michigan, Ann Arbor, MI, USA. 7Department of Agricultural and Applied Economics, University of Georgia, Athens, GA, USA. 8Food, Agricultural, and Environmental Sciences Library, The Ohio State University, Columbus, OH, USA. 9Department of Global Development, Cornell University, Ithaca, NY, USA. 10Centre for Evidence Based Agriculture, Harper Adams University, Newport, UK. ✉e-mail: vinnyricciardi@gmail.com The 2015 Sustainable Development Goals (SDGs) have motivated the development community to focus on improving the livelihoods and climate resilience of small-scale farms. To focus research, programmatic efforts and donor funding towards supporting small-scale farmers, SDG 2.3 aims to "double the agricultural productivity and incomes of small-scale food producers [by 2030]"1. Critical to achieving SDG 2.3 is to sustain farmers’ production and incomes through climate shocks and stressors. As climate projections indicate that drier regions will become drier and that droughts are likely to increase2, small-scale farmers in already water scarce regions will require additional support3. Tackling water scarcity in small-scale farming systems remains a top priority for development organizations, researchers and donors4,5. As precipitation patterns change with climate change, the livelihoods of small-scale farmers who lack access to water and technologies will become even more marginalized6,7. Small-scale farmers represent >80% of the world’s farms8 and are major contributors to the food system9–11; farms of <5 ha produce nearly 50% of the global food supply11. Finding options for this large and diverse set of farmers to adapt to increasing water scarcity is a growing central tenet to alleviate poverty and to ensure a resilient food system4,5. A key question for donors is which interventions have proven impacts. For instance, improved water access and water-use efficiency have been shown to increase yields and farmer incomes but interventions that are successful in one region do not necessarily work elsewhere. Many interventions (such as water harvesting, soil improvement strategies, drought-resistant crops and livestock breed selection) are heterogeneous and their impacts can vary across physical, social and political dimensions12. This creates a challenge for donors seeking to focus their investments around the goals of increasing food production, improving livelihoods and reducing climate vulnerability for small-scale farmers. Despite the large volume of evidence produced every year in peer-reviewed journals, grey literature and monitoring and evaluation reports, it is difficult to track which interventions have adequate research support. For instance, in water scarce regions it remains unknown if general research trends have kept up to date with policy trends that have shifted from the focus of the Green Revolution era on improving yields to modern development initiatives (for example, SDGs) that include livelihoods, commitment to reducing environmental impacts and gender. To date, the evidence base addressing interventions to improve the livelihoods of small-scale farmers in water scarce regions is limited. Structured evidence syntheses such as systematic reviews and meta-analyses do exist13–20, as do more traditional literature reviews21,22, but all tend to focus on subtopics such as conservation agriculture, irrigation or policy frameworks. This scoping review collates many different types of interventions aimed at small-scale farmers in water scarce regions across lowand middle-income countries (LMICs) to understand the breadth of evidence for available on-farm interventions. Our effort seeks to help donors identify future research funding, to focus the efforts of researchers towards filling knowledge gaps and to build a repository of studies on a broad swath of interventions that may improve the yields and incomes of small-scale farmers. We focus our evidence synthesis on water scarce, small-scale farms across LMICs to assess whether research effort is being conducted in the locations that need it most. Our research identifies countries that have limited research for adaptation solutions, yet A scoping review of research funding for small-scale farmers in water scarce regions

T he 2015 Sustainable Development Goals (SDGs) have motivated the development community to focus on improving the livelihoods and climate resilience of small-scale farms. To focus research, programmatic efforts and donor funding towards supporting small-scale farmers, SDG 2.3 aims to "double the agricultural productivity and incomes of small-scale food producers [by 2030]" 1 . Critical to achieving SDG 2.3 is to sustain farmers' production and incomes through climate shocks and stressors. As climate projections indicate that drier regions will become drier and that droughts are likely to increase 2 , small-scale farmers in already water scarce regions will require additional support 3 .
Tackling water scarcity in small-scale farming systems remains a top priority for development organizations, researchers and donors 4,5 . As precipitation patterns change with climate change, the livelihoods of small-scale farmers who lack access to water and technologies will become even more marginalized 6,7 . Small-scale farmers represent >80% of the world's farms 8 and are major contributors to the food system [9][10][11] ; farms of <5 ha produce nearly 50% of the global food supply 11 . Finding options for this large and diverse set of farmers to adapt to increasing water scarcity is a growing central tenet to alleviate poverty and to ensure a resilient food system 4,5 .
A key question for donors is which interventions have proven impacts. For instance, improved water access and water-use efficiency have been shown to increase yields and farmer incomes but interventions that are successful in one region do not necessarily work elsewhere. Many interventions (such as water harvesting, soil improvement strategies, drought-resistant crops and livestock breed selection) are heterogeneous and their impacts can vary across physical, social and political dimensions 12 . This creates a challenge for donors seeking to focus their investments around the goals of increasing food production, improving livelihoods and reducing climate vulnerability for small-scale farmers. Despite the large volume of evidence produced every year in peer-reviewed journals, grey literature and monitoring and evaluation reports, it is difficult to track which interventions have adequate research support. For instance, in water scarce regions it remains unknown if general research trends have kept up to date with policy trends that have shifted from the focus of the Green Revolution era on improving yields to modern development initiatives (for example, SDGs) that include livelihoods, commitment to reducing environmental impacts and gender. To date, the evidence base addressing interventions to improve the livelihoods of small-scale farmers in water scarce regions is limited. Structured evidence syntheses such as systematic reviews and meta-analyses do exist [13][14][15][16][17][18][19][20] , as do more traditional literature reviews 21,22 , but all tend to focus on subtopics such as conservation agriculture, irrigation or policy frameworks. This scoping review collates many different types of interventions aimed at small-scale farmers in water scarce regions across low-and middle-income countries (LMICs) to understand the breadth of evidence for available on-farm interventions. Our effort seeks to help donors identify future research funding, to focus the efforts of researchers towards filling knowledge gaps and to build a repository of studies on a broad swath of interventions that may improve the yields and incomes of small-scale farmers.
We focus our evidence synthesis on water scarce, small-scale farms across LMICs to assess whether research effort is being conducted in the locations that need it most. Our research identifies countries that have limited research for adaptation solutions, yet high numbers of small-scale farmers in water scarce regions. On the basis of the available evidence, we systematically map research effort onto intervention types to identify which interventions lack research (Box 1). We also assess whether studies examined environmental trade-offs or gender effects to ensure that there is evidence for integrated solutions that reflect the relationship between SDG 2.3 and other development goals. Our scoping review provides a rapid way to identify the breadth of evidence represented in the literature to assist the adaptation of small-scale farmers to water scarcity.

Results
Research needs versus research effort. We found that 76.7% of small-scale farms and 72.4% of small-scale farm area across all LMICs were probably located in water scarce regions. We also found that existing technological penetration of basic irrigation infrastructure for these farms was low. For example, we estimated that fewer than 37.2% of small-scale farms in water scarce regions across all LMICs were irrigated, compared to 43.2% of non-small-scale farms ( Fig. 1 and Supplementary Fig. 1). The largest disparities in irrigation coverage between non-small-scale farms and small-scale farms were in Latin America and the Caribbean, South Asia and Sub-Saharan Africa. Yet, in East Asia and the Pacific and in the Middle East and North Africa regions, small-scale farms had greater irrigation coverage than non-small-scale farms.
We also found a large variation across countries of the number of studies of interventions for farmers living in water scarce regions ( Fig. 2). When comparing where research on interventions was conducted with where high numbers of water scarce, small-scale farms were located, we observed considerable gaps for countries where few studies met our criteria, such as Nigeria, Mali, Uganda, Chad and Ivory Coast (Fig. 3 shows that these countries had 12, 10, 7, 1 and 0 studies, respectively). There were also hotspots of water scarce, small-scale farms with more moderate research effort, which included India, Ethiopia, Kenya, Tanzania and Ghana ( Fig. 3 shows that these countries had 74, 54, 48, 33 and 28 studies, respectively). These differences in the amount of studies are quite large when considering that the included studies ranged from 1962 to 2019.
A promising finding was that 55% of studies tested interventions in controlled field trials and 18% were modelling studies both of which focus on the causal relationships between the interventions and yield and/or incomes. In addition, 28% of studies were from household surveys to determine if interventions designed for water scarce, small-scale farms worked in farmers' local contexts according to farmers' responses on yield and livelihood outcomes. Studies in high-income countries were excluded unless they were relevant to LMICs (for example, studies that used experiments and modelling to mimic water scarce conditions) and those accounted for 2% of the studies we analysed. We found eight meta-analyses and six systematic reviews in our literature search. Five of the metaanalyses found that seed priming, soil management and water management contributed to increased yields in water scarce contexts; the three meta-analyses that examined conservation agriculture showed mixed results in its effects on yield for water scarce contexts (Table 1).

Research focus on yields versus livelihoods.
We found that most articles assessed outcomes associated with yield (91% of articles) as compared to livelihoods (21% of articles examined incomes and/ or expenditures) (Fig. 4). Despite shifts from Green Revolution rhetoric in the 1950s-1970s to language used in the Millennium Development Goals that launched in 2000 and the subsequent 2015 SDGs, research continues to focus on yields compared to livelihoods ( Supplementary Fig. 2).
We checked this research bias towards yields compared to livelihoods by assessing meta-analyses that synthesized the literature on interventions for small-scale farmers in water scarce regions. We found that all eight meta-analyses focused on yield (Table 1). This shows that not only is the focus on yield prevalent in the primary literature but it is also the sole focus of existing research syntheses that quantifies the relationships between interventions and outcome for small-scale farmers in water scarce regions to date.
In addition to outcomes, the coverage and evidence for different kinds of interventions is also diverse. Farm-level interventions represent a toolbox of actions that farmers can take to tackle water scarcity directly, which include drip irrigation, soil improvement strategies, integrated pest management, crop rotation and so on (Fig. 4). While we found higher coverage for interventions around irrigation, tillage, soil amendments, cropping systems, crop varieties and pest management (145-210 articles), we found little work on key interventions of interest, including livestock interventions (43 articles), solutions that protect natural resources at the farm scale, such as buffer strips and contouring (15 articles) and digital technology interventions (three articles) (see Supplementary Table 1 for full definitions of interventions). Changes in the number of studies increased for all interventions over time but the relative share of studies per intervention remained stagnant since the early 1990s ( Supplementary Fig. 3).

Research on environmental impacts and effects on gender.
We found relatively high coverage of research that examined environmental impacts of the intervention assessed (68.9% of interventions and 50.4% of articles; Fig. 5a). Most of these studies assessed

Spatial analysis
To calculate how many small-scale farmers live in water scarce regions, we overlaid spatial datasets of farm size and water scarcity. We then estimated the disparity that small-scale farmers face in accessing irrigation, a critical on-farm resource in water scarce regions. While this was a coarse method that relied on spatial overlays, it enabled us to take a high-level view on where water scarce areas and small-scale farmers are colocated and estimate the number of these farms that probably have access to irrigation.

Evidence synthesis
To identify research trends and gaps, we synthesized academic and grey literature from 26 databases. We focused on studies that tested the effects of on-farm interventions on small-scale farmers' incomes or yields in water scarce regions across LMICs. We used a 'systematic mapping' method 51 assisted by machine learning to quantify the number of studies per type of intervention and to identify countries that had few studies.
The inclusion criteria for our scoping review were that: (1) studies explicitly addressed small-scale farmers (2) studies examined on-farm production management techniques or technologies that explicitly addressed water scarcity, drought adaptation or water efficiency adaptation (3) studies examined the effect on yield or incomes of an on-farm intervention (4) study assessments needed to include either a control case for comparison (temporal or spatial) for identifying the outcomes in the absence of the intervention (this can be from a randomized control trial, pre-post design, postpost design and so on) or a comparison between alternative interventions Full description available in Methods. Evidence synthesis protocol available in the Supplementary Information.
win-wins between farmers and the environment in the context of water scarcity, such as improving water efficiency, increasing water availability (for example, through rainwater harvesting) and managing soil for greater water retention. A smaller share of research (7% of interventions and 6.3% of articles; Fig. 5b) addressed the negative environmental effects of interventions, such as greenhouse gas emissions, biodiversity loss and land conversion/degradation.
We found few studies that considered gender aspects of the effects of interventions on yields and incomes. Only 9.6% of interventions (and 9.6% of articles; Fig. 5a) either assessed gender differences in the outcomes measured or framed their research with a gender lens, with most of these studies examining if the intervention could reduce the time burden many female farmers face. This limited number of studies was uncovered even though we used broad inclusion criteria to include studies not only if they directly addressed gender outcomes but also if they were framed in terms of gender issues.

Discussion
This scoping review quantified the global irrigation gap of small-scale farms and found that water scarce regions needed equitable irrigation infrastructure the most. While our results generally found that small-scale farmers face unequal access to one critical resource in adapting to water scarcity across LMICs, local political, economic and environmental realities may determine irrigation access. These realities may explain the variation in irrigation coverage between small-scale and non-small-scale farmers across regions. Hence, donors and researchers should focus on water scarce regions with the largest disparities in access to critical resources (for example, irrigation or other) and the lowest amount of evidence for the efficacy of on-farm interventions on yields and livelihoods. A limitation to these findings is that our research highlights only one type of disparity that small-scale farmers face (irrigation coverage) but future studies can build on our methods to examine other types of marginalization (for example, poor soil quality, distance to markets, climate exposure, land use, land tenure/governance and so on) to help prioritize research funding towards evidence-backed interventions for disadvantaged small-scale farmers.
We found that the shift in funding priorities for small-scale farmers from improving yields during the Green Revolution to including livelihoods, environmental trade-offs and gender impacts in the SDGs was not always reflected in the research. Yields have    Results are from the eight meta-analyses our scoping review identified to synthesize interventions for small-scale farmers to adapt to water scarcity. Each meta-analysis synthesized the effect of the intervention on small-scale farms' yields. The 'number of studies' refers to the number of studies the meta-analysis included in their synthesis. The 'quality score' was the average score we rated each meta-analysis by using CEESAT 55 . a Increase occurred only in certain conditions. remained the main focus in this research literature since the 1960s ( Supplementary Fig. 2). Yet, yields only provide a partial farm productivity measure based on output per unit of land cultivated. They cannot proxy for productivity measures that incorporate more production factors (for example, total factor productivity), for economic measures (for example, farm profitability or farm household welfare measures based on income, consumption or asset holdings) or for dynamic measures that address the probability that the welfare of a farm household falls below some threshold in a given year (for example, vulnerability or resilience). All of these outcomes can be difficult to measure and even yields, which are relatively straightforward to measure, are subject to bias when self-reported 23 . One cannot conclude whether an intervention has been successful for small-scale farmers without incorporating economic measures that properly account for self-provisioned inputs such as family labour and capture aspects of farmers' livelihoods 24 . Critical environmental trade-offs of interventions need to be tested to ensure that funded projects do not have deleterious effects on local natural resources and, in turn, on small-scale farmers' future livelihoods. While clear environmental and productivity win-wins were prevalent in the literature, such as water-use efficiency [25][26][27] , it was concerning that so few studies jointly addressed cost-cutting solutions to both environmental degradation and productivity, which include, sustainable intensification, minimizing biodiversity loss and solutions across the food-energy-water nexus [28][29][30] . Future research should help identify where and when win-win interventions exist or when environmental trade-offs are inevitable.
Similarly, there was a large research gap on gender dimensions of interventions. Donors should focus funding research that assesses interventions specifically designed to improve women's outcomes, for example by addressing their large time burden. In addition, existing interventions need to be tested to ensure avoidance of unintended consequences on women. We were concerned to find that gender themes were so rarely addressed, even though women comprise 43% of the agricultural labour force, account for two-thirds of livestock keepers in LMICs and are often responsible for very large shares of agricultural tasks (for example, 90% of hand weeding across Sub-Saharan Africa), while consistently farming plots that are smaller than male farmers' plots 31 . Studies examining the gender effects of on-farm interventions in water scarce regions should focus on gender-specific interventions, such as reducing unpaid labour requirements of women 32 , to free up women's time towards the most productive activities. Improving the gender balance of female extension agents or agricultural researchers-for instance, only 24% of African agricultural researchers are female 31may help to better align research priorities towards the needs of female small-scale farmers.
Given the importance of livestock for adaptation in water scarce conditions, climate solutions that protect farm-level natural resources, and the benefits of climate services and digital technology for in-season recommendations and responses, the lack of evidence we found suggests a need for more research addressing these three types of interventions. While the dearth of research in digital technology is probably due to its relatively recent emergence, the low interest in livestock and interventions that protect natural resources have been persistently low throughout the last several decades (Supplementary Fig. 3). The lack of research on livestock interventions to improve productivity or incomes in water scarce regions was particularly concerning considering an estimated 23% of farmers own cattle 33 . Our findings compliment the notion that pastoral systems in LMICs have tended to receive less investment from the international research community and other institutions, compared to livestock systems in high-income countries and cropping systems more broadly 34  has, however, been growing in recent years, driven by increasing concerns about food security, land competition, the vulnerability of remote communities and the environment as well as by a raising awareness regarding the contribution of livestock systems to all these aspects 35,36 , we did not see relative increases in the number of studies examining these systems over time ( Supplementary  Fig. 3). Climate resilience strategies for livestock keepers require increased institutional support through policies that address the issues of land tenure, fragmentation and degradation 35 . The evidence gap we found pertaining to natural resource protection may reflect the focus of this particular literature on different outcome metrics, such as contouring to prevent soil erosion or using buffer strips to improve biodiversity instead of the focused outcomes of improving yields and livelihoods of the SDG 2.3. However, as ecosystem services are an important dimension of a farm's resilience to climate stressors and shocks 37 , natural resource protecting interventions require more evidence for their effects on farmer livelihoods in water scarce regions. Digital solutions for water scarce, small-scale farms is an emerging field but funders should prioritize research grants to test the livelihood impacts of these solutions. There are more applications for small-scale farmers generally 38 but there is limited evidence to apply these interventions even outside of water scarce regions. For example, a meta-analysis carried out in 2019 only found five studies with nine observations that examined digital solutions that improved small-scale farmers' yields (researchers found a 0-8% increase) and the odds of adopting a recommended agricultural input (researchers found a 13-31% increase) 38 . While the goal of our systematic map was not to identify studies suitable for a meta-analysis, we found seven studies that examined digital solutions for farmers in water scarce regions. These studies focused on providing farmers with tools to improve the water-use efficiency of their irrigation systems 39 , climate information to farmers 40 and to aid decisions in nutrient, water and weed management to reduce production risks 41 . Despite the promise of digital solutions in water scarce environments, we suggest that there needs to be better proof of digital technologies that are tested in low-bandwidth settings since many small-scale farmers live outside of 3G and higher

Box 2 | Recommendations
These recommendations address research funding priorities to bolster outcomes for small-scale farmers in water scarce regions.
(1) Geographic focus. Donors should increase their focus on water scarce regions, especially in countries with many small-scale farmers and little research available, when evaluating the impacts of on-farm interventions to improve outcomes for small-scale farmers. (2) Sustainability focus. By requiring funded research to measure key environmental outcomes associated with interventions in water scarce regions, donors can reduce the gap in evidence that quantifies trade-offs between on-farm interventions and deleterious environmental effects, whether they are experienced off-farm, thus harming other farmers, or on-farm, thus harming future livelihoods opportunities. (3) Gender focus. Donors should address the dearth of evidence seeking to improve outcomes for women in water scarce regions by requiring funded research to explicitly address gender themes. (4) Intervention focus. Donors can earmark research funds for addressing digital solutions and livestock interventions, two intervention types that have been neglected by researchers focusing on on-farm interventions to assist small-scale farmers in water scarce regions. Digital solutions comprise an emerging field that holds promise for improving farm management in water scarce regions. Interventions to enhance livestock productivity in mixed farming systems also offer an important opportunity to enhance the productivity of small-scale farmers in water scarce regions. mobile coverage areas 42 . Research in these geographies, metrics and outcomes will bring us closer to meeting the needs of water scarce, small-scale farms on the ground. To achieve the SDG 2.3 goal of bolstering small-scale farmers' yield and livelihoods, a greater research focus is needed in water scarce regions of LMICS. Future research needs to test the effects of on-farm interventions not only on yields but also on more outcomes that are relevant for farmer livelihoods, such as farm profitability, farmer income and resilience. Trade-offs between interventions and the environment need to continue to become part of research designs and donor requirements to ensure that unintended environmental impacts can be avoided, especially in regions with limited water resources and for populations that are systematically disadvantaged from accessing critical on-farm infrastructure. Similarly, gender dimensions of interventions require a tremendous amount of support in research funding since these remain under-researched despite being an SDG cross-cutting theme. We suggest that funders incorporate these outcomes into their impact assessment frameworks for research grants. For specific policy recommendations see Box 2.

Spatial analysis.
To contextualize the disparity small-scale farmers face in accessing critical resources, we quantified the number of small-scale farms with irrigation in water scarce regions in LMICs. We created a spatial layer of small-scale farms and overlaid it with available water scarcity and irrigation spatial layers. Our analysis provided estimates aggregated to the country level. While this was a coarse method that relied on spatial overlays, it enabled us to take a high-level view on where water scarce small-scale farmers live and how many of these farms have access to irrigation.
To create the small-scale farms layer, we implemented the SDG 2.3 definition of small-scale farms through spatial proxies at 10 km 2 resolution. SDG 2.3 defines small-scale farms as the smallest 40% of farms in a country and farms with the lowest 40% of agricultural revenue in a country 43 . We used an available farm size map at 10 km 2 resolution with a global spatial extent 42 . It was created by using a crowd-sourced field size map 44 to downscale the national farm size distributions of the World Census of Agriculture 8 . Each grid cell contains the most common farm size using World Census of Agriculture categories: 0-1 ha, 1-2 ha, 2-5 ha, 5-10, ha, 10-20 ha, 20-50 ha, 50-100 ha, 100-200 ha, 200-500 ha, 500-1,000 ha and farms >1,000 ha. We computed the smallest 40% of farms in a country on the basis of this map. Since there are no comparable spatial data on agricultural revenue, we used a proxy for agricultural revenue. We used a downscaled gross domestic product (GDP) per capita dataset available at 110 km 2 resolution 45 , which we subsetted to cropland area 46 . We computed the lowest 40% of GDP per capita per country using this data. We aggregated the farm size map to the same spatial resolution as the GDP per capita map and overlaid these two layers to find pixels that were both the smallest 40% of farms in each country and in agricultural areas with the lowest 40% of GDP per capita in each country. We estimated the number of farms in each pixel by dividing the agricultural area by the predominant farm size of that pixel. This approach allowed us to identify the poorest agricultural areas with the smallest farms in each country.
We created the water scarcity layer by combining two different measures of water scarcity at 10 km 2 resolution. Each grid cell needed to be either green (rainwater stored in the soil) or blue (fresh surface water and groundwater) water scarce. We defined green water scarcity according to the IPCC definitions of a semi-arid region, which is an area with <250 mm of rainfall in a year 47 . We used the Global Precipitation Climatology Centre rainfall dataset (full v.2018) 48 to calculate the median rainfall per grid cell from 1996 to 2005. We relied on Mekonnen and Hoekstra's blue water scarcity dataset to identify grid cells that had on average at least one month of blue water scarcity a year from 1996 to 2005 49 . Through using subnational and intra-annual green and blue water scarcity data, we implemented a more detailed view on water scarcity than commonly reported country-level water scarcity metrics reported at annual intervals.
To determine the irrigation area for small-scale farms compared to non-small-scale farms, we used the global rainfed and irrigated croplands (GRIPC) layer, which is a global irrigation map at 500 m 2 resolution 50 . The irrigation map was created through remote sensing and calibrated to nationally reported irrigation coverage statistics to represent circa 2005 values. This irrigation map provided the percentage of agricultural area in a grid cell covered by irrigated cultivation, rice paddy or rainfed cultivation. For our analysis, we only used the irrigated and rainfed categories. We excluded paddy cropland (and the corresponding farming populations) from our analysis because GRIPC does not further classify paddy fields as irrigated or rainfed in their pixel level dataset but they do offer that irrigated paddy accounts for 66 Mha of global cropland and non-irrigated paddy accounts for 63 Mha of global cropland in their manuscript; put another way, this near 50/50 split should not have a major effect on our estimates of the share of water scarce areas that are irrigated but future studies would benefit from better spatial data that splits irrigated and non-irrigated paddy. To provide the full range of values, our scoping review did not include paddy and found that 37.22% of crop area farmed by small-scale farmers in water scarce regions is irrigated (versus 43.19% farmed by not small-scale farmers). If we assume all paddy fields are rainfed, then 26.48% of crop area farmed by small-scale farmers in water scarce regions is irrigated (versus 37.24% farmed by not small-scale farmers). If we assume all paddy fields are irrigated, then 55.07% of crop area farmed by small-scale farmers in water scarce regions is irrigated (versus 52.82% farmed by not small-scale farmers).
We overlaid the small-scale farm layer, the water scarcity layer and the irrigation layer to calculate the number of farms with and without irrigation. We repeated this analysis for non-small-scale farms and non-water scarce regions. Results were aggregated to the country level and summed across all LMICs and per world region.
Evidence synthesis. We used a 'systematic map' assisted by machine learning and natural language processing (NLP) to perform our evidence review. Systematic maps (also referred to as 'evidence maps' and 'gap maps') are an emerging type of systematic review that attempt to identify patterns of research, to identify gaps in a field and future priorities for research 51 . A systematic map is not like traditional expert-based or narrative reviews in that it attempts to capture all of the research on a given topic and reduce the authors' biases 51 . Systematic maps can capture the full-breadth of interventions relevant to a population, such as our scoping review that asks: What is the spectrum of farm-level interventions that have been tested to increase small-scale farmers' incomes, yields and productivity in water scarce regions?
Our systematic map method had six steps: (1) forming the research question; (2) querying academic and grey literature databases for relevant studies; (3) screening titles and abstracts to determine if a study should be included in our synthesis; (4) screening the full text of studies that passed step 3 to determine if a study should be included in our synthesis; (5) extracting relevant data from each included study; and (6) summarizing and reporting the results. The protocol for this scoping review was registered on the Open Science Framework (https://osf.io/ c6n4k/) before study selection, which can also be accessed in our Supplementary Information.
The guiding question for this systematic map was: 'What spectrum of farm-level interventions to alleviate water scarcity has been tested to increase small-scale farmers' incomes, yields and productivity?' An exhaustive search strategy was developed and applied to 26 academic and grey literature databases to identify all available research pertaining to on-farm interventions that have been tested to increase small-scale farmers' incomes and yields in water scarce regions in low-and middle-income countries. Search terms included variations of the key concepts in the research question: small-scale farmers, water scarcity, and income, yield and productivity. Searches were performed in the following bibliographic databases: CAB Abstracts and Global Health (access via CAB Direct), Web of Science Core Collection (access via Web of Science), Scopus (access via Elsevier), Agricola (access via EBSCOhost), EconLit (access via EBSCOhost) and ProQuest Dissertations & Theses Global (access via ProQuest). A search of grey literature sources (20 specialist organizations and online databases) was also conducted. Full search strategies for each database, including grey literature, can be accessed in their entirety in our protocol available in our Supplementary Information or at https://osf.io/c6n4k/. Search results were de-duplicated to remove redundant citations identified from multiple sources, resulting in 18,365 unique publication records.
We screened the 18,365 titles and abstracts to include or exclude from our scoping review. Two independent reviewers assessed each title and abstract. If there was disagreement between reviewers on whether the study was to be included, a third independent reviewer decided. Articles needed to meet the following eligibility criteria for inclusion in our systematic map. (1) Studies needed to address small-scale farms explicitly. For inclusion in the evidence synthesis, we defined a small-scale farm to meet two of four dimensions: land size, labour input (especially of family members), market orientation and economic size. (2) Studies examining on-farm production management techniques or technologies explicitly addressing water scarcity, drought adaptation or water efficiency adaptation. For the systematic map, we erred on the side of inclusivity and used a general definition of water scarcity to include a broad range of studies and interventions across a spectrum of agricultural regions as well as the concept of water stress. We considered water scarcity as when there is not enough water to be used by a farmer for agricultural purposes, which includes blue and green water. We considered water stress to be an additional subset of water scarcity where certain farmers are economically disadvantaged due to poor access to water resources. (3) Studies examining the effect of an on-farm intervention on yields or incomes. While interventions that improve small-scale farmers welfare can range across plot-level technologies, farm-level management, collective action, government infrastructure projects and bi/multilateral trade agreements, we only focused on-farm-level interventions to represent a toolbox of actions that farmers can take to tackle water scarcity directly. (4) Studies including either a control case for comparison (temporal or spatial) for identifying counterfactual outcomes in the intervention's absence (for example, by using randomized control trials, pre-post designs, random block designs, modelling and so on) or studies comparing alternative interventions. We included reviews if they were systematic reviews or meta-analyses. The most common reason an article was excluded was because it characterized a farming practice, rather than measuring the effects of the farming practice on yields or livelihoods. Other common reasons that studies were excluded were because the interventions were not explicitly linked to water scarcity, water savings or improving adaptation to drought. Several studies were not in English, which was a limitation of our research.
To assist the time-consuming task of sorting the 18,365 titles and abstracts that we identified in the academic and grey literature databases, we used a machine learning approach. Using NLP and machine learning for this stage of systematic reviews is an emerging method 52 . While studies have used Naive Bayes and Support Vector Machines (SVM) models 52,53 the Google Development Team released the Bidirectional Encoder Representations from Transformers (BERT) model on Tensorhub in 2019 and it has outperformed other NLP models in a variety of tasks 54 . BERT is a deep learning language representation model that is context aware, in which the word in context of the sentence and the sentence in the context of the paragraph are embedded in the structure of the model. The BERT comes pretrained, which speeds up processing time since the end user only needs to fine-tune the model. Our scoping review tests the BERT model against other classification models to assist a systematic review.
Our team manually classified 1,500 titles and abstracts to include or exclude from our scoping review. We split the manually classified titles and abstracts into training and test data to build and validate several machine learning classifiers. We tested the accuracy of Naive Bayes and SVM, where we used a bag-of-words model with term-frequency times inverse document-frequency (TFIDF) to construct the features of the model. We conducted cross-validated grid searches to identify optimal sets of hyperparameters, which included removing stop words (common words in the English language) and stemming (converting the word to the root word); all hyperparameters assessed can be found in the supplemental code provided. We compared these scores to a BERT model that we fine-tuned to minimize binary cross entropy loss in a classification layer. We used the multilanguage base version of BERT (12-layer, 768-hidden, 12-heads, 110 M parameters), which was trained on the top 102 languages with the largest Wikipedias. Our supplemental code details all parameterization (https://github. com/vinnyricciardi/Ricciardi_etal_2020_ceres). For all models, we calculated accuracy, precision, recall and F1 scores through k-fold cross validation (k = 10) (Supplementary Table 3). The goal was for the model to perform better than the level of agreement our team achieved during the manual classification of titles and abstracts. During the manual classification, two random reviewers (out of a team of ten reviewers) classified each title and abstract. If there was disagreement, a third reviewer broke the tie. Reviewers agreed on the classification 82% of the time. Each of the NLP models performed better than 82%. BERT was the best-performing model with an 88% accuracy (Supplementary Table 3). We applied this final model to the 18,365 titles and abstracts to classify each study to be included or excluded from our scoping review. The model included 1,423 studies in our scoping review for full-text review, of which we were able to find and download 1,355 texts.
In the next phase of our assessment, we downloaded the 1,355 full-texts and manually checked if they were to be included or excluded from our scoping review. In the fourth stage of our assessment, for each of the 560 articles included, we manually extracted the location each study was conducted, the type of method, the type of farming system (crop, livestock or mixed), the intervention assessed (see Supplementary Table 1 for full list), the outcome assessed (yield or income) and whether the study measured a cross-cutting theme (gender effects or environmental impacts). This final stage consisted of one reviewer extracting the information from each text, with communication between reviewers to ensure consistency. Supplementary Fig. 4 shows the number of publications included and excluded at each stage. Our final analysis consisted of cross-tabulations and descriptive statistics.
Through our evidence synthesis, we identified eight meta-analyses. We extracted the intervention and outcome assessed, the location(s) that the meta-analysis included, also the number of studies the meta-analysis included. We recorded the main finding of the meta-analysis to understand the impact of the intervention on the outcome (for example, did water harvesting increase, decrease or have no effects on yields of water scarce, small-scale farms). To assess the quality of each meta-analysis, we relied on Collaboration for Environmental Evidence Synthesis Assessment Tool (CEESAT) 55 , which provides a score sheet of 16 questions to appraise the rigour, transparency and limitations of the systematic review or meta-analysis. For each question, the meta-analysis can receive a red score for the lowest score, an amber score, a green score or a gold score for the highest rating. We converted these scores from one to four, averaged the scores and presented them in Table 1. Our Supplementary Information provides our scoring for the CEESAT score sheet.
Reporting Summary. Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability
For reproducibility, updatability and future research to further develop our methods, the data that support the findings of this scoping review are available in figshare: https://doi.org/10.6084/m9.figshare.12867038

Code availability
For reproducibility, updatability and future research to further develop our methods, all analysis codes are available in the public GitHub repository: https:// github.com/vinnyricciardi/Ricciardi_etal_2020_ceres 1 nature research | reporting summary

October 2018
Corresponding author(s): Vincent Ricciardi Last updated by author(s): Mar 28, 2020 Reporting Summary Nature Research wishes to improve the reproducibility of the work that we publish. This form provides structure for consistency and transparency in reporting. For further information on Nature Research policies, see Authors & Referees and the Editorial Policy Checklist.

Statistics
For all statistical analyses, confirm that the following items are present in the figure legend, table legend, main text, or Methods section.

n/a Confirmed
The exact sample size (n) for each experimental group/condition, given as a discrete number and unit of measurement A statement on whether measurements were taken from distinct samples or whether the same sample was measured repeatedly The statistical test(s) used AND whether they are one-or two-sided Only common tests should be described solely by name; describe more complex techniques in the Methods section.
A description of all covariates tested A description of any assumptions or corrections, such as tests of normality and adjustment for multiple comparisons A full description of the statistical parameters including central tendency (e.g. means) or other basic estimates (e.g. regression coefficient) AND variation (e.g. standard deviation) or associated estimates of uncertainty (e.g. confidence intervals) For null hypothesis testing, the test statistic (e.g. F, t, r) with confidence intervals, effect sizes, degrees of freedom and P value noted Give P values as exact values whenever suitable.

For Bayesian analysis, information on the choice of priors and Markov chain Monte Carlo settings
For hierarchical and complex designs, identification of the appropriate level for tests and full reporting of outcomes Estimates of effect sizes (e.g. Cohen's d, Pearson's r), indicating how they were calculated Our web collection on statistics for biologists contains articles on many of the points above.

Software and code
Policy information about availability of computer code Data collection Covidence and Microsoft Excel softwares were used for data collection and storage, respectively.

Data analysis
Python and R were use for all analyses.
For manuscripts utilizing custom algorithms or software that are central to the research but not yet described in published literature, software must be made available to editors/reviewers. We strongly encourage code deposition in a community repository (e.g. GitHub). See the Nature Research guidelines for submitting code & software for further information.

Data
Policy information about availability of data All manuscripts must include a data availability statement. This statement should provide the following information, where applicable: -Accession codes, unique identifiers, or web links for publicly available datasets -A list of figures that have associated raw data -A description of any restrictions on data availability All data generated or analysed during this study are included in this published article (and its supplementary information files).

Field-specific reporting
Please select the one below that is the best fit for your research. If you are not sure, read the appropriate sections before making your selection.

Life sciences Behavioural & social sciences Ecological, evolutionary & environmental sciences
For a reference copy of the document with all sections, see nature.com/documents/nr-reporting-summary-flat.pdf