The problem

The COVID-19 pandemic placed enormous pressure on education systems worldwide. At the peak of the crisis, school closures forced over 1.6 billion learners out of classrooms, exacerbating a learning crisis that existed before the pandemic1. Widespread school closures are not unique to COVID-19 — teacher strikes, summer breaks, earthquakes, viruses such as influenza and Ebola, and extreme weather conditions all result in school closures. The cost of school closures has proven to be substantial, particularly for households of lower socioeconomic status2,3. Reducing learning loss requires outside-school interventions that can effectively deliver instructions to children. However, little evidence exists on how to implement cost-effective learning interventions during school disruptions that can reach as many families as possible.

The solution

The use of mobile phones provides a potential solution to deliver educational instruction when schooling is disrupted, with the advantage of being widely accessible and cost effective4. However, this ‘low-tech’ solution is less commonly used in education relative to ‘high-tech’ approaches that rely on internet-based instruction, despite only 15–60% of households in low- and middle-income countries having internet access. By contrast, it is estimated that 70–90% of households own at least one mobile phone, suggesting that the use of mobile phones has the potential to provide educational instruction in resource-constrained contexts at scale. To examine this possibility, we conducted a randomized controlled trial, with a sample of approximately 4,500 households across Botswana, testing two mobile phone-based methods as low-tech solutions to support parents when educating children during the COVID-19 pandemic. In one treatment arm, SMS messages provided a few basic numeracy ‘problems of the week’; a second treatment arm supplemented these weekly SMS messages with a live 15–20-minute phone call from a teacher to provide a walkthrough of numeracy problems.

We found that SMS messages alone had little effect on household engagement in education and learning. However, a combination of phone calls with SMS interventions resulted in a pronounced improvement, increasing learning by 0.12 standard deviations (Fig. 1) — or up to 0.89 standard deviations of learning per US $100 — which represents one of the most cost-effective learning interventions5. We further developed remote assessments, as a means to measure learning, and found that targeting instruction on the basis of the results of assessments improved learning gains in certain proficiencies, particularly for place value and fractions (Fig. 1). Finally, we found high parental engagement: parents became more confident and accurate in their beliefs about their child’s education. Overall, this study shows that instruction through mobile phones can provide an effective, scalable method for education delivery beyond traditional schooling approaches.

Fig. 1: Effect of low-tech interventions on learning outcomes.
figure 1

The graph shows the effects (in standard deviations) of multiple learning strategies relative to the control (no intervention) group. ‘Average level’ represents results from the Annual Status of Education Report (ASER) 0 to 4 scale corresponding to no operations (0), addition, subtraction, multiplication and division. ‘Place value’ and ‘fractions’ refer to two types of problem. Each group (such as ‘phone + SMS’) refers to randomized treatment groups pooled across the designated category. ‘Targeted’ refers to children in a subset that received additional targeted instruction on the basis of child-specific learning levels; ‘not targeted’ refers to children within a subgroup that did not receive targeted instruction. © 2022, Angrist, N. et al.

Full size image

The implications

Our findings have immediate policy relevance as the COVID-19 pandemic continues to disrupt schooling. Even where schools have re-opened, instruction time has often been reduced owing to social distancing measures, such as double-shift systems in which half of the students attend school in the morning and the other half attend in the afternoon.

Providing additional educational instruction out of school is therefore a current priority. More broadly, our findings have implications for the role of simple, low-tech methods to support education during many forms of school disruption, including teacher strikes, summer holidays, public health crises, weather shocks, natural disasters, and in refugee and conflict settings. In moments in which schooling is disrupted, education systems require resilient approaches to continue to provide education.

Despite our trial including a very large sample size, our data are limited to a single context: the COVID-19 pandemic in Botswana. Future research might involve similar trials to assess how well a low-tech learning approach can be adapted across low- and middle-income countries. We are currently engaged in an active research agenda focused on education in emergencies, which includes a multicontext study testing the adaptability and scalability of remote mobile phone education across five countries: India, Kenya, Nepal, the Philippines and Uganda. Finally, it is important to note that our study evaluates only a subset of potential interventions; other low-tech methods of educational instruction, such as radio and TV, require further investigation.

Noam Angrist1,2

1University of Oxford, Oxford, UK.

2Youth Impact, Gaborone, Botswana.

Expert opinion

“This is a timely and carefully executed and analysed study. The authors provide evidence of a promising, innovative, replicable, potentially scalable and cost-effective intervention to address the massive educational challenge posed by the COVID-19 pandemic. It is a valuable contribution to the literature, although it remains unclear whether the observed short-term gains persist or wane further into the future.” Juan E. Saavedra, University of Southern California, Los Angeles, CA, USA.

Behind the paper

We launched this study within a month of school closures in Botswana, providing some of the first experimental evidence on distance education during the COVID-19 pandemic. This rapid response was enabled by the depth and breadth of presence of Youth Impact in Botswana — an evidence-based nongovernmental organization that provides health and education programmes. Youth Impact provides education services to over 20% of primary schools in the country in partnership with the government, and had experience in running more than 20 rapid randomized trials prior to the pandemic. Our study demonstrates the power of real-time, rigorous evidence to identify effective solutions in a moment of enormous uncertainty and need. The results emerged quickly, were policy-relevant and have been followed by efforts in at least 5 countries reaching over 20,000 students, galvanizing a global and growing evidence base on effective approaches to education in emergencies. N.A.

From the editor

“The challenge of mitigating learning loss during the COVID-19 pandemic is crucial, and this paper by Angrist et al. stands out for its efforts to tackle this problem and test an intervention that could potentially be widely implemented.” Aisha Bradshaw, Senior Editor, Nature Human Behaviour.