Abstract
The wide-scale global movement of school education to remote instruction due to Covid-19 is unprecedented. The use of educational technology (EdTech) offers an alternative to in-person learning and reinforces social distancing, but there is limited evidence on whether and how EdTech affects academic outcomes. Recently, we conducted two large-scale randomized experiments, involving ~10,000 primary school students in China and Russia, to evaluate the effectiveness of EdTech as a substitute for traditional schooling. In China, we examined whether EdTech improves academic outcomes relative to paper-and-pencil workbook exercises of identical content. We found that EdTech was a perfect substitute for traditional learning. In Russia, we further explored how much EdTech can substitute for traditional learning. We found that EdTech substitutes only to a limited extent. The findings from these large-scale trials indicate that we need to be careful about using EdTech as a full-scale substitute for the traditional instruction received by schoolchildren.
Similar content being viewed by others
The wide-scale global movement of school education to remote instruction due to Covid-19 is unprecedented. The use of educational technology (EdTech) offers an alternative to in-person learning and reinforces social distancing, but there is limited evidence on whether and how EdTech affects academic outcomes, and that limited evidence is mixed.1,2 For example, previous studies examine performance of students in online courses and generally find that they do not perform as well as in traditional courses. On the other hand, recent large-scale evaluations of supplemental computer-assisted learning programs show large positive effects on test scores. One concern, however, is that EdTech is often evaluated as a supplemental after-school program instead of as a direct substitute for traditional learning. Supplemental programs inherently have an advantage in that provide more time learning material.
Recently, we conducted two large-scale randomized experiments, involving ~10,000 primary school students in China and Russia, to evaluate the effectiveness of EdTech as a substitute for traditional schooling.3,4 In both, we focused on whether and how EdTech can substitute for in-person instruction (being careful to control for time on task). In China, we examined whether EdTech improves academic outcomes relative to paper-and-pencil workbook exercises of identical content. We followed students ages 9–13 for several months over the academic year. When we examined the impacts of each supplemental program we found that EdTech and workbook exercise sessions of equal time and content outside of school hours had the same effect on standardized math test scores and grades in math classes. As such, EdTech appeared to be a perfect substitute for traditional learning.
In Russia, we built on these findings by further exploring how much EdTech can substitute for traditional learning. We examined whether providing students ages 9–11 with no EdTech, a base level of EdTech (~45 min per week), and a doubling of that level of EdTech can improve standardized test scores and grades. We found that EdTech can substitute for traditional learning only to a limited extent. There is a diminishing marginal rate of substitution for traditional learning from doubling the amount of EdTech use (that is, when we double the amount of EdTech used we do not find that test scores performance doubles). We find that additional time on EdTech even decreases schoolchildren’s motivation and engagement in subject material.
The findings from the large-scale trials indicate that we need to be careful about using EdTech as a full-scale substitute for the traditional instruction received by schoolchildren. There are two general takeaways: First, to a certain extent, EdTech can successfully substitute for traditional learning. Second, there are limits on how much EdTech may be beneficial. Admittedly, we need to be careful about extrapolating from the smaller amount of technology substitution in our experiments to the full-scale substitution in the face of the coronavirus pandemic. However, these studies may offer important lessons. For example, a balanced approach to learning in which schoolchildren intermingle work on electronic devices and work with traditional materials might be optimal. Schools could mail workbooks to students or recommend that students print out exercises to break up the amount of continuous time schoolchildren spend on devices. This might keep students engaged throughout the day and avoid problems associated with removing the structure of classroom schedules. Schools and families can devise creative remote learning solutions that include a combination of EdTech and more traditional forms of learning. Activities such as reading books, running at-home experiments, and art projects can also be used to break up extensive use of technology in remote instruction.
References
Bulman, G. & Fairlie, R. W. in Handbook of the Economics of Education (eds Hanushek, E., Machin, S. & Woessmann, L.) 239–280 (North-Holland, 2016).
Escueta, M., Quan, V., Nickow, A. J. & Oreopoulos, P. Education Technology: An Evidence-Based Review (National Bureau of Economics Research Working Paper No. 23744, 2017).
Bettinger, E. et al. Does EdTech Substitute for Traditional Learning? Experimental Estimates of the Educational Production Function (National Bureau of Economics Research Working Paper, 2020).
Ma, Y., Fairlie, R. W., Loyalka, P. & Rozelle, S. Isolating the “Tech” from EdTech: Experimental Evidence on Computer Assisted Learning in China (National Bureau of Economics Research Working Paper, 2020).
Acknowledgements
We would like to thank the numerous people that helped us with this research.
Author information
Authors and Affiliations
Contributions
R.F.: contributed to analysis and writing. P.L.: contributed to analysis and writing. Both authors are accountable for the accuracy or integrity of all of the work. The authors are co-first authors having provided equal contributions to work.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fairlie, R., Loyalka, P. Schooling and Covid-19: lessons from recent research on EdTech. npj Sci. Learn. 5, 13 (2020). https://doi.org/10.1038/s41539-020-00072-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41539-020-00072-6