School student in an e-learning session. Milan, Italy - March 2020. Credit: Michele D'Ottavio / Alamy Stock Photo.
In March 2020 Italian schools were the first to be closed in Europe in the attempt to curb the spread of SARS-CoV-2. Teaching activities continued remotely until the end of the school year, with a negative impact on students learning. Standardized test scores in March 2021 were on average between 20 and 30% lower than those before the pandemic. A recent global meta-analysis showed that similar estimates were obtained for other high-income countries1.
To investigate the factors at play, a new study published in PLOS ONE2 focused on the use of digital tools in emergency remote teaching and their impact on teacher satisfaction and student performance. The study was coordinated by Tommaso Agasisti, an expert in economics and management of educational services at Politecnico di Milano.
Scientists found that students whose teachers used a wide range of digital tools for educational purposes - rather than only for communicating with students and families - fared better in the standardized tests taken in 2021. Instead, the prevalence of asynchronous teaching, whether based on digital tools or not, was associated with lower levels of teachers’ satisfaction.
Researchers partnered with INVALSI, the public agency appointed by the ministry of education that conducts standardized tests among students each year, to distribute a survey on the use of digital tools by teachers between March and June 2020. INVALSI selected a representative sample of grade 4 and grade 7 classes. Nearly 1,400 teachers responded between July and September.
“We used 12 questions to characterize the attitude of teachers towards digital tools”, explains Marta Cannistrà, PhD student at the Polytechnic and an author of the study. The questions were related to the use of surveys, videos, games, and slides for synchronous teaching, to the use of forums, texts, videos and apps for asynchronous teaching and to the use of WhatsApp, social media, calls and text messages for communication with students and families.
The researchers identified four categories of teachers. The two largest categories were those who integrated digital tools in educational activities (38%), and those resistant to digital (33%), who made a limited use of digital tools and mainly for asynchronous teaching. Nearly 10% of respondents used digital tools intensively both for teaching and communicating with students and families and were defined all-round digital. Finally, 20% of respondents used digital tools but mainly for asynchronous teaching activities.
The researchers were able to profile members of each group. “Previous experiences with digital teaching and training were the two factors that increased the odds of belonging to the integrated digital group”, says Mara Soncin, a postdoctoral researcher at Politecnico and another author of the study.
As for performance, students whose teachers made a large use of digital tools for teaching activities obtained on average a INVALSI standardized test score of around 207, higher than the national average of 200. In contrast, the students of the other three groups of teachers scored on average between 193 and 197. INVALSI scores are obtained by standardizing the test results in three subjects, math, reading and English.
“This association is robust even though integrated digital teachers could be overrepresented in our sample due to a self-selection bias”, observes Agasisti. “We compared the test scores distribution of students whose teachers responded to the survey with those who did not, and we found no statistical difference.”
“The study indicates the need to scale up the public efforts in digital training for teachers,” says Carlo Giovannella a researcher in smart learning ecosystems, at the University of Tor Vergata. “During lockdowns teachers struggled to find the right tools among the many that were proposed, and they were discouraged by the time needed to set up those tools”.
