Introduction

About 15 million babies worldwide (10.6% of all births) are born preterm (<37 weeks gestational age (GA)) every year and preterm birth represents a significant cause of lifelong morbidity.1 Delivery at any gestation other than full term (39–41 weeks GA) may result in altered brain development and risk for adverse neurocognitive outcomes and academic underachievement.2,3 While problems of moderately and late preterm (MLP) children are subtle, they represent ~80–85% of all preterm born children, and even small increases in cognitive abilities may have large effects on academic performance on a population level.4 MLP children have been neglected in follow-up services, although they may benefit most from intervention.5

With regard to specific difficulties, preterm birth has been consistently associated with low attention, working memory, and mathematic scores.6,7 These areas are closely associated with academic achievement. To reduce adverse outcomes, it is timely and essential to invest in the development of interventions that can increase the academic performance of children born preterm.

There is increasing evidence that the start of schooling may be a critical window of opportunity for intervention in preterm populations.8 Some have reported that an adaptive working memory training, Cogmed® (Pearson Education Inc.) may improve working memory up to 7 months post training in very low birth weight (<1500 g) preschool children9 and extremely low birth weight (<1000 g) adolescents.10 However, effects were rebutted by a recent study reporting no long-term benefits of Cogmed in extremely preterm school-aged children.11 Moreover, a population-based randomized controlled clinical trial found no evidence for short- and long-term effects of Cogmed on academic performance among children with working memory difficulties.12 Thus, interventions targeting general working memory skills may not result in beneficial effects on preterm children’s school performance, in particular in mathematics, and more targeted specific intervention may be needed.

Individual mathematic differences are determined early in life.13,14 In elementary school, the math curriculum comprises different domains such as numeration, arithmetic, problem-solving, geometry, and algorithmic computation; however, we still know little about the domain-specific and domain-general cognitive processes involved in each domain.15 There is agreement that complex mathematical problem-solving requires both short-term/working memory (e.g., processing of new information) and long-term memory (e.g., arithmetic fact recall). Development of fast arithmetic fact recall (i.e., computational fluency) comprises automaticity and efficient use of limited cognitive resources, and is thus essential for successful math learning and progress. The assumption underlying this argument is that fluid abilities, such as attention, working memory, and processing speed, are essential for acquiring knowledge.16 To master higher-level math skills, children thus need to transition from finger counting or slowly calculating basic math facts to recalling all four operations (addition, subtraction, multiplication, and division) accurately, quickly, and effortlessly. Such progress to automatized recall is associated with freeing up working memory resources during problem solving.3,17 While fact fluency is requisite for later mathematic success,17,18 inattentive behavior and poor processing speed may inhibit the age-appropriate development of such computational skills.15,19 Preterm children’s processing speed20 and cognitive workload deficits3 may make them vulnerable for delays with basic math fact fluency. Thus, interventions targeting fact fluency may be particularly effective for increasing preterm children’s school success. XtraMath® is such a program, but has never been systematically evaluated. In addition, while cognitive training in general may have beneficial effects,21 there is little research on feasibility and acceptability of computerized interventions at school age,22 especially among preterm children and their parents.11,23,24

In this randomized controlled trial (RCT), we investigated whether an adaptive online math training (XtraMath) would result in short- and long-term (i.e., directly after and 12 months after the intervention) changes in academic performance in preterm elementary school children, compared with an active control condition (Cogmed working memory training). Second, we evaluated children’s and parents’ training satisfaction, motivation, and general feasibility of both trainings.

Methods

Procedure

Participants of this prospective, multicenter RCT were recruited from birth registries of seven neonatal intensive care (NICU) units, level 3, in the German State of North Rhine-Westphalia. The study was conducted according to CONSORT guidelines and the Declaration of Helsinki. The study protocol was approved by the Ethics Committees of the Faculty of Psychology at the Ruhr-University Bochum (Votum 134) and the Medical Faculty of the University of Essen (14-6163-BO). The trial was registered online with the German Clinical Trials Register (DRKS; ID DRKS00007685).

First, participants were screened for eligibility based on the information available in their birth records. Overall, N = 1026 infants were born preterm between October 1, 2008 and September 30, 2009 in the participating centers. Of these, n = 494 did not meet the predefined inclusion criteria (see Fig. 1 for details). Accordingly, the parents of n = 532 children were invited via mail to participate in the RCT. In line with German medical records data protection regulations, only those who actively responded and agreed to a screening interview could be interviewed via telephone (n = 88). Of these, n = 7 did not meet the inclusion criteria, n = 16 parents declined to participate in the trial, and n = 65 agreed. Participants (parents and children) provided written informed consent and assent before being randomized.

Fig. 1: Randomization flow chart.
figure 1

Note. HIE hypoxic–ischemic encephalopathy, PVL periventricular leukomalacia, IVH intraventricular hemorrhage, ODD oppositional defiant disorder.

Full size image

Inclusion/exclusion criteria

Preterm children (GA < 37 weeks) who started elementary school in August 2015 were included. Only children and parents with sufficient German language abilities to participate in standardized assessments were included; their place of residence was <100 km distance from at least one of the study centers to increase feasibility of participation. Preterm children with non-correctable handicaps (London Handicap Scale, function scale >2), moderate/severe cerebral palsy, hemiparesis, or intraventricular hemorrhage >grade 2 combined with significant leucomalacia/hydrocephalus were excluded to allow participants to successfully complete the training. Families with twins or higher multiples were excluded as parental supervision of ≥2 children’s completion of daily training after school was not considered feasible. Children with a diagnosis of oppositional defiant disorder (ODD) or conduct problems were excluded as they were expected to resist daily training. Children whose school entry was delayed were excluded as their academic progress (primary outcome) could not be assessed.

Intervention

Children were stratified according to gender and GA, and then randomly assigned to either the adaptive math or working memory training group using central blockwise randomization. A.Z. generated the random allocation sequence, and research nurses enrolled participants and assigned them to intervention groups. In addition, children were grouped into four blocks and started assessments and trainings successively during their first year of formal schooling.

Intervention group

Children in the intervention group played an online, computerized math program (XtraMath®) that continuously adapts to children’s abilities. XtraMath aims to help students transition from counting or calculating the basic math facts to automatically recalling them.25 The program uses timed activities with 3-s thresholds to encourage students to answer questions as quickly as possible combined with spaced repetition. XtraMath has four operation components: addition, subtraction, multiplication, and division. Children usually progress through these operations sequentially, based on their mastery (e.g., once children achieve 100% mastery in addition, they will start with subtraction, and so on). Each of the four components starts with a placement quiz that determines the individual initial mastery score and the subsequent math problems the child is presented with. Parents receive weekly progress report emails and they can check training progress online. The XtraMath training itself is not language-dependent, while the instructions are available in nine languages, including German. Children randomized into the intervention group used the XtraMath standard protocol (at least 5×/weekly practice for 10–15 min) for 5 weeks. In this study, only one-digit addition tasks were used (0 + 0 to 9 + 9) in order to standardize training content in line with child age and training duration.

Active control group

For the purpose of rigorous trial evaluation, an adaptive computerized working memory training, Cogmed (version JM for children aged 4–6 years), that has recently been shown to not facilitate school success12 was administered to the preterm children randomized into the active control group. The control training comprised several games designed as fairground rides (e.g., Ferris wheel, rollercoaster). Children were instructed to recall visual–spatial distributed sequences. The training started at a low level of complexity and was administered according to its standard protocol, 5×/weekly for 15–20 min/day for 5 weeks.

In both groups, a trained coach provided weekly in-person and phone-based technical and motivational support for parents during the 5-week intervention period.

Outcome measures

Children’s academic performance was assessed via teacher ratings and standardized tests, administered by trained psychologists before (baseline), directly after (post), and 12 months after the intervention (follow-up). All teachers were blind to children’s intervention group memberships. Blind assessment of secondary outcomes was not possible because post-intervention questionnaires on training motivation and general feasibility differed by training program.

Primary outcome

Participants’ teachers were asked to compare the individual child’s performance to the class average expected performance levels, using the Teacher Academic Attainment Scale (TAAS).26 The TAAS is a brief and psychometrically sound teacher report of achievement in the following three core dimensions: mathematics, reading/writing, and attention/concentration in class. Ratings are completed on five-point scales (1–5), with 3 representing average class performance. The TAAS has been validated in general and preterm populations, shows high concurrent correlations with achievement tests (range: r = 0.69–0.82), high test–retest reliability (r = 0.77), and has high sensitivity and specificity in identifying learning difficulties.26 The TAAS was administered at baseline, post test, and at the 12-month follow-up to assess training-induced changes in individual trajectories of academic achievement. Students in North Rhine-Westphalia usually keep their main subjects’ classroom teachers throughout the 4 years of elementary school. Thus, the three TAAS ratings were completed by the same teacher for each child, while all participating children were at different schools. The two primary hypotheses (post test versus baseline and 12 months versus baseline) were ordered hierarchically: post test first, 12 months second. Because of the hierarchical testing, no adjustment for multiplicity was necessary.

Secondary outcomes

In addition to teacher ratings, children performed standardized tests of school achievement in mathematics (DEMAT 1+, DEMAT 2+).27,28 At baseline and post test, the DEMAT 1+ (Deutscher Mathematiktest für erste Klassen) was used, and at the 12-month follow-up, the DEMAT 2+ (Deutscher Mathematiktest für zweite Klassen) was administered. Both tests comprise different tasks (e.g., number range, addition and subtraction, geometry) based on the respective math curricula.

For the process analysis, n = 60 children answered 16 questions on their experiences with the online training at the post-intervention assessment; 14 of these were adapted from the Intrinsic Motivation Inventory (IMI)29,30 and translated into German, and two were self-created. The IMI assesses participants’ experiences related to a target activity. Items used in this study were chosen from the four IMI subscales interest/enjoyment, perceived competence, effort/importance, and value/usefulness. Children responded on an adapted three-point Likert-type scale (1 = no, 2 = sometimes, and 3 = yes). Factor structure (principal component analysis) and reliability analyses revealed that 11 of the 16 items loaded on one single factor that explained 36.9% of the overall variance (Cronbach’s α = 0.81), and these were included in final analyses. Supplementary Table S1 (online) displays these items ordered by IMI subscale. Individual child responses were z-standardized and averaged into an index scale of Training Motivation.

In addition, n = 59 parents completed an in-house questionnaire containing 16 items assessing training satisfaction, motivation, and general feasibility. Z-standardized items were averaged and combined into index scales of Training Satisfaction (Cronbach’s α = 0.74), Child’s Motivation (α = 0.81), and General Feasibility (α = 0.76; for details see Supplementary Table S2 (online).

Perinatal variables

Information on child sex, GA, birth weight, infections, and brain injury was drawn from birth records.

Social variables

Parental education was assessed as part of a screening interview performed with the parents during the recruitment phase. According to International Standard Classification of Education categories, all parents had medium to high education. Mothers’ and fathers’ scores were thus dichotomized into medium versus high education. In addition, parents completed questionnaires on their child’s behavior and family background information at baseline, post intervention, and 12 months follow-up.

Data analysis

This is a randomized, multicenter trial with two arms. The primary analysis is conducted on the intention-to-treat (ITT) population, control of compliance was conducted via log-files. Sixty-five participants were included, allowing to detect a group difference with an effect size of d = 0.74. Values of participants who were lost to follow-up were not imputed, resulting in a final sample size of XtraMath n = 29 versus controls n = 27 (TAAS)/n = 25 (DEMAT). The primary hypothesis was that children who were randomized into the adaptive XtraMath versus the working memory training would have more academic growth, assessed as TAAS change from baseline (calculated as a difference score of (1) TAAS at post test, directly after the intervention, minus TAAS at baseline; and (2) TAAS at the 12-month follow-up minus TAAS at baseline). Two bootstrapped linear regression analyses were carried out with the dependent variables of academic growth, intervention as the main factor, and baseline TAAS as covariate. The type I error was set to 5% two-sided (no adjustment because of hierarchical testing), yielding >80% statistical power to detect effects of d = 0.78 and R2 = 0.16, respectively. Analyses were repeated controlling for child sex, GA, and intervention block. Results remained stable and there were no effects of confounding variables, due to stratification, thus unadjusted results are reported to preserve statistical power. The secondary math test outcomes were assessed with corresponding regression analyses. A process analysis was carried out using bootstrapped independent-samples t tests to examine differences between training programs on process analysis index scales.

Results

Descriptive sample characteristics

Table 1 shows descriptive characteristics of the two intervention groups at baseline. There were no differences with regard to child sex, age at assessments, or GA due to stratification. Baseline z-standardized TAAS scores were equally distributed across the two groups (TAAS T1: mean (SD) XtraMath: −0.03 (1.15) versus Cogmed: 0.03 (0.84), mean difference: 0.06 [95% confidence interval (CI): −0.44 to 0.56], p = 0.811). The same was true for the math test scores (DEMAT T1: mean (SD) XtraMath: 0.19 (0.97) versus Cogmed: 0.09 (0.98), mean difference: −0.10 [95% CI: −0.66 to 0.44], p = 0.721). In addition, we confirmed that children’s working memory (WISC digit span backwards score: mean (SD) XtraMath: 5.29 (1.24) versus Cogmed: 5.15 (1.10), mean difference: −0.14 [95% CI: −0.77 to 0.50], p = 0.666) and arithmetic (K-ABC calculation standard score: mean (SD) XtraMath: 107.71 (10.14) versus Cogmed: 108.11 (8.99), mean difference: 0.40 [95% CI: −4.79 to 5.59], p = 0.879) abilities at baseline did not differ between the two groups.

Table 1 Descriptive characteristics of preterm children randomized into intervention groups.
Full size table

With regard to training compliance across the 5-week intervention period, log-files showed that, on average, children in the XtraMath group completed fewer sessions per week (4.41 (0.98)) than children in the Cogmed group (5.11 (0.47) mean difference: 0.70 [95% CI: 0.28–1.12], p = 0.002). However, preliminary analyses showed that there was no association between training compliance and primary or secondary outcomes, and main analyses were conducted on the intention-to-treat (ITT) population as planned.

Training effects on academic performance growth

Bootstrapped linear regressions with intervention as main factor and baseline TAAS (T1) as covariate showed that total academic growth to post test was significantly higher in the XtraMath group (B = 0.25 [95% CI: 0.04–0.50], p = 0.039; R2 = 0.14), but this difference was not sustained at the 12-month follow-up (B = 0.00 [−0.31 to 0.34], p = 0.996; R2 = 0.14). Figure 2 suggests that, directly after the intervention (post test), children in the XtraMath group showed more academic growth than children in the active control group; however, these short-term gains were not sustained.

Fig. 2: Teacher-rated Academic Attainment Scores (TAAS) by intervention group (XtraMath (n = 29, bolded) versus Cogmed (n = 27)) at baseline, post test, and 12 months follow-up.
figure 2

Note: Bolded bars represent the XtraMath intervention group. Unadjusted values are shown.

Full size image

There were no relevant group differences in math test score growth according to training condition (baseline DEMAT entered as covariate) post test: B = 0.05 [−0.20 to 0.29], p = 0.697; 12-month follow-up: B = 0.16 [−0.13 to 0.49], p = 0.346); however, inspection of Fig. 3 suggests that children in the XtraMath group achieved stable performance, while children in the active control group (Cogmed), on average, showed a tendency for decreasing math test performance after participating in the training.

Fig. 3: Math test scores (DEMAT) by intervention group (XtraMath (n = 28, bolded) versus Cogmed (n = 25)) at baseline, post test, and 12 months follow-up.
figure 3

Note: Bolded bars represent the XtraMath intervention group. Unadjusted values are shown.

Full size image

Process analysis

Overall, acceptance of both trainings was high. For example, most children reported the training was fun (mean (SD) Cogmed: 2.55 (0.69), XtraMath: 2.58 (0.72), range 1 (low) –3 (high)), and parents rated integration into everyday life as easy (Cogmed: 2.79 (0.69), XtraMath: 2.97 (0.61), range 1–4). Bootstrapped independent-samples t tests showed that, on average, parents in the XtraMath group reported higher general feasibility of the training compared with the Cogmed group (see Table 2), representing a medium-sized effect, d = 0.56.

Table 2 Comparison of training programs on process analysis index scales.
Full size table

Discussion

This multicenter RCT examined the efficacy and feasibility of an adaptive online math training (XtraMath) to improve short- and long-term academic performance in preterm elementary school children, compared with an active control condition (Cogmed adaptive working memory training). Results revealed that the computerized math training promoted higher short-term academic performance growth than the active control condition, but group differences in academic growth were not sustained to 12 months follow-up. Overall, parents in the math training group rated the training as more feasible than in the control group. First, although differences between training groups were not significant at 12 months after the completion of the intervention, the findings of this study provide tentative novel evidence that computerized trainings targeting specific math skills may help support preterm children’s success in school short term. XtraMath has been specifically designed to improve automatic recall of arithmetic facts; however, the short-term training effect found here applied to academic performance across all main subjects, suggesting potential broader transfer effects to other domains.

Complex mathematical problem solving requires both working memory (e.g., processing of new information) and long-term memory (e.g., involving arithmetic fact retrieval). XtraMath involves a simultaneous combination of multiple cognitive processes required in mental arithmetic, such as processing speed, working memory, visual–spatial skills, and fact retrieval.15,19,31 Developing fast arithmetic fact recall comprises automaticity and efficient use of limited cognitive resources, and is thus essential for successful math learning. The assumption underlying this argument is that fluid abilities, such as working memory and processing speed, are essential for acquiring knowledge,16 but these are not the only requirements. To master higher-level math skills, students need to obtain and understand numeric symbols and rules, and transition from finger counting or calculating basic math facts to recalling operations accurately, quickly, and effortlessly. Such progress to automatized retrieval (i.e., recall) has been associated with freeing up working memory resources during problem solving17 and with functional changes in the left inferior parietal cortex.32 Training arithmetic fact recall may help preterm children who are struggling with mathematics become more fluent and provide a foundation for successfully mastering more complex math problems later in life.

There is mixed evidence in intervention research regarding the cost-effectiveness of training intensity, scope, and duration. The data presented here are from an efficacy study, as the intervention was implemented under ideal conditions in highly motivated families and with the support of a specifically trained coach.33 Training effectiveness under real-life conditions has yet to be evaluated.

Our findings suggest that computerized trainings represent a motivating and feasible avenue towards intervention for school-aged preterm children. Results revealed small differences in training acceptance. XtraMath training tasks may be more similar to children’s everyday activities at school than the more playful approach of Cogmed, which may have contributed to differences in parent-rated feasibility of both trainings. Overall, the adaptive, open-access XtraMath training may be more cost-effective and easier to implement into everyday life. Nevertheless, the experiences with recruiting families of school-aged preterm children into this randomized trial suggest that a majority of German parents may be reluctant to participate in computerized interventions, a potentially culturally specific attitude. As Fig. 1 shows, only 88 out of 532 parents of preterm children who fulfilled the inclusion criteria showed interest to participate in a computerized training. Of these 88, however, 16 voiced critical concerns about increased daily screen time as part of the intervention and refused to participate.

With regard to teachers’ roles in supporting preterm children at school, previous research has shown that teachers have poor knowledge about preterm children’s specific needs and difficulties, and it is crucial to provide strategies for supporting their specific needs in the classroom.34 Applying and rigorously evaluating different approaches to improve preterm children’s academic performance provides the best avenue towards improving education and health services for this population. Approaches may include novel e-learning resources to increase teachers’ knowledge of preterm birth and how to support preterm children,35 and recommendations for specific changes in classroom teaching methods, as well as adaptive computerized interventions, as presented here.

In general, and even more in light of recent global developments related to the COVID-19 epidemic, teachers are increasingly challenged to integrate technologies and tools that support distance learning in education. Specialized training programs may help children obtain crucial abilities; however, teachers and parents often struggle to identify innovative high-quality resources, integrate the new materials in regular routines,36 and worry about potential negative consequences of extended screen exposure.37 Computerized trainings can complement but not substitute classroom teaching. With distance and classic in-school learning alike, the aim is to design a well-rounded curriculum that incorporates educational technology elements, but does not solely rely on them.37 Accordingly, the strength of a computational fluency program such as XtraMath may be that it trains foundational skills while leaving strategy instruction to teachers.

Strengths and limitations

This study has several methodological strengths, including a prospective design investigating short- and long-term intervention-induced effects and stratified random assignment of participants to training conditions. The intervention was compared with an active control condition that also adapted to children’s learning progress, which helped control effects of maturation and practice, as well as irrelevant training aspects such as expectancy effects.38 Despite the relatively low recruitment rate discussed above, a high retention rate was achieved, with 86% of children participating in the 12-month follow-up. The primary outcomes were teacher ratings, while previous studies on the effectiveness of computerized trainings on academic functioning used standardized tests of school performance.11,12 In this study, teachers who were blind to intervention group membership rated participants’ academic attainment to evaluate the feasibility of computerized home-based interventions with an ecologically valid measure. Teachers are highly experienced in comparing an individual child’s performance to expected grade levels in school. Moreover, parents are particularly interested in outcomes that make a difference in real life, such as education and later life chances, rather than changes in standardized test scores. Participants were born in multiple centers across a densely populated region of Germany, increasing the generalizability of findings. Children were born very preterm as well as MLP and stratified across groups, thereby extending the range of GA for evaluating intervention efficacy in comparison to previous studies.9,10,11,39 Finally, data were analyzed with an ITT approach, including all participants as randomized irrespective of their training compliance.

This study also has limitations. Blind assessment of secondary outcomes was not possible because post-intervention questionnaires were different for the two training groups. Second, the sample size was small and statistical power was thus only sufficient to detect a large effect size; however, bootstrapping was employed in order to help alleviate some statistical power limitations. Finally, some child characteristics, such as fine motor skills and computer affinity, which may have influenced training performance, were not assessed and accounted for in analyses.

In conclusion, findings of this RCT do not provide evidence that adaptive computerized math training better supports preterm children’s long-term academic performance than working memory training. However, children in the XtraMath training group showed significantly more short-term academic growth than children in the active control group. Overall, results of this study suggest that home-based delivery of computerized training as intervention for school-aged preterm children is feasible. More research on individualized interventions and classroom teaching strategies that cater to preterm children’s specific educational needs is warranted.