Studies of school-based anti-obesity interventions have yielded inconsistent results. Using growth screening data from a school administrative database, we re-evaluated an obesity prevention strategy that was previously reported to have a beneficial effect on weight status of a sample of students in grades 5–7.
Ten K-8 schools (five control and five intervention) participated in a 2-year cluster-randomized trial of a multi-component nutrition education intervention. We obtained student height and weight data for 6 consecutive school years and imputed missing baseline and follow-up measurements (53% and 55%, respectively) and defined the target population based on the intent-to-treat principle. We analyzed changes in body mass index (BMI) Z-scores via mixed-effects linear regression and in the prevalence of overweight/obesity via conditional logistic regression. We also assessed incidence and remission of overweight/obesity and long-term effects.
We analyzed data for 8186 (96%) K-8 students in the 10 schools (4511 in intervention; 3675 in control). From baseline to the end of the intervention period, mean increases in BMI Z-score were 0.10 and 0.09 in the control and intervention groups, respectively (P=0.671). The prevalence of overweight/obesity increased by 3% in both groups (P=0.926). There was no significant intervention effect on the incidence or remission of overweight/obesity. Among 5469 students who attended study schools during both years of the intervention, there was no significant intervention effect. Furthermore, there was no long-term effect among students with up to 2 years of data beyond the end of the intervention.
Using routinely collected data for the entire target population, we failed to confirm earlier findings of an intervention effect observed in a subset of students in grades 5–7. Volunteer bias in the prior evaluation and/or measurement error in the routinely collected data are potential reasons for the discrepant findings.
Over the past three decades, rates of overweight and obesity have risen in children and adolescents. Overweight and obesity in youth are associated with significant co-morbidities,1, 2, 3 increased healthcare costs4, 5 and a shortened life-span.6, 7 Although obese children often become obese adults,8, 9, 10 those who achieve normal weight by adulthood appear to have cardiovascular disease risk similar to those who were normal weight throughout childhood and into adulthood.11 Thus, preventing overweight and obesity in childhood has the potential to ameliorate the immediate effects of excess adiposity and to diminish the risks of type 2 diabetes and cardiovascular disease in individuals who achieve normal weight by adulthood.11, 12
The energy imbalance that ultimately leads to excessive weight gain in children and adolescents involves a complex interplay of environmental, social, behavioral and individual (genetic) factors affecting energy intake (nutrition) and energy output (physical activity, sedentary behavior). School-based interventions appear to be logical and practical approaches for modifying food intake and energy output in children and adolescents,13, 14, 15, 16 with the potential for creating life-long healthy behaviors that would support achievement and maintenance of normal weight.17 Randomized controlled trials conducted to determine the efficacy of such school-based interventions have yielded mixed results and offer limited data that could inform public health strategies for addressing the obesity epidemic.18 Most have been limited to a narrow age range,17, 19, 20, 21, 22 and even when interventions were applied across entire schools or communities, the evaluations were done using volunteer samples drawn from the target population.23, 24, 25, 26 Although such trials are often used to generate evidence of efficacy, the selected evaluation samples may not accurately represent the population where the intervention was applied. Even if an intervention is shown to be efficacious in a particular age group, the findings provide little information on which interventions will be effective when applied to the full range of school-age children. Ideally, the effectiveness of population-based interventions should be evaluated with longitudinal data from a sample representative of all exposed and unexposed individuals, using the intent-to-treat principle to avoid bias.
To address these issues, we re-evaluated a cluster-randomized trial of a multi-faceted school-based nutrition education intervention conducted school-wide in 10 K-8 schools of the School District of Philadelphia (SDP). A previous report evaluated the intervention effect only among a subset of students in grades 5 through 7, and found a modest impact on overweight incidence and prevalence, but no effect on obesity or body mass index (BMI) Z-scores.24 In this paper, we present analyses of longitudinal height and weight data from the administrative database maintained by the SDP that represent a more complete evaluation of the short-term and long-term impact of the intervention in the entire target student population, as well as within age, sex and race subgroups.
Materials and methods
Study design and study population
The School Nutrition Policy Initiative (SNPI) was a multi-component program developed by The Food Trust, a community-based nutrition advocacy organization, to address the needs of public schools of the School District of Philadelphia (SDP).27 The SNPI was designed to prevent overweight and obesity among schoolchildren in grades K through 8, and included the following components: (i) school self-assessment; (ii) teacher nutrition education training; (iii) student nutrition education by the trained teachers; (iv) school nutrition policy changes; (v) social marketing and (vi) parent and community outreach. The program was implemented during the 2003–04 and 2004–05 school years as a cluster-randomized trial in 10 SDP schools (5 intervention and 5 control). A total of 27 schools were organized into 5 clusters, based on school size and type of food service. Two schools from each cluster were then randomized to the intervention or the control group.24
For purposes of the analyses for this report, we defined five time periods: pre-baseline (Pre-B), baseline (B), intervention (I), immediate follow-up (F1), and long-term follow-up (F2) (Figure 1). Although the intervention took place during the entire 2003–04 and 2004–05 school years, we counted the second half of the second school year in the F1 period for consistency with a prior report (Figure 1, dashed red line).24 The main objective of the study was to estimate immediate (short-term) intervention effects (B to F1). Secondary objectives were to evaluate the immediate intervention effects by grade, sex and race, and to estimate any longer term effects (F1 to F2).
Following the intent-to-treat principle of randomized trials, we defined the target population as all students who were enrolled for at least 1 day at any of the 10 SNPI schools during the first intervention year (9/8/03 to 6/16/04). We subsequently excluded students with total SDP enrollment of less than 45 calendar days in the entire year and students who were recorded as being absent for at least a third of their enrollment time, since those most likely were students not attending these schools who had not been cleared from the rosters (Table 1). We also excluded students who were younger than 4 or older than 18 years of age, as well as those with invalid data on sex, date of birth or race (Table 1). Again following the intent-to-treat principle, SNPI group (control or intervention) for each student was defined according to the first SNPI school where the student was enrolled during the 2003–04 school year (first intervention year), irrespective of subsequent transfers.
The SDP maintains a database (Copyright 2010 by the School District of Philadelphia, all rights reserved) of all students enrolled in its schools, with information on dates of enrollment and attendance at each school, student sociodemographics and growth and health variables. Periodic height and weight measurements, made on average every other year, are typically performed by Pennsylvania State Certified school nurses, using stadiometers and medical scales. Under a limited data use agreement with Thomas Jefferson University, the SDP provided de-identified data for 6 consecutive school years (2001–02 through 2006–07), with student records linked across the years by a study-specific identifier generated by an SDP analyst. The analyses presented in this paper were approved by the Institutional Review Board of Thomas Jefferson University.
Body mass index (BMI) was computed as the ratio of weight in kilograms over height in meters squared. BMI Z-scores and percentiles were then computed based on CDC’s age- and sex-specific growth charts.28, 29 Weight status was defined as underweight (BMI <5th percentile), normal weight (BMI 5th and <85th percentile), overweight (BMI 85th and <95th percentile), or obese (BMI 95th percentile).30
For the immediate (short-term) intervention effects, we compared intervention and control groups on the change of BMI Z-score and the change in the prevalence of overweight/obesity from baseline to immediate follow-up (B to F1). We also compared the two SNPI groups on the incidence and remission of overweight or obesity (from B to F1). For the long-term intervention effects, we evaluated the same outcomes from immediate to long-term follow-up (F1 to F2).
Each student had between zero and five BMI measurements (up to one per study period). Therefore, the main analyses used multiple imputation for missing measurements. Missing BMI Z-scores were imputed via a Markov Chain Monte Carlo algorithm, a multivariate normal imputation method that exploits the means, s.d. and correlations of the BMI Z-scores, as well as various study and student characteristics.31, 32 Imputations were generated separately for the control and intervention groups, and the imputation model included terms for school, grade, sex, race/ethnicity, eligibility for free- or reduced-cost meal and special education status. We generated 30 multiply imputed datasets. These were then analyzed and the results of those analyses appropriately combined to yield the results presented in this paper.31 All main analyses controlled for school pair, grade, sex, race/ethnicity, eligibility for free- or reduced-cost meals, and special education status. Imputations and all main analyses were performed in Stata 11.2 (StataCorp., College Station, TX, USA).
The BMI Z-score was analyzed via mixed-effects regression. Fixed effects included terms for time (F1 vs B), study group, the six covariates listed above and all interactions with time. Random effects were included for the intercept and the time slope to account for the within-student correlation of the measurements over time. An additional random effect for school contributed very little and was omitted. The main parameter of interest was the mean difference of BMI Z-score (change from B to F1). The intervention effect is a difference of differences, that is, the mean B-to-F1 change in the intervention group minus the mean B-to-F1 change in the control group, which corresponds to the interaction term between time and group (fixed effect). The covariate main effects control for the impact of the covariates on the baseline BMI Z-scores, while the interactions of the covariates with time control for potentially different trajectories of BMI Z-scores over time among different subgroups of students.
The prevalence of overweight or obesity was analyzed via conditional logistic regression, with the robust variance estimator to account for potential clustering within school. Predictors were time (F1 vs B), as well as the interactions of study group and the six covariates with time. The main effects of group and the six covariates do not vary by time, and hence are not estimable and were not included in the model. The interpretation of the model is essentially similar to that for the BMI Z-score. The main parameter of interest is the odds ratio for time (prevalence odds at F1 over prevalence odds at B). The intervention effect is a ratio of odds ratios, that is, the odds ratio for time in the intervention group over the odds ratio for time in the control group, which corresponds to the interaction between time and group.
Incidence of overweight or obesity was defined as any forward progression into the overweight or obese categories between B and F1 (‘at risk population’: students who were underweight, normal weight, or overweight at B). Similarly, remission of overweight or obesity was defined as any backward movement of students who were either overweight or obese at B (‘at risk population’). Analyses for both incidence and remission were based on logistic regression, with the robust variance estimator to account for potential clustering within school.
Analytical sample and available measurements
Table 1 summarizes the target and analysis study populations. Of the 8504 students in the target population, 8186 had valid attendance and sociodemographic data (3675 in the control and 4511 in the intervention). Of this analysis sample, 7565 students had at least one usable BMI measurement after data checking and corrections (3537 in the control vs 4028 in the intervention). About a quarter of the students had BMI measurements for both the B and F1 periods, although the intervention group students were more likely than control group students to have missing BMI measurements (observed BMI at baseline: 58% in control, 38% in intervention; observed BMI at immediate follow-up: 52% in control, 39% in intervention). SDP measurements were taken throughout the B and F1 periods (Figure 1). Of the 2077 students who had BMI measurements at both B and F1, the time between the two measurements ranged from 18 to 41 months (mean=30 months).
Except for race, baseline characteristics for the control and intervention groups were similar. The proportions of non-Hispanic White and African American students were greater in the control group and the proportion of Hispanic students was greater in the intervention group (Table 2).
Immediate effects of intervention
Table 3 summarizes the results regarding the immediate (short-term) intervention effects. There was very little difference between the control and intervention groups with respect to the B-to-F1 change in BMI Z-scores or change in the prevalence of overweight/obesity (adjusted P-values=0.671 and 0.926, respectively, Table 3). Compared with the control group, the intervention group had a slightly higher incidence but also a higher remission rate for overweight/obesity, although these differences were not significant (adjusted P-values=0.536 and 0.204, respectively, Table 3). We also investigated the prevalence, incidence and remission of overweight and obesity separately, but found no significant intervention effects (results not shown).
Table 4 summarizes the results regarding the association of various student characteristics with the change in BMI Z-score and the change in the prevalence of overweight/obesity. These are results from the same multivariable models that assessed the intervention effect (that is, the analyses included the adjusted intervention effects shown in Table 3). With respect to BMI Z-scores, males gained less than females (crude B-to-F1 change=0.12 for females, 0.07 for males; adjusted P-value for the difference=0.019), and Hispanics gained less than other races (crude B-to-F1 change=0.13 for whites, 0.10 for African Americans, 0.02 for Hispanics, 0.12 for Asians and 0.19 for other racial groups; adjusted global P-value for the differences=0.067). However, these differences were small and did not translate into meaningful differences in the change in prevalence of overweight/obesity (adjusted P-values=0.704 for sex and 0.598 for race, Table 4).
We also conducted subgroup analyses for the intervention effect by sex, grade and race/ethnicity. Table 5 summarizes the unadjusted results of these subgroup analyses. In grades K-4, the average BMI Z-score increased slightly more in the control than in the intervention schools (0.14 vs 0.08, intervention effect=−0.06, unadjusted P=0.072, Table 5). In contrast, in grades 5–8, the average BMI Z-score increased somewhat less in the control than in the intervention group (0.06 vs 0.11, intervention effect=0.05, unadjusted P=0.127). This heterogeneity of the intervention effect was significant both in unadjusted (P=0.019, Table 5) and adjusted (P=0.036) analyses. A similar pattern of heterogeneity of intervention effect was seen for the change of prevalence of overweight/obesity (unadjusted P=0.076, Table 5; adjusted P=0.105). The lack of an intervention effect in the higher grades is in disagreement with the original SNPI evaluation that found significant effects of the intervention on the incidence and prevalence of overweight in grades 5 through 7.24
We also conducted a ‘per-protocol’ analysis that included 5469 of the 8186 students (67%) who attended a school in their original study group during both intervention years. These analyses excluded students who were in the eighth grade during the first intervention year and graduated into high school during the second intervention year; all students who switched from a control to an intervention school or vice versa; all students who moved to a non-SNPI school; and all students who left the SDP altogether. The results were very similar to the results for the full cohort. Finally, analyses carried out only among the 2077 complete cases (those with observed data at both B and F1) yielded results that were similar to those obtained in the full study sample with imputed data.
Long-term effects of the intervention
Between the immediate follow-up period (F1) and the long-term follow-up period (F2), there were no significant differences between control and intervention groups in the change of BMI Z-scores (adjusted mean difference for intervention vs control=0, P=0.827) or in the change of the prevalence of overweight/obesity (adjusted odds ratio for intervention vs control=1.08, P=0.671). Similarly, the intervention did not significantly impact the incidence or remission of overweight/obesity during this later follow-up window (P=0.267 and 0.118, respectively).
We used longitudinal growth data available in an administrative database maintained by a large urban school district to conduct a full evaluation of a school-based obesity prevention strategy. The use of such routinely collected data combined with the multiple imputation analytic approach allowed us to include in the analyses almost all students in the participating schools, minimizing biases related to follow-up losses and maximizing power for detecting any existing intervention effects. In our analyses, we compared control and intervention schools with respect to changes in BMI Z-score, changes in the prevalence of overweight and obesity, and the incidence and remission rates for overweight and obesity, and found that the intervention had no significant effect on any of these measures. These findings are in disagreement with the previous report by Foster et al.,24 who found a modest intervention effect on the incidence and prevalence of overweight among students who were in grades 5 through 7 during the first year of the 2-year intervention, but no intervention effect on the incidence and prevalence of obesity or on BMI Z-scores. Even when we conducted subgroup analyses by grade, we did not find any meaningful intervention effects among the higher grades, in direct contrast to the results reported by Foster et al.24 There are two main reasons for the differences in these two sets of findings: selection bias in the earlier Foster et al.24 evaluation and measurement error (misclassification) in the SDP data.
Foster et al.24 analyzed data only from students in grades 5–7. Furthermore, they only obtained consent from about 70% of the target population in those grades and only 63% of the students with baseline measurements were available at the end of the 2-year study period, raising the potential of selection bias. Students in this group and their parents may have differed from the full target population (for example, SES, race/ethnicity, or even other unmeasured characteristics) and may have been more likely than non-participants to alter behavior based on lessons and distributed materials. Furthermore, students who left the 10 SNPI schools in the second intervention year (approximately 20% of the target population) were lost to the analysis of Foster et al.24 Children who were available for the second year measurements may have had better school attendance or may have been from more stable households (that is, not changed address or school catchment area during study period) and thus may have been more likely to implement nutrition changes promoted by the intervention. This, then, might be another source of selection bias in the report by Foster et al.24 In contrast, our analyses included almost the entire target population and would not be subject to similar biases.
Furthermore, assessors who obtained the BMI data used in Foster et al.24 were not blind to the intervention/control status of schools and were involved in the design and execution of the study. SDP staff/nurses who obtained the BMI measurements for the SDP database were probably also aware of their school’s status, but might have been less susceptible to observer bias.
On the other hand, in contrast to the Foster et al.24 data collection, the SDP height and weight measurements were not collected for research purposes and their accuracy is unknown. Even random measurement error independent of the control/intervention status of a school could have introduced a bias towards the null. The random error that is introduced through the imputation of data for students who were missing either baseline or follow-up measurements might have further exacerbated this non-differential misclassification.
Finally, Foster et al.24 used a relatively narrow window for their baseline and follow-up measurements, with a typical time lag of about 24 months between the two (spring 2003 and spring 2005, respectively). The corresponding SDP measurements were obtained over wider intervals and had a longer time lag, about 30 months on average, between baseline and follow-up. This variability of the timing of the measurements and the longer time lag between baseline and follow-up may have led to attenuation of intervention effects in the SDP data.
We evaluated a cluster-randomized trial of the efficacy of a school-based multi-component nutrition education intervention to prevent overweight and obesity using routinely collected height and weight data with imputation of missing values. Our analyses included 96% of the target population in grades K through 8 and found no meaningful effects of the intervention on BMI Z-scores, or on the prevalence, incidence or remission of overweight and obesity. In contrast, a prior evaluation of the intervention relied on measurements from volunteer participants in grades 5 through 7 and found a modest intervention impact on overweight, although it identified no intervention effect on obesity or BMI Z-scores. To the extent that selection bias is a credible reason for the difference between the current study results and those of the prior report, 24 these results reinforce the importance of evaluating population-based interventions in a representative sample of the affected population. On the other hand, validation of routinely collected height and weight measurements in schools is urgently needed, perhaps by direct comparison with the data collected in the previous research evaluation by Foster et al.24
This work was supported by a grant from The Thrasher Research Fund. We also acknowledge the help and cooperation of the School District of Philadelphia and of Dr Gary D Foster, who provided key information about the intervention. The study was approved by the Institutional Review Board of Thomas Jefferson University.
About this article
Systematic review and meta-analysis of the association between childhood overweight and obesity and primary school diet and physical activity policies
International Journal of Behavioral Nutrition and Physical Activity (2013)