Randomized design evidence of the attendance benefits of the EPA School Bus Rebate Program

Abstract

Approximately 25 million children ride buses to school in the United States. While school buses are the safest school transport from an accident perspective, older buses often expose students to high levels of diesel exhaust. Because these exposures can adversely impact health, which may lead to more missed school, the US Environmental Protection Agency (EPA) has spent millions of dollars to hasten the transition of school bus fleets to cleaner vehicles. Here, we leveraged the randomized allocation of the EPA’s 2012–2017 School Bus Rebate Program funding to causally assess the district attendance impacts of upgrading buses. Districts randomly selected for funding had greater attendance improvements after the lottery than unselected districts, resulting in over 350,000 estimated additional student days of attendance each year (95% confidence interval = −70,678 to 772,865) due to the use of EPA funds. Attendance improvements were greatest when the oldest buses were replaced and for districts with high ridership on applicant buses. Extrapolating our results nationwide, we expect that the replacement of all pre-2000 model year school buses would lead to more than 1.3 million additional student days of attendance per year (95% confidence interval = 247,443 to 2,406,511). Given the importance of attendance to educational success, we conclude that increasing the pace at which older, highly polluting buses are replaced positively impacts student attendance.

Main

Approximately 25 million children ride buses to school each day in the United States¹. While school buses are the safest means to transport children to school from a traffic accident perspective², the use of older school buses means students often experience high exposures to diesel exhaust during their commutes^3,4,5. Diesel exhaust can enter school buses indirectly via leaky cabins or directly through open windows or doors, resulting in exposures to pollutants inside school buses that are as high as ten times those in ambient air^3,4,5,6,7. Thus, even relatively short commutes on school buses can dominate students’ daily air pollution exposures⁸. This is of great concern given that exposures to traffic-related pollutants are understood to induce inflammation, reduce lung function and increase asthma attacks⁹, which can lead to missed days of school^10,11.

Importantly, however, not all school buses generate the same exposures to diesel exhaust. The US Environmental Protection Agency (EPA) reports that diesel particulate matter (PM) filters, for example, lower emissions of PM from buses by 60–90%¹². Similarly, testing of school buses in Washington¹³, Alabama¹⁴ and Colorado¹⁵ has shown that technologies such as diesel oxidation catalysts (DOCs) and closed crankcase ventilation (CCV) systems can reduce particle concentrations inside of bus cabins by 50–60%. While this work suggests that school districts should retrofit older buses with these technologies or replace them with newer buses that incorporate these technologies, retrofits cost nearly US$10,000 per bus¹⁶. New buses are even more costly at approximately US$100,000 to US$300,000 per bus¹⁷. Thus, the average school bus is on the road for 16 years before being decommissioned¹⁸ and millions of children ride older, highly polluting buses^19,20.

To hasten the transition of school districts to cleaner vehicles, the EPA set aside funding to replace or retrofit old, highly polluting school buses under the National Clean Diesel Rebate Program, which was authorized by the Diesel Emissions Reduction Act of 2010 ref. ²¹. Using a random lottery approach to allocate funds, the EPA’s School Bus Rebate Program (Table 1) has awarded, on average, over US$7 million dollars per year to replace or retrofit school buses since the programme began in 2012, and the programme continues to distribute funds^{21,22,23,24,25,26,27,28}. Although there are documented health and attendance benefits of local school bus retrofit programs in two states^13,29,30, the effectiveness of the EPA’s School Bus Rebate Program has yet to be evaluated.

Table 1 Summary of the EPA School Bus Rebate Program according to year

In this study, we took advantage of the randomized allocation of funding for school bus replacements and retrofits to causally assess the impacts of upgrading buses on student attendance through the EPA’s national School Bus Rebate Program. Specifically, we used classical intent-to-treat (ITT) analyses for randomized controlled trials to compare the change in school district-level attendance rates after versus before the 2012 through 2017 lotteries by funding selection status. (No funding was awarded for school bus replacements or retrofits in 2013, but for brevity we refer to 2012–2017 as our analysis years throughout.) We used overall district attendance rates because rates were not available for only school bus riders. This level of aggregation probably dilutes the treatment effect even though school bus emissions may impact non-riders by idling near to where students play, study and/or wait for other transport^31,32,33. Similarly, the EPA has strengthened emissions standards for vehicles over time so we expected improvements in air quality to vary by the age of bus replaced. Therefore, in the secondary analyses, we evaluated the potential for heterogeneity of effect by quartiles of estimated ridership on applicant buses and by model years of the replaced buses.

Results

Among the 2,816 EPA applicants (90% of all entrants) who met our inclusion criteria and had complete data (Methods), 383 were selected for funding (Table 2). When comparing the districts in terms of size, demographic make-up and a proxy for socio-economic status (that is, free or reduced-price lunch eligibility) at baseline, we found no statistical differences between the winning and losing districts. That said, the districts not selected for funding were slightly larger geographically (281 versus 249 square miles for selected districts), included more schools (15 versus 13 for selected districts) and had more students (9,296 versus 8,335 for selected districts). The baseline attendance rates, however, were similar between the two sets of applicants, that is, 94.90% for the losing districts and 94.75% for the winning districts.

Table 2 Characteristics of school district entrants^a at baseline^b according to lottery status

Nationwide, we found that districts selected for the School Bus Rebate funding had, on average, a 0.06 percentage point (pp) higher attendance rate in the year after the lottery (95% confidence interval (CI) = −0.01 to 0.13) than districts that were not selected for funding (Table 3). In an average size district of 10,000 students, this translates to approximately six additional students attending school each day. The impacts of winning the lottery were even larger for districts with higher levels of ridership on the buses requested for replacement, with impacts reaching as high as 0.14 pp (95% CI = −0.05 to 0.32) for the highest estimated ridership group, translating to 14 additional students attending school each day in an average size district of 10,000 students. Funding also had a much greater impact for districts that replaced older buses. For example, districts replacing a pre-1990 model year bus had the largest improvements in attendance at 0.45 pp (95% CI = 0.26 to 0.65), translating to 45 additional students attending school each day in an average size district of 10,000 students. Districts who replaced 1990–1999 model year buses had a 0.10 pp improvement in attendance (95% CI = −0.03 to 0.23) whereas there was little improvement for applicants that replaced model year 2000 and newer buses (−0.03 pp; 95% CI = −0.16 to 0.09).

Table 3 Impact of clean buses on attendance overall and by ridership and model year of the replaced buses

Based on our main results, we estimated that in the first year after the funding lotteries, the upgrade of older buses through the EPA’s School Bus Rebate funding programmes resulted in over 350,000 estimated additional student days of attendance between 2012 and 2017 (95% CI = −70,678 to 772,865) (Table 4). Notably, this is probably an underestimate of the total impact because it does not incorporate any sustained impacts of the funding over time. Furthermore, extrapolation of our data suggests that funding to replace all pre-2000 model year buses in school districts nationwide would lead to more than 1.3 million additional student days of attendance each year (95% CI = 247,443 to 2,406,511) (Table 4), that is, approximately 400,000 from the replacement of pre-1990 buses and approximately 900,000 from the replacement of 1990s model year buses.

Table 4 Estimated and extrapolated 1-year student attendance impacts (mean and 95% confidence interval) of the US EPA School Bus Rebate Program funding among students in the EPA lottery winner districts only and among all US students if all pre-2000 model year buses were replaced

Our results were robust across multiple alternative specifications of our model in sensitivity analyses, including using the change in attendance rate as the dependent variable and adjusting for covariates such as free and reduced-price lunch eligibility (Supplementary Table 1).

Discussion

In this national analysis of the impacts of the EPA’s School Bus Rebate Program, we found evidence that upgrading older, more polluting school buses with newer, cleaner buses was associated with increases in school district attendance rates. Across all districts selected for school bus rebate funding, we found that there were six more students per day in attendance per 10,000 students in the year after the lottery compared to districts that were not selected for funding. These benefits were closer to 14 and 45 more students per day in attendance when a larger proportion of students were impacted by the programme and when the oldest, most polluting buses were replaced, respectively.

Our findings are noteworthy because the randomized allocation of funding by the EPA allows for a causal interpretation of the attendance impacts of school districts switching to cleaner buses. In addition, this is the first evaluation, to our knowledge, to assess the effectiveness of the national EPA School Bus Rebate Program. This programme was designed to reduce diesel emissions from school buses given the known health impacts of diesel exhaust and we have now demonstrated that it has sizeable impacts on student attendance across the nation. Specifically, we estimate that there were over 350,000 additional student days of attendance in the districts winning the lottery in the year after purchasing a new bus, not to mention any sustained impacts in future years. These are meaningful impacts given that nearly 8 million students (16%) missed more than 15 days of school in the 2015–2016 school year³⁴ and that school attendance has repeatedly been associated with student achievement^{35,36,37,38,39,40}. Excessive school absenteeism has also been linked to substance use, grade retention and school dropout^41,42,43, the latter of which can have economic and health consequences in adulthood⁴⁴. Importantly, the attendance impacts of this programme would probably be of greatest consequence in lower-income areas because those school districts probably have fewer resources available to replace or retrofit older buses in the absence of programmes such as this one.

Our results that indicate greater improvements for the oldest model years of the buses replaced are important but unsurprising given the improvement in emissions standards over time. For context, US EPA exhaust emissions standards for PM from buses had an approximate sixfold improvement for 1991–1997 model year buses compared to 1990 and older model year buses, with smaller absolute improvements beginning in 1998, 2004 and 2007⁴⁵. Although this indicates that the overall attendance impacts of this programme may decline over time if there are no further improvements to cleaner bus technologies, the value of this programme will probably continue for many years given that the average school bus is on the road for 16 years before being decommissioned¹⁸. This implies that, even now in the 2020s, almost 3 million students probably ride a pre-2000 model year bus to school²⁰. Replacement of pre-1990 model year buses alone could lead to over 400,000 additional student days of attendance each year in the US while replacement of 1990s model year buses could result in over 900,000 additional student days of attendance each year.

Notably, our results are for the school district level and thus include both bus riders and non-bus riders. This raises the possibility that the benefits are not identifiable to individual students. Nonetheless, we expect that our findings probably underestimate the true impacts to students who were directly affected by the change to new, cleaner buses. This hypothesis is supported by results from a cohort study in the Washington State Puget Sound that evaluated the impacts of school bus retrofits¹³ and found that students experienced a 5–15% reduction in the risk of absenteeism in the previous month and less lung inflammation when they were riding newer, cleaner than older, more polluting buses¹³. Using binomial distribution assumptions, these results are approximately 1.25 times stronger than our primary result, which again is unsurprising given that these findings are at the student level whereas our results are averaged across the whole district. Further evidence of the dilution of the impacts for individual riders comes from our finding of a near dose–response relationship across quartiles of estimated bus ridership levels on the buses requested for replacement with an order of magnitude greater increase in attendance in winning districts with the highest estimated ridership than in districts in the lowest quartile of estimated bus ridership.

Our findings are also consistent with three other ecological studies that have used observational study designs to evaluate the benefits of cleaner school buses. Using a difference-in-difference design, Beatty & Shimshack³⁰ found that school bus retrofits in Washington State were associated with reduced community-wide hospitalizations for bronchitis, asthma and pneumonia in at-risk populations, all of which can lead to missed days of school. Austin et al.²⁹ found that school bus retrofits in Georgia led to district-level improvements on student physical activity tests and notable gains on standardized testing. They also estimated a 0.03 pp increase in attendance for a district with the average level of fleet retrofits in their study (19%). Finally, one unpublished study has looked at national-level impacts of a separate EPA competitive grant programme, which also funds school bus retrofits and replacements⁴⁶. The author reported similar gains on standardized testing as those seen in Austin et al.²⁹ and very small improvements in local air quality; attendance was not studied. Notably, ours and these studies have hypothesized that the attendance impacts are largely driven by reductions in asthma exacerbations among asthmatic students and fewer respiratory symptoms in general among all students as a result of reductions in emissions from the newer, cleaner school buses. However, other mechanisms, such as students preferring to ride on newer buses or potential reductions in noise levels on newer buses, might also have a role in the programme’s attendance impacts (Supplementary Fig. 1).

Unlike this study, which leveraged the randomized allotment of clean bus funding to estimate the causal impact on school districts of switching to cleaner school buses, previous studies relied on districts self-selecting bus replacements. This lack of random assignment raises the possibility that school districts that adopted cleaner buses or times when cleaner buses were used are fundamentally different from districts or times without cleaner buses due to some other characteristics that are important to health and student attendance. In contrast, our design reduces concerns of confounding by measured or unmeasured school district-related characteristics. This is a key strength of this study compared to all other studies in this area.

Limitations of our study include the relatively small number of entrants who were ultimately awarded funding, which reduced power for this analysis. Although we had intended to include the 2018 lottery, the ‘after’ year for that lottery was disrupted due to the global coronavirus disease 2019 pandemic. Another issue was that 29 districts selected for funding ultimately did not purchase a clean bus due to difficulty acquiring matching funds. In using an ITT approach, which is the common approach for randomized controlled trials, we treated these applicants as lottery winners in our analysis. Similarly, districts not selected for funding could have replaced or retrofitted buses outside of this programme but they were treated as lottery losers in our analysis. These ITT specifications retain the benefits of randomization but mean that we have estimated a lower bound for the true association between being selected to receive clean bus rebate funding and school district attendance rates⁴⁷. Our national impact analysis assumes a linear scaling of our primary result to students across all winning districts while our model-year extrapolation assumes that the fraction of US students who would be impacted by the replacement of older buses is the same as the fleet fraction of the older buses, which might not be correct if district bus routing prioritizes newer buses. Finally, while we have demonstrated that the School Bus Rebate Program is effective at improving district attendance rates in the year after the lottery, we have not performed any formal economic cost–benefit analyses of the programme nor evaluated the sustained benefits of this programme over time.

Overall, we find evidence that the EPA’s School Bus Rebate Program has improved district-level student attendance, especially in the districts that removed the oldest buses and those with high levels of bus ridership on the impacted buses. Given the importance of attendance to student educational success, our results suggest that this programme is both successful at increasing the pace at which older, highly polluting buses are removed from use and that those actions have positive impacts on district-level student attendance levels.

Methods

Programme evaluated

Starting in 2012, the EPA’s School Bus Rebate Program provided funding to replace diesel-powered school buses with older engines with new diesel, alternate fuel, battery, hybrid or electric school buses^{21,22,23,24,25}. In 2015–2017 funding was added for retrofits of school buses with DOCs and CCVs^23,24,25. Additional funding was added in 2016 and 2017 for EPA-verified fuel operated heaters onboard buses to reduce idling for heat^24,25 (Table 1 and Supplementary Table 2).

The EPA’s eligibility criteria allowed school districts and private bus transport companies who serviced school districts to apply for funding for up to five or ten buses, depending on programme year. They could enter up to two applications depending on fleet size and programme year. There were also specific age requirements for the engines eligible to be replaced in each funding cycle and for the type and age of eligible replacement engines (Table 1 and Supplementary Table 2).

The deadline for each of the rebate programmes was the end of the calendar year, at which point the EPA randomly selected applicants for funding using a random number generator until all available funds were exhausted. Because some EPA regional offices had additional funding for school bus replacements, these offices awarded funding to additional applicants based on the randomized rank of applicants who did not receive funding from the EPA national programme (Supplementary Table 3). No restrictions were placed on the number of years that an applicant could enter the lottery.

The EPA notified all applicants at the end of the school year if they were selected for funds. Winners then purchased their replacement buses or installed retrofits in the summer after the lottery and used their new buses for the first time at the start of the next school year. For example, all 2012 applicants who won the lottery replaced their buses in the summer of 2013 and began using the new buses at the start of the 2013–2014 school year, which we refer to throughout this analysis as the ‘after’ lottery year. For 2012 applicants, the 2012–2013 school year would then be the ‘before’ year. All awardees were required to submit proof of new bus purchases and of scrappage of their old buses.

EPA funding applicant data

We obtained data on all applicants for the 2012 through 2017 lotteries from the EPA under a Freedom of Information Act request. The data for all applicants included: the lottery selection status; school district served; number of buses requested to be replaced; and funding requested. For districts that were awarded funding, we additionally received information on the number of buses and engines replaced or retrofitted, as well as engine model year of the replaced (that is, baseline) buses, although this information was most often averaged across all replaced buses in a district.

School district information

School district information came from the US Department of Education’s yearly Local Education Agency (School District) Universe Survey Data. These publicly available data include the number of students (total and by grade and race/ethnicity), number of schools and urbanicity (that is, city, suburb, town, rural) of each district. The land area of each school district was provided in the National Center for Education Statistics School District Geographic Relationship files for the school years of 2013–2014, 2015–2016 and 2017–2018. As a proxy for district socio-economic status, we used data on the number of students in a school who were eligible for the free or reduced-price lunch programme during the baseline school year from the US Department of Education yearly Public Elementary/Secondary School Universe Survey Data, which we aggregated to the district level.

School district attendance data

We collected 2012–2013 through 2018–2019 school year annual attendance rates for school districts that applied for funding from each state’s Department of Education, either from public websites or through individual data requests with a state. Annual attendance rates reflect the average number of students present at all schools in a district across all days of a school year divided by the number of students serviced by that district. We linked annual attendance rate data for both the school year before and after the purchase of new buses to EPA entrants to have the most proximate data to an entrant’s lottery selection status inform the analysis and to reduce the influence of trends.

Data exclusions

Across all five lottery years, there were a total of 3,153 entrants to the EPA School Bus Rebate Program (Table 1). District-level attendance data were not available for Hawaii (n = 8), Puerto Rico (n = 9) and Pennsylvania (n = 71) for any years; Alabama (n = 18), Arizona (n = 11) and Montana (n = 14) for the 2015–2016 through 2018–2019 school years; New Jersey (n = 44) for 2012–2013 and 2013–2014; and North Dakota (n = 16) for 2012–2013. We restricted our analyses to entrants that served individual school districts because they can be linked to school attendance records; therefore, we excluded private bus transport companies (n = 31) and school district consortium entrants (n = 14) who represented multiple school districts. We also excluded entrants who represented private schools (n = 5), non-traditional (for example, special education and technology centres) (n = 22) and tribal schools (n = 5) because attendance data were not consistently available for these schools. Finally, to prevent outliers from impacting our findings, we excluded entrants with changes in attendance rates of five percentage points or more between the before and after lottery school years (n = 18). This cut-off was consistent with the literature for reasonable levels and was consistent with errors^13,48,49. Ultimately, we evaluated associations using attendance data from 90% of the EPA applicants.

Statistical design and analysis

We first compared means (using independent, two-sample t-tests) and proportions (using Pearson’s chi-squared test) of baseline-measured characteristics of the winning and losing districts in our analytical dataset to check for balance among the applicants by selection status. The baseline year was the ‘before’ year for each entrant, which is described above. Then, to evaluate the impact of the EPA’s School Bus Rebate Program on attendance, we used classical ITT analyses for randomized controlled trials because the EPA awarded funding randomly to applicants.

Our primary analysis used model 1, where Attendance_it+1 is the continuous attendance rate for each applicant school district i in the school year after the year t lottery (that is, the 2012 or 2014–2017 lotteries) at which time the new buses were in use. Winner_it is an indicator equal to 1 if school district i was selected to receive funding in lottery year t and 0 if not. To account for any time-invariant differences that occurred by chance between winning and losing districts, we adjusted for Attendance_it, the attendance rate for school district i in the school year of lottery t before the new buses were in use. This adjustment supports causal conclusions with the greatest efficiency by focusing on within-area differences between the before and after randomization levels^50,51. We also adjusted for applicants who submitted more than one application within a lottery year using the indicator MultiEntrant_it because the later lottery years allowed districts with large fleets to submit up to two applications. Similarly, because some EPA regions provided additional funding for the purchase of clean buses, we included fixed effects for the EPA regions (Region_i). To maximize power, we combined data from all lottery years but included fixed effects for lottery year (Time_it) to adjust for any potential confounding over time that may have occurred as the percentages of lottery winners changed by year. Because school districts are not limited to entering the lottery in only one year, we estimated associations and 95% CIs using general estimating equations with robust standard errors clustered at the state level to account for any potential correlation in the data. β₁ is the model outcome of interest, the ITT effect:

$$\begin{array}{l}{\mathrm{Attendance}}_{it + 1} = \beta _0 + \beta _1{\mathrm{Winner}}_{it} + \beta _2{\mathrm{Attendance}}_{it}\\ + \beta _3{\mathrm{MultiEntrant}}_{it} + \beta {\mathrm{Region}}_i + \beta {\mathrm{Time}}_{it} + {\it{\epsilon }}_{it}\end{array}$$

(1)

Our primary analysis used overall district attendance rates because rates were not available for only school bus riders. This level of aggregation probably dilutes the treatment effect even though school bus emissions may impact non-riders when buses idle in close proximity to where students play, study or wait for other transport^31,32,33. Therefore, we evaluated effect modification of our main association by the estimated fractions of children who are likely to ride the buses requested for replacement (that is, the fraction that would be impacted by the treatment). With no databases of school bus ridership rates at the district level, we estimated this fraction by multiplying the number of buses requested for replacement by 72 (the capacity for a standard school bus) and dividing by the total student enrolment for a district at baseline. We evaluated quartiles of this fraction as the interaction terms in our model.

We further evaluated heterogeneity of the attendance impacts by age of the replaced buses. To do so, we replaced the Winner_it indicator in model 1 with three indicator variables for winners who replaced pre-1990, 1990–1999 or 2000 and newer model year buses. Finally, to ensure that our findings were robust to our analytic choices, we tested the sensitivity of our primary results to alternatively modelling the difference in attendance rates before and after the lottery rather than controlling for the previous year’s attendance rate and by including further adjustment for free and reduced-price lunch eligibility (Supplementary Table 1).

Estimated national impact analysis

We estimated the nationwide impacts of the EPA’s School Bus Rebate Program on student attendance by multiplying the total number of US students in winning districts at baseline by the observed effect estimate (that is, 0.06 pp; Table 3) and by the number of days in the school year (that is, 180). To identify the potential national impacts of replacing only the oldest buses, we used the EPA’s Age Distribution Tool for MOVES2014 to estimate what fraction of the US school bus fleet was pre-1990 and, separately, 1990–1999 in calendar year 2021 (ref. ¹⁶). We then applied these fractions to the total count of all US students at the mid-point of our analysis period (that is, 50,115,178 in the school year 2015–2016 (ref. ⁵²)) to estimate the attendance benefits based on the observed effect sizes for buses of those model years.

All data processing and analyses were carried out in SAS v.9.4.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.