Introduction

Societies and decision-makers around the globe are deploying unprecedented non-pharmaceutical interventions (NPIs) to manage the COVID-19 pandemic. These NPIs have been shown to slow the spread of COVID-191,2,3,4, but they also create enormous economic and social costs (for example5,6,7,8,9,10). Thus, different populations have adopted wildly different containment strategies11, and local decision-makers face difficult decisions about when to impose or lift specific interventions in their community. In some contexts, these decision-makers have access to state-of-the-art models, which simulate potential scenarios based on detailed epidemiological models and rich sources of data (for example12,13).

In contrast, many local and regional decision-makers do not have access to state-of-the-art epidemiological models, but must nonetheless manage the COVID-19 crisis with the resources available to them. With global public health capacity stretched thin by the pandemic, thousands of cities, counties, and provinces—as well as many countries—lack the data and expertise required to develop, calibrate, and deploy the sophisticated epidemiological models that have guided decision-making in regions with greater modeling capacity14,15,16. In addition, early evidence suggests a need to adapt models to a local context, particularly for developing countries, where disease, population and other characteristics are different from developed countries, where models are primarily being developed17,18,19.

Here, we aim to address this “modeling-capacity gap” by developing, demonstrating, and testing a simple approach to forecasting the impact of NPIs on infections. This approach is built on two main insights. First, we show that passively collected data on human mobility, which has previously been used to measure NPI compliance20,21,22,23,24,25,26, can also effectively forecast the COVID-19 infection response to NPIs up to 10 days in the future. Second, we show that basic concepts from econometrics and machine learning can be used to construct these 10-day forecasts, effectively emulating the behavior of more sophisticated epidemiological models, including those which incorporate mobility data27,28.

This approach is not a substitute for more refined epidemiological models. Rather, it represents a practical and low-cost alternative that may be easily adopted in many contexts when the former is unavailable. It is designed to enable any individual with access to standard statistical software to produce forecasts of NPI impacts with a level of fidelity that is practical for decision-making in an ongoing crisis.

Data

Our study links information on non-pharmaceutical interventions (NPIs, shown in Fig. 1a) to patterns of human mobility (Fig. 1b) and COVID-19 cases (Figure 1c-d). All data were obtained from publicly available sources. We provide a brief summary of these data here; full details are provided in “Supplementary file 1: Appendix A”.

Figure 1
figure 1

Data on mobility measures, COVID-19 infections and home isolation policy adoption. (a) Home isolation policy adoption, (b) Change in time spent at home, (c) Infection growth rate, and (d) Total confirmed cases are displayed at the county, state and country level. (e) Illustrative example of different mobility measures in California. We utilize data on trips both within and between counties (Facebook and Baidu) as well as the purpose of the trip (Google) and the average distance traveled (SafeGraph).

Full size image
Figure 2
figure 2

Empirical estimates of the effect of NPIs on mobility measures. Markers are country specific-estimates, whiskers show the 95% confidence interval. (a) Estimated combined effect of all policies on number of trips between counties (left) and time spent in specific places (right). (b) Estimated effects of individual policy or policy groups on mobility measures, jointly estimated for each country. (c) Estimated effect of lockdown on mobility the 80 countries which experienced such policy, jointly estimated for each type of mobility.

Full size image

Non-pharmaceutical interventions

We obtain NPI data from two sources. At the sub-national level, we use the NPI dataset compiled by Global Policy Lab2,29. For each sub-national region in five countries, we observe the fraction of the population treated with NPIs in each location on each day. We aggregate 13 different policy actions into four general categories: Shelter in Place, Social Distance, School Closure, and Travel Ban. At the national level, we compiled data on national lockdown policies from the Organisation for Economic Co-operation and Development (OECD)—Country Policy Tracker30, and crowed-sourced information on Wikipedia and COVID-19 Kaggle competitions31.

Human mobility

We source publicly-available data on human mobility from Google, Facebook, Baidu and SafeGraph. These private companies provide free aggregated and anonymized information on the movement of users of their online platform (Fig. 1e). Data from Google indicates the percentage change in the amount of time people spend in different types of locations (e.g., residential, retail, and workplace)32. These changes are relative to a baseline defined as the median value, for the corresponding day of the week, during Jan 3–Feb 6, 2020. Facebook provides estimates of the number of trips within and between square tiles (of resolution up to 360m\(^2\)) in a region33. We aggregate these data to show trips between and within sub-national units. Baidu provides similar data, indicating movement between and within major Chinese cities34,35. Lastly, SafeGraph dataset gives us information on average distance travelled from home by millions of devices across the US36.

COVID-19 cases

For each sub-national and national unit, we obtain the cumulative confirmed cases of COVID-19 from the data repository compiled by the Johns Hopkins Center for Systems Science and Engineering (CSSE)37. The World Health Organization (WHO) provides similar data at the national level but at the moment of writing this paper, no such data are available at the sub-national level38.

Linking data sets

The availability of epidemiological, policy, and mobility data varies across subnational units and countries included in the analysis. We distinguish between three different levels of aggregation for administrative regions - denoted “ADM2” (the smallest unit), “ADM1”, “ADM0.” Our global analysis is conducted using ADM0 data. The country-specific analysis is determined by data availability. Results are provided at the prefecture (ADM2) and province level (ADM1) in China; the regional (ADM1) level in France; the province (ADM2) and region (ADM1) level in Italy; the province (ADM1) level in South Korea; and the county (ADM2) and state (ADM1) level in the United States.

We merge the sub-national NPI, mobility, and epidemiological data based on administrative unit and day to form a single longitudinal (panel) data set for each country. We merge the daily country-level observations to construct a longitudinal data sets for the portion of the world we observe.

Methods

We briefly summarize our methodology below. This discussion is meant to be accessible to a general audience, including policymakers who do not necessarily have advanced training in statistics. Full details, including model equations and estimation methods, are provided in “Supplementary file 1: Appendix B”.

Models

We decompose the impact of an NPI on infections (\(\frac{\Delta infections}{\Delta NPI}\)) into two components that can be modeled separately: the change in behavior associated with the NPI, and the resulting change in infections associated with that change in behavior:

$$\begin{aligned} \frac{\Delta infections}{\Delta NPI} = \frac{\Delta behavior}{\Delta NPI} \times \frac{\Delta infections}{\Delta behavior}. \end{aligned}$$
(1)

We construct models to describe each of these two factors. The “behavior model” describes how mobility behavior changes in association with the deployment of NPIs (\(\frac{\Delta behavior}{\Delta NPI}\)). The “infection model” describes how infections change in association with changes in mobility behavior (\(\frac{\Delta infections}{\Delta behavior}\)). Both models are “reduced-form” models, commonly used in econometrics, that characterize the behavior of these variables without explicitly modeling the underlying mechanisms that link them (cf.2). Instead, these models emulate the output one would expect from more sophisticated and mechanistically explicit epidemiological models—without requiring the underlying processes to be specified. While this reduced-form approach does not provide the same epidemiological insight that more detailed models do, they demand less data and fewer assumptions. For example, they can be fit to local data by analysts with basic statistical training, not necessarily in epidemiology, and they do not require knowledge of fundamental epidemiological parameters—some of which may differ in each context and can be difficult to determine. The performance of these simple, low-cost models can then be evaluated via cross-validation, i.e., by systematically evaluating out-of-sample forecast quality.

Behavior model

For each country, we separately estimate how daily sub-national mobility behavior changes in association with the deployments of NPIs using a country-specific model. In the global model, we pool data across countries and estimate how mobility in each country changes in association with national exposure to NPIs. Each category of mobility on each day is assumed to be simultaneously influenced by the collection of NPIs that are active in that location on that day. A panel multiple linear regression model is used to estimate the relative association of each category of mobility with each NPI. Our approach accounts for constant differences in baseline mobility between and within each sub-national unit—such as differences due to regional commuting patterns, culture, or geography, and differences in mobility across days of the week. These effects are not modeled explicitly but instead are accounted for non-parametrically. “Supplementary file 1: Appendix B.1” contains details of the modeling approach.

Infection model

As with the behavior model, we model the daily growth rate of infections at the local, national, and global scale. In each location, we model the daily growth rate of infections as a function of recent human mobility and historical infections. The approach does not require epidemiological parameters, such as the incubation period or \(R_0\), nor information on NPIs.

In practice, we estimate a distributed-lag model where the predictor variables are mobility rates in that location for the prior 21 days, and the dependent variable is the daily infection growth rate, constructed as the first-difference of log confirmed infections. This approach captures the intuition that human mobility is a key factor in determining rates of infection, but does not require parametric assumptions about the nature of that dependency. The model also accounts for constant differences in baseline infection growth rates within each locality—such as those due to differences in local behavior unrelated to mobility, differences across days of the week, and changes in how confirmed infections are defined or tested for. This approach is also robust to incomplete rates of COVID-19 testing, uneven patterns of testing across space, and gradual changes in testing over time2—see “Supplementary file 1: Appendix B.2” for details.

We fit the model using historical data from each location, and follow stringent practices of cross-validation to ensure that the models are not ‘overfit’ to historical trends. The accuracy of the forecast is then evaluated against actual infections observed during the forecast period, but which were not used to fit the model. Models are fit at the finest administrative level where data are available and forecasts are aggregated to larger regions to evaluate the ability of the model to predict infections at different spatial scales. “Supplementary file 1: Appendix B.2” contains details of the modeling approach.

In principle, such future forecasts can be used by decision-makers who are able to influence local mobility through policy and/or NPIs, perhaps informed either by a behavioral model or observation. Here, we test the quality of the infection model to generate forecasts by simulating and evaluating what a forecaster would have predicted had they generated a forecast at a historical date. In the forecasts presented here, we assume that mobility remains at the level observed during the forecast period—although in practice we expect that decision-makers would simulate different forecasts under different mobility assumptions to inform NPI deployment and policy-making.

Results

We first present results from our behavior model, characterizing the mobility response of different populations to different NPIs. We then evaluate the infection model’s ability to forecast COVID-19 infections based on these same mobility measures. We conclude by discussing how these models could be used to guide policy decisions at local and regional scales.

Mobility response to NPIs

We estimate the reduction in human mobility associated with the deployment of NPIs by linking comprehensive data on policy interventions to mobility data from several different countries at multiple geographic scales. We find that the combined impact of all NPIs reduced mobility between administrative units (Facebook/Baidu) by 73% on average across the countries with sub-national policy data (Fig. 2a). The combined effects were of similar magnitude in China (− 78%, se = 8%), France (− 88%, se = 27%), Italy (− 85%, se = 12%), and the US (− 69%, se = 6%); no significant change was observed in South Korea, where mobility was not a direct target of NPIs (for example39). Excluding South Korea, we estimate that all policies combined were associated with a decrease in mobility by 81% . The general consistency of these magnitudes across countries holds for alternative measures of mobility: using Google data we find that all NPIs combined result in an increase in time spent at home by 28% (se = 2.9), 24% (se = 1.3), and 26% (se = 1.3) in France, Italy, and the US, respectively. This was achieved, in part, by reducing time spent at workplaces by an average of 59.8% and time in commercial retail locations by an average of 78.8%.

Figure 3
figure 3

Short term prediction of COVID-19 cases. Solid line is the recorded number of COVID-19 infections, markers show data in our training sample (blue) and our predictions estimated using mobility measures (orange) versus a model without mobility (green). Model with no mobility measures consistently over-predict the number of infections and drift away quickly from the observed data. (a) This pattern is confirmed when aggregating locally estimated predictions (left) at the state (middle) and country (right) level. (b) Similarly, predictions obtained from country level estimates are significantly more accurate when a measure of mobility is included.

Full size image
Figure 4
figure 4

Evaluation of forecast errors for the infection model. For Italy, US and China, forecasts are evaluated at the finest administrative level (ADM2), as well as aggregated to larger regions (ADM1). For each ADM2 region and forecast length, the mean is taken over all available forecast dates, and the error is evaluated using that mean. Boxplots display the distribution of these percentage errors for each ADM2 region. These are then aggregated to ADM1 level (right panel), for both models including and excluding mobility variables. Similarly, for data fitted at a global level (bottom-most plot), for each country and forecast length, the mean is taken over all forecast dates.

Full size image

We estimate the impact of each individual NPI on total trips (Facebook/Baidu) and quantity of time spent at home and other locations (Google) accounting for the estimated impact of all other NPIs. Travel bans are significantly associated with large mobility reductions in China (− 70%, se = 7%) and Italy (− 82%, se = 25%), where individuals stayed home for 10% more time, but not in the US (Fig. 2b). School closures were associated with moderate negative impacts on mobility in the US (− 26%, se = 10%) and increased time at home (4.6%, se = 0.7%) but slight positive impacts in Italy (33%, se = 7%) and France (15%, se = 7%). Other social distancing policies, such as religious closures, had no consistent impact on total trips but were associated with individuals spending more time at home in the US (11.5%, se = 1.6%) and more time in retail locations in Italy (17.6%, se = 4.8%). Similarly, the national emergency declaration was associated with significant mobility reductions in China (- 62.6 %, se = 12.7 %). Shelter-in-place orders were associated with large reductions in trips for the US (− 60.8%, se = 8%), Italy (− 38.4%, se = 35%), and France (− 91.2%, se = 13.6%), and large increases in the fraction of time spent in homes (8.9%, 22.1%, 28%, respectively). Shelter in place orders did not appear to have large impacts in South Korea or China. This is consistent with earlier policies (such as the Emergency Declaration) restricting movement in China earlier than the shelter in place orders, while mobility in South Korea was never substantially affected by NPIs.

Globally, we find evidence that lockdown policies were associated with substantial reductions in mobility (Fig. 2c). Across 80 countries, the average time spend in non-residential locations decreased by 40% (se = 2%) in response to NPIs. Time spent in retail locations is the most impacted category, declining 49.9% (se = 2%). Some of the variation in response across countries (grey dots) likely reflects different social, cultural, and economic norms; measurement error; and statistical variability. In “Supplementary file 1: Appendix C”, we disaggregate this effect temporally, and find that the most significant reductions occur during the first eight days after a lockdown (Figure S1c).

In “Supplementary file 1: Appendix C”, we further exploit the granular resolution of the mobility data to investigate whether localized policies also impacted neighboring regions (Figure S1). In the USA and Italy, the impact of NPIs on mobility was highly localized, with little evidence of spatial spillover effects (“Supplementary file 1: Appendix C - Figure S1a”). In China, the evidence is more mixed, with some evidence of spillovers between neighboring cities (“Supplementary file 1: Appendix C - Fig S1b”).

Forecasting infections based on mobility

We find that mobility data alone are sufficient to meaningfully forecast COVID-19 infections 7–10 days ahead at all geographic scales – from counties and cities (ADM2), to states and provinces (ADM1), to countries (ADM0) and the entire world. Furthermore, identical models that exclude mobility data perform substantially worse, suggesting an important role for mobility data in forecasting.

Figure 3 illustrates the performance of model forecasts in several geographic regions and at multiple scales. The true infection rate is shown as a solid line; data used to train each model are depicted in blue dots, and the forecast of our model is shown in orange, contrasted against a model with no mobility data in green. Forecasts that account for current and lagged measures of mobility generally track actual cases more closely than forecasts that do not account for mobility. For example, a forecast made for the period 4/06/2020–4/15/2020 for California-Los Angeles on 4/15/2020 without mobility projects 30,716 cases, while the same forecast accounting for mobility would be 12,650 cases, much closer to the 10,496 that was observed. Figure 3b depicts projected cases for the entire world based on this reduced-form approach, estimated using country-level data mobility data from Google.

Figure 4 summarizes model performance across all administrative subdivisions of each of the three countries we consider for the forecast analysis (China, Italy, and the United States). We show the distribution of model errors over all ADM2 and ADM1 regions at forecast lengths ranging from 1 to 10 days. Table 1 summarizes each distribution using the median.

Table 1 Median percentage error for each model and day of forecast, as plotted in Fig. 4. The error is presented for each model and geographical region, and for 1 to 10 day forecasts.
Full size table

In all geographies and at all scales, models with mobility data perform better than models without. In general, sub-national forecasts in China benefit least from mobility data, but forecasts in Italy and the US are substantially improved by including a single measure of mobility for the 21 days prior to the date of the forecast. At the local (ADM2) level in Italy, the MPE is −1.73% and 13.27% for five and ten days in the future when mobility is accounted for, compared to 45.81% and 167.97% when it is omitted. In the US, MPE is 7.00% (5-day) and 20.75% (10-day) accounting for mobility, and 23.79% and 79.47% omitting mobility. In China, MPE is 4.18% (5-day) and 131.09% (10-day) accounting for mobility, and 16.83% and 128.80% omitting mobility. At the regional (ADM1) level, MPE rates are similar but extreme errors are reduced, largely because positive and negative errors cancel out. Country-level forecasts, which use country-level mobility data from Google, benefit relatively less than sub-national model from including mobility information, in part because baseline forecast errors are smaller. For countries in our sample, MPE is 6.35% (5-day) and 15.24% (10-day) accounting for mobility, and 11.46% and 31.12% omitting mobility.

Model application in decentralized management of infections

Our results suggest that a simple reduced-form approach to estimating model (1) may provide useful information and feedback to decision-makers who might otherwise lack the resources to access more sophisticated scenario analysis. We imagine the approach can be utilized in two ways. First, a decision-maker considering an NPI (either deploying, continuing, or lifting) could develop an estimate for how that NPI might affect behavior, based on our analysis of different policies above (Fig. 2). Using these estimated changes in mobility, they could then forecast changes in infections using the infection model described above—but fit to local data.

Table 2 provides an example calculation for how a novel policy that increased residential time (observed in Google data) would alter future infections, using estimates from the global-level model. For example, a policy that increases residential time by 5% in a country is predicted to reduce cumulative infections ten days later, to 82.5% (CI: (78.2, 87.0)) of what they would otherwise have been. Similar tabulations can be generated by fitting infection models using recent and local data, which would flexibly capture local social, economic, and epidemiological conditions.

Table 2 Example: Estimating the effect of a mobility-reducing policy on infections for the global model (unit of observation is a country day). The values in the table are the ratio of the cumulative number of cases after up to 10 days, if residential time over baseline was increased in a country at day 0 by \(\Delta\) = 1%, 5%, 10% or 20% from their original values. These values are estimated using coefficients of the mobility variables derived from the pooled global model (details in “Supplementary file 1: Appendix B.2”). Note that each column compares to the value on its first row (indicated by the value 1). An example interpretation is: if a country increases residential time by 5%, cumulative infections ten days later is predicted to be 82.5% of what they would have been with no change in mobility.
Full size table

A second way that a decision-maker could use our approach would be to actually deploy a policy without ex ante knowledge of the effect it will have on mobility, instead simply observing mobility responses that occur after NPI deployment using these publicly available data sources. Based on these observed responses, they could forecast infections using our behavior model.

Discussion

The COVID-19 pandemic has led to an unprecedented degree of cooperation and transparency within the scientific community, with important new insights rapidly disseminated freely around the globe40. However, the capacity of different populations to leverage new scientific insights is not uniform. In many resource-constrained contexts, critical decisions are not supported by robust epidemiological modeling of scenarios. Here we have demonstrated that freely available mobility data can be used in simple models to generate practically useful forecasts. The goal is for these models to be accessible to a single individual with basic training in regression analysis using standard statistical software. The reduced-form model we develop generally performs well when fit to local data, except in China where it cannot account for some key factors that contributed to reductions in transmission.

A key insight from our work is that passively observed measures of aggregate mobility are useful predictors of growth in COVID-19 cases. However, this does not imply that population mobility itself is the only fundamental cause of transmission. The measures of mobility we observe capture a degree of “mixing” that is occurring within a population, as populations move about their local geographic context. This movement is likely correlated with other behaviors and factors that contribute to the spread of the virus, such as low rates of mask-wearing and/or physical distancing. Our approach does not explicitly capture these other factors—and thus should not be used to draw causal inferences—but is possible that our infection model performs well in part because the easy-to-observe mobility measures capture these other factors by proxy.

The simple model we present here is designed to provide useful information in contexts when more sophisticated process-based models are unavailable, but it should not necessarily displace those models where they are available. In cases where complete process-based epidemiological models have been developed for a population and can be deployed for decision-making, the model we develop here could be considered complementary to those models. Future work might determine how information from combinations of qualitatively distinct models can be used to optimally guide decision-making.

We also note that the reduced-form model is designed to forecast infections in a certain population at a restricted point in time. It achieves this by capturing dynamics that are governed by many underlying processes that are unobserved by the modeler. However, because these underlying mechanisms are only captured implicitly, the model is not well-suited to environments where these underlying dynamics change dramatically. In such circumstances, process-based models will likely perform better. The reduced-form approach presented here can still be applied in such circumstances, but it may be necessary to refit the model based on data that is representative of current conditions. Similarly, when our reduced-form model is applied to a new population, it should be fit to local data to capture dynamics representative of the new population.

The approach we present here depends critically on the availability of aggregate mobility data, which is currently provided to the public by private firms that passively collect this information. At the time of writing, these mobility datasets are publicly available in 135 and 152 countries for Google and Facebook, respectively. In lower-resource settings, where use of smartphones is less common, the users who generate mobility data may not be as representative of the total population as in wealthy nations, but prior work suggests that biases in phone ownership may not dramatically bias estimates of overall population mobility41. In such contexts, anonymized metadata from mobile phone operators is increasingly being made available for research and policy interventions42,43, and offers a promising source of data for public health applications44.

We hypothesize that the approach we develop here might skillfully forecast the spread of other diseases besides COVID-19. If true, this suggests our approach could provide useful information to decision-makers for managing other public health challenges, such as influenza or other outbreaks, potentially indicating a public health benefit from firms continuing to made mobility data available—even after the COVID-19 pandemic has subsided.