Abstract
Despite a number of successful approaches in predicting the spatiotemporal patterns of the novel coronavirus (COVID-19) pandemic and quantifying the effectiveness of non-pharmaceutical interventions starting from data about the initial outbreak location, we lack an intrinsic understanding as outbreak locations shift and evolve. Here, we fill this gap by developing a country distance approach to capture the pandemic’s propagation backbone tree from a complex airline network with multiple and evolving outbreak locations. We apply this approach, which is analogous to the effective resistance in series and parallel circuits, to examine countries’ closeness regarding disease spreading and evaluate the effectiveness of travel restrictions on delaying infections. In particular, we find that 63.2% of travel restrictions implemented as of 1 June 2020 are ineffective. The remaining percentage postponed the disease arrival time by 18.56 days per geographical area and resulted in a total reduction of 13,186,045 infected cases. Our approach enables us to design optimized and coordinated travel restrictions to extend the delay in arrival time and further reduce more infected cases while preserving air travel.
Introduction
Given 41,570,883 confirmed cases of COVID-19 and 1,134,940 deaths worldwide as of 23 October 20201,2,3, the need for world to deploy non-pharmaceutical interventions prior to comprehensive consideration is urgent4,5,6. Today’s high population density and the high volume, speed, and non-locality of human mobility provide perfect conditions for an epidemic to spread7,8,9,10 and simultaneously raise the challenges related to the development of non-pharmaceutical intervention strategies on the timescale that modern diseases spread11,12,13. Although the practice of quarantine and social distancing protocols can drastically reduce the virus propagation locally6,14,15,16, the global COVDI-19 pandemic patterns are shaped by the global mobility network (GMN), which determines when and where the disease arrives globally17,18. Consequently, the straightforward way to diminish the international importation of COVID-19 involves imposing radical travel restrictions (i.e., entry bans, global travel bans, and lockdowns)19,20,21, which reduce the entry of airline passengers into a country. According to available data as of 1 June 2020, 187 geographical areas imposed the entry bans, 111 geographical regions imposed global travel bans, and 38 geographical areas imposed the full lockdowns to prevent their citizens and tourists from traveling overseas5,22. However, researchers demonstrated that these travel restrictions were only effective at the beginning of the outbreak23. When such interventions fail to control the initial outbreak, they instead disrupt the healthcare aid and support, business, and cause extensive and profound social and economic damage24,25. Therefore, it is crucial to assess and impose effective travel restrictions to avoid unnecessary and costly responses to COVID-19 from the side of governments26.
Measuring the effectiveness of travel restrictions often relies on the specific epidemic models4,27,28,29,30, which require accurate estimation of the disease’s epidemiological parameters, such as the basic reproductive number (R0). However, the parameter estimations are often unreliable due to the daily changes in underreported cases and various errors arising from the lack of diagnosis tests11,31,32. Furthermore, these models are difficult to calibrate due to incomplete information (i.e., partial network topology33 or unknown dynamics19,34). Overall, it is unclear how much detail is required to achieve a certain level of predictive accuracy. Human mobility plays a significant role in understanding hidden spatiotemporal spreading patterns35 and enables us to predict the arrival time17,36,37 and number of infected cases38. In particular, effective distance, a method introduced by Brockmann and Helbing17, measures the mobility from the initial outbreak location (OL) to the target geographical area by discarding other redundant connections. This method allows to predict the arrival time and epidemic wavefront without knowing the epidemiological parameters and has already been demonstrated useful in the 2009 H1N1 pandemic and the 2003 SARS epidemic. Additionally, the initial OL’s mobility outflow is a vital predictor for the log-transformed cumulative infections in the destination locations38,39, and was validated by the Wuhan’s outflow to each prefecture in mainland China in the early stage of the COVID-19 pandemic.
Here, by simulating the spread of disease with the meta-population susceptible–infected–removed (SIR) model, we show that the effective distance method is successful in predicting the evolution of the pandemic when Mainland China is the only OL. As shown in Fig. 1a, b, the effective distance accurately estimates countries’ arrival times with an R-squared value R2 = 0.87 and log-transformed cumulative infected cases with R2 = 0.88. However, when multiple OLs appeared, effective distance weakly correlated with disease dynamics with R2 ≤ 0.4 (see in Fig. 1c, d). We argue that the observed underperformance is due to the effective distance’s underlying assumption of a single OL, which neglects other OLs’ potential in exporting viruses to other areas through air travel. The presence of varying OLs is essentially related to the variability in countries’ health responses to COVID-19. In fact, the time-varying nature of OLs poses challenges in capturing the ongoing pandemic’s spatiotemporal pattern and evaluating the effects of travel restrictions. This feature prompts new mathematical tools to advance the understanding of global disease dynamics under unprecedented non-pharmaceutical interventions. To address this gap, we propose a country distancing method that captures global disease dynamics when OLs evolve. Our approach takes inspiration from the connection laws in electric circuits with resistances in series and parallel configurations. We apply this approach to quantify the effectiveness and efficiency of existing travel restrictions and design optimized and coordinated travel restrictions to maximize their impact in slowing infections.
a, b depict the global disease dynamics, i.e., arrival times Tm and log-transformed infected cases Im(t) (e.g., t = 50) as functions of effective distance from the initial OL, respectively. However, when the single OL evolves to multiple OLs (e.g., five OLs), see c and d, the effective distance from the initial OL fails to correlate with arrival times Tm and log-transformed infected cases Im(t), with an R-squared value (R2) < 0.4. Circles represent geographical areas, and those belonging to the same continent are presented in the same color. Both arrival time and infected cases are obtained from the meta-population susceptible–infected–removed model, with epidemiological parameters provided by the literature55,56.
Results
Mathematical framework of country distancing
We tested the international spread of the disease on the GMN (see Supplementary Table 1). The GMN, which is provided by the Official Aviation Guide, is presented by a matrix of airline passenger influx F among 250 geographical areas, representing countries, dependent territories, and special areas of geographical interest. Here, Fmn (Fmn ∈ F) expresses the airline passenger influx from area n to the area m. For a single OL, the diseases may propagate to a geographical area through distinct paths, the shortest of which predicts the infection to the destination17. For area n and its connected area m, their effective distance is \({d}_{m| n}=1-{\mathrm{log}}\,{P}_{mn}\), indicating that a larger fraction of air travel Pmn (\({P}_{mn}=\frac{{F}_{mn}}{{\sum }_{k}{F}_{kn}}\)) means a smaller distance, and vice versa. Then, for an arbitrary area m that can be reached by n through a path τ, the effective distance is the sum of effective lengths along the links of the shortest path, \({d}_{m| n}=\mathop{\min }\limits_{\tau }{\sum }_{(i,j)\in \tau }{d}_{j| i}\). We call this series connection law (see Fig. 2a) because it is analogous to the effective resistance in series circuits defined as R = ∑iRi, where Ri is the resistance that is connected along a chain.
By deriving the series and parallel connection law (a, b) for global disease transmission, we derive the country distancing method to capture the global disease spread dynamics, i.e., the arrival times and new infected cases. Rn and Rc represent the resistances in the circuit. Nodes n, c, and m represent the geographical areas and the directed links between the nodes represent the traffic flow. The disease propagates from one OL (e.g., n) to destination m, on par with series connection law; whereas the disease propagates from multiple OLs (e.g., n and c) to destination m, on par with parallel connection law. c, d separately show the R-squared values (R2) for evaluating global disease dynamics versus country distancing and effective distance. Country distancing correlates strongly with remaining arrival times Tm(t) (c) and log-transformed new infected cases Im(t) − Im(t − 1) (d) when multiple OLs are present. The R2 values for country distancing keep above than 0.7 although OLs are shifting, whereas the R2 values for effective distance decline sharply given that multiple OLs emerge. e, f are the slopes of linear correlations for c and d, representing the speed of arrival times and speed of infection. The shaded areas are the 95% confidence intervals for the speeds. Outbreak locations are defined as the geographic areas whose active confirmed case greater than 0.01% population. On 1 June 2020, 146 geographical areas are OLs. Similar to Fig. 1, both arrival time and infected cases are obtained from the meta-population susceptible–infected–removed model.
The number of OLs may grow or shrink, which escalates or diminishes the importation risk for the other geographical areas and simultaneously increases/reduces geographical areas’ distance to the risky sources, respectively. Using the meta-population SIR model, the disease evolution on this model becomes segmented according to the presence of OLs (see “Methods” and Supplementary Note 2). Motivated by the effective distance17 and propagation processes simulated by the meta-population SIR model with multiple OLs, we derive a formula that examines areas’ closeness to all existing OLs through the parallel connection law. For example, the disease propagates from two OLs, n and c, to the destination geographical area, m, with effective distances dm∣n, and dm∣c, respectively (see Fig. 2b). The overall likelihood of transmitting from both OLs satisfies \({e}^{{d}_{m| \left\{n,c\right\}}}\propto \frac{1}{\frac{1}{{e}^{{d}_{m| n}}}+\frac{1}{{e}^{{d}_{m| c}}}}\). This process is similar to the effective resistance in parallel circuits that \(R=\frac{1}{\frac{1}{{R}_{n}}+\frac{1}{{R}_{c}}}\). For the general case, we derive the series and parallel connection law for global disease transmission and formulate it as country distancing (see Supplementary Note 3):
where NI is the OL set and M is the total number of geographical areas (M = 250). Note that a large set of OLs may lead to small country distancing, and a larger portion of passenger influx leads to a smaller effective distance and may further lead to smaller country distancing.
Global disease dynamics for the evolving outbreak locations
Two fundamental properties describe the main spread dynamics of the COVID-19 pandemic: the arrival time (Tm), i.e., the date of the first confirmed case, and the cumulative infected cases [Im(t)] in a geographical area m. As numerous undetected, missing, undiagnosed, or unreported COVID-19 cases result in biased arrival times and infected cases in the collected real-world dataset31,32,40, we simulate the spread of COVID-19 by adopting the meta-population SIR model41 in the segmented time interval according to the presence of different OL sets. The model is validated by (1) defining the OL NI(t) as the geographic areas having greater than 0.01% of the infected population at time t; (2) ratifying the strengths for entry bans, global travel ban, and lockdown in 90% effective in limiting travels according to the International Civil Aviation Organization42; and (3) assuming domestic trips alter the local infection rate. The simulated data resemble the arrival times and infected cases within the given ranges as of 1 June 2020 (see Supplementary Table 3). We conducted extensive sensitivity analysis and model validation using different combinations of parameters, as shown in Supplementary Note 2.
Evidence shows that human mobility determines arrival times12,17,43 and infected cases38 when there is only one OL. However, these approaches are not suitable for the presence of multiple OLs because it is unclear how each OL contributes to the arrival time and infected cases in each geographical area. Our approach compresses multiple OLs to a single one and calculates the corresponding effective mobility using the series and parallel connection law for global disease transmission. As demonstrated in Fig. 2c, d, country distancing correlates with the simulated arrival times and the logarithm of the simulated infected cases with a coefficient of determination R2 > 0.7 regardless of the number of OLs at time t. In addition, R2 for country distancing is consistently far above its value for effective distance, which is ≤0.3. For a robustness check, we also test the predictability of country distancing for different criteria of selecting OLs in Supplementary Figs. 1–5. All the result indicates that instead of effective distance, country distancing is an excellent predictor of global disease dynamics, especially for the evolving OLs. Thus, for each time t, we formulate functional linear relationships between country distancing and remaining arrival times Tm(t) [Tm(t) = Tm − t] and new infected cases Im(t + 1) − Im(t) as:
where v(t) and u(t) are the slopes (speeds), as shown in Fig. 2e, f, representing the rates of change of arrival times/infected cases relative to country distancing. The different OLs and temporal infection rates exhibit variability in speeds and further estimating the effectiveness of travel restrictions (see Supplementary Notes 3 and 4).
Status quo of existing travel restrictions
As of 1 June 2020, 625 travel restrictions have been imposed worldwide. An overview of implemented travel restrictions in given geographical areas is shown in Fig. 3a. As shown in Supplementary Table 2 and Supplementary Fig. 6, 184 geographical areas imposed 476 entry bans from 21 January 2020 to 17 March 2020. These areas that imposed entry bans deny access to noncitizens at some specific geographical regions such as mainland China, South Korea, Japan, Iran, and the Schengen Area. From 11 March 2020, when approximately half of the countries were infected worldwide, entry bans were not sufficient to lower the risk of coronavirus importation from the infected regions44. Consequently, 111 geographical areas imposed the global travel ban to prevent overseas travelers from entering their areas except for their residents, and 38 geographical areas imposed full/national lockdowns to deter people entering and exiting their countries.
a shows geographic areas’ implemented travel restrictions. Each pie in the global map represents the share of entry bans (in red), global travel ban (in orange), and lockdown (in blue) that the area imposed. For example, New Zealand imposed three entry bans, one global travel ban, and one lockdown, whereas Japan imposed four entry bans. b illustrates how the entry ban, global travel ban, and lockdown reduce airline passenger influx. When the colored areas (in red, orange, and blue) enforce the entry ban, the global travel ban, or the lockdown, the passenger influx on corresponding colored links is reduced (dotted links). The outward/inward arrows indicate restricting travelers exiting/entering. c shows the implementation time of existing travel restrictions versus their induced global country distancing increase. Each dot represents the travel restriction. Among the marks, for example, “US-CN” represents the entry ban (“-”) imposed by the United States (US) on mainland China (CN); “*KP” represents the global travel ban (“*”) imposed by North Korea (KP); and “@CN” represents the lockdown (“@”) imposed by mainland China (CN). For clear visualization, geographical areas are presented by two-letter codes. Please see Supplementary Table 5 for geographical areas’ two-letter codes. The marginal charts of c show the standardized box plots of travel restrictions’ implementation time and country distancing increase on the top and right, respectively.
Measuring effectiveness and efficiency of travel restrictions using country distancing
According to the International Civil Aviation Organization42, 90% of passenger seats were lost in the second quarter of 2020. As illustrated in Fig. 3b, for the area n that implemented the travel restrictions (see Eqs. (5)–(7)), (1) the entry ban reduces the passenger influx from banned areas to n with strength α = 90%; (2) the global travel ban reduces the passenger influx from all neighboring areas to enter n with strength β = 90%; (3) the lockdown reduces the passenger influx entering/leaving n with strength γ = 90% for full lockdown. In other words, travel restrictions induce a reduction in passenger influx, leading to larger country distancing (named country distancing increase, see Eqs. (8)–(15)) in Fig. 3c. We map the country distancing increase at each geographic area to the changes in arrival times (named as arrival time delay (ATD)) and changes in infected cases (named as infected case reduction (ICR)) according to the linear relationships in Eq. (2). Given the slopes of the linear correlations, namely, v(t) and u(t) in Eq. (2), the effectiveness of travel restrictions in slowing disease dynamics could be quantified by the world’s average ATD and total ICR with the loss of air travel caused by them.
Through all existing travel restrictions, 36.3% of passenger influx is reduced from GMN, the arrival times of the disease in the world are delayed by 18.56 (95% CI, 15.84–21.28) days on average, and infected cases are reduced by 13,186,045 (95% CI, 3,992,055–40,148,831) cases. However, ~63.2% of travel restrictions are ineffective, resulting in zero country distancing increase. Figure 4a, b shows the effectiveness and efficiency of the remaining 36.8% travel restrictions by representing their ATD/ICR against the cost of losing passenger influx. The means of lost passenger influx (vertical line) and ATD/ICR (horizontal line) divide the travel restrictions into four blocks, i.e., effective and efficient block (top left), effective and inefficient block (top right), ineffective and efficient block (bottom left), and ineffective and inefficient block (bottom right). In addition to the lockdowns imposed by mainland China, the lockdowns imposed by Spain (@ES), New Zealand (@NZ), and South Africa (@ZA) produce crucial ATD for the world. Concurrently, lockdowns established by Italy (@IT) and Spain (@ES) as well as the global travel ban enforced by Turkey (*TR) carry significant ICR for the world. However, these extreme travel restrictions also aggravate the loss of passenger influx. Several travel restrictions, e.g., the entry ban imposed by the United States (US-CN), Hong Kong (HK-CN), and Italy (IT-CN) on mainland China produce comparable ATD or ICR but with considerably less loss of passenger influx. Moreover, 505 (80.8%) travel restrictions generate <0.01 days of ATD and <1000 cases of ICR for the world, suggesting the ineffectiveness of travel restrictions.
a, b depict existing travel restrictions’ arrival time delay (ATD) and infected case reduction (ICR) at the cost of losing passenger influx. Effective and efficient travel restrictions pursue high ATD or ICR with a low cost of passenger influx. Circles represent travel restrictions, and their size is proportional to the number of geographical areas influenced by the travel restrictions. The vertical and horizontal dashed lines individually represent the average of lost passenger influx and the average of total ATD/ICR for all travel restrictions. The markers for travel restrictions are the same as those in Fig. 3. The two-letter codes for geographical areas can be found in Supplementary Table 5.
Quantification of the impact of travel restrictions through country distancing increase
Figure 4 indicates that entry bans to the OLs are most effective and efficient. The areas implementing such entry bans and their descendants in the shortest path tree are distant from the OLs. As shown in Fig. 5a, by implementing an entry ban on mainland China on 31 January 2020, the United States increased country distancing by 40.96 in total. This means that the entry ban produces 8.10 days of ATD for the 28 areas, which are descendants of the United States (shaded area). By imposing a global travel ban on 21 January 2020 (see Fig. 5b), North Korea’s country distancing rises by 2.30 (13.53 days of ATD, 0 cases of ICR) but induces no country distancing change for other countries, because North Korea is a leaf node. The findings suggest that the travel restrictions imposed by the areas with more descendant areas in the shortest path tree are more effective in distancing the world from coronavirus importation risk. The OLs, which are the sources of the shortest path tree, have the most significant influence. By imposing the national lockdown on 8 February 2020 (see Fig. 5c), mainland China increased the country distancing by 522.6 in total. Specifically, the lockdowns produced 10.81 (95% CI, 9.23–12.39) days of ATD per geographical area and 6,518,277 (95% CI, 1,948,391–19,955,176) ICR in total, representing 58.26% of ATD and 49.43% of ICR generated by all travel restrictions. However, as the number of OLs grows (multiple OLs), as shown in Fig. 5d–f, the distance to OLs is reduced. Substantial intervention efforts cause relatively smaller country distancing increases and fewer health benefits in delaying infection. For example, see Fig. 5d, e, the entry ban imposed by the United States to Japan on 1 March 2020 and the global travel ban imposed by New Zealand on 19 March 2020 both leads to <0.07 ATD and <2000 ICR. The same phenomenon is observed in Italy, one of the 38 sources in the shortest path tree, which imposed a national lockdown on 23 March 2020 (see Fig. 5f). This lockdown led to a 3.75 increase in country distancing. The lockdown also produces 0.007 (95% CI, 0.006–0.008) days of ATD per geographical area and 13,590 (95% CI, 10,569–17,191) cases of ICR in total, accounting for nearly 0% of ATD and 0.1% of ICR generated by all travel restrictions. Our findings indicate that two factors determine the effectiveness of travel restrictions. One is the geographical areas’ position that imposed travel restrictions on the shortest path tree; another is the implementation date. Insufficient consideration of both factors leads to ineffectiveness and further inefficiency of travel restrictions.
Corresponding to Figs. 3 and 4, the impact of six example travel restrictions on each geographical area are visualized through the shortest path tree and global maps. a Entry bans imposed by the United States to Mainland China. b Global travel ban imposed by North Korea. c Lockdown imposed by Mainland China. d Entry bans imposed by the United States to Japan. e Global travel ban imposed by New Zealand. f Lockdown imposed by Italy. The paths from root (OLs) to other nodes (areas) construct the shortest path tree. The area that imposed travel restriction is marked with a green star, whereas the areas with no country distancing increase are in gray. The star area’s travel restriction directly increases its descendants nodes’ distance from the OL. As the number of OLs grows in d–f (multiple OLs merge to the root), the geographical areas’ country distancing is reduced. The colors of the areas in the trees correspond to the colors on the corresponding maps. See Supplementary Fig. 12 for the shortest path tree before any travel restrictions are implemented.
Geographic areas’ contributions and gains
Geographical areas implemented different travel restrictions. By integrating the health benefits of travel restrictions enforced by the same area, we find that mainland China (CN) has a dominant contribution followed by Australia (AU), the United States (US), New Zealand (NZ), the Netherlands (NL), and Russia (RU), as shown in Fig. 6a (also see Supplementary Fig. 10). This finding suggests that mainland China is the most influential area in mitigating the spread of COVID-19. On the other hand, Italy (IT), Germany (DE), Hong Kong (HK), Turkey (TR), and Taiwan (TW) contribute a great number of ICRs by imposing travel restrictions. Figure 6b, c shows the gains of ATD and ICR in geographic areas due to all collected travel restrictions. Note that Tuvalu (TV), Niue (NU), Lesotho (LS), Tonga (TO), and North Korea (KP), which were not infected until 1 June 2020, were the areas with the most ATD, i.e., 125, 73, 73, 71, and 70 days, respectively. Otherwise, Italy (IT), Hong Kong (HK), Germany (DE), South Korea (KR), and Taiwan (TW), which are infected at the early stage of the outbreak, are the areas with the highest ICR, i.e., 9,781,603, 649,010, 630,612, 298,068, and 210,577 cases, respectively.
According to existing travel restrictions until 1 June 2020, (a) geographic areas contributed to increasing other countries’ distances from the outbreak locations. The size of the circle in each area is proportional to its contribution to the global country distancing increase. The curved lines depict the distancing from one area to the destination area. b, c show the geographic areas’ gains in terms of days of arrival time delay and counts of infected case reduction.
Optimal and coordinated travel restrictions
We find that most of the existing travel restrictions are ineffective for two reasons: (1) the travel restrictions are imposed by geographical areas in an uncoordinated manner out of self-interest, failing to contribute to the global public interest; (2) the sole travel restriction is not enacted in an optimal time and at optimal locations for the most significant self-interest. Furthermore, these inefficient travel restrictions have created a substantial unnecessary loss of passenger influx, ultimately damaging the global economy and social stability25. These findings prompted us to design the strategic plans for when and where to impose each travel restriction tailored to the real-time national context. Specifically, we formulate a bi-objective optimization problem of maximizing travel restrictions’ the effectiveness (country distancing increase) and efficiency (the loss of airline passenger influx) (see Eq. (16)). If governments worldwide coordinately implemented the optimal travel restrictions, the infection could be dramatically delayed and avoided.
Using the Non-dominated Sorting Genetic Algorithm (NSGA-II), we obtain non-dominated solutions for each entry ban and present the approximate optimal solution, which has the largest country distancing increase in Fig. 7. Our numerical results show that the optimal and coordinated travel restrictions significantly outperform the existing travel restrictions in the three features, i.e., lost passenger influx, the average ATD, and the total ICR. As shown in Fig. 7a–c, the coordinated travel restrictions reach of an average ATD of 62.39 days (95% CI, 53.36–71.43), with 7.81% of lost passenger influx. The ICR of the coordinated travel restrictions is 19,147,287 (95% CI, 9,761,103–46,165,850) cases. Unlike the existing travel restrictions suggesting that mainland China contributes most ATD and ICR for the world, the coordinated travel restrictions work as a whole-of-government and whole-of-society approach with many geographical areas contributing substantially to an increased ATD and ICR for the world45. As shown in Fig. 7d, mainland China occupies small portions of the ATD and ICR contributions, which decline to 1.75% and ~0% from 58.26% and 49.43%, respectively, when a coordinated approach is adopted. Concurrently, other geographical areas contribute more when using a coordinated approach. For example, the United States’ portion of ATD and ICR contributions could rise to 9.36% and 2.95% from 4.89% and 0.46%, respectively.
The comparisons are performed for arrival time delay (a), infected case reduction (b), and lost passenger influx (c). The shaded areas in a, b present the 95% confidence intervals for the estimated value according to the speeds of arrival times/infected cases (Fig. 2e, f). d Pie charts illustrate the comparisons of the proportion of geographical areas’ contributions to inducing arrival time delays and infected case reductions. Each solution of optimized travel restriction, which comprises a set of coordinated entry bans, is the non-dominated solution determined using the non-dominated sorting genetic algorithm II. The markers for travel restrictions a–c are the same as in Fig. 3 and the two-letter codes for geographical areas can be found in Supplementary Table 5.
Discussion
We developed a country distancing method that captures the global disease dynamics when OL evolve. Our method enables us to systematically quantify the effectiveness and efficiency of travel restrictions (i.e., entry bans, global travel bans, and lockdowns), which delay or prevent infections at the cost of loss of air travels. Our analysis confirms findings from existing studies4,35,39,46, which conclude that travel restrictions are more effective at the early stage of the pandemic. Our study reveals that the coexistence of multiple OLs substantially weakens the effectiveness of travel restrictions. In addition, travel restrictions’ effectiveness is related to the geographical areas’ capability to cut possible disease propagation paths from existing OLs to other countries. By maximizing the health impact of travel restrictions, in other words, increasing the country distancing from OLs and minimizing the loss of air travel, we find that the well deployment of entry bans to OLs via global joint efforts as early as possible is sufficient to fight effectively against COVID-19. The optimized and globally coordinated travel restrictions enable the sustainable suppression of transmission at a low level47,48, without the use of radical approaches, such as the global travel bans.
Three main limitations can lead one to overestimate/underestimate travel restrictions’ effectiveness: (1) incomplete and biased travel restrictions data; (2) homogeneous assumptions on the strengths of different travel restrictions; (3) ignorance that the combined effect of travel restrictions and local anti-contiguous policies6, such as social distancing policy, work from home, and school closure, has a substantial effect on reducing the rate of global spread21,49. Understandably, these three limitations could be overcome with more available data of the daily GMN. Furthermore, COVID-19 undertesting and underreporting50 would overvalue the impact of travel restrictions on slowing infection. Nevertheless, the approach in this study and its implications may help to control the spread of COVID-19.
The key advantage of the country distancing approach is that it captures the global diffusion pattern despite the heterogeneous responses of governments to the pandemic and varying OLs. The idea of quantifying the health impact of travel restrictions is useful in the ongoing pandemic and future diseases. Although many geographical areas attempt to ease their travel restrictions by escalating the testing and mandatory quarantines for the incoming travelers, it is clear that addressing how people/infected people move is key to controlling the disease. Pinpointing the significance of controlling OLs provides insights to properly distribute and administer a COVID-19 vaccine when it becomes available. In summary, the use of country distancing approach complements existing studies highlighting the significance of mobility from both the OLs and hubs in GMN. Furthermore, this study also seeks to balance optimally positive health effects of travel restrictions with their negative impact on individuals’ free movement. Knowing that the COVID-19 pandemic is more than a health crisis and may last until 202251, geographical areas are continuously enduring the coronavirus importation risk from other infected areas and enduring social instability. We emphasize the necessity of a global joint effort to implement travel restrictions when it seems impossible to curb the spread of COVID-19 with isolated actions. Although limitations exist, this study indicates that whole-of-government and whole-of-society approaches are necessary to fight against coronavirus and strengthen pandemic preparedness in the future45.
Methods
Model with multiple outbreak locations
We adopt the meta-population SIR model17,52,53 to simulate the spread of disease in GMN G when OLs are changing. In the meta-population SIR model, each geographical area n has a population size of Ωn. The evolving disease within the population is governed by three states (i.e., susceptible \({s}_{n}=\frac{{S}_{n}}{{{{\Omega }}}_{n}}\), infectious \({i}_{n}=\frac{{I}_{n}}{{{{\Omega }}}_{n}}\), removed \({r}_{n}=\frac{{R}_{n}}{{{{\Omega }}}_{n}}\)), and the disease evolution between populations is described by the travel influx coupling term pmn. For each set of OL NI(t):
with the initial condition given as:
from t to t + 1. We show the definitions of the parameters as below:
-
\({N}_{I}(t)=\{n| \forall n\in N,{I}_{n}^{o}(t)\ge \theta \}\) are the defined OL set, consisting of the geographical areas whose reported cases \({I}_{n}^{o}(t)\) are greater than the threshold θ. The time range for NI(t) is from 8 December 2019 to 1 June 2020. For the initial outbreak date t0 = 2019 − 12 − 08, mainland China is the OL with \({I}_{n}^{o}({t}_{0})=1\).
-
a(t) is the infection rate, and b(t) is the remove rate, which is defined as the sum of recovery rate and death rate.
-
\(c(t)=\frac{{\sum }_{m,n}{F}_{mn}(t)}{{\sum }_{n}{{{\Omega }}}_{n}}\) is the mobility rate.
-
\({p}_{mn}(t)=\frac{{F}_{mn}(t)}{{\sum }_{k}{F}_{kn}({t}_{0})}\) is the fraction of travel influx, where the influx matrix Fmn(t) is altered by travel restrictions; see “Passenger influx reduction”.
-
\(\sigma ({i}_{n}/\varepsilon )=\frac{{({i}_{n}/\varepsilon )}^{\eta }}{{({i}_{n}/\varepsilon )}^{\eta }+1}\) is a sigmoid Hill-type function. ε is the threshold for the disease at a location that can take off when in > ε (\(\varepsilon \approx \frac{M}{{\sum }_{n}{{{\Omega }}}_{n}}=3.242\times 0.{1}^{8}\)). η (η = 4) is the gain parameter for the threshold ε.
-
\({S}_{k}^{o}(t)\), \({I}_{k}^{o}(t)\), and \({R}_{k}^{o}(t)\) are the observed susceptible cases, infected cases, and removed cases at OL k at the start time t.
By adding up the daily new infections caused by different sets of OLs from 8 December 2019 to 1 June 2020, we finally obtain two fundamental properties of COVID-19 in the iterated simulations, i.e., the arrival time Tm [the date t when Im(t) ≥ 1] and the infected cases Im(t) for each area m. If the OL does not evolve and the influx matrix F(t) and epidemiological rate a(t) and b(t) remain the same for all t, this model is reduced to the classical model as introduced in the literature17.
Passenger influx reduction
We present the travel restrictions by a tuple \(\{{\mathbb{S}},{\mathbb{T}},{\mathbb{{N}_{I}}},{\mathbb{{N}_{S}}},{\mathbb{{F}_{S}}}\}\), where \({\mathbb{S}}=\left\{1,2,...,s,...,S\right\}\) is the set of occurrence orders of travel restrictions, and \({\mathbb{T}}=\left\{{t}^{1},{t}^{2},...,{t}^{s},...,{t}^{S}\right\}\) (ts ∈ T) is the set of occurrence dates of travel restrictions. By definition, \({\mathbb{{N}_{I}}}\) is the set of OLs that occur at time ts when sth travel restrictions are implemented, \({\mathbb{{N}_{I}}}=\left\{{N}_{I}^{1},{N}_{I}^{2},...,{N}_{I}^{s},...,{N}_{I}^{S}\right\}\), and \({\mathbb{{N}_{S}}}=\left\{{n}^{1},{n}^{2},...,{n}^{s},...,{n}^{S}\right\}\) is the set of geographical areas ns that impose sth travel restrictions. Specifically, \({\mathbb{{F}_{S}}}=\left\{{{\bf{F}}}^{1},{{\bf{F}}}^{2},...,{{\bf{F}}}^{s},...,{{\bf{F}}}^{S}\right\}\) is influx matrix updated by the sth travel restrictions. For each travel restriction, we assume that:
-
(1)
An entry ban leads to a α decline in passenger influx from banned areas m to ns, i.e.,
$${F}_{{n}^{s}m}^{s}={F}_{{n}^{s}m}^{s-1}(1-\alpha ).$$(5) -
(2)
A global travel ban results in a reduction in passenger influx from neighboring areas m to ns by β, i.e.,
$${F}_{{n}^{s}m}^{s}={F}_{{n}^{s}m}^{s-1}(1-\beta ).$$(6) -
(3)
A lockdown reduces passenger influx by γ from neighboring areas m to ns and passenger influx from ns to neighboring areas m, i.e.,
$${F}_{m{n}^{s}}^{s}= \, {F}_{m{n}^{s}}^{s-1}(1-\gamma );\\ {F}_{{n}^{s}m}^{s} = \, {F}_{{n}^{s}m}^{s-1}(1-\gamma ).$$(7)
The other weighted links that are not influenced by sth travel restrictions remain the same, i.e., \({F}_{mn}^{s}={F}_{mn}^{s-1}\). Notably, F0 = F. Thus, \({\sum }_{m,n}{F}_{mn}^{s}-{F}_{mn}^{0}\) is the lost passenger influx until the sth travel restriction is imposed.
Country distancing increase
Based on the influx matrix Fs and the set of OLs \({N}_{I}^{s}\), we could measure the country distancing of geographical area m when sth travel restriction is imposed with:
where \({d}_{m| ({n}_{i},{F}^{s})}\) is the effective distance from source ni to destination location m based on the influx-fraction matrix Ps. Here, \({P}_{mn}^{s}=\frac{{F}_{mn}^{s}}{{\sum }_{k}{F}_{kn}}\), which is derived from Fs and F. We know that travel restrictions increase the country distancing by decreasing passenger influx in Fs, whereas the number of OLs \({N}_{I}^{s}\) decreases country distancing by promoting importation risk from multiple OLs. To better understand the impact of travel restrictions on country distancing, we exclude the influence of OLs \({D}_{m| {N}_{I}^{s}}\):
where:
Given that travel restrictions, which are collected until 1 June 2020, continuously reduce the airline passengers, the country distancing difference with and without the sth travel restriction is always non-negative. In detail:
Because \({{\Delta }}{D}_{mk}^{s}\ge 0\), we call the non-negative difference \({{\Delta }}{D}_{mk}^{s}\) as the country distancing increase at geographical area m caused by sth travel restriction imposed by area k.
Arrival time delay and infected case reduction
Country distancing is a good predictor for the arrival times and cumulative infected cases, as illustrated in Eq. (2). Given the country distancing increase (\({{\Delta }}{D}_{mk}^{s}\)) at area m caused by travel restrictions s implemented by area k, we could map the country distancing increase (\({{\Delta }}{D}_{mk}^{s}\)) to the delayed infection—ATD and ICR. Specifically, ATD, i.e., the difference in arrival times with and without sth travel restriction is denoted as:
where v(ts) is the speed for arrival times at time ts when sth travel restriction is implemented. It should be noted that the ATD is only applicable to uninfected areas. See Algorithm 1 in Supplementary Note 3 for the details. On the other hand, ICR, i.e., the difference of newly infected cases with and without travel restrictions for area m at time t is:
where \({I}_{m}^{\star }(t)={I}_{m}^{o}(t+1)-{I}_{m}^{o}(t)\) is the observed new infected increase in area m from day t + 1 to day t and \({D}_{m}^{\lozenge}(t)={\sum }_{s\in \{s| {t}^{s}\le t\}}{{\Delta }}{D}_{mk}^{s}\) is the accumulative country distancing increase due to the travel restrictions implemented by time t. Thus, as of the end date (tS = 1 June 2020), the accumulative ICR at area m is:
by assuming that daily ICRs in geographical areas are independent. For sth travel restrictions implemented by area k, the ICR at area m is:
Given that travel restrictions continuously reduce the airline passengers and continuously distance geographical areas, the country distancing increase accumulates over time. Simultaneously, the ICR accumulates over time and grows exponentially as the country distancing increase accrues. However, as time progresses, more or less OLs are present, an the intertwined effect of OLs disables existing travel restrictions. See Algorithm 2 in the Supplementary Note 3 for the details on obtaining ICR when multiple OLs are present. Substituting the upper and lower bounds of the 95% confidence interval for v(t) and u(t) in Eqs. (12) and (13), we obtain the upper and lower limits for ATD and ICR.
Thus, for sth travel restriction implemented by area k, we could calculate its global average ATD as \(\frac{{\sum }_{m}{\sum }_{k}{{\Delta }}{T}_{mk}^{s}}{M}\) and total ICR as \({\sum }_{m}{\sum }_{k}{{\Delta }}{I}_{mk}^{s}\). For each geographic area k, we could calculate its total contribution of ATD as \(\frac{{\sum }_{s}{\sum }_{m}{{\Delta }}{T}_{mk}^{s}}{M}\) and the total contribution of ICR as \({\sum }_{s}{\sum }_{m}{{\Delta }}{I}_{mk}^{s}\). Furthermore, the gain of ATD in geographic area k is \(\frac{{\sum }_{s}{\sum }_{m}{{\Delta }}{T}_{km}^{s}}{M}\) and the gain of ICR in geographical area k is \({\sum }_{s}{\sum }_{m}{{\Delta }}{I}_{km}^{s}\). Because the two matrices \({{\Delta }}{T}_{mk}^{s}\) and \({{\Delta }}{I}_{mk}^{s}\) are asymmetric, i.e., \({{\Delta }}{T}_{mk}^{s}\ne {{\Delta }}{T}_{km}^{s}\) and \({{\Delta }}{I}_{mk}^{s}\ne {{\Delta }}{I}_{km}^{s}\), such that countries’ contributions differ from their gains.
Bi-objective problem
To address the ineffectiveness and inefficiency of the existing travel restrictions, we formulate the coordinated and targeted travel restrictions as a bi-objective problem of maximizing their influence on delaying infection (maximizing the entry bans’ country distancing increase) and minimizing the loss of airline passenger influx in GMN:
For the sth entry ban implemented by area k, which reduces the passenger influx from arbitrary area n to area k, we find its corresponding optimized solution \({\theta }^{s^{\prime} }\) (\({\theta }^{s^{\prime} }\in {{{\Theta }}}^{s^{\prime} }\)) by selecting one airline link from the GMN. The set \({{{\Theta }}}^{s^{\prime} }\) is the solution set for the \(s{^{\prime} }^{{\rm{th}}}\) entry ban. Then each airline link \((n,k)\in {\theta }^{s^{\prime} }\) follows the formulation \({F}_{kn}^{s^{\prime} }={F}_{kn}^{s^{\prime} -1}(1-\alpha )\). The optimal solution should ensure that the lost passenger influx is minimal while maximizing the global country distancing increase. To solve the problem of minimizing the loss of passenger influx and maximizing country distancing increase, we adopt the NSGA-II54, a well-known fast sorting and elite multi-objective genetic algorithm. This algorithm can find the solutions that are not dominated by any other solutions and are closer to the true Pareto optimal front in the solution space. Given that we have found that effective travel restrictions prevent air travels from OLs, we prioritize the solution of entry bans to OLs. The procedure for generating non-dominated fronts follows the algorithm proposed in the literature54. Based on experiments, we finally choose a population of size 100, a crossover probability of 0.5, and a mutation probability of 0.5 to solve the problems of balanced travel restrictions. The algorithm terminates after it runs 1000 generations.
Data availability
All data needed to evaluate the paper’s conclusions are presented in the Supplementary Note 1. Additional data related to this paper may be requested from the authors.
References
World Health Organization. Coronavirus disease 2019 (covid-19): situation report. 2020.
Pinotti, F. et al. Lessons learnt from 288 covid-19 international cases: importations over time, effect of interventions, underdetection of imported cases. medRxiv https://doi.org/10.1101/2020.02.24.20027326 (2020).
Sun, J. et al. Covid-19: epidemiology, evolution, and cross-disciplinary perspectives. Trend. Mol. Med. 26, 483–495 (2020).
Chinazzi, M. et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (covid-19) outbreak. Science 368, 395–400 (2020).
Salcedo, A. & Cherelus, G. Coronavirus travel restrictions, across the globe. The New York Times (2020).
Hsiang, S. et al. The effect of large-scale anti-contagion policies on the covid-19 pandemic. Nature 584, 1–9 (2020).
McCloskey, B. et al. Mass gathering events and reducing further global spread of covid-19: a political and public health dilemma. Lancet 395, 1096–1099 (2020).
Dalziel, B. D., Pourbohloul, B. & Ellner, S. P. Human mobility patterns predict divergent epidemic dynamics among cities. Proc. R. Soc. B Biol. Sci. 280, 20130763 (2013).
Belik, V., Geisel, T. & Brockmann, D. Natural human mobility patterns and spatial spread of infectious diseases. Phys. Rev. X 1, 011001 (2011).
Saker, L. et al. Globalization and Infectious Diseases: A Review of the Linkages. Tech. Rep. (World Health Organization, 2004).
Kucharski, A. J. et al. Early dynamics of transmission and control of covid-19: a mathematical modelling study. Lancet Infect. Dis. 20, 553–558 (2020).
Adiga, A. et al. Evaluating the impact of international airline suspensions on the early global spread of covid-19. medRxiv https://doi.org/10.1101/2020.02.20.20025882 (2020).
Tang, B. et al. Estimation of the transmission risk of the 2019-ncov and its implication for public health interventions. J. Clin. Med. 9, 462 (2020).
Wilder-Smith, A. & Freedman, D. Isolation, quarantine, social distancing and community containment: pivotal role for old-style public health measures in the novel coronavirus (2019-ncov) outbreak. J. Travel Med. 27, taaa020 (2020).
Aleta, A. et al. Modelling the impact of testing, contact tracing and household quarantine on second waves of covid-19. Nat. Hum. Behav. 4, 1–8 (2020).
Della Rossa, F. et al. A network model of Italy shows that intermittent regional strategies can alleviate the covid-19 epidemic. Nat. Commun. 11, 1–9 (2020).
Brockmann, D. & Helbing, D. The hidden geometry of complex, network-driven contagion phenomena. Science 342, 1337–1342 (2013).
Colizza, V., Barrat, A., Barthélemy, M. & Vespignani, A. The role of the airline transportation network in the prediction and predictability of global epidemics. Proc. Natl Acad. Sci. USA 103, 2015–2020 (2006).
Anderson, R. M., Heesterbeek, H., Klinkenberg, D. & Hollingsworth, T. D. How will country based mitigation measures influence the course of the covid-19 epidemic? Lancet 395, 931–934 (2020).
Peto, J. et al. Universal weekly testing as the UK covid-19 lockdown exit strategy. Lancet 395, 1420–1421 (2020).
Karatayev, V. A., Anand, M. & Bauch, C. T. Local lockdowns outperform global lockdown on the far side of the covid-19 epidemic curve. Proc. Natil Acad. Sci. USA 117, 24575–24580 (2020).
Wikipeida. Travel restrictions related to the 201920 coronavirus pandemic. 2020.
Economist. The new coronavirus could have a lasting impact on global supply chains. 2020.
Habibi, R. et al. Do not violate the international health regulations during the covid-19 outbreak. Lancet 395, 664–666 (2020).
Ferretti, L. et al. Quantifying sars-cov-2 transmission suggests epidemic control with digital contact tracing. Science 368, eabb6936 (2020).
Heymann, D. The need for a coordinated international pandemic response. Bull. World Health Organ. 98, 378–379 (2020).
Mateus, A. L., Otete, H. E., Beck, C. R., Dolan, G. P. & Nguyen-Van-Tam, J. S. Effectiveness of travel restrictions in the rapid containment of human influenza: a systematic review. Bull. World Health Organ. 92, 868–880D (2014).
Dehning, J. et al. Inferring change points in the spread of covid-19 reveals the effectiveness of interventions. Science 369, eabb9789 (2020).
The Novel Coronavirus Pneumonia Emergency Response Epidemiology Team. The epidemiological characteristics of an outbreak of 2019 novel coronavirus diseases (covid-19)china, 2020. China CDC Weekly 2, 113–122 (2020).
Worby, C. J. & Chang, H.-H. Face mask use in the general population and optimal resource allocation during the covid-19 pandemic. medRxiv https://doi.org/10.1101/2020.04.04.20052696 (2020).
Li, R. et al. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (sars-cov-2). Science 368, 489–493 (2020).
Lau, H. et al. Internationally lost covid-19 cases. J. Microbio. Immunol. Infect. 53, 454–458 (2020).
Jiang, C., Gao, J. & Magdon-Ismail, M. True nonlinear dynamics from incomplete networks. In Proc. of the AAAI Conference on Artificial Intelligence, vol. 34, 131–138 (AI Magazine, 2020).
Mangili, A. & Gendreau, M. A. Transmission of infectious diseases during commercial air travel. Lancet 365, 989–996 (2005).
Bajardi, P. et al. Human mobility networks, travel restrictions, and the global spread of 2009 h1n1 pandemic. PLoS ONE 6, e16591 (2011).
Gautreau, A., Barrat, A. & Barthelemy, M. Global disease spread: statistics and estimation of arrival times. J. Theor. Biol. 251, 509–522 (2008).
Hufnagel, L., Brockmann, D. & Geisel, T. Forecast and control of epidemics in a globalized world. Proc. Natl Acad. Sci. USA 101, 15124–15129 (2004).
Jia, J. S. et al. Population flow drives spatio-temporal distribution of covid-19 in China. Nature 582, 1–11 (2020).
Kraemer, M. U. et al. The effect of human mobility and control measures on the covid-19 epidemic in china. Science 368, 493–497 (2020).
Wu, S. L. et al. Substantial underestimation of sars-cov-2 infection in the united states. Nat. Commun. 11, 1–10 (2020).
Anderson, R. M. & May, R. M. Infectious Diseases of Humans: Dynamics and Control (Oxford University Press, 1992).
International Civil Aviation Organization. https://www.icao.int/sustainability/Pages/Economic-Impacts-of-COVID-19.aspx. 2021.
Lin, S., Huang, J., He, Z. & Zhan, D. Which measures are effective in containing covid-19? empirical research based on prevention and control cases in china. medRxiv https://doi.org/10.1101/2020.03.28.20046110 (2020).
Wells, C. R. et al. Impact of international travel and border control measures on the global spread of the novel 2019 coronavirus outbreak. Proc. Natl Acad. Sci. USA 117, 7504–7509 (2020).
World Health Organization. Covid19 strategy update. 2020.
Tian, H. et al. An investigation of transmission control measures during the first 50 days of the covid-19 epidemic in china. Science 368, 638–642 (2020).
Ruktanonchai, N.W. et al. Assessing the impact of coordinated covid-19 exit strategies across Europe. Science 369, 1465–1470 (2020).
Li, R. et al. Global covid-19 pandemic demands joint interventions for the suppression of future waves. Proc. Natil Acad. Sci. USA 117, 26151–26157 (2020).
Cooper, B. S., Pitman, R. J., Edmunds, W. J. & Gay, N. J. Delaying the international spread of pandemic influenza. PLoS Med. 3, e212 (2006).
ODriscoll, M. et al. Age-specific mortality and immunity patterns of sars-cov-2. Nature 590, 1–9 (2020).
Kissler, S. M., Tedijanto, C., Goldstein, E., Grad, Y. H. & Lipsitch, M. Projecting the transmission dynamics of sars-cov-2 through the postpandemic period. Science 368, 860–868 (2020).
Colizza, V., Pastor-Satorras, R. & Vespignani, A. Reaction-diffusion processes and metapopulation models in heterogeneous networks. Nat. Phys. 3, 276–282 (2007).
Rvachev, L. A. & Longini, I. M. Jr A mathematical model for the global spread of influenza. Math. Biosci. 75, 3–22 (1985).
Deb, K., Pratap, A., Agarwal, S. & Meyarivan, T. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evol. Comput. 6, 182–197 (2002).
Cooper, I., Mondal, A. & Antonopoulos, C. G. A sir model assumption for the spread of covid-19 in different communities. Chaos Soliton. Fract. 139, 110057 (2020).
Li, Q. et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N. Engl J. Med. 382, 1199–1207 (2020).
Acknowledgements
This work was supported in part by the Department of Mechanical Aerospace and Nuclear Engineering Department at Rensselaer Polytechnic Institute, Troy, NY. J. G. acknowledges the support of National Science Foundation under Grant No. 2047488, and the Rensselaer-IBM AI Research Collaboration.
Author information
Authors and Affiliations
Contributions
L.Z., M.D., and J.G. conceived the project and designed the experiments; L.Z. collected the dataset and analyzed the data; L.Z. and J.G. carried out theoretical calculations; L.Z. and W.W. performed the experiments on optimizations; L.Z., M.D., and J.G. wrote the manuscript; all authors edited the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhong, L., Diagne, M., Wang, W. et al. Country distancing increase reveals the effectiveness of travel restrictions in stopping COVID-19 transmission. Commun Phys 4, 121 (2021). https://doi.org/10.1038/s42005-021-00620-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s42005-021-00620-5
This article is cited by
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.