Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Leveraging artificial intelligence for pandemic preparedness and response: a scoping review to identify key use cases

Abstract

Artificial intelligence (AI) represents a valuable tool that could be widely used to inform clinical and public health decision-making to effectively manage the impacts of a pandemic. The objective of this scoping review was to identify the key use cases for involving AI for pandemic preparedness and response from the peer-reviewed, preprint, and grey literature. The data synthesis had two parts: an in-depth review of studies that leveraged machine learning (ML) techniques and a limited review of studies that applied traditional modeling approaches. ML applications from the in-depth review were categorized into use cases related to public health and clinical practice, and narratively synthesized. One hundred eighty-three articles met the inclusion criteria for the in-depth review. Six key use cases were identified: forecasting infectious disease dynamics and effects of interventions; surveillance and outbreak detection; real-time monitoring of adherence to public health recommendations; real-time detection of influenza-like illness; triage and timely diagnosis of infections; and prognosis of illness and response to treatment. Data sources and types of ML that were useful varied by use case. The search identified 1167 articles that reported on traditional modeling approaches, which highlighted additional areas where ML could be leveraged for improving the accuracy of estimations or projections. Important ML-based solutions have been developed in response to pandemics, and particularly for COVID-19 but few were optimized for practical application early in the pandemic. These findings can support policymakers, clinicians, and other stakeholders in prioritizing research and development to support operationalization of AI for future pandemics.

Introduction

Given the pace of globalization, future pandemics are likely to follow novel coronavirus disease 2019 (COVID-19), although their frequency is uncertain. Half a year into the pandemic, it was estimated that 59–92% of COVID-19 deaths in the USA could have been avoided if the pandemic had been managed differently and mortality rates were similar to those in countries with moderate rates of COVID-19 deaths, such as Norway or Canada1.

Despite a significantly lower mortality rate compared with severe acute respiratory syndrome (SARS), caused by a related coronavirus (SARS-CoV) with a case fatality rate of 11%2, COVID-19 has resulted in exponentially more harm. The virus spread rapidly and widely around the world, in a way SARS-CoV did not, from asymptomatic and mild cases resulting in undetected spread and leading to a higher number of deaths overall. If pandemics are to be managed effectively, policymakers, clinicians, and other stakeholders need access to data and recommendations in near-real time, including models to weigh the relative risks and benefits of various interventions. Notably, there have been numerous conflicting projection models for COVID-19, but few were accurate for this novel pathogen.

Policymakers and governments have many choices for population-level health interventions, which are critical to control spread early on. Non-pharmaceutical interventions include implementing travel bans, closing businesses, shutting schools, mandating masks, and allocating scarce supplies such as personal protective equipment (PPE) and testing. Implementation, timing, enforcement, and cessation all represent additional choices. Many of these decisions are still based on expert recommendations, rather than data-driven models. With these decisions come difficult tradeoffs, as many have serious economic consequences as well as direct health implications. For example, implementing restrictions (e.g., stay-at-home orders) during a pandemic may reduce infection-related morbidity and mortality, but the associated economic decline, social isolation, and delayed medical care also adversely affect public health and welfare.

Optimally managing a pandemic necessitates rapid feedback cycles of data-driven learning to respond effectively at each step. Policymakers must make initial decisions about which interventions are most likely to protect public health, and make mid-course adjustments, including updating policies and recommendations as more data become available. Clinicians must determine how to diagnose, triage, and care for infected patients under uncertainty, given the possibility that the pathogen may behave differently from known infections; rapidly studying and disseminating information about symptoms, disease progression, and responses to treatments are critical for reducing harm.

Data have always been important for healthcare and public health decision-making; however, data have been especially instrumental in efforts to tackle COVID-19 worldwide. Unprecedented levels of global collaboration have initiated data-sharing efforts from traditional sources such as those from health services, and non-traditional ones including transportation records and personal data from smartphones. These early strides in data sharing are critical for artificial intelligence (AI) where performance improves with large, inclusive, historical and real-time datasets. Innovations are rapidly advancing the application of data, advanced analytics, and machine learning (ML) to help manage the COVID-19 pandemic.

The objective of this scoping review was to synthesize available literature describing the use of AI to inform clinical and public health decision-making for pandemic preparedness and response. This review had two parts: an in-depth review of studies that leveraged ML techniques, and a limited review of studies that applied traditional modeling approaches. The in-depth review identified key use cases for ML alongside data sources and types of ML well suited for each use case. The limited review highlighted additional areas where ML could be leveraged for improving the accuracy of estimations or projections.

Methods

This scoping review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR)3.

Search strategies

Five databases (PubMed [NCBI], Embase [Elsevier], Web of Science [Clarivate], IEEE Xplore [IEEE], and the ACM Guide to Computing Literature [ACM]) were searched without date limits on May 4, 2020, to identify relevant peer-reviewed literature. Two main concepts of AI and pandemics were mapped to the most relevant controlled vocabulary using Medical Subject Headings (MeSH), and free-text terms were included. Although the search strategy captured the published literature on all pandemics, additional MeSH terms and keywords were added to focus on the COVID-19 pandemic and the most recent past pandemic of influenza A subtype H1N1 (H1N1) in 2009. The search also captured relevant literature about the SARS global outbreak caused by SARS-CoV in 2003. Two preprint servers (medRxiv and bioRxiv) were searched from January 1 to May 27, 2020, to locate relevant research that had not yet been published. The main concepts of AI and COVID-19 were captured using free-text terms. Reference lists of included structured reviews were hand searched to identify further relevant studies.

In addition, a structured Google search was conducted to locate grey literature describing the application of AI for the management of COVID-19. Reputable trade and commercial publications were also reviewed to identify emerging and proprietary AI solutions. Peer-reviewed, preprint, and grey literature search strategies are provided in Supplementary Notes13.

Inclusion and exclusion criteria

This scoping review had two parts: an in-depth review focused on the use of ‘complex’ ML for preparedness or response to viral respiratory pandemics as well as the SARS global outbreak, and a limited review describing the use of traditional modeling approaches. ‘Complex’ ML (hereafter referred to as ML) included neural networks, tree-based algorithms, support vector machines, and natural language processing. Traditional approaches included compartmental, simulation, statistical, and time series models. The Glossary provides a detailed listing of complex and traditional models (Box 1). Although categorization could be considered somewhat arbitrary, models were categorized as complex if they were generally less explainable, required increased computing power, or could more effectively manage irregularly sampled or high-dimensional data. Various publications have summarized these methods and offer insights about strengths and weaknesses4,5,6. All study designs were considered for inclusion. Articles were excluded if they did not report on original research or describe a structured review of the literature, did not focus on human populations, or were not published in the English language. Studies reporting on public opinion, vaccine uptake or adverse events, molecular docking, genomic sequencing, or applications in robotics were also excluded. Detailed inclusion and exclusion criteria are provided in Supplementary Table 1.

The Google search focused on grey literature describing the application of proprietary AI solutions by governments or industry for COVID-19 response, and other emerging applications not yet captured by the peer-reviewed and preprint literature. The same exclusion criteria were applied.

Screening and data abstraction

Articles were screened in two stages using Covidence (Australia), a web-based review management tool. Articles were first screened for relevance based on the information provided in the title and abstract and then evaluated for inclusion based on the full text. Articles were screened by one reviewer at each stage. For articles that described the use of ML, the following criteria were abstracted into standardized forms: citation information; relevant use cases; respiratory pandemic (or SARS); population under study (i.e., region); purpose of the models (e.g., surveillance or prediction); type of ML models; outcomes of interest (e.g., infections or deaths); and data sources. Given the volume of relevant peer-reviewed and preprint literature reporting on traditional modeling approaches, data abstraction was not completed for studies included in the limited review. Manuscript details are provided in Supplementary Tables 2 and 3. Similarly, data were not abstracted for relevant grey literature.

In-depth review of studies that applied machine learning techniques

The characteristics of studies that reported on the use of ML were summarized. Examples from the peer-reviewed, preprint, and grey literature were categorized into a framework of use cases related to public health and clinical practice. Each use case was narratively synthesized. Commonly used data sources and ML techniques were summarized in tabular form. Emerging use cases were identified as opportunities for future work.

Limited review of studies that used traditional modeling approaches

The number of peer-reviewed articles and preprints that described traditional modeling approaches was reported to highlight the large volume of literature compared with manuscripts describing the application of ML. The objectives of these models and data sources were summarized in tabular form to identify additional areas where ML could be leveraged to provide more accurate estimations or projections.

Results

From 8070 unique peer-reviewed and preprint records, 183 reported on the use of ML and met the inclusion criteria for the in-depth review (Supplementary Table 2). A modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram is provided in Fig. 1. The review of the grey literature identified one additional use case not captured by the peer-reviewed or preprint literature and provided supporting examples for other use cases. Overall, the in-depth review identified six key use cases where ML was used for pandemic preparedness and response, as well as emerging areas beyond management of infectious disease, such as impacts of a pandemic on mental health or chronic conditions (Table 1).

Fig. 1: Study selection flow diagram.
figure 1

Modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram showing disposition of articles.

Table 1 Number of manuscripts included in the in-depth review by use case and respiratory pandemic or SARS global outbreaka.

The search also identified 1167 manuscripts that described the use of traditional modeling approaches and met the inclusion criteria for the limited review (Supplementary Table 3). A synthesis of the findings is presented in Box 2 and Table 2.

Table 2 Areas where machine learning could be leveraged for improving the accuracy of estimations or projections and potential data sources identified through the review of traditional approaches.

Forecasting infectious disease dynamics and effects of interventions

ML can be leveraged to improve the accuracy of estimations and projections to inform decision-making about the management of pandemics. Forty studies used ML to identify factors influencing spread of disease, fit epidemic curves, and forecast infectious disease dynamics or effects of interventions (40/183 studies [Supplementary Table 2]; 22%).

Most COVID-19 estimations and forecasts (32/33 studies; 97%) relied on relatively simplistic publicly available data sources such as counts from the Johns Hopkins COVID-19 map, Worldometer, and the World Health Organization as well as data released by country-specific Centers for Disease Control and Prevention, where ML may not provide much benefit compared with traditional modeling approaches. For example, one study used publicly available Worldometer and Google Trends data to project COVID-19 infections; however, traditional linear regression was shown to outperform a recurrent neural network-based model7.

Early in the pandemic, when data were limited, ML was used to augment traditional modeling approaches. Four studies used neural networks8,9,10,11 and one used a random forest algorithm12 to provide data-driven estimates of parameters for compartmental or statistical models. Two studies compared the performance of neural network-augmented models with traditional Susceptible-(Exposed)-Infected-Recovered models and showed that the augmented models provided better approximations of the true epidemic curve resulting in more accurate forecasts8,10.

Another approach involved augmenting sparse data. One study explored multiple approaches including random forests and variations of neural networks to forecast COVID-19 infections, deaths and effects of non-pharmaceutical interventions13. The models were trained using historical SARS data and fine-tuned using limited COVID-19 data. Similarly, a study published one month after the COVID-19 outbreak in Wuhan combined three strategies to develop an accurate ML model to forecast suspected infections by augmenting a 14-day COVID-19 dataset with other data sources, selecting the most appropriate model from a panel of models, and fine-tuning the parameters14. The final model used a polynomial neural network and showed significantly lower error compared with traditional time series modeling including autoregressive integrated moving average and exponential growth models.

As the pandemic progressed and more data became available, ML was leveraged to analyze temporal COVID-19 data and many studies integrated additional data sources such as health and demographic information, and geographic characteristics such as population density or climate. The most common techniques were variations of neural networks (Supplementary Table 2). These models forecasted various short- (e.g., 10 days15) or longer-term (e.g., 24 days16) outcomes including infections, deaths, and effects of non-pharmaceutical interventions; spread of COVID-19 across the globe17; and regional vulnerability to COVID-1918. Although many studies compared the relative performance of various ML techniques, these models were rarely evaluated against traditional approaches.

Similar ML-based approaches were also explored following SARS and H1N1 using historical data (Table 3). Most studies included in the limited review used traditional approaches to forecast infectious disease dynamics or effects of non-pharmaceutical interventions (Box 2). ML could be used to address other use cases presented in Table 2 with the advantage of integrating additional data sources and more effectively modeling irregularly sampled or high-dimensional data to provide more accurate predictions.

Table 3 Machine learning approaches explored in response to past or hypothetical pandemics and the SARS global outbreak by use case.

Surveillance and outbreak detection

The grey literature highlighted many examples where ML was used for outbreak detection. For example, industry-based companies and Boston Children’s Hospital HealthMap (USA) were among the first outside of China to report the emerging risk of COVID-19 by leveraging natural language processing (NLP) to translate and analyze foreign news reports19.

Sixteen studies reported on the use of ML for surveillance or outbreak detection (16/183 studies [Supplementary Table 2]; 9%). However, only three studies focused on COVID-19. Two preprints used NLP and deep neural networks to mine and analyze Twitter posts for personal reports of potential exposure to COVID-1920,21. Another preprint described leveraging data from smartphone-connected thermometers to monitor rates of influenza-like illness and flag higher than expected rates22. Both approaches tracked potential exposures or symptoms in real time coupled with precise geolocation information to understand where outbreaks were occurring. These data sources could also be used to forecast influenza-like illness rates.

Similar approaches were explored in response to or following SARS and H1N1 (Table 3). In addition, clinical information, such as electronic health record (EHR) data from emergency departments, was shown to be an informative data source for monitoring rates of influenza-like illness using historical H1N1 data. A smartphone application (app) was also developed for syndromic surveillance in public spaces.

Real-time monitoring of adherence to public health recommendations

The peer-reviewed and preprint literature did not provide examples of how ML was used in real time to improve adherence to public health recommendations; all the examples were found in the grey literature. This use case was not explored in response to past pandemics or the SARS global outbreak.

Early in the COVID-19 pandemic, some countries such as China and Russia leveraged existing AI-based facial-recognition software and cameras to identify individuals who were not compliant with mandated self-isolation or quarantine23,24. This technology also advanced to accurately identify those wearing a mask for mass public monitoring24. On a smaller scale, contactless verification of employees was proposed for returning to work25.

To address privacy concerns, alternatives based on facial detection rather than recognition were developed to help businesses, schools, and workplaces reopen safely. Numerous companies developed computer vision-based solutions to monitor and improve adherence to public health recommendations such as wearing masks, social distancing, and hand sanitization by analyzing closed-circuit surveillance videos using neural networks (Fig. 3)26,27. Clients were able to receive daily summaries or real-time alerts to help improve adherence to protect employees and visitors. Additional features included tracking store capacity and prioritizing areas for timely sanitation26.

Fig. 2: Example of projected hospital resource use for COVID-19 patients in the USA using traditional modeling approaches.
figure 2

The image shows forecasts for use of hospital beds, intensive care beds and ventilators over the next four months from the Institute for Health Metrics and Evaluation124. Image courtesy of the University of Washington, available under Public License and used with permission.

Fig. 3: Example of a computer vision solution for real-time monitoring of adherence to social distancing26.
figure 3

The image shows the movement of people through a public space and estimates compliance with distancing by at least 6 feet. Image courtesy of Aura Vision, used with permission.

Similar computer vision systems were developed for hospitals to monitor interactions with COVID-19 patients at the bedside and document which employees entered the room and for how long, whether there was close contact with the patient, and if PPE was secure28. As a next step, industry was developing computer vision applications to monitor healthcare PPE inventory in real time29.

The review identified one related preprint where ML was used to help decision-makers understand adherence to non-pharmaceutical interventions in near real time (1/183 studies [Supplementary Table 2]; <1%). Deep neural networks were used for travel mode detection to calculate various population-level mobility and social distancing metrics reported daily on the COVID-19 Impact Analysis Platform30.

Real-time detection of influenza-like illness

Computer vision solutions were also developed to detect influenza-like illness consistent with viral respiratory pandemic symptoms for mass screening (8/183 studies [Supplementary Table 2]; 4%). However, only two studies focused on COVID-19.

The first COVID-19 study employed computer vision to assess for both fever and cyanosis with 97% and 77% accuracies, respectively31. Similar approaches were developed following SARS, however, the types and quality of sensor data and ML techniques improved over time (Table 3). The grey literature showed that thermal scanners were widely deployed for COVID-19 in hospitals and public spaces29, although underlying data sources, models and performance may have varied. ‘Pandemic’ drones were also developed to detect influenza-like illness remotely including fever, increased heart and respiratory rates, as well as more overt symptoms such as coughing32.

The other COVID-19 study developed a smartphone app that differentiated COVID-19 coughs from other types using convolutional neural networks (CNN) and a support vector machine, and demonstrated promising accuracy33. The grey literature search identified another app that was under development and aimed to detect COVID-19 by analyzing voice recordings34.

Data from wearable devices were also leveraged for early detection of COVID-19. A press release reported that algorithms integrating data collected by the Oura Ring (Oura Health Ltd, Finland) with patient-reported data from a COVID-19 monitoring app were able to detect subclinical signs of infection up to 3 days prior to onset of classic symptoms such as fever or cough with 90% accuracy35.

Triage and timely diagnosis of infections

The most common use case for the application of ML for pandemic response was triage and timely diagnosis of symptomatic cases (87/183 studies [Supplementary Table 2]; 48%). Most studies developed algorithms or tools in response to COVID-19 (78/87; 90%), and one study conducted a systematic review of these tools36. Eight studies (9%) reported on the development of similar tools following H1N1 or SARS (Table 3).

The use of ML for detection or estimation of disease severity based solely on chest imaging made up the bulk of COVID-19 original research (65/78 studies [Supplementary Table 2]; 83%). Most studies relied on open-source datasets and leveraged some variation of CNNs for image segmentation, classification to differentiate between COVID-19 and other common lung infections, or estimation of disease severity (Table 4). The algorithms showed varying performance with AUCs ranging from 0.81 to >0.99, which could have been impacted by size or quality of the data source, type of imaging, approaches to image processing, types of ML used, and fine-tuning of parameters.

Table 4 Characteristics of studies that developed machine learning-based algorithms and tools for COVID-19 diagnosis or estimation of disease severity based solely on chest imaging (n = 65).

ML algorithms incorporating other information beyond imaging were also developed to help prioritize patients with a higher likelihood of COVID-19 for isolation and testing. Nine studies developed models using combinations of standard variables such as patient demographics, vital signs, clinical symptoms, comorbidities, and known exposure history, as well as CT images37,38,39,40,41,42,43,44,45; most also included the results from routine bloodwork (8/9 studies; 89%). The algorithms were developed using a wide array of ML approaches and showed varying performance with AUCs ranging from 0.84 to >0.99.

Similarly, one preprint used results from routine bloodwork to estimate COVID-19 disease severity46. Another study used a transformer neural network to identify symptoms documented in unstructured clinical notes from an EHR during the week leading up to COVID-19 testing, which could be used to inform development of triage tools47.

Although studied in research settings, these types of clinical decision support tools were not widely available to assist clinicians with timely diagnosis of COVID-19. There were a few notable exceptions. One article described the rapid development and deployment of an ML-based COVID-19 diagnostic system that screened CT images across 16 hospitals in China48. More strategically, some healthcare software companies, such as RADLogics, Inc., have adapted existing solutions to accurately detect COVID-19 from CT images and quantify the extent of infection in a clinically interpretable way (Fig. 4)49,50.

Fig. 4: Example of a ML solution for detecting and estimating the extent of COVID-19 infection based on CT images.
figure 4

The image on the left shows a patient infected with COVID-19. The image on the right shows a patient negative for COVID-19. Images courtesy of RADLogics, Inc., used with permission.

In contrast, the grey literature reported widespread development and implementation of AI-based chatbots for large-scale public triage by governments and healthcare organizations during the COVID-19 pandemic51. These tools were not described in the peer-reviewed or preprint literature, and as a result, the underlying models and appropriateness of chatbot recommendations were generally not known. Only one preprint reported on the Symptoma chatbot (Austria), which was shown to have 96% accuracy for detecting COVID-1952. Another study used gradient boosting to develop a model for COVID-19 triage for testing using data collected from national symptom surveys and demonstrated an AUC of 0.7353.

Prognosis of illness and response to treatment

ML models were also commonly developed to predict which patients were at higher risk of COVID-19-related deterioration (31/183 studies [Supplementary Table 2]; 17%), including one systematic review36. This use case was not explored in response to past pandemics.

Original research focused on predicting progression to severe disease, intensive care admission, ventilator use, or mortality (Table 5). Almost half of the studies used data routinely captured by an EHR or obtained through a quick patient history, and explored various classification algorithms (13/30 studies; 43%)54,55,56,57,58,59,60,61,62,63,64,65,66. Nine studies compared the performance of complex ML with simple logistic regression54,55,56,57,58,59,60,61,62. In four studies, logistic regression was found to have similar or better performance55,57,58,60, and of three studies that developed a clinical prediction tool, two selected logistic regression as the final model for simplicity and interpretability55,60. Similarly, another study developed a model using extreme gradient boosted decision trees, but deferred to an explainable single tree for the final model66.

Table 5 Characteristics of studies that developed machine learning-based algorithms and tools to predict COVID-19-related deterioration (n = 30).

On the other hand, ML can help to make sense of large amounts of complex, or unstructured data. Thirteen studies used CT or X-ray images to predict deterioration alone or in combination with clinical information (Supplementary Table 2) with AUCs ranging from 0.70 to 0.97. Another study predicted which hospitalized patients would be admitted to intensive care by analyzing unstructured EHR notes using the proprietary NLP- and neural network-based EHRead (Savana, Madrid) extraction technology67. The data were then used to classify patients using a decision tree yielding an AUC of 0.76.

Existing tools, such as the Epic Deterioration Index (Epic Systems Corporation, USA), were used to predict COVID-19-related deterioration; the index showed moderate performance with an AUC of 0.7968. Other prognostic systems were studied in research settings, but these tools were not widely available to assist clinicians early in the pandemic but slowly came to market. For example, CLEWICU (CLEW Medical Ltd., Israel) received Emergency Use Authorization from the U.S. Food and Drug Administration (FDA) in June 2020 for use in hospitals to predict respiratory failure and hemodynamic instability in COVID-19 patients69.

Although there were no known specific treatments for COVID-19 early on, one preprint described an algorithm for prediction of patients’ response to treatment based on age, chronic conditions, respiratory or organ failure, and treatment plan to guide the use of limited healthcare resources70. The best performing model had an AUC > 0.99 and was developed using a CNN for image interpretation coupled with a support vector machine to integrate clinical data for prediction of response to treatment.

Emerging areas beyond management of infection

The COVID-19 pandemic affected health broadly beyond the outcomes of the infectious disease itself. Literature describing the use of ML in other domains, such as mental health and chronic conditions, was emerging even early in the pandemic (6/183 studies [Supplementary Table 2]; 3%). One study reported on the short-term mental health impacts of COVID-19 through sentiment analysis of social media posts before and shortly after the initial outbreak in Wuhan71. Another study used ML to group related literature on the impact of coronaviruses on people with intellectual disabilities72.

In response to the large body of COVID-19-related literature, the COVID-19 Open Research Dataset was released with a ‘call to action’ for academic and industry researchers to develop AI techniques to rapidly analyze the literature to address important knowledge gaps73. Three responses to this call were identified: two studies identified themes in the literature using k-means clustering or lexical link analysis, and another study used NLP to generate summaries of the relevant literature72,74,75. Two other studies described the use of NLP and/or neural networks to mine other sources of publicly available literature and summarize the results76,77.

To ensure that impactful, high-quality research could be used to guide pandemic response, ML techniques were used to identify promising research uploaded to preprint servers for expedited peer-review78. Reviews were published in the open-access journal Rapid Reviews: COVID-19. These approaches aimed to quickly create evidence bases that could be used to inform public health and clinical decision-making. In addition, neural networks were also applied to limit the spread of misinformation about the COVID-19 pandemic79.

Discussion

We performed a scoping review of the peer-reviewed, preprint and grey literature, and identified six key use cases where ML was leveraged for pandemic preparedness or response. We also found that the sources of data and types of ML that were useful varied by use case (Table 6). While there were many examples of novel solutions, most were still at the research or developmental stage and had not been widely used to inform clinical or public health decisions early in the COVID-19 pandemic. For example, despite numerous publications demonstrating good to excellent performance in diagnosing COVID-19 from lung imaging, practical prospective clinical applications of these algorithms were rare; possibly, due to many algorithms being developed based on the availability of data and knowledge of ML, rather than to address specific clinical or public health-driven questions. However, some existing products were adapted or modified for implementation; these included computer vision for real-time monitoring of adherence to public health recommendations or detection of influenza-like illness, as well as specific tools for triage such as chatbots. A few ML solutions received FDA Emergency Use Authorization for use in clinical settings to detect COVID-19 or predict infection-related deterioration69,80. Most examples of tools that were rapidly implemented were identified through the grey literature and were developed by health systems or industry.

Table 6 Commonly used data sources and types of machine learning suitable for each use case based on studies from the in-depth review about past pandemics, SARS, and COVID-19.

Given the relative technological limitations during past pandemics and SARS, understandably most research relied on traditional modeling approaches. However, the limited review included in our study highlighted that there was still a strong reliance on traditional approaches in response to COVID-19 and identified additional areas where ML could be leveraged to improve performance (Table 2). ML is well positioned to complement traditional modeling in the following ways: (1) Integration of diverse sources of information: ML methods are better at integrating diverse and complex sources of data than traditional statistical regression models; (2) Combination of different types of models: ensemble learning or data augmentation methods can be used to combine different types of prediction models to achieve better accuracy81, or more granular models; (3) Temporal modeling: while traditional time series modeling or statistical methods can be effective for dealing with regularly sampled and low-dimensional temporal data, data from a pandemic tend to be irregularly sampled and high dimensional, where ML methods such as neural networks could substantially improve performance.

In addition to the use cases described in this article, ML approaches also played a key role in other aspects of pandemic response. One area was genome sequencing, where ML was used for classification of COVID-19 viral genomes, which allowed for rapid detection of unknown mutations and supported contact tracing by determining the genetic origin of each case82,83. On a molecular level, ML was used to understand the underlying structures of associated proteins and molecular docking processes84. This knowledge could inform vaccine development or identification of effective drug treatments.

Challenges of employing machine learning

The performance of ML algorithms depends on the availability and accessibility of vast amounts of data, conditions that are subject to technology infrastructure and interoperability, and privacy and data-sharing laws. In many cases, even the most basic infrastructure necessary to transmit data between healthcare organizations was lacking. For example, based on 2018 data, 41% of US hospitals were not able to electronically report surveillance data to public health agencies85.

Moreover, when datasets do exist, a lack of comprehensive and diverse data is a critical challenge. In instances where training data systematically exclude parts of the population (e.g., asymptomatic cases due to lack of testing; or individuals who do not have access to data collection using consumer-centric technologies, such as wearables or smartphones, or reliable Internet service), the applicability of the model to wider populations could be compromised. Data quality could be further compromised by incomplete or inconsistent labeling of racial, ethnic, and other demographic information86. Based on biased or limited samples, ML algorithms may inadvertently increase disparities by misrepresenting the burden of disease and inappropriately informing resource allocation.

ML algorithms and tools also face challenges at deployment. Health systems and public health experts must exercise caution when applying models in different contexts. Algorithms trained in a specific health system, cultural or socio-economic context may not provide similar performance for populations with different characteristics. Algorithms must undergo critical evaluation and re-calibration when implemented across settings which requires time, as well as financial and human resources.

Interpretability of ML solutions can also limit approval, implementation, or adoption of these tools in real-world settings. There is generally a tradeoff between model complexity and interpretability that needs to be considered particularly in healthcare settings, given ethical and legal implications of decision-making. A few studies identified through the review compared ML with traditional approaches and in some cases simpler models demonstrated similar or better performance while offering the additional benefit of interpretability, highlighting the importance of comparing ML-based algorithms and solutions with traditional approaches to ensure that increased complexity adds value. However, for many use cases highly complex models were necessary for tasks like image interpretation, and many of the tools described in this review offered some level of interpretability by highlighting physical locations where people were not social distancing (Fig. 3) or areas on chest imaging contributing to detection of COVID-19 (Fig. 4).

Overarching lessons

These findings have several overarching lessons. First, past pandemics and the SARS global outbreak were followed by spurts of research, followed by rapid declines in research support for approaches that could have enabled better management of COVID-1987,88. As such, longitudinal support is essential for this work. Second, while ML was explored widely early in the COVID-19 pandemic as evidenced by the preprint search, almost all this work was at the research or developmental stage, and real-world applications were limited. Third, support for development of large and comprehensive databases preferably at the national, and even international level, containing health data would be extremely valuable for many purposes, with pandemic management being among the most important89. Fourth, tools should be developed that allow modeling of multiple scenarios to make better choices about the wide array of options that need to be considered, from choices about school closures to care management for the elderly, to distribution of scarce resources like ventilators and PPE.

Limitations of the study

This study has several limitations. Each record was evaluated by one reviewer due to the large number of studies identified through the search. As a scoping review, the goal was to provide an overview of key use cases for ML rather than a comprehensive evaluation of specific data sources or ML approaches. Future work is warranted to assess the risk of bias and usability of these solutions in practical settings.

The review included preprint articles to capture the breadth of the rapidly growing body of literature about the COVID-19 pandemic. However, the preprint articles were not peer-reviewed and results should be interpreted with caution.

The grey literature search included reputable trade and commercial publications to identify applications of proprietary AI solutions by governments or industry for COVID-19 response, and other emerging applications not yet captured by the peer-reviewed and preprint literature. Although not the standard approach, it was considered appropriate given the context, where trade and commercial publications have been a valuable source of information throughout the COVID-19 pandemic.

Conclusions

Important ML-based solutions have been developed in response to pandemics and particularly for COVID-19 but few were optimized for practical clinical or public health application early in the pandemic. These findings can support policymakers, clinicians, and other stakeholders in prioritizing operationalization of AI for future pandemics.

Data availability

All data generated or analyzed during this study are included in this published article and its supplementary information.

References

  1. Bilinski, A. & Emanuel, E. J. COVID-19 and excess all-cause mortality in the US and 18 comparison countries. JAMA 324, 2100–2102 (2020).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  2. World Health Organization. Consensus document on the epidemiology of severe acute respiratory syndrome (SARS). https://apps.who.int/iris/bitstream/handle/10665/70863/WHO_CDS_CSR_GAR_2003.11_eng.pdf?sequence=1&isAllowed=y (2020).

  3. Tricco, A. C. et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann. Intern. Med. 169, 467–473 (2018).

    PubMed  Article  Google Scholar 

  4. Bi, Q., Goodman, K. E., Kaminsky, J. & Lessler, J. What is machine learning? A primer for the epidemiologist. Am. J. Epidemiol. 188, 2222–2239 (2019).

    PubMed  Google Scholar 

  5. Badillo, S. et al. An introduction to machine learning. Clin. Pharmacol. Ther. 107, 871–885 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  6. Modern machine learning algorithms: strengths and weaknesses. EliteDataScience https://elitedatascience.com/machine-learning-algorithms (2017).

  7. Ayyoubzadeh, S. M., Ayyoubzadeh, S. M., Zahedi, H., Ahmadi, M. & Kalhori, S. R. N. Predicting COVID-19 incidence through analysis of Google Trends data in Iran: data mining and deep learning pilot study. JMIR Public Health Surveill. 6, e18828 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  8. Dandekar, R. & Barbastathis, G. Quantifying the effect of quarantine control in Covid-19 infectious spread using machine learning. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.03.20052084v1 (2020).

  9. Uhlig, S., Nichani, K., Uhlig, C. & Simon, K. Modeling projections for COVID-19 pandemic by combining epidemiological, statistical, and neural network approaches. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.17.20059535v1 (2020).

  10. Yu, Y. et al. COVID-19 Asymptomatic infection estimation. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.19.20068072v1 (2020).

  11. Distante, C., Gadelha Pereira, I., Garcia Goncalves, L. M., Piscitelli, P. & Miani, A. Forecasting Covid-19 outbreak progression in Italian regions: a model based on neural network training from Chinese data. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.09.20059055v1 (2020).

  12. Watson, G. L. et al. Pandemic velocity: forecasting COVID-19 in the US with a machine learning & Bayesian time series compartmental model. PLoS Comput. Biol. 17, e1008837 (2021).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  13. Kafieh, R. et al. COVID-19 in Iran: a deeper look into the future. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.24.20078477v1 (2020).

  14. Fong, S. J., Li, G., Dey, N., Crespo, R. G. & Herrera-Viedma, E. Finding an accurate early forecasting model from small dataset: a case of 2019-nCoV novel coronavirus outbreak. Int. J. Interact. Multimed. Artif. Intell. 6, 132–140 (2020).

    Google Scholar 

  15. Al-qaness, M. A. A., Ewees, A. A., Fan, H. & El Aziz, M. A. Optimization method for forecasting confirmed cases of COVID-19 in China. J. Clin. Med. 9, https://doi.org/10.3390/jcm9030674 (2020).

  16. Suzuki, Y., Suzuki, A., Nakamura, S., Ishikawa, T. & Kinoshita, A. Machine learning model estimating number of COVID-19 infection cases over coming 24 days in every province of South Korea (XGBoost and MultiOutputRegressor). Preprint at https://www.medrxiv.org/content/10.1101/2020.05.10.20097527v1 (2020).

  17. Ibrahim, M. R. et al. Variational-LSTM autoencoder to forecast the spread of coronavirus across the globe. PLoS ONE 16, e0246120 (2020).

    Article  CAS  Google Scholar 

  18. Mehta, M., Julaiti, J., Griffin, P. & Kumara, S. Early stage machine learning–based prediction of US county vulnerability to the COVID-19 pandemic: machine learning approach. JMIR Public Health Surveill. 6, e19446 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  19. Heaven, W. D. AI could help with the next pandemic—but not with this one. MIT Technology Review. https://www.technologyreview.com/2020/03/12/905352/ai-could-help-with-the-next-pandemicbut-not-with-this-one/ (2020).

  20. Golder, S. et al. Extending A chronological and geographical analysis of personal reports of COVID-19 on Twitter to England, UK. Preprint at https://www.medrxiv.org/content/10.1101/2020.05.05.20083436v1 (2020).

  21. Klein, A. et al. A chronological and geographical analysis of personal reports of COVID-19 on Twitter. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.19.20069948v2 (2020).

  22. Chamberlain, S. D. et al. Real-time detection of COVID-19 epicenters within the United States using a network of smart thermometers. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.06.20039909v1 (2020).

  23. Dixon, R. In Russia, facial surveillance and threat of prison being used to make coronavirus quarantines stick. The Washington Post. https://www.washingtonpost.com/world/europe/in-russia-facial-surveillance-and-risk-of-jail-seek-to-make-coronavirus-quarantines-stick/2020/03/24/a590c7e8-6dbf-11ea-a156-0048b62cdb51_story.html (2020).

  24. Yang, Y. & Zhu, J. Coronavirus brings China’s surveillance state out of the shadows. Reuters. https://www.reuters.com/article/us-china-health-surveillance/coronavirus-brings-chinas-surveillance-state-out-of-the-shadows-idUSKBN2011HO (2020).

  25. Pascu, L. LG CNS collaborates with SenseTime on biometric entry service unaffected by masks. Biometric Update. https://www.biometricupdate.com/202002/lg-cns-collaborates-with-sensetime-on-biometric-entry-service-unaffected-by-masks (2020).

  26. COVID-19 Solutions Suite. Aura Vision, https://auravision.ai/covid-solutions/ (2020).

  27. Dave, P. Companies bet on AI cameras to track social distancing, limit liability. Reuters. https://www.reuters.com/article/us-health-coronavirus-surveillance-tech-idUSKCN22914R (2020).

  28. Infection Control & Prevention. INSPIREN. https://inspiren.com/solutions/infection-control-prevention/ (2020).

  29. Yao, R. COVID caught on camera: Startup’s sensors keep hospitals safe. The Official NVIDIA Blog. https://blogs.nvidia.com/blog/2020/05/19/fever-covid-hospitals-gpus/ (2020).

  30. Zhang, L. et al. An interactive covid-19 mobility impact and social distancing analysis platform. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.29.20085472v1 (2020).

  31. Hegde, C. et al. AutoTriage - an open source edge computing raspberry pi-based clinical screening system. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.09.20059840v2 (2020).

  32. Meisenzahl, M. ‘Pandemic drones’ could single people out in a crowd for coughing, sneezing, or running a temperature, developers say — here’s how they work. Business Insider. https://www.businessinsider.com/draganfly-pandemic-drone-will-detect-people-infected-with-coronavirus-2020-4 (2020).

  33. Imran, A. et al. AI4COVID-19: AI enabled preliminary diagnosis for COVID-19 from cough samples via an app. Inf. Med. Unlocked 20, 100378, https://doi.org/10.1016/j.imu.2020.100378 (2020).

    Article  Google Scholar 

  34. Record your voice to help beat COVID. COVID Voice Detector. https://cvd.lti.cmu.edu/ (2020).

  35. WVU Rockefeller Neuroscience Institute announces capability to predict COVID-19 related symptoms up to three days in advance. WVU Medicine. https://wvumedicine.org/news/story?headline=wvu-rockefeller-neuroscience-institute-announces-capability-to-predict-covid-19-related-symptoms-up- (2020).

  36. Wynants, L. et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 369, m1328, https://doi.org/10.1136/bmj.m1328 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  37. de Moraes Batista, A. F., Miraglia, J. L., Donato, T. H. R. & Chiavegatto Filho, A. D. P. COVID-19 diagnosis prediction in emergency care patients: a machine learning approach. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.04.20052092v2 (2020).

  38. Chen, Y. et al. An interpretable machine learning framework for accurate severe vs non-severe COVID-19 clinical type classification. Preprint at https://www.medrxiv.org/content/10.1101/2020.05.18.20105841v1 (2020).

  39. de Freitas Barbosa, V. A. et al. Heg.IA: an intelligent system to support diagnosis of Covid-19 based on blood tests. Preprint at https://www.medrxiv.org/content/10.1101/2020.05.14.20102533v1 (2020).

  40. Feng, C. et al. A novel artificial intelligence-assisted triage tool to aid in the diagnosis of suspected COVID-19 pneumonia cases in fever clinics. Ann Transl Med 9, 201, https://doi.org/10.21037/atm-20-3073 (2021).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  41. Mei, X. et al. Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat. Med. 26, 1224–1228 (2020).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  42. Brinati, D. et al. Detection of COVID-19 infection from routine blood exams with machine learning: A feasibility study. J. Med. Syst. 44, 1–12 (2020).

    Article  CAS  Google Scholar 

  43. Zoabi, Y., Deri-Rozov, S. & Shomron, N. Machine learning-based prediction of COVID-19 diagnosis based on symptoms. NPJ Digit Med. 4, 1–5 (2021).

    Article  Google Scholar 

  44. Wu, J. et al. Rapid and accurate identification of COVID-19 infection through machine learning based on clinical available blood test results. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.02.20051136v1 (2020).

  45. Soares, F. et al. A novel specific artificial intelligence-based method to identify COVID-19 cases using simple blood exams. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.10.20061036v3 (2020).

  46. Yu, H. et al. Data-driven discovery of a clinical route for severity detection of COVID-19 pediatric cases. Preprint at https://www.medrxiv.org/content/10.1101/2020.03.09.20032219v2 (2020).

  47. Wagner, T. et al. Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis. eLife 9, e58227, https://doi.org/10.7554/eLife.58227 (2020).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  48. Jin, S. et al. AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical AI system. Appl Soft Comput 98, 106897, https://doi.org/10.1016/j.asoc.2020.106897 (2021).

    Article  PubMed  Google Scholar 

  49. Gozes, O. et al. Rapid AI development cycle for the coronavirus (COVID-19) pandemic: Initial results for automated detection & patient monitoring using deep learning CT image analysis. Preprint at https://arxiv.org/abs/2003.05037 (2020).

  50. Gozes, O. et al. Coronavirus detection and analysis on Chest CT with deep learning. Preprint at https://arxiv.org/abs/2004.02640 (2020).

  51. Vanian, J. How chatbots are helping in the fight against COVID-19. Fortune. https://fortune.com/2020/07/15/covid-coronavirus-artificial-intelligence-triage/ (2020).

  52. Martin, A. et al. An artificial intelligence-based first-line defence against COVID-19: digitally screening citizens for risks via a chatbot. Sci. Rep. 10, 1–7 (2020).

    Article  CAS  Google Scholar 

  53. Shoer, S. et al. A prediction model to prioritize individuals for a SARS-CoV-2 test built from national symptom surveys. Med 2, 196–208 (2021).

    PubMed  Article  Google Scholar 

  54. Pourhomayoun, M. & Shakibi, M. Predicting mortality risk in patients with COVID-19 using machine learning to help medical decision-making. Smart Health 20, 100178, https://doi.org/10.1016/j.smhl.2020.100178 (2021).

    Article  PubMed  Google Scholar 

  55. Gong, J. et al. A tool for early prediction of severe Coronavirus Disease 2019 (COVID-19): a multicenter study using the risk nomogram in Wuhan and Guangdong, China. Clin. Infect. Dis. 71, 833–840 (2020).

    CAS  PubMed  Article  Google Scholar 

  56. Jiang, X. G. et al. Towards an artificial intelligence framework for data-driven prediction of coronavirus clinical severity. Comput. Mater. Contin. 63, 537–551 (2020).

    Article  Google Scholar 

  57. Wollenstein-Betech, S., Cassandras, C. G. & Paschalidis, I. C. Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: hospitalizations, mortality, and the need for an ICU or ventilator. Med Inf. 142, 104258, https://doi.org/10.1016/j.ijmedinf.2020.104258 (2020).

    Article  Google Scholar 

  58. Das, A. K., Mishra, S. & Gopalan, S. S. Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ 8, e10083, https://doi.org/10.7717/peerj.10083 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  59. Heldt, F. S. et al. Early risk assessment for COVID-19 patients from emergency department data using machine learning. Sci. Rep. 11, 4200 (2021).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  60. Hu, C. et al. Early prediction of mortality risk among patients with severe COVID-19, using machine learning. Int. J. Epidemiol. 49, 1918–1929 (2020).

    Article  Google Scholar 

  61. Vaid, A. et al. Machine learning to predict mortality and critical events in a cohort of patients with COVID-19 in New York City: Model development and validation. J. Med. Internet Res. 22, e24018, https://doi.org/10.2196/24018 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Yadaw, A. S. et al. Clinical features of COVID-19 mortality: development and validation of a clinical prediction model. Lancet Digit Health 2, e516–e525 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  63. Sarkar, J. & Chakrabarti, P. A machine learning model reveals older age and delayed hospitalization as predictors of mortality in patients with COVID-19. Preprint at https://www.medrxiv.org/content/10.1101/2020.03.25.20043331v1 (2020).

  64. Barda, N. et al. Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nat. Commun. 11, 4439 (2020).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  65. Al-Najjar, H. & Al-Rousan, N. A classifier prediction model to predict the status of Coronavirus COVID-19 patients in South Korea. Eur. Rev. Med. Pharmacol Sci. 24, 3400–3403 (2020).

    CAS  PubMed  Google Scholar 

  66. Yan, L. et al. A machine learning-based model for survival prediction in patients with severe COVID-19 infection. Preprint at https://www.medrxiv.org/content/10.1101/2020.02.27.20028027v3 (2020).

  67. Izquierdo, J. L., Ancochea, J., Savana COVID-19 Research Group & Soriano, J. B. Clinical Characteristics and prognostic factors for intensive care unit admission of patients with COVID-19: Retrospective study using machine learning and natural language processing. J. Med. Internet Res. 22, e21801 (2020).

  68. Singh, K. et al. Evaluating a widely implemented proprietary deterioration index model among hospitalized COVID-19 patients. Ann. Am. Thorac. Soc. https://doi.org/10.1513/AnnalsATS.202006-698OC (2020).

  69. CLEW receives FDA emergency use authorization (EUA) for its predictive analytics platform in support of COVID-19 patients. CLEW. https://clewmed.com/clew-receives-fda-emergency-use-authorization-eua-for-its-predictive-analytics-platform-in-support-of-covid-19-patients/ (2020).

  70. Elghamrawy, S. M. & Hassanien, A. E. Diagnosis and prediction model for COVID-19 patient’s response to treatment based on convolutional neural networks and whale optimization algorithm using CT images. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.16.20063990v1 (2020).

  71. Li, S., Wang, Y., Xue, J., Zhao, N. & Zhu, T. The impact of COVID-19 epidemic declaration on psychological consequences: a study on active Weibo users. Int. J. Environ. Res. Public Health 17, 2032 (2020).

    CAS  PubMed Central  Article  Google Scholar 

  72. Tummers, J., Catal, C., Tobi, H., Tekinerdogan, B. & Leusink, G. Coronaviruses and people with intellectual disability: an exploratory data analysis. J. Intellect. Disabil. Res. https://doi.org/10.1111/jir.12730 (2020).

  73. Call to action to the tech community on new machine readable COVID-19 dataset. The White House. https://trumpwhitehouse.archives.gov/briefings-statements/call-action-tech-community-new-machine-readable-covid-19-dataset/ (2020).

  74. Joshi, B., Bakarola, V., Shah, P. & Krishnamurthy, R. deepMINE - natural language processing based automatic literature mining and research summarization for early-stage comprehension in pandemic situations specifically for COVID-19. Preprint at https://www.biorxiv.org/content/10.1101/2020.03.30.014555v1 (2020).

  75. Zhao, Y. & Zhou, C. C. Applying lexical link analysis to discover insights from public information on COVID-19. Preprint at https://www.biorxiv.org/content/10.1101/2020.05.06.079798v1 (2020).

  76. Wagner, T. et al. Real-time biomedical knowledge synthesis of the exponentially growing world wide web using unsupervised neural networks. Preprint at https://www.biorxiv.org/content/10.1101/2020.04.03.020602v1 (2020).

  77. Awasthi, R. et al. CovidNLP: A web application for distilling systemic implications of COVID-19 pandemic with natural language processing. Preprint at https://www.medrxiv.org/content/10.1101/2020.04.25.20079129v1 (2020).

  78. Rapid Reviews: COVID-19, publishes reviews of COVID-19 preprints. Rapid Rev. COVID-19, https://rapidreviewscovid19.mitpress.mit.edu/ (2020).

  79. Using AI to detect COVID-19 misinformation and exploitative content. Facebook. https://ai.facebook.com/blog/using-ai-to-detect-covid-19-misinformation-and-exploitative-content/ (2020).

  80. RADLogics announces FDA clearance and validation for ai-powered application to support chest X-ray triage and prioritization. PRWeb. https://www.prweb.com/releases/radlogics_announces_fda_clearance_and_validation_for_ai_powered_application_to_support_chest_x_ray_triage_and_prioritization/prweb17410713.htm (2020).

  81. Reich, N. UMass Amherst team develops COVID-19 Forecast Hub. Office of News & Media Relations|UMass Amherst. https://www.umass.edu/newsoffice/article/umass-amherst-team-develops-covid-19 (2020).

  82. Randhawa, G. S. et al. Machine learning using intrinsic genomic signatures for rapid classification of novel pathogens: COVID-19 case study. PLoS ONE 15, e0232391 (2020).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  83. Gudbjartsson, D. F. et al. Spread of SARS-CoV-2 in the Icelandic population. N. Engl. J. Med. 382, 2302–2315 (2020).

    CAS  PubMed  Article  Google Scholar 

  84. Chavan, R., Samant, L., Bapat, S. & Chowdhary, A. Protein modeling and docking of curcurin against neuraminidase, hemagglutinin proteins of pandemic influenza H1N1/2009. J. Pharm. Sci. 7, 70–75 (2015).

    Google Scholar 

  85. Holmgren, A. J., Apathy, N. C. & Adler-Milstein, J. Barriers to hospital electronic public health reporting and implications for the COVID-19 Pandemic. J. Am. Med. Inform. Assoc. 27, 1306–1309 (2020).

    PubMed  PubMed Central  Article  Google Scholar 

  86. The problem with COVID-19 artificial intelligence solutions and how to fix them (SSIR). Stanford Social Innovation Review. https://ssir.org/articles/entry/the_problem_with_covid_19_artificial_intelligence_solutions_and_how_to_fix_them (2020).

  87. Branswell, H. Fluctuating funding and flagging interest hurt coronavirus research. STAT. https://www.statnews.com/2020/02/10/fluctuating-funding-and-flagging-interest-hurt-coronavirus-research/ (2020).

  88. Berry, K. et al. The economic case for a pandemic fund. EcoHealth 15, 244–258 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  89. Bates, D. W., Heitmueller, A., Kakad, M. & Saria, S. Why policymakers should care about “big data” in healthcare. Health Policy Technol. 7, 211–216 (2018).

    Article  Google Scholar 

  90. Bai, Y. P. & Jin, Z. Prediction of SARS epidemic by BP neural networks with online prediction strategy. Chaos Solitons Fractals 26, 559–569 (2005).

    Article  Google Scholar 

  91. Jiang, C. L., Che, Y. Q., Dong, M. & Zhu, Q. A prediction method with more precision on SARS epidemic transmission. In Proc 11th Joint International Computer Conference (World Scientific Publ Co Pte Ltd, 2005).

  92. Mei, S. et al. Individual decision making can drive epidemics: a fuzzy cognitive map study. IEEE Trans. Fuzzy Syst. 22, 264–273 (2014).

    Article  Google Scholar 

  93. Lopez, D. et al. Assessment of vaccination strategies using fuzzy multi-criteria decision making. Proc. Fifth Int. Conf. Fuzzy Neuro Comput. 415, 195–208 (2015).

    Google Scholar 

  94. Aviso, K. B. et al. Allocating human resources in organizations operating under crisis conditions: A fuzzy input-output optimization modeling framework. Resour. Conserv. Recycl. 128, 250–258 (2018).

    Article  Google Scholar 

  95. Tessmer, H. L., Ito, K. & Omori, R. Can machines learn respiratory virus epidemiology?: A comparative study of likelihood-free methods for the estimation of epidemiological dynamics. Front. Microbiol 9, 343 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  96. Achrekar, H., Gandhe, A., Lazarus, R., Yu, S.-H. & Liu, B. Twitter improves seasonal influenza prediction. In Proc International Conference on Health Informatics (HEALTHINF-2012), 61–70, https://doi.org/10.5220/0003780600610070 (2012).

  97. Damianos, L. et al. MiTAP for SARS detection. In Demonstration Papers at HLT-NAACL 2004, 13–16 (2004).

  98. J. Pei et al. Improving prediction accuracy of influenza-like illnesses in hospital emergency departments. In Proc 2013 IEEE International Conference on Bioinformatics and Biomedicine. 602–607, https://doi.org/10.1109/BIBM.2013.6732566 (2013).

  99. López Pineda, A. et al. Comparison of machine learning classifiers for influenza detection from emergency department free-text reports. J. Biomed. Inf. 58, 60–69 (2015).

    Article  Google Scholar 

  100. Lampos, V. & Cristianini, N. Tracking the flu pandemic by monitoring the social web. In Proc 2nd International Workshop on Cognitive Information Processing. 411–416. https://doi.org/10.1109/CIP.2010.5604088 (2010).

  101. Aramaki, E., Maskawa, S. & Morita, M. Twitter catches the flu: detecting influenza epidemics using Twitter. In Proc 2011 Conference on Empirical Methods in Natural Language Processing. 1568–1576 (2011).

  102. Culotta, A. Towards detecting influenza epidemics by analyzing Twitter messages. In SOMA 2010 Proceedings of the 1st Workshop on Social Media Analytics, Association for Computational Linguistics. https://doi.org/10.1145/1964858.1964874 (2010).

  103. Signorini, A., Segre, A. M. & Polgreen, P. M. The use of Twitter to track levels of disease activity and public concern in the U.S. during the Influenza A H1N1 Pandemic. PLoS ONE 6, e19467 (2011).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  104. Collier, N., Son, N. T. & Nguyen, N. M. OMG U got flu? Analysis of shared health messages for bio-surveillance. J. Biomed. Semant. 2, S9 (2011).

    Article  Google Scholar 

  105. Jain, V. K. & Kumar, S. An effective approach to track levels of influenza-A (H1N1) pandemic in India using twitter. Procedia Computer Sci. 70, 801–807 (2015).

    Article  Google Scholar 

  106. Jain, V. K. & Kumar, S. Rough set based intelligent approach for identification of H1N1 suspect using social media. Kuwait J. Sci. 45, 8–14 (2018).

    Google Scholar 

  107. Al-garadi, M. A., Khan, M. S., Varathan, K. D., Mujtaba, G. & Al-Kabsi, A. M. Using online social networks to track a pandemic: a systematic review. J. Biomed. Inf. 62, 1–11 (2016).

    Article  Google Scholar 

  108. Huang, H., Sun, Y. (SH)-H-3: A symptom surveillance system in high spatial resolution using smartphones. 2016 IEEE Wireless Health 64, https://doi.org/10.1109/WH.2016.7764557 (2016).

  109. Ng, E. Y. K., Chong, C. & Kaw, G. J. L. Classification of human facial and aural temperature using neural networks and IR fever scanner: a responsible second look. J. Mech. Med. Biol. 5, 165–190 (2005).

    CAS  Article  Google Scholar 

  110. Ng, E. Y. K. Is thermal scanner losing its bite in mass screening of fever due to SARS? Med. Phys. 32, 93–97 (2005).

    PubMed  Article  Google Scholar 

  111. Ng, E. Y. K. & Chong, C. ANN-based mapping of febrile subjects in mass thermogram screening: facts and myths. J. Med. Eng. Technol. 30, 330–337 (2006).

    CAS  PubMed  Article  Google Scholar 

  112. Quek, C., Irawan, W. & Ng, E. A Cognitive Interpretation of Thermographic Images Using Novel Fuzzy Learning Semantic Memories. Handbook on Decision Making. In: Jain L.C., Lim C.P. (eds) Handbook on Decision Making. Intelligent Systems Reference Library, Vol 4, 427–452. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13639-9_17 (2010).

  113. Quek, C., Irawan, W. & Ng, E. Y. K. A novel brain-inspired neural cognitive approach to SARS thermal image analysis. Expert Syst. Appl. 37, 3040–3054 (2010).

    Article  Google Scholar 

  114. Sun, G. H. et al. Applications of infrared thermography for noncontact and noninvasive mass screening of febrile international travelers at airport quarantine stations. Appl. Infrared Biomed. Sci. 347–358, https://doi.org/10.1007/978-981-10-3147-2_19 (2017).

  115. Xuanyang, X., Yuchang, G., Shouhong, W. & Xi, L. Computer aided detection of SARS based on radiographs data mining. In. Conf. Proc. IEEE Eng. Med Biol. Soc. 2005, 7459–7462 (2005).

    PubMed  Google Scholar 

  116. Xie, X. Y. et al. Mining X-ray images of SARS patients. Data Min. Theory Methodol. Tech. Appl. 3755, 282–294 (2006).

    Google Scholar 

  117. Yao, J., Dwyer, A., Summers, R. M. & Mollura, D. J. Computer-aided diagnosis of pulmonary infections using texture analysis and support vector machine classification. Acad. Radiol. 18, 306–314 (2011).

    PubMed  PubMed Central  Article  Google Scholar 

  118. Mendis, B. S. U., Gedeon, T. D. & Koczy, L. T. Learning generalized weighted relevance aggregation operators using Levenberg-Marquardt method. In Proc 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06), 34–34 https://doi.org/10.1109/HIS.2006.264917 (2006).

  119. Biswas, S. K., Sinha, N., Purakayastha, B. & Marbaniang, L. Hybrid expert system using case based reasoning and neural network for classification. Biol. Inspired Cogn. Archit. 9, 57–70 (2014).

    Google Scholar 

  120. Biswas, S. K., Sinha, N., Baruah, B. & Purkayastha, B. Intelligent decision support system of swine flu prediction using novel case classification algorithm. Int J. Knowl. Eng. Data Min. 3, 1–19 (2014).

    Article  Google Scholar 

  121. Mansiaux, Y. & Carrat, F. Detection of independent associations in a large epidemiologic dataset: a comparison of random forests, boosted regression trees, conventional and penalized logistic regression for identifying independent factors associated with H1N1pdm influenza infections. BMC Med. Res. Methodol. 14, 1–10 (2014).

    Article  Google Scholar 

  122. Raghav, R. S. & Dhavachelvan, P. Bigdata fog based cyber physical system for classifying, identifying and prevention of SARS disease. J. Intell. Fuzzy Syst. 36, 4361–4373 (2019).

    Article  Google Scholar 

  123. Bates, D. W., Auerbach, A., Schulam, P., Wright, A. & Saria, S. Reporting and implementing interventions involving machine learning and artificial intelligence. Ann. Intern. Med. 172, S137–S144 (2020).

    PubMed  Article  Google Scholar 

  124. IHME|COVID-19 Projections. Institute for Health Metrics and Evaluation. https://covid19.healthdata.org/ (2020).

  125. Murray, C. J. Forecasting COVID-19 impact on hospital bed-days, ICU-days, ventilator-days and deaths by US state in the next 4 months. Preprint at https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v1 (2020).

  126. IHME COVID-19 Forecasting Team. Modeling COVID-19 scenarios for the United States. Nat. Med. 27, 94–105 (2020).

    Article  CAS  Google Scholar 

  127. Tsunoda, K., Shinya, K. & Suzuki, Y. Investigation of efficient protection from an influenza pandemic using CARMS. Artif. Life Robot. 16, 1–4 (2011).

    Article  Google Scholar 

  128. Laguzet, L. & Turinici, G. Individual vaccination as Nash Equilibrium in a SIR model with application to the 2009-2010 Influenza A (H1N1) Epidemic in France. Bull. Math. Biol. 77, 1955–1984 (2015).

    CAS  PubMed  Article  Google Scholar 

  129. Beeler, M. F., Aleman, D. M. & Carter, M. W. Estimation and management of pandemic influenza transmission risk at mass immunization clinics. In Proc 2011 Winter Simulation Conference, 1117–1124, https://doi.org/10.1109/WSC.2011.6147834 (2011).

  130. Chen, S. C. & Liao, C. M. Modelling control measures to reduce the impact of pandemic influenza among schoolchildren. Epidemiol. Infect. 136, 1035–1045 (2008).

    PubMed  Article  Google Scholar 

  131. Chin, E. T. et al. Frequency of routine testing for Coronavirus Disease 2019 (COVID-19) in high-risk healthcare environments to reduce outbreaks. Clin. Infect. Dis. https://doi.org/10.1093/cid/ciaa1383 (2020).

Download references

Acknowledgements

The authors would like to thank Dr. Wenyu Song and Ms. Zoe Co for assistance with data abstraction. A.S. is supported by a Fellowship Award from the Canadian Institutes of Health Research. This work has been supported by IBM Watson Health (Cambridge, MA), which is not responsible for the content or recommendations made.

Author information

Authors and Affiliations

Authors

Contributions

A.S., M.K., K.J.T.C., J.H., G.P.J., K.R. and D.W.B. were responsible for study conception and design; A.S. and P.A.B. developed the peer-reviewed literature search; A.S. and M.K. developed the preprint and Google searches; A.S., M.K., A.A., and A.L.B. reviewed the literature; A.S., M.K. and A.A. analyzed and interpreted data; A.S., M.K., A.L.B., K.J.T.C., J.H. and D.W.B. drafted the original manuscript; and A.A., G.P.J. and K.R. reviewed the draft and provided critical feedback. All authors contributed to and approved the final manuscript.

Corresponding author

Correspondence to Ania Syrowatka.

Ethics declarations

Competing interests

D.W.B. consults for EarlySense, which makes patient safety monitoring systems. He receives cash compensation from CDI (Negev), Ltd, which is a not-for-profit incubator for health IT startups. He receives equity from ValeraHealth which makes software to help patients with chronic diseases. He receives equity from Clew which makes software to support clinical decision-making in intensive care. He receives equity from MDClone which takes clinical data and produces deidentified versions of it. He receives equity from AESOP which makes software to reduce medication error rates. K.J.T.C. and G.P.J. are employed by IBM Watson Health. J.H. is employed by IBM Research. K.R. was employed by IBM Watson Health and is employed by CVS Health. A.L.B. reported receiving consulting fees from Aledade Inc. and was previously employed there. The other co-authors have no disclosures.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Syrowatka, A., Kuznetsova, M., Alsubai, A. et al. Leveraging artificial intelligence for pandemic preparedness and response: a scoping review to identify key use cases. npj Digit. Med. 4, 96 (2021). https://doi.org/10.1038/s41746-021-00459-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41746-021-00459-8

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing