The coronavirus disease 2019 (COVID-19) pandemic has overwhelmed healthcare services, faced with the twin challenges in acutely meeting the medical needs of patients with COVID-19 while continuing essential services for non-COVID-19 illnesses. The need to re-invent, re-organize and transform healthcare and co-ordinate clinical services at a population level is urgent as countries that controlled initial outbreaks start to experience resurgences. A wide range of digital health solutions have been proposed, although the extent of successful real-world applications of these technologies is unclear. This study aims to review applications of artificial intelligence (AI), telehealth, and other relevant digital health solutions for public health responses in the healthcare operating environment amidst the COVID-19 pandemic. A systematic scoping review was performed to identify potentially relevant reports. Key findings include a large body of evidence for various clinical and operational applications of telehealth (40.1%, n = 99/247). Although a large quantity of reports investigated applications of artificial intelligence (AI) (44.9%, n = 111/247) and big data analytics (36.0%, n = 89/247), weaknesses in study design limit generalizability and translation, highlighting the need for more pragmatic real-world investigations. There were also few descriptions of applications for the internet of things (IoT) (2.0%, n = 5/247), digital platforms for communication (DC) (10.9%, 27/247), digital solutions for data management (DM) (1.6%, n = 4/247), and digital structural screening (DS) (8.9%, n = 22/247); representing gaps and opportunities for digital public health. Finally, the performance of digital health technology for operational applications related to population surveillance and points of entry have not been adequately evaluated.
The coronavirus disease 2019 (COVID-19) pandemic has crippled both economies and health systems, killing more than 1 million people, with threats of resurgence even as many nations control initial outbreaks1,2. Many health systems are overwhelmed3,4, with this trend being more pronounced in front-line emergency services and mental health services5,6,7. Conservative modelling has indicated that certain health systems are particularly vulnerable, including many developing countries in Asia with limited healthcare capacity, along with shortages of beds in hospitals and intensive care units (ICUs) in African countries8,9. Health systems need to rapidly re-organize resources and restructure clinical services at a population level to minimize the risk of healthcare-associated transmission, as well as meet public health requirements for continued surveillance, risk mitigation, and containment2,4.
Digital health technologies, such as telehealth, artificial intelligence (AI) and big data predictive analytics, offer substantial promise to mitigate the effects of COVID-19 by enhancing population-level public health responses. Some of these digital solutions have already been piloted and deployed to address the challenges of COVID-1910,11. However, while there have been exciting isolated reports of real-world development and validation of these digital solutions, recent literature has also highlighted significant challenges in deployment and scale-up, and limitations of clinical trials that are of varied quality and design12,13. Therefore, it is presently unclear what digital health solutions, if any, have been successfully deployed and applied in the public health responses to the COVID-19 pandemic.
This manuscript is a systemic review of digital health applications for population-level public health responses during the first 6 months of the pandemic. We used a scoping review approach to map out the range and nature of evidence, in order to answer our fundamental question: “What forms of digital health had been applied for public health responses to COVID-19?”.
We retrieved an initial 1904 unique records by the search. All titles and abstract information available in the database were reviewed during screening, and 1559 reports were excluded. 345 full-text reports were then assessed for eligibility. The 345 reports identified on screening originated from over 15 countries and regions (Supplemental Fig. 1). After full-text articles were assessed for eligibility, 247 reports were included in this scoping review for data charting and analysis (Screening flow diagram in Supplemental Fig. 2). The study design and other key features of the included articles are described in Table 1. Only 20 articles (8.1%) investigated patient and/or provider acceptance of these technologies, whereby 17 studies focused primarily on acceptance while three cross-sectional studies had included assessment of acceptance. The technology domains that were most frequently described for responses to COVID-19 were AI (44.9%, n = 111/247), telehealth (40.1%, n = 99/247), and big data (36.0%, n = 89/247).
The complete spectrum of CAs and OAs of these digital health technology domains in the context of COVID-19 are detailed in Table 2. These are further visualized in the form of a Spider diagrams (Fig. 1) and detailed matrix cross-tabular table (Supplemental Fig. 3) to map the evidence for these digital health solutions, indicating the percentage of reports that have topical coverage of each clinical and/or operational application within each technology domain.
Despite a large number of reports describing promising AI and big data applications in the pandemic13, we found that minimal investigations for patient and/or provider acceptance have been reported. This is despite numerous reports of these tools being widely applied for surveillance or interpretation of chest imaging scans for operational efficiencies in overloaded healthcare services. Furthermore, few investigations of IoT solutions have been reported despite a large number of these solutions being deployed at the population level for the monitoring of high risk patients under quarantine, such as returning travelers or contacts of confirmed cases. That said, there have been a surprisingly large number of reports regarding applications of DCs such as digital messaging communications platforms, national/organizational websites for information dissemination (1-way), or online health communities (OHCs) that facilitate discussions (2-way). These categories of reports are further detailed in Table 3 using the report assessment criteria defined in the methodology section (screening reports).
There was a paucity of reports describing the performance of digital health technologies applied at points of entry and national laboratories. The varying quality of study design and methods of analyses are further depicted in the form of a bubble plot in Fig. 2 using the same report assessment criteria, highlighting the shortage of investigations for DM, IoT, DS and DC, as well as methodological limitations despite the large number of reported studies for AI and big data applications. Notably, there was a surprising prominence of reports about DCs (Fig. 2) for triage14,15,16, co-ordination16,17,18, and public health communication addressing misinformation, resource availability, and evolving guidelines19,20,21,22.
Our systematic scoping review provides an overview of digital health technology that were used for clinical and/or operational applications for population-level public health responses in the first 6 months of the COVID-19 pandemic. Our findings build on recent editorial and perspective articles that provide a subjective narrative overview of various digital health topics that could be potentially applied in public health responses to COVID-1910,11,23. The systematic and pragmatic approach of this scoping review provides a map of existing reports at this critical juncture of the pandemic as countries develop population health strategies for safe re-opening. We believe that this serves as a crucial reference for public healthcare systems regarding potential impact and relevance of different digital technologies to prioritise resources and efforts to address the challenges presented by COVID-19. In addition, we highlight significant gaps in the literature that can be addressed through the conduct of research concurrently with the deployment of these solutions.
The need for rapid adoption of digital health technology has been suggested and driven by the unprecedented scale in the impact of the COVID-19 pandemic due to an increasingly connected global ecosystem with mass travel, urban overcrowding, and information from social and digital media11. These factors did not feature as prominently in previous major infectious disease outbreaks such as severe acute respiratory distress syndrome (SARS) and middle east respiratory syndrome (MERS)24,25. In particular, the deluge of misinformation during this pandemic has drowned out official information, in what has been dubbed an “infodemic” by WHO26. Coupled with evolving recommendations as scientists gradually uncover more information about this virus23, the infodemic has needlessly fueled growing paranoia and anxiety among the public7, as well as confusion for patients with chronic diseases who seek to continue the care for their medical problems27. In this regard, DC is at the forefront to address the infodemic and provide transparent information and updates, as reflected in the prominence of relevant reports (Fig. 2).
While there are a range of digital health technology and the maturity of some (e.g., AI) has paved the way for the digitization of clinical and operational responses to contain the pandemic28, there remain significant challenges and gaps in adoption, scale-up and integration into healthcare systems, even in developed countries29. For example, there continues to be ethical concerns with population-level deployment of these tools, particularly in the case of surveillance technologies without individual consent, presenting new ethical and privacy concerns that need to be addressed30,31. Moreover, although vulnerable regions with limited health system capacity are likely to benefit the most from scalable digital tools32, many have barriers to technology implementation illustrated in earlier technology reports33. These regions will require concerted support and public health coordination for the year ahead23, at least until a safe and effective vaccine or treatment is readily available. Without this, limiting the human toll and addressing infectious reservoirs will remain a formidable challenge, potentially crippling these fragile health systems.
The scoping review approach provided for a detailed account of the spectrum of relevant literature at this critical juncture of the pandemic. Gaps in the literature that have been identified in this review include assessments of digital health technologies for operational applications at points of entry and for population surveillance (Fig. 1). Furthermore, although there was a large quantity of reports investigating applications of AI and big data, limitations in study design curb generalizability and translation. These results indicate that there is a pressing need for more investigations of IoT, DC, DM, and DS digital health technologies, as well as underscore the need for better quality studies of digital health such as AI and big data applications using prospective, pragmatic study designs (Fig. 2)34.
The strengths of this review include its timeliness in the context of the ongoing pandemic, systematic article inclusion and data extraction, as well as the scoping review approach for an in-depth analysis of the literature. Added benefits of this review include an a priori protocol and involvement of stakeholders with relevant experience developing digital health and deployment in clinical services before and during COVID-1935. We have specifically included in this review only digital technologies with applications for population-level public health responses during COVID-19. Our search strategy was limited to reports that self-identified with relevant search terms (Supplementary note 1) selected to improve the yield of reports about digital technologies with relevance to population-level public health responses to COVID-19 given the timeliness of this topic We have not included other digital technologies such as those regarding fitness trackers, augmented reality (AR) or virtual reality (VR) digital health tools29.
The limited number of studies investigating patient and provider acceptance of these tools (<10% of reports) also highlights the need for greater participatory research involving stakeholders to increase the likelihood of sustained adoption beyond the pandemic26,36. This is advocated on the basis of a growing body of evidence surrounding the complexity of digital health solutions due to their interactions with operational and interpersonal aspects of clinical care beyond the target condition(s)30,36,37,38. Therefore, digital health solutions may need to be evaluated as clinical care pathway interventions rather than isolated tools, in order to achieve holistic assessment and inform successful implementation39.
In conclusion, our study provides a rapid scoping review of digital health applications described in the first six months of the pandemic, highlighting potential applications and gaps in the literature for the consideration of clinicians, administrators, and researchers. More studies investigating specific applications of digital health to develop relevant scalable public health responses are highlighted, in particular, the pressing need for researchers to formally evaluate digital health applications for population surveillance and points of entry. Finally, there is a general need for better methodological design in the investigation of digital health applications prospectively using pragmatic approaches to better inform public health responses. The use of participatory approaches in the deployment and assessment of these tools will also yield crucial insights to enable sustained adoption during and beyond the pandemic.
We conducted a systematic review in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines extension for scoping reviews (PRISMA-ScR). The review was pre-registered in open science framework (OSF, registration number: osf.io/8nbgj). To be included in the review, papers needed to provide original descriptions of clinical and/or operational applications of digital health technology or solutions in the context of COVID-19 for population-level public health responses. All English-language peer-reviewed reports and pre-prints published within the first 6 months of the pandemic are included. Pre-prints are included due to the extremely current nature of this topic. The completed PRISMA-ScR checklist is included (Supplemental Table 1).
Search strategy and selection criteria
To identify potentially relevant reports, databases were searched from the time of the initial announcement from WHO regarding a cluster of cases of pneumonia in Wuhan on 31 Dec 201940, to 1 July 2020. The search was conducted on 2 July 2020 and exported to Microsoft excel for screening and charting. Electronic bibliographic databases of published research in Pubmed including MEDLINE, IEE explore, and databases for research pre-prints including medrXiv (health sciences), arXiv (engineering), and bioRxiv (biology), given the cross-disciplinary nature of the search topic involving both health sciences and information systems. The search strategies were drafted and refined through study team discussion. Search terms selected for the literature search include the digital health technology domains and the target application context of the pandemic using Boolean operators (OR/AND). The final detailed search strategy for Pubmed is included in this publication (Supplementary note 1).
Randomized-controlled trials, cohort studies, case–control studies, case series, descriptions of technology solutions or case reports of digital health technology and solutions for clinical and/or operational applications in COVID-19 are included. Reports were excluded if they did not fit into the conceptual framework of clinical and/or operational applications for public health responses applied to this study, such as descriptions of digital health for residency training or continuing medical education. Editorials, perspective articles, narrative or other reviews without original data, and study protocols are also excluded.
Study selection was determined by review of available information from study title and abstract in the indexed database for relevance to digital health clinical and/or operational applications for public health responses in the context of COVID-19. Data charting was completed based on all accessible information in the study manuscript. A standardized study screening manual, including a data charting form along with an explanation and elaboration document in the form of a coding manual (Supplementary note 2) was developed by the study team by group consensus.
The report quality assessment criteria used in this study were extrapolated from distillation of the oxford center for evidence-based medicine (OCEBM) construct41, to facilitate greater granularity and relevance to translation for this review. This was done with an aim to provide practical information for decision makers to inform ongoing responses to the pandemic and identify gaps in the literature for researchers looking to evaluate ongoing applications of digital health technologies. Studies are thereby categorized based on the strength of evidence, ranging from case reports to the ideal randomized-controlled trial (RCT) methodology, as well as the translational relevance depending whether prospective or retrospective data was used, and whether an intention-to-treat approach to evaluate the technology “as offered” was adopted to reduce bias and missing data39,42,43.
The coding manual (Supplementary note 2) details how digital health technology or solutions described in these reports were characterized based on technology domains10, including artificial intelligence (AI), big data analytics, internet of things (IoT), telehealth, digital platforms for communication (DC), digital solutions for data management (DM), and digital structural screening (DS). The coding of clinical applications (CAs) were indicated based on clinical priorities for patients with COVID-19 such as detection, triage, developing tests/treatment, as well as continuing care for patients with non-COVID ailments44,45. Finally, the coding of potential relevance to operational applications (OAs) were indicated based on descriptions of the 9-pillars of country-level public health responses as recommended by the WHO46.
Data charting and analysis
To increase consistency of study screening among reviewers, reviewer 1 (DG) piloted the study screening manual for database search and study selection based on title/abstract information available in the databases. Subsequently, reviewer 2 (RT) independently cross-checked study selection for 10% of all articles identified in the database search, using a computer-generated random sequence (www.randomizer.org). Both reviewers then discussed results, and amended the screening manual before the data charting step.
Subsequently, both reviewer 1 and 2 independently piloted the study screening manual for evaluating the eligibility of 10% of all identified full-text reports using a computer-generated random sequence (www.randomizer.org), along with complete data charting for the included articles. Both reviewers then discussed results and amended the screening manual. Finally, reviewers 1 and 2 independently completed assessment of the remaining full-text reports for eligibility along with data charting for all included reports.
Any disagreements on study selection and data charting during pilot testing were resolved by consensus, or otherwise tie-breaker by reviewer 3 (YT) if needed. Where interrater agreement was low (Cohen’s kappa coefficient < 0.8), a repeat sampling of 10% of all relevant reports was conducted with disagreements resolved by consensus. Data from the coding of included studies were analysed quantitatively whereby missing data were handled by pairwise deletion without imputation. We grouped studies by technology domains and summarized the clinical and operational applications described to identify key trends in the literature and knowledge gaps for future research. All findings are synthesized using a narrative review approach. Key results are summarized using Spider diagrams, matrix cross-tabular table of the published clinical/operational applications as well as a bubble plot of the reports depicting the strength of evidence and translational relevance for each technology domain.
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
All included reports from which data was generated and/or analysed in this systematic review are included in the published article and Supplemental Information.
World Health Organisation (WHO). Coronavirus disease (COVID-2019) situation reports. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports/ (2020).
Fontanarosa, P. B. & Bauchner, H. COVID-19-looking beyond tomorrow for health care and society. JAMA 323, 1907–1908 (2020).
Legido-Quigley, H. et al. Are high-performing health systems resilient against the COVID-19 epidemic? Lancet 395, 848–850 (2020).
Walensky, R. P., Rio, Del & From, C. Mitigation to containment of the COVID-19 pandemic: putting the SARS-CoV-2 genie back in the bottle. JAMA 323, 1889–1890 (2020).
Uppal, A. et al. Critical care and emergency department response at the epicenter of the COVID-19 pandemic. Health Aff (Millwood) 39, 1443–1449 (2020).
Rajkumar, R. P. COVID-19 and mental health: a review of the existing literature. Asian J. Psychiatr. 52, 102066 (2020).
Chew, A. M. K. et al. Digital health solutions for mental health disorders during COVID-19. Front. Psychiatry 11, 582007 (2020).
Barasa, E., Ouma, P. O. & Okiro, E. A. Assessing the hospital surge capacity of the Kenyan health system in the face of the COVID-19 pandemic. PLoS ONE 15, e0236308 (2020).
Verma, V. R., Saini, A., Gandhi, S., Dash, U. & Koya, M. S. F. Projecting Demand-Supply Gap of Hospital Capacity in India in the face of COVID-19 pandemic using Age-Structured Deterministic SEIR model. medRxiv 2020.05.14.20100537. Preprint at https://doi.org/10.1101/2020.05.14.20100537 (2020).
Ting, D. S. W., Carin, L., Dzau, V. & Wong, T. Y. Digital technology and COVID-19. Nat. Med. 26, 459–461 (2020).
Rasmussen, S. A., Khoury, M. J. & Del Rio, C. Precision public health as a key tool in the COVID-19 response. JAMA 324, 933–934 (2020).
Piovani, D., Pansieri, C., Peyrin-Biroulet, L., Danese, S. & Bonovas, S. A snapshot of the ongoing clinical research on COVID-19. F1000Res. 9, 373 (2020).
Murray, C. J. L., Alamro, N. M. S., Hwang, H. & Lee, U. Digital public health and COVID-19. Lancet. Public Health 5, e469–e470 (2020).
Annis, T. et al. Rapid implementation of a COVID-19 remote patient monitoring program. J. Am. Med Inf. Assoc. 27, 1326–1330 (2020).
Espinoza, J., Crown, K. & Kulkarni, O. A guide to chatbots for COVID-19 screening at pediatric health care facilities. JMIR Public Health Surveill. 6, e18808 (2020).
Judson, T. J. et al. Rapid design and implementation of an integrated patient self-triage and self-scheduling tool for COVID-19. J. Am. Med Inf. Assoc. 27, 860–866 (2020).
Perez-Alba, E., Nuzzolo-Shihadeh, L., Espinosa-Mora, J. E. & Camacho-Ortiz, A. Use of self-administered surveys through QR code and same center telemedicine in a walk-in clinic in the era of COVID-19. J. Am. Med Inf. Assoc. 27, 985–986 (2020).
Menni, C. et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat. Med. 26, 1037–1040 (2020).
Yuan, E. J. et al. Where to buy face masks? Survey of applications using Taiwan’s open data in the time of coronavirus disease 2019. J. Chin. Med Assoc. 83, 557–560 (2020).
Lu, Y. & Zhang, L. Social media WeChat infers the development trend of COVID-19. J. Infect. 81, e82–e83 (2020).
Zamberg, I. et al. A mobile health platform to disseminate validated institutional measurements during the COVID-19 outbreak: utilization-focused evaluation study. JMIR Public Health Surveill. 6, e18668 (2020).
Hua, J. & Shaw, R. Corona Virus (COVID-19) "Infodemic" and Emerging Issues through a Data Lens: the case of China. Int. J. Environ. Res. Public Health 17, 2309 (2020).
Whitelaw, S., Mamas, M. A., Topol, E., Van & Spall, H. G. C. Applications of digital technology in COVID-19 pandemic planning and response. Lancet Digit Health 2, e435–e440 (2020).
Peeri, N. C. et al. The SARS, MERS and novel coronavirus (COVID-19) epidemics, the newest and biggest global health threats: what lessons have we learned?. Int. J. Epidemiol 49, 717–726 (2020).
Leung, G. M. & Leung, K. Crowdsourcing data to mitigate epidemics. Lancet Digit Health 2, e156–e157 (2020).
Li, L. W., Chew, A. M. K. & Gunasekeran, D. V. Digital health for patients with chronic pain during the COVID-19 pandemic. Br. J. Anaesth. 125, 657–660 (2020).
Rosenbaum, L. The untold toll—the pandemic’s effects on patients without Covid-19. N. Engl. J. Med. 382, 2368–2371 (2020).
Whitelaw, S., Mamas, M. A., Topol, E. & Van Spall, H.G.C. Applications of digital technology in COVID-19 pandemic planning and response. Lancet Digit Health 2, e435–e440 (2020).
Meskó, B., Drobni, Z., Bényei, É., Gergely, B. & Győrffy, Z. Digital health is a cultural transformation of traditional healthcare. Mhealth 3, 38 (2017).
Sustained suppression. Nat. Biomed. Eng. 4, 479–480 (2020).
Cohen, I. G., Gostin, L. O. & Weitzner, D. J. Digital smartphone tracking for COVID-19: public health and civil liberties in tension. JAMA 323, 2371–2372 (2020).
Mallapaty, S. Scientists fear coronavirus spread in countries least able to contain it. Nature 578, 348 (2020).
Gunasekeran, D. V. & Wong, T. Y. Artificial intelligence in ophthalmology in 2020: a technology on the Cusp for translation and implementation. Asia Pac. J. Ophthalmol. (Philos.) 9, 61–66 (2020).
Evangelista, L., Steinhubl, S. R. & Topol, E. J. Digital health care for older adults. Lancet 393, 1493 (2019).
Feldmann, J., Puhan, M. A. & Mütsch, M. Characteristics of stakeholder involvement in systematic and rapid reviews: a methodological review in the area of health services research. BMJ Open 9, e024587 (2020).
Gunasekeran, D. V. Technology and chronic disease management. Lancet Diabetes Endocrinol. 6, 91 (2018).
Greenhalgh, T., Wherton, J., Shaw, S. & Morrison, C. Video consultations for covid-19. BMJ 368, m998 (2020).
Greenhalgh, T. et al. Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework. BMC Med 16, 66 (2018).
Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med 25, 44–56 (2019).
World Health Organization (WHO). Timeline of WHO’s response to COVID-19. https://www.who.int/news-room/detail/29-06-2020-covidtimeline (2020).
OCEBM Levels of Evidence Working Group* “The Oxford Levels of Evidence 2”. Oxford Centre for Evidence-Based Medicine. https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (2020).
Little, R. J. et al. The prevention and treatment of missing data in clinical trials. N. Engl. J. Med. 367, 1355–1360 (2012).
McCoy, C. E. Understanding the intention-to-treat principle in randomized controlled trials. West J. Emerg. Med. 18, 1075–1078 (2017).
Olivia, Li,J. P. et al. Preparedness among ophthalmologists: during and beyond the COVID-19 pandemic. Ophthalmology 127, 569–572 (2020).
Heymann, D. L. et al. COVID-19: what is next for public health? Lancet 395, 542–545 (2020).
World Health Organisation (WHO). COVID‑19 strategic preparedness and response: operational planning guidance to support country preparedness and response. https://www.who.int/publications/i/item/draft-operational-planning-guidance-for-un-country-teams (2020).
D.G. reports equity investment in digital health solutions AskDr, Doctorbell (acquired by Mobile Health), VISRE, and Shyfts, and appointments as physician leader (telemedicine) in Raffles Medical Group and senior lecturer (medical innovation) at the National University of Singapore. T.W. holds patents of deep learning systems for detection of eye diseases. T.W. is the deputy group chief executive officer (research and education) of Singapore Health Services, a consultant & advisory board for Allergan, Bayer, Boehringer-Ingelheim, Genentech, Merck, Novartis, Oxurion (formerly ThromboGenics), Roche, and co-founder of plano and EyRiS. The remaining authors R.T. and Y.T. declare no competing financial interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Gunasekeran, D.V., Tseng, R.M.W.W., Tham, YC. et al. Applications of digital health for public health responses to COVID-19: a systematic scoping review of artificial intelligence, telehealth and related technologies. npj Digit. Med. 4, 40 (2021). https://doi.org/10.1038/s41746-021-00412-9