Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Digitizing clinical trials


Clinical trials are a fundamental tool used to evaluate the efficacy and safety of new drugs and medical devices and other health system interventions. The traditional clinical trials system acts as a quality funnel for the development and implementation of new drugs, devices and health system interventions. The concept of a “digital clinical trial” involves leveraging digital technology to improve participant access, engagement, trial-related measurements, and/or interventions, enable concealed randomized intervention allocation, and has the potential to transform clinical trials and to lower their cost. In April 2019, the US National Institutes of Health (NIH) and the National Science Foundation (NSF) held a workshop bringing together experts in clinical trials, digital technology, and digital analytics to discuss strategies to implement the use of digital technologies in clinical trials while considering potential challenges. This position paper builds on this workshop to describe the current state of the art for digital clinical trials including (1) defining and outlining the composition and elements of digital trials; (2) describing recruitment and retention using digital technology; (3) outlining data collection elements including mobile health, wearable technologies, application programming interfaces (APIs), digital transmission of data, and consideration of regulatory oversight and guidance for data security, privacy, and remotely provided informed consent; (4) elucidating digital analytics and data science approaches leveraging artificial intelligence and machine learning algorithms; and (5) setting future priorities and strategies that should be addressed to successfully harness digital methods and the myriad benefits of such technologies for clinical research.


Clinical trials are required for causal estimation of the efficacy and the safety of new medical treatments, drugs, and devices. However, traditional clinical trials pose challenges that can hinder the efficient conduct of research to develop a knowledge base supporting products for patient communities. Current operational inefficiencies relating to the identification, recruitment, data acquisition, and follow-up of participants inflate costs, increase participant burden, and extend the already-long clinical trial timelines, all of which contribute to low clinical trial participation: for example, only about 8% of cancer patients enroll in cancer trials1,2. For people who do not live close to a research site or who have mobility or scheduling constraints, participating can be expensive, or even impossible3, thus increasing disparities in access to research and limiting the diversity of participants in a trial.

What makes a clinical trial “digital”?

Enter the concept of digital clinical trial, which holds promise as a way to overcome current clinical trial challenges. A digital trial is one that uses technology to improve recruitment and retention, data collection, and analytics (see Fig. 1).

Fig. 1: Summary of the key elements of a digital clinical trial.

Three elements discussed in this paper include digital recruitment and retention, digital health data collection, and digital analytics.

The opportunity to streamline clinical trial costs and efforts through the utilization of digital technologies, while moving more towards a patient-centered trial experience, has captured the interest of the United States (US) National Institutes of Health (NIH) and the National Science Foundation (NSF). The NIH and NSF held a workshop in April 2019 in Bethesda, Maryland bringing together US experts in clinical trials, digital technology, and digital analytics to discuss strategies to implement the use of digital technologies while considering the challenges4. In addition, the NIH made pragmatic trials, which include the use of digital technologies, a priority5. NIH leaders urge trialists to responsibly adopt digital technologies and other pragmatic features6 that disrupt and create efficiencies in a way that preserves the strength of our randomized clinical trials enterprise7. The value of digital technologies has also been recognized in two laws passed by Congress that require multiple federal initiatives8,9.

There is an important opportunity now to harness digital technology to accelerate the pace at which we generate evidence through clinical trials. Digital technology can improve trial efficiency by enhancing and supporting the role of investigators and study teams. Many trials can be done entirely without in-person visits and in some cases participants might not even meet their study teams. However, trials that that involve serious illnesses, extensive procedures like advanced imaging and biopsies and interventions with significant risk will require close monitoring and oversight by competent clinicians and investigators.

Rethinking data collection in the digital health era

Digital health technologies already enable user-friendly measurement of participant health markers, including physical activity10, sleep11, heart rate12, medication adherence13, and respiration patterns/rate14. Many of these technologies have not been developed for research use, but are a part of the burgeoning health and wellness industry. As such, many will need more evaluation and validation before they are fully ready for use in the clinical research setting. For example, some may not have the precision of standard medical devices, but given the longitudinal nature of the data, may produce useful information when evaluated against validated measures.

Despite this, digital health technologies provide the research community with new tools that can greatly enhance the clinical trial enterprise, including the opportunity to run trials liberated from the anchor of “brick and mortar” research centers. A fully digital trial will enable access for potential participants regardless of where they live or work. For investigators, this means more efficient and real-time remote monitoring and multiple opportunities for interactive patient management and assessment (passive and active), which means that more intensive work can be done with budgets that may not require greater resources. This also means investigators can investigate traditionally collected endpoints alongside more novel endpoints—ones that can be captured in an automated fashion and over extended periods. Doing so has the potential to reduce the burden on both research teams and on study participants. Consider the opportunities of pairing geo-location apps on smartphones with electronic health record (EHR) data to identify a study endpoint such as real-time emergency medical appointments and hospitalizations. As such, available and emerging technology represents a potentially transformative approach and evidence for the need for greater efficiency has never been highlighted so well as now, during the time of the COVID-19 pandemic. Indeed, one such randomized trial evaluating a 5-day course of hydroxychloroquine for asymptomatic volunteers following exposure to someone with confirmed infection was launched, enrolled via social and traditional media, consented via electronic means, and all data collected electronically, and published within 90 days15. As such, although digital trials are still in their infancy, the ability to more widely operationalize them has taken on new urgency. Recent advances specific to each of these elements are described below in detail.

Digital recruitment and retention in clinical trials

As discussed above, recruitment, informed consent, and retention of participants is a major impediment to the timely completion of clinical trials. Enrollment rates may be severely curtailed if enrolling a representative population takes priority16,17. Often, to meet enrollment milestones, populations for whom services are hard-to-reach are excluded, which may undermine external validity. Multiple reasons exist to help explain this lack of enrollment, including lack of clinician involvement, financial difficulties18 and overly restrictive eligibility criteria19. The use of digital tools to aid in recruitment, especially for purposes of therapeutic intent trials, is key to more efficient trials, but to move forward, collaboration with the Office of Human Research Protections (OHRP) is critical. While guidance has been issued with respect to the use of new technologies in the informed consent process20, similar guidance for recruiting potential study volunteers is lacking. Such guidance is necessary to address the use of social media and other novel communication tools as recruitment tools by local institutional review boards (IRBs), while allowing investigators to react and respond in real-time. For example, requiring review of tweets planned for use in support of a clinical trial might be reasonable; but requiring IRB review of revisions or responses based on real-time feedback may be detrimental to the original intent of why social media outreach was used. Despite these unchartered waters, efforts are being made to help investigators reach potential study volunteers. For example, the SWOG Cancer Research Network proposes the incorporation of a social media toolkit for new clinical trials, aimed at helping investigators and their institutions raise awareness of these studies21.

Participant retention may be adversely impacted for other reasons, including concerns regarding randomization and assignment to a placebo arm, lack of understanding related to poorly executed informed consent, lack of renumeration, and a general lack of understanding of the clinical trial process22. Digital technologies have the potential to help address all of these issues. Proposed interventions range from tracking applications to allow for more efficient participant outreach, improved opportunities to participate in trials without having to return to the research center, and the use of video and other visual formats to aide informed consent23. Where payments are envisioned, efforts to provide payment should maximize use of currently available technologies, bearing in mind the importance of any relevant revenue and taxation limits for study volunteers.

Beyond this, it is widely known that disparities exist even among those who enroll in clinical trials. For example, Black Americans represented only 2.5% and 4% of enrollees in trials involving cardiology or cancer immunotherapy24,25. Whether these estimates are accurate is a matter of concern, because almost 85% of clinical trials included in a recent review did not report race/ethnicity data26. These findings point to ongoing issues addressing disparities in clinical trials, and potentially another facet that may be addressed through digital technology. Solutions may vary but a potentially actionable solution could focus on the creation of more effective tailored communication strategies to overcome barriers, such as mistrust, access, and fear of human experimentation.

Another demographic group under-represented in clinical trials is older adults. For example, estimates are that 40% of all cardiology clinical trials exclude older adults27, despite older age being a well-established factor associated with cardiovascular risk. Similarly, heart failure is primarily a disease affecting the elderly, but those enrolled in clinical trials tend to be much younger than the actual population of patients in that condition16. The lack of older adults in trials is especially striking given that older adults are the fastest growing demographic group in many parts of the world and who may be differentially impacted by treatments because of their age28. Trials that incorporate remote monitoring to reduce in-person visits could improve research participation for patients who require assistance to get to study sites, including many older patients.

Fortunately, work to harness new technologies to revolutionize the clinical trial enterprise has begun demonstrating that these efforts are feasible. In addition to the randomized trial for volunteers exposed to COVID-19 above, within oncology, the Metastatic Breast Cancer Project29 was developed in collaboration with women and men with metastatic breast cancer, physicians, researchers, and advocates. The project harnesses engagement through social media and addresses the barrier to recruitment and retention through online consent and bidirectional communication that supports information sharing throughout the trial timeframe. As reported in 2017, almost 3000 volunteers signed up for the registry in its first year with 95% completing the required survey about their cancer, treatments, and demographics29.

In addition, recent work supported by the National Library of Medicine and ClinicalTrials.gov30 has been exploring whether standardized language and machine readable templates can use be used to create a platform for trial recruitment. The templates would be analyzed using artificial intelligence to translate clinical trial requirements and information into common language so that potential practitioners and participants could easily find studies in which they could participate.

Digital health data collection

Following recruitment of participants for a clinical trial, data must be collected as part of the trial process. Digital data in clinical trials can take many forms, ranging from clinical and demographic data, sensed physiological and activity data, patient reported outcomes and images collected via a smart phone or tablet, electronic medical record data from a vendor API, and biological samples drawn at home or in a local lab31. Aside from the biological samples, examples of digital health data collection are described in this section.

Digital monitoring and biomarkers

The term digital biomarker typically refers specifically to objective measures of physiologic, pathologic, anatomic, behavioral, social or activity characteristics, and patient self-report that can be acquired using digital technology and “evaluated as an indicator of normal biologic processes, pathologic processes, or biological responses to therapeutic interventions”32,33. While many digital biomarkers are still being validated, over time they may provide in-depth information regarding physiological processes central to informing diagnostics, dosing titration and as endpoints for clinical trials. Examples include wearable sweat sensing for glucose and lactate, as well as electrolytes34, cardiogenic chest wall vibrations to assess clinical status of patients with heart failure35, or structural health markers for knee joint injury obtained through a brace36.

Digital technologies can also be leveraged to enable the collection of data that could not be obtained previously. For example, Zhan et al. required subjects to perform five cognitive tasks on a smartphone application, then merged the task data to form a validated mobile Parkinson’s disease score37. Brogioli and colleagues validated wearable sensors to generate a neurological classification in spinal cord injury38. Recently Food and Drug Administration (FDA) approved digital approaches to detect rhythm abnormalities, such as atrial fibrillation, through sensors in the Apple Watch, including electrodes for electrocardiography and optical sensors for photoplethysmography39,40.

Safety monitoring in digital clinical trials

As digital technology finds its way in to routine clinical trial processes, researchers are increasingly pairing new tools with traditional biomarker assessments as a way to validate their safety and usability32. Harnessing the ability for digital tools to collect data continuously, which can be transmitted directly to researchers, might improve the detection of infrequent events or those that are situation-specific and unlikely to occur during a study visit. The speed at which adverse and safety events can be identified and reported may have a significant impact on the timeliness of completion and reporting of clinical trials.

Nevertheless, critical issues remain that will impact the implementation of digital tools, including their evaluation to ensure they meet standards for reliability and validity, which can be harder to establish for detection of fleeting, momentary or novel events. As important, the expectations on study participants, investigators, and trial sponsors need to be better defined. Attempts are being made to do just this. For example, collaborative recommendations are provided by the Clinical Trials Transformation Initiative (CTTI) for dealing with data collection from mobile technologies, as summarized in Table 141.

Table 1 Clinical trials transformation initiative (CTTI) recommendations for dealing with data collection from mobile technologies (Adapted from ref. 41).

Data security (privacy and security)

It is clear that digital health technology has created challenges in the efforts to modernize standards for privacy, safety, ethics and regulation. Digital technology and remote monitoring heighten the need for security measures to protect against data breaches during collection, transmission, and/or storage of data. While identifiable patient data require protection, some measures, such as GPS/location data42, could put trial participants at particular risk for legal actions and economic loss due to stigma41. To this end, the FDA has embraced cybersecurity as a component of medical device certification43. Even as we strive to increase patient access, the risks and benefits of sharing individual and/or pooled patient data must be conveyed to all stakeholders. Breach of confidential databases remains a risk, although the use of distributed ledgers, such as blockchain, or decentralized databases, could mitigate risk.

Digital trials may benefit from guidance for IRBs, especially as it pertains to consenting requirements, reporting, and oversight. For example, a site-less study may include virtual visits across state lines or national borders rather than at a specific physical study site, and differences in state regulations pertaining to investigator oversight must be considered in order for trials to effectively operate. In the absence of explicit cross-state telehealth licensing agreements, an Interstate Medical Licensure Compact may address this issue, and encouraging all states to participate may be critical44. Further, options for documentation of informed consent will need to incorporate remote consent technologies, while maintaining certainty that consent covers data collection, transmission, sharing, and security.

Data analytics

Real-world data

The digital transformation of healthcare data presents marked opportunities to improve clinical trials, from trial matching to data collection, using real world data collected in EHRs45, medical devices, and technology in Internet of Things, as well as other sources. Truly flexible, adaptable, extensible, and scalable clinical trial infrastructure that utilizes the EHR is now possible; the health record is converging on a few forms of interoperability that will underpin an apps-driven information economy46. For example, the SMART (Substitutable Medical Applications, Reusable Technologies) on FHIR (Fast Healthcare Interoperability Resources) API enables medical researchers, clinical providers, and patients to connect apps to the health system across EHR platforms47. As part of such an automated real-world data system, data from sensors, mobile devices, patient-generated data and patient-reported outcomes could become more routine trial markers and endpoints48. The SMART/HL7 Flat FHIR/Bulk Data Export API49 provides a standardized approach to readily create population-level datasets from EHR. A principal challenge will be data quality, which is currently being addressed throughout the research ecosystem50, including by the FDA. For example, data mapping, which focuses attention on the standardization and validation of data is being explored, and efforts to create common data models are also addressing this challenge51,52 as are emerging technologies to automate processes.

Advanced analytics

Machine learning and artificial intelligence enables the development of advanced analytic methods that can be brought to bear on multiple facets of the conduct of digital clinical trials. For example, automatically inferred clusters or sub-types can be used to bootstrap the personalization of supervised detection methods, providing greater detection accuracy. Similarly, supervised and unsupervised learning methods can be used to passively detect context and infer proximal outcomes while leveraging reinforcement learning methods to optimize intervention component selections based on the inferred context. These approaches can be used to match participants to studies, to improve digital data extraction and computational phenotyping, and to augment efforts to interpret the trial findings.

Optimizing trial methods

Digital approaches also lend themselves to a range of methods, such as micro-randomization, to optimize and personalize clinical trial methods53,54,55. Micro-randomized trials56,57, a method that repeatedly randomizes provision of an intervention to each person each time that intervention may be provided, can identify factors such as timing and dose for optimally delivering digital trial tools such as reminders and engagement strategies for recruiting, enrolling, and retaining trail participants. In addition, such methods can be used to personalize the delivery of such trial tools to maximize their effectiveness for each participant58. Since randomizations intended to optimize trial technologies are often decoupled from the randomizations used to investigate the primary trial question, in many cases both the health and trial technology optimization questions can be investigated concurrently. Over time, such optimization experiments will result in a robust evidence base about how to most effectively recruit, consent, and engage participants in digital trials, greatly enhancing this important innovation in the conduct of biomedical research.

Strategies for the future

The current methods for conducting clinical trials are not sustainable, and will leave a chasm between the need for evidence to inform health and healthcare and the availability of that evidence. New strategies for the future of clinical trials are needed. Suggested strategies for disruptive clinical trials that were suggested by participants in the NIH/NSF Workshop are summarized in Table 24.

Table 2 Research opportunities and potential action Items from the NIH/NSF workshop on digital clinical trials.


The creation of a digital clinical trial enterprise in the US will require empirical research on the risks and benefits of digital trials. This also includes research addressing the privacy and security concerns, training protocols, and potential negative impact of using technology. Efforts should also target development of an evidence base of effective intervention components that can readily inform future intervention development. Implementation will also require research that is higher in risk than past clinical trials as there is a steep learning curve for digital technologies in the biomedical space. Most of those that now develop these technologies are not partnered with clinical researchers, while clinical researchers do not have the skills that support digital clinical trials. To develop the knowledge needed to spur a robust digital clinical trial enterprise will require substantial commitment amongst leadership facing initial challenges, and is required for growth.

The clinical trial research team of the future will have a different composition compared with those of today. Research teams will be transformed to include computer scientists and engineers as critical partners for both technology development, but also data collection, analysis, security, and privacy. These new teams will also have increased technological and data science knowledge among all involved with conducting trials, so that they have a common language with their technical partners and know which questions they should be asking. This includes investigators, research team, clinical staff, regulatory officials, which are all necessary to successfully develop and implement digital trials41. In the future, simply purchasing app development from a third party vendor will not be enough to sustain a digital clinical trial. Diverse partnerships within the team must go deeper to develop tools that engage and sustain participants safely and effectively.

Education on digital technology for IRBs should continue. This builds off the work of the Public Responsibility in Medicine and Research group that has been providing training in digital technologies for health for almost a decade. In addition, members of data safety and monitoring boards (DSMBs) should also be trained on the safe and effective use of technologies, including the risks that technology introduces into the system. Teams should also be informed of the state-of-the-art work being done by FDA and National Institute of Standards and Technology (NIST) on cybersecurity and privacy in medical settings and other best practices.

In summary, the traditional clinical trials system cannot keep pace with the need to evaluate new drugs, devices, and health system interventions or to conduct much needed comparative effectiveness research. Over the last two decades, people have been increasingly embracing innovative solutions built around the reach, convenience and value digital technologies bring them. Many interactions in the past that required face-to-face contact with specific professionals, such as a travel agent or bank teller, are now routinely carried out by individuals in their home on their computer or smartphone. Although the health care and clinical trials enterprises have been slower to adopt digital alternatives to traditional systems, recently, a range of digital tools that can enable the reengineering of health care and clinical research have become available and continue to expand rapidly. Key components, such as medical grade personal sensors, individual access and control of health records, and most challenging, personalized digital communications, are already being incorporated in clinical research, although still most commonly in small pilot programs. Many challenges remain and lessons will be learned as digital research is moved into the mainstream, but knowledge of the innumerable benefits to clinical research reinforces the view that now is the time to support digital methods with a focus on learning the most effective and efficient methods.


  1. 1.

    Murthy, V. H., Krumholz, H. M. & Gross, C. P. Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA 291, 2720–2726 (2004).

    CAS  Article  Google Scholar 

  2. 2.

    Unger, J. M., Vaidya, R., Hershman, D. L., Minasian, L. M. & Fleury, M. E. Systematic review and meta-analysis of the magnitude of structural, clinical, and physician and patient barriers to cancer clinical trial participation. J. Natl. Cancer Inst. 111, 245–255 (2019).

    Article  Google Scholar 

  3. 3.

    Anderson, D. Digital R&D: Four Ways to Maximize Patient Engagement in Clinical Trials, Deloitte Consulting LLP. 25 Jun 2018.

  4. 4.

    NIH NHLBI. Digital Clinical Trials Workshop: Creating a Vision for the Future. Accessed 10 Jun 2020.

  5. 5.

    National Institutes of Health, National Center for Complementary and Integrative Health. The NIH Collaboratory Launches a New Resource on Methods and Best Practices for Pragmatic Clinical Trials. Accessed 23 Mar 2020.

  6. 6.

    National Institutes of Health, Office of Science Policy. Clinical Trials Policy. Accessed 16 Aug 2019.

  7. 7.

    Lauer, M. S. & Bonds, D. Eliminating the “expensive” adjective for clinical trials. Am. Heart J. 167, 419–420 (2014).

    Article  Google Scholar 

  8. 8.

    US Senate Help Committee, 21st Century Cures Act. US House Energy and Commerce Committee. Accessed 4 Aug 2019.

  9. 9.

    US Food and Drug Administration. Prescription Drug User Fee Act (PDUFA). Accessed 4 Aug 2019.

  10. 10.

    Case, M. A., Burwick, H. A., Volpp, K. G. & Patel, M. S. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA 313, 625–626 (2015).

    CAS  Article  Google Scholar 

  11. 11.

    Toon, E., Davey, M. J., Hollis, S. L., Nixon, G. M. & Horne, R. S. et al. Comparison of commercial wrist-based and smartphone accelerometers, actigraphy, and PSG in a clinical cohort of children and adolescents. J. Clin. Sleep Med. 12, 343–350 (2016).

    Article  Google Scholar 

  12. 12.

    Shcherbina, A., Mattsson, C. M., Waggott, D., Salisbury, H. & Christle, J. W. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Pers. Med. 7, 3 (2017).

    Article  Google Scholar 

  13. 13.

    Park, L. G., Howie‐Esquivel, J. & Dracup, K. A quantitative systematic review of the efficacy of mobile phone interventions to improve medication adherence. J. Adv. Nurs. 70, 1932–1953 (2014).

    Article  Google Scholar 

  14. 14.

    AL‐Khalidi, F. Q., Saatchi, R., Burke, D., Elphick, H. & Tan, S. Respiration rate monitoring methods: a review. Pediatr. Pulmonol. 46, 523–529 (2011).

    Article  Google Scholar 

  15. 15.

    Boulware, D. R, Pullen, M. F., Bangdiwala, A. S., Pastick K. A., Lofgren S. M. et al. A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19. New Eng. J. Med. (2020).

  16. 16.

    Kennedy-Martin, T., Curtis, S., Faries, D., Robinson, S. & Johnston, J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials 16, 495 (2015).

    Article  Google Scholar 

  17. 17.

    Gold, M., Amatniek, J., Carrillo, M. C., Cedarbaum, J. M. & Hendrix, J. A. et al. Digital technologies as biomarkers, clinical outcomes assessment, and recruitment tools in Alzheimer’s disease clinical trials. Alzheimer’s Dement. 4, 234–242 (2018).

    Article  Google Scholar 

  18. 18.

    Nipp, R. D., Hong, K. & Paskett, E. D. Overcoming barriers to clinical trial enrollment. Am. Soc. Clin. Oncol. Educ. Book. 39, 105–114 (2019).

    Article  Google Scholar 

  19. 19.

    Kim, E. S., Bruinooge, S. S., Roberts, S., Ison, G. & Lin, N. U. et al. Broadening eligibility criteria to make clinical trials more representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. J. Clin. Oncol. 35, 3737 (2017).

    CAS  Article  Google Scholar 

  20. 20.

    US Food and Drug Administration (FDA). Use of Electronic Informed Consent Questions and Answers: Guidance for Institutional Review Boards, Investigators, and Sponsors. Accessed 10 Jun 2020.

  21. 21.

    Gunturu, K. S., Dizon, D. S., Johnson, J., Mercurio, A. M. & Mason, G. et al. Clinical trials in the era of digital engagement: A SWOG call to action. JCO Clin. Cancer Informat. 4, 254–258 (2020).

    Article  Google Scholar 

  22. 22.

    Fogel, D. B. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp. Clin. Trials Commun. 11, 156–164 (2018).

    Article  Google Scholar 

  23. 23.

    Liu, P., Astudillo, K., Velez, D., Kelley, L., Cobbs-Lomax, D. et al. Use of mobile health apps in low-income populatoins: a prospective study of facilitators and barriers. medRxiv. (2019)

  24. 24.

    Cohen, L. L., Cousins, L. & Martin, S. Procedural pain distraction. (eds McGrath, P. J., Stevens, B. J., Walker, S. M. & Zempsky, W.) (Oxford Textbook of Paediatric Pain, 2013).

  25. 25.

    Nazha, B., Mishra, M., Pentz, R. & Owonikoko, T. K. Enrollment of racial minorities in clinical trials: old problem assumes new urgency in the age of immunotherapy. Am. Soc. Clin. Oncol. Educ. Book. 39, 3–10 (2019).

    Article  Google Scholar 

  26. 26.

    Geller, S. E., Koch, A. R., Roesch, P., Filut, A. & Hallgren, E. et al. The more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials. Acad. Med. 93, 630 (2018).

    Article  Google Scholar 

  27. 27.

    Farkouh, M. E. & Fuster, V. Time to welcome the elderly into clinical trials. Nat. Clin. Pract. Cardiovas. Med. 5, 673–673 (2008).

    Article  Google Scholar 

  28. 28.

    Shenoy, P. & Harugeri, A. Elderly patients’ participation in clinical trials. Perspect. Clin. Res. 6, 184 (2015).

    Article  Google Scholar 

  29. 29.

    Wagle, N., Painter, C., Krevalin, M., Oh, C. & Anderka, K. et al. The metastatic breast cancer project: a national direct-to-patient initiative to accelerate genomics research. J. Clin. Oncol. 34(8-suppl) (2016).

  30. 30.

    Bentley, F., Tollmar, K., Stephenson, P., Levy, L. & Jones, B. et al. Health Mashups: Presenting statistical patterns between wellbeing data and context in natural language to promote behavior change. ACM Trans. Comput.-Human Interaction 20, 1–27 (2013).

  31. 31.

    Dhruva, S. S., Ross, J. S., Akar, J. G., Caldwell, B. & Childers, K. et al. Aggregating multiple real-world data sources using a patient-centered health data sharing platform: an 8-week cohort study among patients undergoing bariatric surgery or catheter ablation of atrial fibrillation. medRxiv. 3, 19010348 (2019).

  32. 32.

    Coravos, A., Khozin, S. & Mandl, K. D. Developing and adopting safe and effective digital biomarkers to improve patient outcomes. NPJ Digital Med. 2, 14 (2019).

    Article  Google Scholar 

  33. 33.

    Kalali, A., Richerson, S., Ouzunova, E. Westphal, R. & Miller, B. Digital biomarkers in clinical drug development. In Handbook of Behavioral Neuroscience. Vol. 29, 229–238 (Elsevier, 2019).

  34. 34.

    Gao, W., Emaminejad, S., Nyein, H. Y. Y., Challa, S. & Chen, K. et al. Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis. Nature 529, 509 (2016).

    CAS  Article  Google Scholar 

  35. 35.

    Inan, O. T., Baran Pouyan, M., Javaid, A. Q., Dowling, S. & Etemadi, M. et al. Novel wearable seismocardiography and machine learning algorithms can assess clinical status of heart failure patients. Circ. Heart Fail. 11, e004313 (2018).

    Article  Google Scholar 

  36. 36.

    Whittingslow, D. C., Jeong, H.-K., Ganti, V. G., Kirkpatrick, N. J. & Kogler, G. F. et al. Acoustic emissions as a non-invasive biomarker of the structural health of the knee. Ann. Biomed. Eng. 48, 1–11 (2019).

  37. 37.

    Zhan, A., Mohan, S., Tarolli, C., Schneider, R. B. & Adams, J. L. et al. Using smartphones and machine learning to quantify Parkinson disease severity: the mobile Parkinson disease score. JAMA Neurol. 75, 876–880 (2018).

    Article  Google Scholar 

  38. 38.

    Brogioli, M., Popp, W. L., Schneider, S., Albisser, U. & Brust, A. K. et al. Multi-day recordings of wearable sensors are valid and sensitive measures of function and independence in human spinal cord injury. J. Neurotrauma 34, 1141–1148 (2017).

    Article  Google Scholar 

  39. 39.

    U.S. Food and Drug Administration. ECG App: Electrocardiograph Software for Over-the-Counter Use. Accessed 10 Jun 2020.

  40. 40.

    U.S. Food and Drug Administration. Irregular Rhythm Notification Feature: Photoplethysmograph Analysis Software for Over-the-Counter Use. Accessed 10 Jun 2020.

  41. 41.

    Clinical Trials Transformation Initiative. CTTI Recommendations: Advancing the Use of Mobile Technologies for Data Capture & Improved Clinical Trials. Accessed 5 Feb 2020.

  42. 42.

    Epstein, D. H., Tyburski, M., Craig, I. M., Phillips, K. A. & Jobes, M. L. et al. Real-time tracking of neighborhood surroundings and mood in urban drug misusers: application of a new method to study behavior in its geographical context. Drug alcohol Depend. 134, 22–29 (2014).

    Article  Google Scholar 

  43. 43.

    US Food and Drug Administration (FDA). The FDA’s Role in Medical Device Cybersecurity. Accessed 10 Jun 2020.

  44. 44.

    Interstate Medical Licensure Compact. Accessed 10 Jun 2020.

  45. 45.

    Mandl, K. D. & Kohane, I. S. A 21st-Century Health IT System-creating a real-world information economy. N. Engl. J. Med. 376, 1905–1907 (2017).

    Article  Google Scholar 

  46. 46.

    Mandl, K. D., Mandel, J. C. & Kohane, I. S. Driving innovation in health systems through an apps-based information economy. Cell Syst. 1, 8–13 (2015).

    CAS  Article  Google Scholar 

  47. 47.

    Mandel, J. C., Kreda, D. A., Mandl, K. D., Kohane, I. S. & Ramoni, R. B. et al. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J. Am. Med. Inform. Assoc. 23, 899–908 (2016).

    Article  Google Scholar 

  48. 48.

    Sayeed, R., Gottlieb, D. & Mandl, K. D. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. npj Digital Med. 3, 1–8 (2020).

    Article  Google Scholar 

  49. 49.

    Meeting to Advance Push Button Population Health. SMART/HL7 Bulk Data Export/FLAT FHIR. Accessed 7 Feb 2020.

  50. 50.

    Duke University. Duke University Margolis Center for Health Policy, Determining Real-World Data’s Fitness for Use and the Role of Reliability. Accessed 7 Feb 2020.

  51. 51.

    OHDSI. Observational Health Data Sciences and Informatics OMOP Common Data Model. Accessed 25 Mar 2020.

  52. 52.

    pcornet: The National Patient-Centered Clinical Research Network. Real-World Data. Accessed 25 Mar 2020.

  53. 53.

    Rabbi, M., Klasnja. P., Choudhury, T., Tewari, A. & Murphy, S. Optimizing mHealth interventions with a bandit. Digital Phenotyping and Mobile Sensing. 277–291 (Springer, 2019).

  54. 54.

    Hekler, E. B., Michie, S., Pavel, M., Rivera, D. E. & Collins, L. M. et al. Advancing models and theories for digital behavior change interventions. Am. J. Prevent. Med. 51, 825–832 (2016).

    Article  Google Scholar 

  55. 55.

    Aung, M. H., Matthews, M. & Choudhury, T. Sensing behavioral symptoms of mental health and delivering personalized interventions using mobile technologies. Depression Anxiety 34, 603–609 (2017).

    Article  Google Scholar 

  56. 56.

    Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A. & Almirall, D. et al. Microrandomized trials: an experimental design for developing just-in-time adaptive interventions. Health Psychol. 34, 1220 (2015).

    Article  Google Scholar 

  57. 57.

    Liao, P., Klasnja, P., Tewari, A. & Murphy, S. A. Sample size calculations for micro‐randomized trials in mHealth. Stat. Med. 35, 1944–1971 (2016).

    Article  Google Scholar 

  58. 58.

    Liao, P., Greenewald, K., Klasnja, P. & Murphy, S. Personalized heartsteps: a reinforcement learning algorithm for optimizing physical activity. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1–22 (2020).

    Article  Google Scholar 

Download references


The workshop that resulted in this work was funded through the National Institutes of Health (NIH), National Heart, Lung and Blood Institute (NHLBI), and the National Science Foundation (NSF) under Grant Number 1922762. The paper was also supported in part by both the Extramural and Intramural research programs of the NIH (NHLBI and NIDA). S.R.S. is supported in part by a grant from the National Center for Advancing Translational Sciences/NIH (grant UL1TR001114). Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the NHLBI, the NIH, or the NSF. We thank W. Nilsen (NSF) for her contributions to this effort; her input was invaluable. We also acknowledge the following people for their contributions to and/or participation in the NIH/NSF Digital Clinical Trials Workshop: D. Goff, W. Cleaveland, A. Abernathy, N. Pereira, C. Lipset, J. Wilbanks, G. Alterovitz, S. Shore, B. Mynatt, R. Harrington, A. Hernandez, S. Grinspoon, L. Belen, K. Velarde, S. Czaja, H. Taylor, B. Rao, T. Zayas-Caban, B. Patel, I. Prichard, A. Kravets, D. Wolff-Hughes, J. Bamdad, B. Newsome, R. Roper, C. Sasiela, H. Nicastro, A. Lee, N. Redmond, V. Sachdev, P. Schum, S. Czajkowski, A. Oh, T. Lash, R. Conroy, C. Aklin, B. Jean-Francois, V. Florance, N. Langhals, C. Lungu, D. Whittingslow, and N. Gurel.

Author information




All authors participated in the NIH/NSF Digital Clinical Trials Workshop held in Bethesda, MD in April 2019 where the concepts described in this manuscript were discussed and formed, and participated in the writing, editing, and review of the manuscript. O.T.I. acquired funding from the NSF for the workshop.

Corresponding author

Correspondence to O. T. Inan.

Ethics declarations

Competing interests

M.T. reports grants from Janssen Inc, personal fees from Medtronic Inc, grants from AstraZeneca, other from AliveCor, personal fees from Abbott, grants from Boehringer Ingelheim, personal fees from Precision Health Economics, personal fees from iBeat Inc, grants and personal fees from Cardiva Medical, personal fees from iRhythm, grants from Bristol Myers Squibb, grants from American Heart Association, grants from SentreHeart, personal fees from Novartis, personal fees from Biotronik, personal fees from Sanofi-Aventis, personal fees from Pfizer, grants from Apple, outside the submitted work; and Dr. M.T. is an editor for JAMA Cardiology. B. Spring is on the Scientific Advisory Board for Actigraph. O.T.I. is a Scientific Advisor for Physiowave, Inc. and a Co-Founder and Scientific Advisor for Cardiosense, has research funding for his lab from Hill-Rom Services, Inc. and Murata Americas, and has IP licensed by Physiowave, TandemLaunch, and Cardiosense. Dr. K.D.M.’s research lab receives philanthropic support from Eli Lily and he has consulted on using real world evidence for Merck. M.J.P. is supported by the NIH to develop and maintain a research platform to support digital trials and mHealth data collection. S.R.S. has served as a medical advisor for Janssen, Otsuka, Livongo, DynoSense, BioSense, and Spry Health. H.R. has received donations of OCT catheters for a research study from Abbott Vascular and telemetry monitors from BioTelemetry, Inc. Dr. Krumholz works under contract with the Centers for Medicare & Medicaid Services to support quality measurement programs; was a recipient of a research grant, through Yale, from Medtronic and the Food and Drug Administration to develop methods for post-market surveillance of medical devices; was a recipient of a research grant with Medtronic and is the recipient of a research grant from Johnson & Johnson, through Yale University, to support clinical trial data sharing; was a recipient of a research agreement, through Yale University, from the Shenzhen Center for Health Information for work to advance intelligent disease prevention and health promotion; collaborates with the National Center for Cardiovascular Diseases in Beijing; receives payment from the Arnold & Porter Law Firm for work related to the Sanofi clopidogrel litigation, from the Martin/Baughman Law Firm for work related to the Cook Celect IVC filter litigation, and from the Siegfried and Jensen Law Firm for work related to Vioxx litigation; chairs a Cardiac Scientific Advisory Board for UnitedHealth; was a participant/participant representative of the IBM Watson Health Life Sciences Board; is a member of the Advisory Board for Element Science, the Advisory Board for Facebook, and the Physician Advisory Board for Aetna; and is the co-founder of HugoHealth, a personal health information platform, and co-founder of Refactor Health, an enterprise healthcare AI-augmented data management company.R.C. reports employment with Verily Life Sciences and Google Health, board membership at Cytokinetics, and status of former commissioner of the US FDA.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Inan, O.T., Tenaerts, P., Prindiville, S.A. et al. Digitizing clinical trials. npj Digit. Med. 3, 101 (2020).

Download citation

Further reading


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing