Filter By:

Article Type
  • Generalization – the ability of AI systems to apply and/or extrapolate their knowledge to new data which might differ from the original training data – is a major challenge for the effective and responsible implementation of human-centric AI applications. Current debate in bioethics proposes selective prediction as a solution. Here we explore data-based reasons for generalization challenges and look at how selective predictions might be implemented technically, focusing on clinical AI applications in real-world healthcare settings.

    • Lea Goetz
    • Nabeel Seedat
    • Mihaela van der Schaar
    CommentOpen Access
  • AI holds the potential to transform healthcare, promising improvements in patient care. Yet, realizing this potential is hampered by over-reliance on limited datasets and a lack of transparency in validation processes. To overcome these obstacles, we advocate the creation of a detailed registry for AI algorithms. This registry would document the development, training, and validation of AI models, ensuring scientific integrity and transparency. Additionally, it would serve as a platform for peer review and ethical oversight. By bridging the gap between scientific validation and regulatory approval, such as by the FDA, we aim to enhance the integrity and trustworthiness of AI applications in healthcare.

    • Michel E. van Genderen
    • Davy van de Sande
    • Jeroen van den Hoven
    CommentOpen Access
  • Digital health technologies (DHTs) can transform neurological assessments, improving quality and continuity of care. In the United States, the Food & Drug Administration (FDA) oversees the safety and efficacy of these technologies, employing a detailed regulatory process that classifies devices based on risk and requires rigorous review and post-market surveillance. Following FDA approval, DHTs enter the Current Procedural Terminology, Relative Value Scale Update Committee, and Centers for Medicare & Medicaid Services coding and valuation processes leading to coverage and payment decisions. DHT adoption is challenged by rapid technologic advancements, an inconsistent evidence base, marketing discrepancies, ambiguous coding guidance, and variable health insurance coverage. Regulators, policymakers, and payers will need to develop better methods to evaluate these promising technologies and guide their deployment. This includes striking a balance between patient safety and clinical effectiveness versus promotion of innovation, especially as DHTs increasingly incorporate artificial intelligence. Data validity, cybersecurity, risk management, societal, and ethical responsibilities should be addressed. Regulatory advances can support adoption of these promising tools by ensuring DHTs are safe, effective, accessible, and equitable.

    • Neil A. Busis
    • Dilshad Marolia
    • Scott N. Grossman
    CommentOpen Access
  • Recent developments in large language models (LLMs) have unlocked opportunities for healthcare, from information synthesis to clinical decision support. These LLMs are not just capable of modeling language, but can also act as intelligent “agents” that interact with stakeholders in open-ended conversations and even influence clinical decision-making. Rather than relying on benchmarks that measure a model’s ability to process clinical data or answer standardized test questions, LLM agents can be modeled in high-fidelity simulations of clinical settings and should be assessed for their impact on clinical workflows. These evaluation frameworks, which we refer to as “Artificial Intelligence Structured Clinical Examinations” (“AI-SCE”), can draw from comparable technologies where machines operate with varying degrees of self-governance, such as self-driving cars, in dynamic environments with multiple stakeholders. Developing these robust, real-world clinical evaluations will be crucial towards deploying LLM agents in medical settings.

    • Nikita Mehandru
    • Brenda Y. Miao
    • Ahmed Alaa
    CommentOpen Access
  • In the medical literature, promising results regarding accuracy of medical AI are presented as claims for its potential to increase efficiency. This elision of concepts is misleading and incorrect. First, the promise that AI will reduce human workload rests on a too narrow assessment of what constitutes workload in the first place. Human operators need new skills and deal with new responsibilities, these systems need an elaborate infrastructure and support system that all contribute to an increased amount of human work and short-term efficiency wins may become sources of long-term inefficiency. Second, for the realization of increased efficiency, the human-side of technology implementation is determinate. Human knowledge, competencies and trust can foster or undermine efficiency. We conclude that is important to remain conscious and critical about how we talk about expected benefits of AI, especially when referring to systemic changes based on single studies.

    • Karin Rolanda Jongsma
    • Martin Sand
    • Megan Milota
    CommentOpen Access
  • Historically, the Centers for Medicare and Medicaid Services (CMS) has formed partnerships with select private sector entities, including large traditional hospital and health system networks, nursing homes, and payer groups. However, innovations from technology-enabled services companies and digital technology companies are uniquely poised to aid CMS in addressing key barriers toward advancing its mission of improving healthcare access and equity. There are four pivotal opportunity areas where partnerships with technology businesses and tools would enhance the work of CMS: (1) improving consumer awareness about CMS programs, (2) mitigating access gaps through virtual care programs, (3) streamlining the complexity of different payer plan models, and (4) using technology-enabled services to address social risk factors without imposing additional burdens on providers. We offer examples of digital and technology-enabled solutions that improve patient access to care and close equity gaps, as well as propose specific recommendations for CMS to advance and expand the reach and impact of these solutions. Namely, these recommendations include partnerships with private sector companies that can educate and support consumers about their benefits, the extension of telehealth reimbursement parity for virtual care solutions, allowing for cross-state licensure across plans and reimbursement for care coordination services that alleviate provider burden to screen and address patients’ social determinants of health needs. We argue that CMS has an imperative role in leveraging the innovations of technology-enabled services and digital health technologies to lower healthcare access barriers, mitigate provider burden, stimulate innovation, and close equity gaps at the patient, provider, and innovator levels.

    • Shobha Dasari
    • Raihana Mehreen
    • Andrey Ostrovsky
    CommentOpen Access
  • The integration of Quality Management System (QMS) principles into the life cycle of development, deployment, and utilization of machine learning (ML) and artificial intelligence (AI) technologies within healthcare settings holds the potential to close the AI translation gap by establishing a robust framework that accelerates the safe, ethical, and effective delivery of AI/ML in day-to-day patient care. Healthcare organizations (HCOs) can implement these principles effectively by embracing an enterprise QMS analogous to those in regulated industries. By establishing a QMS explicitly tailored to health AI technologies, HCOs can comply with evolving regulations and minimize redundancy and rework while aligning their internal governance practices with their steadfast commitment to scientific rigor and medical excellence.

    • Shauna M. Overgaard
    • Megan G. Graham
    • Nicoleta J. Economou-Zavlanos
    CommentOpen Access
  • As result of the pandemic-related increase in telehealth and the 21st Century Cures Act, technology is playing an increasing role in healthcare. This has led to organizational investments in the “digital front door” of healthcare. The promise that these technologies can revolutionize care by better connecting us to our patients, overcoming analog barriers to care, and addressing health disparities is grounded in “techno-optimism.” We advocate for organizational leaders to inform their digital health equity strategies with a balanced measure of “techno-skepticism”, grounded in implementation science, that can ensure alignment between health technology and health equity.

    • Jorge A. Rodriguez
    • Courtney R. Lyles
    CommentOpen Access
  • At a time of diminishing clinical skills and diagnostic endeavours for clinicians worldwide, a method of rapid automated microscopy coupled with pH measurement is introduced with early clinical success results in women with common vulvovaginal symptoms facilitating rapid diagnosis and enhanced therapeutic measures.

    • J. D. Sobel
    CommentOpen Access
  • Sleep recordings are visually classified in stages by experts in the field, based on consensus international criteria. This procedure is expensive and time-consuming. Automatic sleep scoring systems have, progressively over the years, demonstrated good levels of accuracy. Although the performance of these algorithms is believed to be high, however, there remains widespread skepticism in their daily use in clinical and scientific practice. In this comment to a recent publication of NPJ Digital Medicine, we express the reasons why we think the sleep expert should remain the central pivot in the pendulum between visual and automatic methodology, trying to find a new balance in the scientific debate.

    • Vincenzo Muto
    • Christian Berthomier
    CommentOpen Access
  • Digital health technologies (DHTs) enable remote data collection, support a patient-centric approach to drug development, and provide real-time data in real-world settings. With increasing use of DHTs in clinical care and development, we expect a growing body of evidence supporting use of DHTs to capture endpoint data in clinical trials. As the body of evidence grows, it will be critical to ensure that available prior work can be leveraged. We propose a framework to reuse analytical and clinical validation, as well as verification data, generated for existing DHTs. We apply real life case studies to illustrate our proposal aimed at leveraging prior work, while applying the V3 framework (verification, analytical validation, clinical validation) and avoiding duplication. Utilizing our framework will enable stakeholders to share best practices and consistent approaches to employing these tools in clinical studies, build on each other’s work, and ultimately accelerate evidence generation demonstrating the reproducibility and value add of these new tools.

    • Amy Bertha
    • Rinol Alaj
    • Sven Reimann
    CommentOpen Access
  • Artificial Intelligence-supported digital applications (AI applications) are expected to transform radiology. However, providers need the motivation and incentives to adopt these technologies. For some radiology AI applications, the benefits of the application itself may sufficiently serve as the incentive. For others, payers may have to consider reimbursing the AI application separate from the cost of the underlying imaging studies. In such circumstances, it is important for payers to develop a clear set of criteria to decide which AI applications should be paid for separately. In this article, we propose a framework to help serve as a guide for payers aiming to establish such criteria and for technology vendors developing radiology AI applications. As a rule of thumb, we propose that radiology AI applications with a clinical utility must be reimbursed separately provided they have supporting evidence that the improved diagnostic performance leads to improved outcomes from a societal standpoint, or if such improved outcomes can reasonably be anticipated based on the clinical utility offered.

    • Franziska Lobig
    • Dhinagar Subramanian
    • Oisin Butler
    CommentOpen Access
  • The generalizability of predictive algorithms is of key relevance to application in clinical practice. We provide an overview of three types of generalizability, based on existing literature: temporal, geographical, and domain generalizability. These generalizability types are linked to their associated goals, methodology, and stakeholders.

    • Anne A. H. de Hond
    • Vaibhavi B. Shah
    • Tina Hernandez-Boussard
    CommentOpen Access
  • Digital tools are transforming mental health care. The promise of this transformation to improve outcomes has not yet been realized fully. While some have become skeptical, this article argues that we are just at the end of Act 1, with several opportunities and challenges ahead.

    • Thomas Insel
    CommentOpen Access
  • Postpartum mental health conditions are a public health concern, affecting a large number of reproductive-age women and their families. Postpartum depression alone affects at least 14% of new mothers and their families. However, very little has been written about how advances in digital mental health can benefit women in the postpartum period, or how those advances may poorly serve this vulnerable population. This manuscript takes a high-level view of the advances in different areas of digital mental health, including telehealth, apps, and digital phenotyping. In this comment, we explore ways in which digital interventions for postpartum mental health may help with connection to treatment, accessibility, agency, and ease of access. We also note particular concerns for how digital postpartum mental health may encounter issues of low-quality resources, ethical considerations, and equity considerations. We provide suggestions for how to leverage the promise and avoid the pitfalls of digital mental health for postpartum women.

    • Natalie Feldman
    • Sarah Perret
    CommentOpen Access
  • This paper reviews the current state of patient safety and the application of artificial intelligence (AI) techniques to patient safety. This paper defines patient safety broadly, not just inpatient care but across the continuum of care, including diagnostic errors, misdiagnosis, adverse events, injuries, and measurement issues. It outlines the major current uses of AI in patient safety and the relative adoption of these techniques in hospitals and health systems. It also outlines some of the limitations of these AI systems and the challenges with evaluation of these systems. Finally, it outlines the importance of developing a proactive agenda for AI in healthcare that includes marked increased funding of research and evaluation in this area.

    • David C. Classen
    • Christopher Longhurst
    • Eric J. Thomas
    CommentOpen Access
  • Digital technology is increasingly important in people’s lives, particularly for new parents as it allows them to access information, stay connected to peers and offers them seductive solutions for improving infant sleep and parental well-being. Digital technology has been developed to support parents in the following four ways: (1) providing digital information on infant sleep, (2) offering targeted support for night-time care, (3) managing infant sleep and (4) monitoring infant sleep and safety. Evidence on the effectiveness of these strategies is varied and there are concerns regarding the reliability of information, use of personal data, commercial exploitation of parents, and the effects of replacing caregiver presence with digital technology.

    • Helen L. Ball
    • Alice-Amber Keegan
    CommentOpen Access