Introduction

There is a fraught and tortuous pathway between exciting new digital medicine concepts and their making a real and safe impact on standard care. Perhaps the degree of excitement and otherworldliness of digital technology is directly proportional to the likelihood of taking false turns and entering cul-de-sacs in development. One approach to avoiding pitfalls while bringing exciting and challenging technologies to healthcare is through focused conferences which bring together the stakeholders critical to progress. On one side are the enthusiasts, innovators, early adopters, and entrepreneurs, and on the other the regulators, implementation scientists, and healthcare providers, including those from outside the bubble of the innovation in question.

An example of this event type is the SHIFT MEDICAL (https://shiftmedical.eu) conference on MXR, which has run in Heidelberg, Germany since 2020. MXR merges the physical and digital worlds and encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR), and is certainly another universe when compared to traditional tools for medical practice. The meeting’s focus is on in-person colleague-to-colleague networking and open knowledge exchange between researchers, healthcare providers, businesses, governments, and the general public. The conference was one of the first to go “fully virtual” in 2020 but dropped this in 2022, instead adopting a fully in-person and selected attendance format. The format seems almost a reaction to the hype one might associate with VR in medicine, with instead a focus on ethical, regulatory, technological, data safety, and evidence-based principles, and is exactly what the doctor ordered for MXR.

The application of MXR technology in medicine?

The interrelated concepts, QR, MR, and VR that make up MXR are explained in Fig. 1.

Fig. 1: The range of MXR modalities.
figure 1

Definition and description of the interrelated concepts of MXR, AR, MR, and VR.

Full size image

MXR approaches are deployed using a mix of head-mounted glasses, headsets, and screens. There has been a hype of the general and medical potential of these technologies in the last years, but what is their true role in medicine and how can development be better driven to meet this?

The role of MXR in medical training

Almost all current application areas of MXR in the medical field were presented at the conference. The most advanced in implementation was the field of medical training, with products gradually being rolled out and evaluated, showing promising results in nursing education1 and in the training of healthcare professionals and students2,3. Approaches are also being researched for emergency medicine training in complex scenarios (https://cordis.europa.eu/project/id/101021775).

The application of MXR in care

Beyond education, applications of VR were presented for aiding patient care, from cognitive rehabilitation post-stroke4 to physical rehabilitation5 and psychological therapies, e.g., to treat specific phobias, depression, and addiction6. Further in-development approaches on show were for VR as a tool for surgery planning7,8 and in radiology image viewing9. Some of these are already approved as medical devices in the EU and the US, a subset of which are on the market for everyday patient care. In contrast, AR can complement existing non-digitized procedures thanks to the simultaneous presentation of real-world and digital objects. There were presentations on AR approaches for precise organ transplantation through anatomical overlays and size matching in the operating room10 or as information-presenting interfaces on care units. Other AR applications discussed were in interventional cardiology and radiology, for assisting in planning procedures involving catheters and guiding the placement of such, with an enhancing effect on precision and efficiency10,11.

The path to wider utilization

Where MXR is applied in medical devices, it is justified that it must navigate a rigorous regulatory pathway from laboratory to clinic. The main technological challenge presented was related to display technology: current flat panel displays have a physically limited depth presentation12. New, multifocal display technologies were described which could lead to closer-to-reality image perception, resulting in improved image quality and better depth perception13. Yet, the issues of headset weight, battery life, and the constraints of wired headsets remain. From a user’s perspective, poor image quality, non-ergonomic designs (e.g., weight), and unexpected or unfamiliar control conventions could limit the acceptance14. The FDA brought to the meeting a focus on the critical theme of usability and how this links to display and image quality15. The novelty of MXR technologies presents a unique set of challenges here, and new standardized evaluation methods and frameworks are needed to determine safety and effectiveness. The FDA recognizes the need for quantitative evaluation methodologies and is actively working to develop assessment methodologies they term ‘regulatory science tools.’ The collection of available regulatory science tools is freely available on the FDA’s website (https://www.fda.gov/medical-devices/science-and-research-medical-devices/catalog-regulatory-science-tools-help-assess-new-medical-devices) and can be used by both device developers and evaluators.

A model for MXR and a model format for effective conferences

A clear message from the conference—the journey of MXR products from prototype to clinic is fraught with technical and regulatory challenges and the implementation in clinical settings is still in its early stages. These challenges are such that there should be ‘healthy skepticism’ on whether a comprehensive and widespread introduction of MXR in the healthcare sector will happen. This needs consideration of cost-effectiveness, safety, and the resolution of the known technical challenges. Standardized tests and frameworks and further development of adapted and balanced regulatory pathways will be instrumental in navigating the journey toward the clinical adoption of MXR. If you wish to get involved in the MXR movement’, can you? Although this conference is by application or invitation only this is not a barrier to those wishing to engage in MXR. This conference has a straightforward application process, and the selected group of attendees fits the concept of moving the theme forward through interdisciplinary work ‘at the coal face’ of implementation, rather than providing an audience for ‘leading experts’ to present long and largely pre-published work at a large audience.

Current and inspiring meeting designers and driven technologist-scientists could learn a lot from this format. There is much work to do at conferences like SHIFT MEDICAL, and not only for MXR but for many challenges in digital medicine and beyond.