Interventional healthcare will evolve from an artisanal craft based on the individual experiences, preferences and traditions of physicians into a discipline that relies on objective decision-making on the basis of large-scale data from heterogeneous sources.
Future advances in surgical care require, to an increasingly greater extent, a close partnership between caregivers, patients, technology and information systems. This will be enabled by data science, an emerging interdisciplinary field that aims to extract knowledge from data. Yet despite tremendous progress made over the past decade, there has been a delay in the introduction of large-scale data science into interventional medicine (for example, surgery, interventional radiology, gastroenterology and radiotherapy). This delay can partly be attributed to the fact that today only a fraction of patient-related data and information is digitized and stored in a structured and standardized manner — for instance, in registries1,2. Moreover, diversity in caregiver training, experience, and routine institutional practices has elicited variation in the methods and means of perioperative care. Without data providing insight into actual practice, disparity in outcomes is an inevitable consequence.
In this Comment, we introduce surgical data science on the basis of discussions held in a two-day international workshop (www.surgical-data-science.org/workshop2016) that brought together leading researchers working in the related field of computer-assisted and robot-assisted interventions. Our consensual opinion is that increasing access to large amounts of complex data throughout the patient-care process, complemented by advances in data-science and machine-learning techniques, has set the stage for a new generation of analytics that will support decision-making and quality improvement in interventional medicine. Here, we provide a consensual definition for surgical data science, identify associated challenges and opportunities, and provide a roadmap for advancing the field (Box 1).
Surgical data science will pave the way from artisanal to data-driven interventional healthcare, with concomitant improvements in quality and efficiency of care.
A key element will be to institutionalize a culture of continuous measurement, assessment and improvement using evidence from data as a core component.
An actionable path would involve the support and nurturing of efforts in surgical data science through best practice, comprehensive data registries, and active engagement and oversight.
Surgical data science should be established as a central element in the educational programmes of hospitals that teach and train future interventionalists, with the goal of instilling data-science thinking and data-driven approaches into future generations of clinical faculty and practicing surgeons.
What is surgical data science?
Surgical data science aims to improve the quality of interventional healthcare and its value through the capture, organization, analysis and modelling of data. It encompasses all clinical disciplines in which patient care requires intervention to manipulate anatomical structures with a diagnostic, prognostic or therapeutic goal, such as surgery, interventional radiology, radiotherapy and interventional gastroenterology. Data may pertain to any part of the patient-care process (from initial presentation to long-term outcomes), may concern the patient, caregivers and/or technology used to deliver care, and are analysed in the context of generic domain-specific knowledge derived from existing evidence, clinical guidelines, current practice patterns, caregiver experience and patient preferences. Data may be obtained through medical records, imaging, medical devices or sensors that may either be positioned on patients or caregivers, or integrated into the instruments and technology used to deliver care. Improvement may result from understanding processes and strategies, predicting events and clinical outcome, assisting physicians in decision-making and planning execution, optimizing the ergonomics of systems, controlling devices before, during and after treatment, and from advances in prevention, training, simulation and assessment. Surgical data science builds on principles and methods from other data-intensive disciplines, such as computer science, engineering, information theory, statistics, mathematics and epidemiology, and complements other information-enabled technologies such as surgical robotics, smart operating rooms and electronic patient records.
The evolution of surgical practice
It has been said that “surgery is a profession defined by its authority to cure by means of bodily invasion”3. Yet surgical practice has significantly evolved throughout the centuries (Fig. 1). It underwent revolutionary changes with the introduction of anaesthesia and antiseptics in the nineteenth century. At this point, surgeons typically relied on minimal instrumentation as well as on their own knowledge and clinical experience, which to some extent were augmented by learning from peers and from the few medical books available. In the twentieth century, advances in surgery centred on professionalization, systematic measurement of outcomes of care, and minimally invasive access to surgical sites. Surgery was further transformed with the introduction of multimodal medical imaging4, the development of surgical microscopes and endoscopes, and ultimately the emergence of computer- and robot-assisted interventions5. Despite rapid advances and the increased expectation for outcomes and safety from patients, hospitals and insurers, it has been estimated that 9 million6 of an estimated 300 million surgical procedures carried out per year worldwide7 will encounter major complications. Importantly, the seamless integration of computational support tools in the surgical environment — that is, devices or methods that enhance situation awareness, ergonomics and minimization of cognitive workload — remains a challenge. Although the digital revolution has brought access to an almost unlimited number of electronic patient records, such an avalanche of data is typically unstructured, undergoes limited quality control, and has almost no direct integration with computer-assisted interventional systems.
Future advances in surgery will continue to be motivated by safety, effectiveness and efficiency of care. We believe that the next fundamental changes will be from clinician intuition to explicit, data-derived computational models, from subjective to objective decision-making, and from qualitative to quantitative assessment. This will enable personalized treatment and will ensure that the evolution of surgery is centred on patients and caregivers. Within this vision, surgical data science will evolve to observe all that is occurring within and around the treatment process. It will provide the surgeon with quantitative support to aid decision-making and surgical actions and, importantly, will link decisions to patient outcomes. For the patient, this will mean having access to the best surgical care, with less variability arising from unique patient characteristics and from the choice of surgeon or care facility. Ultimately, surgical data science will offer the opportunity to create ‘superhuman’ surgery, which will move beyond the data associations that individuals are able to perceive, detect and maintain, to the realm of vast data types and sizes that can only be exploited through modern computing solutions.
Although surgical data science is related to biomedical data science, its unique characteristic is the focus on procedural data. It pertains to (i) the patient, (ii) the effectors involved in the manipulation of the patient, including physicians, the anaesthesia team, nurses, and devices including robots, (iii) sensors for perceiving patient- and procedure-related data such as images, vital signs, medical-device data and motion data, and (iv) domain knowledge, including factual knowledge, such as (hospital-specific) standards related to the clinical workflow, previous findings from studies or clinical guidelines, and practical knowledge from previous procedures.
Key clinical applications
Data science may impact interventional care throughout the entire patient-care pathway. In what follows, we discuss some of the opportunities for impact in the context of surgery.
The quality of surgical care is affected to some extent by decisions made by caregivers and patients throughout the care pathway. Traditionally, surgeons relied on their experience to play a major role in consequential decisions, such as whether to operate and the type of surgery to perform8.
A principal issue in this context, however, is that human capability limits the processing of large amounts of disparate data9,10. Computational reduction of data to contextually useful information could be a highly useful tool to support the caregiver. In fact, increasingly more studies in the related field of computer-aided diagnosis are demonstrating the strong potential of machine-learning techniques to complement human cognition11,12. As a first step towards such data-driven approaches, the surgical decision-making model has gradually evolved to be informed by predictive analytics based on systematic data capture and curation through patient registries. However, currently available registry-based analytics to support surgical decision-making rely on cross-sectional measures of a subset of patient characteristics before surgery13. Furthermore, registries rarely capture the full record of the patient-care pathway, and the amount of data that they are missing varies14. A data-science approach to decision support relies not only on continuously updating predictive analytics throughout the patient-care process but also on more comprehensive and unconventional sources of data15,16,17. Moreover, surgical decisions may be optimized by modelling individual patients within the context of population-level data and other multimodal data sources18,19. Overall, surgical data science reinforces the importance of integration of such decision support into patient-care workflows via user-friendly data products.
Surgical data science enables context-aware assistance that can be applied throughout the patient-care pathway. In the operating room, the application of context-aware assistance can include the monitoring of procedures to predict the remaining duration, to facilitate scheduling or to anticipate needs for resources20. Similarly, autonomous assistance can provide surgeons with timely information through surgical-phase recognition21,22, and with decision support through patient-specific simulations23 and collaborative robots24. Context-aware assistance improves the safety, quality and efficiency of care, and can augment the performance of providers when integrated into surgical-care pathways.
Surgical education and certification ensure that competent surgeons provide care, and are thus a critical element in assuring care quality. Poor surgical technical skill is associated with an increased risk of re-admission, re-operation and death25,26. Technical skills and errors are also associated with non-technical skills such as decision making27. Surgical data science can be transformative for surgical training through objective computer-aided skill evaluation28, robot-assisted active learning of technical skills29, patient- and context-specific simulation training and assessment, and surgical coaching30,31. Additional data analytics such as surgical-process modelling, detection of constituent activities, and errors and skill deficits can facilitate targeted feedback32,33. Surgical data science thus represents the new frontier for surgical training in a complex patient-care environment with limited resources.
Advancing our vision of surgical data science will require overcoming several challenges related to the acquisition and analysis of highly heterogeneous multimodal data.
Surgical data science relies on access, on a large scale, to high-quality data that documents both the patient-care process and patient outcomes. Although large annotated datasets for advancing research and practice exist (for example, ImageNet34), these are lacking in documentation of surgery despite the fact that quality improvement can be inherently achieved through outcome measurement (for example, by using patient registries). This paucity of datasets may be attributed to a multitude of regulatory, technical and sociological factors. On the one hand, concerns related to privacy and confidentiality of both patients and caregivers pose important legal and ethical issues that must be addressed for data science to be possible. On the other hand, although large amounts of data are routinely available during interventional care, they are not captured and annotated using standardized protocols35. And although international healthcare-terminology standards for biomedical data science (in particular, Foundational Model of Anatomy36, Gene Ontology37 and SNOMED-CT) are well established, ontologies to describe activities and other aspects of interventional-care processes are lacking. Furthermore, data annotation is resource-intensive. Although some aspects of data annotation for interventional-care processes can be crowd-sourced to untrained individuals38, other aspects may require specific expertise. Ultimately, data should be collected as a matter of best practice in a consistent and longitudinal manner by using tools that smoothly integrate into the clinical workflow. Partners and clear short-term ‘win scenarios’ that will build interest and trust in the area need to be identified so that hospitals, insurers and practitioners understand the value of creating these resources39.
Machine learning has made rapid strides, primarily by exploiting the confluence of large datasets, the availability of crowd-sourced knowledge at scale, and massive computation40. Surgical data science could, in principle, take advantage of these advances by marrying organized data collection in the operating room with electronic medical records, augmented with both traditional and patient-reported outcomes. However, analysis of data from interventions faces unique challenges. First, a substantial aspect of surgical data science involves modelling the orchestrated actions of medical teams and the response of the patients to such actions. During surgery, for example, the head surgeon, anaesthetists, assistant surgeons, circulators and nurses play crucial roles at the different procedural steps, and their smooth dynamic collaboration and coordination are essential to the success of the overall process. This is in strong contrast to the field of radiological data science11,12,41,42, where data interpretation can typically be performed ‘offline’ — that is, within timescales that span minutes to hours or even days. Second, anatomical manipulation during surgery is frequently irreversible, with errors resulting in serious complications or even death. Therefore, the robustness and reliability of the methods are crucial43. It also raises the requirement for transparency of reasoning (explainable artificial intelligence). Third, key decisions, and alterations in therapeutic trajectory, often need to be made within minutes or seconds44, emphasizing the need for rapid computation times. Furthermore, although the diagnostic process follows a rather regular flow of data acquisition, and established companies such as Google Inc. and IBM have started developing biomedical data-science techniques to support it, the surgical process varies significantly from case to case and is highly specific to procedure, patient and surgeon45. The heterogeneity in the data resulting from different hardware, imaging protocols (such as OR.NET and MD PnP), context, training, care guidelines and physicians is a great challenge to overcome, not only for the development of data-analysis methods but also for the validation of new methodology and systems. In addition, procedural data must be holistically analysed with other heterogeneous data — including genetics data, biomarker data, patient demographics, and imaging and pre- and intraoperative data — to enable the move from eminence-based to knowledge-based and data-driven medicine. In this context, shared tools for optimizing discovery and for training researchers46 exist.
Still, advances in data acquisition and analysis alone will not be sufficient to fully realize the potential of surgical data science to improve patient care. In particular, changing the healthcare culture to one that welcomes and exploits the use of data analytics requires the buy-in of multiple stakeholders.
Caregivers and researchers
Currently, a major hurdle preventing surgeons from dedicating time to developing data-science approaches is the balance between research and clinical practice. An obstacle for engineers is that promising solutions for problems in surgical data science may not be perceived as necessarily innovative from a computer-science and engineering perspective. A key action to address these issues would be the establishment of surgical data science as a career path in hospitals, academic medical centres and governmental organizations focused on healthcare policy and oversight. Scientific incentives — such as access to a joint pool of (potentially multi-centre) interventional data, co-authorship on publications, or the opportunity to address long-standing research challenges that cannot be taken on by individual teams or institutions — could then encourage the acquisition and sharing of high-quality curated data. Although access to such shared databases may be restricted to those that contribute data, other important building blocks for surgical data science will be funded open-science initiatives (such as the Alzheimer’s Disease Neuroimaging Initiative47; http://adni.loni.usc.edu). In this context, better incentives from journals, funding agencies and academic institutions — such as the publishing and acknowledgement of data-description papers — will be crucial in compensating for the high costs incurred when generating open data. Furthermore, societal support is required to reach the long-term goal of comprehensive data registries. Of note, many surgical societies (such as the American College of Surgeons, the Americas Hernia Society and the Society of Thoracic Surgeons) encourage or require their members to use patient registries. To ensure optimal reimbursement from the Centers for Medicare and Medicaid Services, participating physicians must now document and track various key patient-care quality measures, using a variety of patient registries. Many United States insurance companies (most of which model their reimbursement practices on those of the Centers for Medicare and Medicaid Services) have also begun requesting evidence of surgeon participation in such quality databases, with a view to providing payment for care provided.
Although the healthcare industry has lagged behind other sectors (such as banking) in the use of big data, most big healthcare companies have set up active departments for machine learning and begun developing software and hardware solutions that support the capture of anonymized patient information. Analysis of such data gathered across many sites worldwide could lead to powerful new solutions within operating rooms of the future (Fig. 2), making industry one of the main stakeholders in the field of surgical data science. However, even for industry stakeholders, the path towards scalable analytics is currently impeded by the diversity of data types, data quality and system interfaces present in interventional devices. Data at different points along a treatment pathway are fundamentally dependent on the manufacturer, distributor, and equipment or facility manager. Standards are a partial solution, and market opportunities will be needed to motivate companies to implement and use open-data interfaces and create long-term value and impact. One value point could be to identify opportunities for data federation, where multiple systems rely on each other. However, this is challenging to negotiate, and may face additional hurdles regarding regulation in cases where systems, individually certified, may need to also have joint certification. Identifying new markets or exploiting surgical data science to perform cost optimization, and hence motivate buyers from industry indirectly rather than directly, will be the more likely drivers.
Patients are increasingly demanding detailed information about their treatment choices, mostly as a result of broad access to information and of difficulties in identifying what is relevant. This is pushing medical professionals to provide more detailed explanations to patients. However, such information needs surgical data science to remain statistically correct and up to date. Patients will want reliable information from large datasets that use validated methodologies. They will expect a new level of sophistication around predicting their own unique outcomes on the basis of many factors, including their personal physiologic and co-morbidity profile. Surgical data science must play such crucial roles in detailed data management, analysis, and presentation to the end user — whether patient, caregiver or industry partner — in a cogent, user-friendly way that will ultimately improve the quality and safety of treatments.
Dissemination and impact
Surgical data science will enable fundamental understanding of surgical procedures, their variability, crucial parameters, hidden structures, dependencies, optimal pathways, the importance of each parameter, the keys to success and failure of methodologies, and the basic principles driving surgical education, training and practice.
Also, surgical data science could change the education and training of millions of physicians across the world. It should facilitate the methods through which the next generation of medical students learn from complex data, without restricting them to a particular book or teacher. We expect that distinct career pathways will evolve, for the training of surgical data scientists and for embedding them into clinical research teams.
Moreover, surgical data science has the potential to enable medical companies to fully optimize their solutions on the basis of large amounts of data. In fact, most large healthcare companies are moving away from building solutions based on predefined customer requirements, and are investing in approaches that record and analyse the interactions of customers with company systems. This replaces the common practice of defining requirements by interviewing key opinion leaders or special focus groups. Such machine learning-based definitions result in user interfaces and products that are adapted to the entire market rather than to particular groups.
The end point for discoveries through surgical data science is their effective translation into patient-care workflows, which can involve commercialization of data products and services. Effective translation will be possible when the various stakeholders collaborate from the inception of data products through to their translation. Although such collaborations are difficult to forge, they are possible and are becoming increasingly common. For example, Intuitive Surgical Inc. (Sunnyvale, CA, USA) provides access to kinematic information from their da Vinci surgical system to collaborating academic or clinical institutions through a dedicated da Vinci Research application programming interface. Various institutions have used this interface to perform research on motion analysis of surgical instruments and computer-vision applications. The productive research outputs have stimulated the emergence of the da Vinci Research Kit — an open platform for surgical robotics led by Intuitive Surgical Inc. and Johns Hopkins University — and have been enabled by da Vinci systems decommissioned from clinical use. Developing with a platform providing access to features for telemanipulation, visualization and instrument control has been a key stimulus for the surgical-robotics community. Intuitive Surgical, in turn, gets insight into research coming out of academic institutions.
Surgical data science can be disseminated through its impact on a wide range of products, from medical training and education to surgical imaging, instrumentation, user interfaces and advanced patient-information systems. We hope that holistic data analysis will eventually usher the next generation of interventional healthcare (Box 1).
The authors thank the Transregional Collaborative Research Center (SFB/TRR) 125: Cognition-Guided Surgery, funded by the German Research Foundation (DFG), for sponsoring the workshop that served as basis for the manuscript (www.surgical-data-science.org/workshop2016). We also thank N. L. Rodas, M. A. Cypko and all other workshop participants for their valuable input during the workshop, and C. Feldmann for preparing the figures. We acknowledge the support of the European Research Council (ERC-2015-StG-37960), the US National Institutes of Health (NIH-R01EB01152407S1, NIH/NIBIB P41 EB015902, NIH/NCI U24CA180918, NIH/NIBIB P41 EB015898, NIH/NIBIB R01EB014955, NIH R01-DE025265), the US Department of Defense (DOD-W81XWH-13-1-0080), the Royal Society (UF140290) and the Link Foundation Fellowship in Advanced Simulation and Training.