Introduction

Widespread adoption of screening colonoscopy has coincided with a steady decline in the incidence of and mortality associated with colorectal cancer in the United States1. However, post-colonoscopy cancers do occur and are primarily attributed to missed cancers or pre-cancerous polyps on preceding colonoscopy2. Thus, maximizing the quality of endoscopic inspection is essential in reducing missed lesions and post-colonoscopy cancers. The adenoma detection rate (ADR), or the proportion of screening colonoscopies in which at least one adenomatous polyp is found, has become the established surrogate of colonoscopy quality3. Multiple studies have demonstrated that performance of ADR correlates with decreased incidence of post-colonoscopy cancers3,4 and that providing endoscopists with feedback related to their ADR can improve their ADR performance5,6. This has led to the elevation of ADR as a priority quality indicator (QI) and the recommendation that all endoscopists measure their ADR to assure delivery of high-quality and high-value care7.

The data needed to determine ADR requires coupling of endoscopic and pathology findings which are separated by both time and location within the electronic health record (EHR). Thus, measuring ADR requires time-intensive chart review either by individual endoscopists or designated staff. Other methods to measure ADR, including the use of natural language processing (NLP) to automatically extract data from endoscopy and pathology text, have been described 8,9,10,11,12,13,14; though these require complex data management systems that are likely not available to many endoscopists and are not easily transferable from one practice to another15. Alternatives to labor intensive manual review and complex NLP based strategies are needed. We therefore aimed to develop and validate a method using embedded EHR functions in Epic, the most widely adopted EHR in the US marketplace16, to facilitate both ADR reporting and feedback.

Materials and methods

Many EHRs contain functions that allow for automated capture of structed data from EHR text notes17. We worked with an Epic analyst to develop a tool using this EHR functionality to capture total ADR (tADR), or the proportion of all colonoscopies regardless of indication in which at least one adenoma is found. Total ADR has previously been shown to correlate with true ADR18. The primary Epic functions we utilized were the SmartList and Reporting Workbench. A SmartList is an item of text that can be embedded in a note to produce data that can be pulled into a report. In addition to the SmartList, the build elements necessary for our report included a procedure grouper (an Epic umbrella identifier to capture all the different ways colonoscopies are labeled in the EHR) to identify colonoscopy procedures to the report, a custom Epic property known as results interpreter to interrogate the endoscopist who performed the procedure, and a summary view within the report to display tADR to the endoscopist (Table 1). The SmartList was built to record a simple yes or no adenoma response in a note and store it to a SmartData element (Table 1, Fig. 1B). The number of yes responses are calculated as a percentage of all colonoscopies and reported as a simple pie chart visible to the endoscopist in the EHR (Fig. 1C). The total count of colonoscopies is determined by looking to the colonoscopy grouper and pulling in all completed procedures for a rolling 90-day lookback, then using the custom results interpreter property to filter the list to the endoscopist(s) of interest thus providing the endoscopist with a real-time tally of their tADR over the last 90 days. Ninety days was chosen to assure that at least 100 colonoscopies were included in each report tally as previous studies analyzing effects of feedback on ADR improvement have utilized at least 100 colonoscopies/provider to assess change19,20 The report could be personalized to each endoscopist within our system by changing the name assigned in the results interpreter property as well as the lookback time interval based on endoscopist volume. The components of the report needed to determine tADR are displayed in Table 1.

Table 1 Build Components and Report Components required for the Epic based tADR tool.
Full size table
Figure 1
figure 1

(A) Workflow timeline for utilizing the SmartList tool. When the report is run, all colonoscopies from the previous 90 days are included and linked to instances of SmartList use in the patient’s chart to calculate the tADR in the report. (B) The SmartList as displayed in a simple note. (C) The pie-chart display of the Reporting Workbench results representing the tADR. This figure was constructed from a screenshot from Epic Clinical Software, Version May 2021 (https://www.epic.com/software).

Full size image

To minimize workflow the clinical staff were instructed to embed the SmartList in a note at time of pathology review, and one provider was selected to prospectively pilot the tool over a 6-month testing timeframe (allowing for 2 90-day Plan Do Study Act (PDSA) cycles). Provider workflow and accompanying EHR functions are displayed in Fig. 1A. Two PDSA cycles were necessary, with iterative expansion of the EHR based colonoscopy grouper until the report captured all colonoscopies performed by our provider over the validation timeframe a greater than 95% sensitivity. A final 90-day audit of colonoscopies performed by the pilot endoscopist was then conducted against a gold standard manual chart review to assess the sensitivity, specificity and accuracy of the report’s ability to capture tADR. Informed consent was waived for this study by the Colorado Multiple Institutions Review Board (COMIRB) because all patient data was de-identified and there was no direct patient contact, nor change in the standard of medical care. This study was reviewed and approved by COMIRB (Protocol Number 19-1378) and was carried out in accordance with all relevant guidelines and regulations.

Results

One hundred and forty-five colonoscopies were included in the final 90-day validation cohort. A manual chart review found that 78 colonoscopies had at least one adenoma (ADR 53.8%). Of the 145 colonoscopies, our EHR method correctly identified 143 (98.6%). The report identified 76/78 of colonoscopies in which at least one adenoma was found and all (67/67) colonoscopies where no adenoma was found (97.4% sensitivity, 100% specificity and 98.6% accuracy for detection of adenomas). The ADR generated by the EHR report was not clinically or statistically different from the ADR determined by gold standard manual chart review (53.1% vs. 53.8%, p = 0.912). The reason the two colonoscopies with adenomas were not captured by our report is because the endoscopist mistakenly did not utilize the SmartList at time of pathology review.

Discussion

The ADR is a high-value quality indicator because it is readily measured and is an established surrogate for the rare outcome post-colonoscopy cancer risk. Thus, it is imperative that all endoscopists have the means to know and track their ADR, and tools to facilitate more widespread measurement of ADR have the potential to support the delivery of high-value care. Here, we describe a novel method, leveraging pre-existing infrastructure within a widely adopted EHR, to do just that. In a pilot testing environment, our tool demonstrated a high sensitivity and accuracy compared to a manual chart review, similar to what studies evaluating NLP methods report8,9,10,11,12,13,14. To our knowledge, this is the first description of a method using EHR functionality to accurately capture endoscopic QIs.

There remain weaknesses to the tool we propose. It still requires effort from clinical staff to open a note and utilize the SmartList at the time of pathology result documentation. This requires several extra clicks beyond the normal workflow and in our pilot study resulted in 2/78 adenoma positive results being incorrectly classified. Additionally, our tool in its present form does not account for colonoscopy indication, completeness or prep quality. However, as previously noted tADR is an accurate surrogate for ADR, and others have even proposed that it may be a preferred colonoscopy QI as it simplifies measurement and may prevent gaming the ADR metric by changing colonoscopy indication18,21. Furthermore, additional macro selections (for instance for indication, bowel preparation quality, polyp histology, such as serrated lesions) can certainly be added to future iterations of our tool. These features can be customized based on a specific practices’ priorities in quality metric tracking and reporting (for instance if this information is not tracked elsewhere). The adaptability and customizability of our tool is a great strength, particularly if professional societies add additional lesion detection rates/benchmarks (such as serrated lesion detection rates) to quality metrics that should be measured.

We believe the tool we propose has several benefits. Because our tool relies largely on pre-existing abilities imbedded within an EHR, it does not require access to specialized data management systems often needed to adopt NLP based solutions. In addition, while NLP methods are often successful at individual institutions, adapting those tools across more diverse clinical settings has proven challenging15. Our tool can be scaled for use by anyone using the Epic EHR. Our tool provides real-time feedback within the EHR related to QI performance, allowing endoscopists to confront their own performance in the same interface in which they regularly manage patient care. While our tool does require some minimal effort from clinical staff, this is largely within normal clinical workflow and remains far less than what is required for manual extraction. Finally, though our tool was built using the Epic EHR, multiple EHRs have similar discrete data macro functionalities which could allow a similar tool to be developed in different systems both in the United States and Europe17,22.

Further work is needed to validate this tool among a greater proportion of endoscopists and ideally among multiple centers using the Epic EHR. Reassuringly, Smartlists are already widely used among everyday documentation in Epic. Additionally, prior research has demonstrated excellent adoption of Smartlists in post-colonoscopy EHR documentation23. Adjusting the structure of the Reporting Workbench algorithm by utilizing other macro data tools like additional SmartLists or flowsheets within the EHR may also allow for capture of additional data such as colonoscopy indication, prep quality and even allow for use of a similar tool to capture QIs in other endoscopic arenas.

This pilot study demonstrates the potential to leverage existing EHR functionality to achieve accurate measurement and feedback of tADR, a reliable surrogate for ADR. This tool may present an easily adoptable alternative to complex NLP based systems or time-intensive chart review to facilitate QI measurement and assure delivery of high-value care.