Abstract
Three-dimensional (3D) reconstruction of capsule endoscopic images has been attempted for a long time to obtain more information on small bowel structures. Due to the limited hardware resources of capsule size and battery capacity, software approaches have been studied but have mainly exhibited inherent limitations. Recently, stereo camera-based capsule endoscopy, which can perform hardware-enabled 3D reconstruction, has been developed. We aimed to evaluate the feasibility of newly developed 3D capsule endoscopy in clinical practice. This study was a prospective, single-arm, feasibility study conducted at two university-affiliated hospitals in South Korea. Small bowel evaluation was performed using a newly developed 3D capsule endoscope for patients with obscure gastrointestinal bleeding, suspected or established Crohn’s disease, small bowel tumors, and abdominal pain of unknown origin. We assessed the technical limitations, performance, and safety of the new capsule endoscope. Thirty-one patients (20 men and 11 women; mean age: 44.5 years) were enrolled. There was no technical defect preventing adequate visualization of the small bowel. The overall completion rate was 77.4%, the detection rate was 64.5%, and there was no capsule retention. All capsule endoscopic procedures were completed uneventfully. In conclusion, newly developed 3D capsule endoscopy was safe and feasible, showing similar performance as conventional capsule endoscopy. Newly added features of 3D reconstruction and size measurement are expected to be useful in the characterization of subepithelial tumours.
Similar content being viewed by others
Introduction
Since its development in 2000, capsule endoscopy has played major roles in the evaluation of small bowel disease1. However, current capsule endoscopy has many limitations, and continuous research and development have been performed to overcome these limirations2,3. One of the major drawbacks is the inability of capsule endoscopy to measure the accurate size of the lesion. In addition, due to its passive movement and uncontrolled air insufflation, characterization of excavated or polypoid lesions is difficult in some cases (e.g., differentiation of mass and mucosal bulges)4. To solve these problems, the three-dimensional (3D) reconstruction of the capsule image has been attempted but has shown limited utility so far4,5. A typical 3D-imaging system requires multiple independent cameras to reconstruct three-dimensional structures. However, due to the limited hardware resources pertaining to capsule size and battery capacity, there has been no hardware-enabled 3D reconstruction to date. Instead, software approaches (e.g., Shape-form-Shading, SfS) have been used to reconstruct 3D small bowel structures6,7. Software approaches have the fundamental limitation of estimating accurate and robust 3D information without newly added image information. Even though this tool has shown some usefulness in the interpretation of capsule images, it has had limited clinical significance due to its inherent limitations4,5.
Recently, IntroMedic Company (Seoul, South Korea) has developed a new stereo camera-based capsule endoscope, which is the first hardware-enabled 3D capsule endoscopy device. This instrument can estimate the geometric structure of the small bowel, making it possible to reconstruct the 3D structure and accurately estimate the size of an object within a 10% relative error to the actual size8.
In this study, we aimed to evaluate the feasibility of this newly developed 3D capsule endoscope in clinical practice.
Patients and Methods
Study design and participants
This study was a prospective, single-arm, feasibility study conducted at two university-affiliated hospitals in South Korea (Dongguk University Ilsan Hospital and Kangwon National University Hospital). Small bowel evaluation was performed using MiroCam® MC4000 (IntroMedic® Co., Seoul, South Korea) capsule endoscopy. Inclusion and exclusion criteria were the same as those for the current application of capsule endoscopy. Capsule endoscopy was performed in patients with obscure gastrointestinal bleeding, suspected or established Crohn’s disease, small bowel tumours, and abdominal pain of unknown origin. Capsule endoscopy was not performed in patients in whom gastrointestinal obstruction or stenosis was suspected, nor was it performed in patients with swallowing disorders. The study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Dongguk University Ilsan Hospital Institutional Review Board (IRB no. DUIH2019-06-001-001). For participants under the age of 18 years, informed consent was obtained from a parent.
Capsule endoscopy device description
MiroCam® MC4000 is a newly developed stereo camera-based wireless capsule endoscope consisting of two cameras that are displaced by approximately 4 mm, four LED lights, a wireless transmitter, and a battery (Fig. 1)8. The size (10.8 mm*24.5 mm) and weight (3.4 g) are similar to those of previous versions of the MiroCam® capsule endoscope, which are commercially available9. The depth range is up to 30 mm, the view angle is 170 degrees and the frame rate is 2*2 frames per second. For stereo matching, we computed the matching cost using the Census-based Hamming distance and absolute difference of intensities (AD-CENSUS)10. To consider the smoothness constraint with neighbouring pixels, we applied the SGM (Semi-Global Matching) to minimize the cost function with the constraint of the direct attenuation model8,11. With the aid of dual image sensors and a depth estimation algorithm, MC 4000 can render the 3D structure of the small bowel and estimate the size of the lesion within 10% relative error to the actual size, which has been demonstrated by a large bowel phantom model and plastic shaft phantom model8,12. Video clips showing the function of 3D reconstruction and size measurement are included (see Supplementary Videos S2 and S3).
Capsule endoscopy procedures
Twelve hours of overnight fasting and bowel cleansing with purgatives (polyethylene glycol with ascorbic acid or oral sodium sulfate solution) were performed for all patients. Patients are allowed to drink clear fluids two hours after capsule ingestion. Written informed consent was obtained from all patients prior to the examination.
Capsule endoscopy video evaluation
Captured images were evaluated by each hospital’s experienced reader (with more than 300 cases of capsule endoscopy performance). Images were analysed using the software MiroView (ver. 4.1, IntroMedic). Tens of thousands of images per patient were evaluated at a maximum speed of 10 frames per second in a single view mode and 20 frames per second in a double or quadruple view mode. Capsule image interpretation was performed according to the established guidelines13,14,15. If an abnormal finding was detected, the reader stopped displaying images and interpreted the lesion in the context of several images back and forth around the culprit image.
The quality of bowel preparation was categorized as follows: excellent, visualization of ≥75% of the mucosa; good, visualization of 50–74% of the mucosa; fair, visualization of 25–49% of the mucosa; poor, visualization of ≤24% of the mucosa. Excellent or good was considered adequate bowel preparation, and fair or poor was considered inadequate. Complete examination was defined as the capsule passing through the ileocaecal valve into the caecum during the recording time. Capsule retention was defined as the capsule remaining in the digestive tract for a minimum of two weeks or one that required directed intervention or therapy to aid its passage. The completion rate was calculated as the ratio of the successfully completed procedures to the total capsule endoscopy procedures. The retention rate was the ratio of the procedures with capsule retention to the total capsule endoscopy procedures. The detection rate was calculated as the ratio of the positive-detection procedures over the total capsule endoscopy procedures.
Study endpoints
The primary objective of this study was to check the feasibility of the newly developed capsule in clinical practice. For this purpose, we assessed (1) the technical limitations and failures (problems associated with the functioning of the capsule) and (2) the performance and safety of the new capsule by analysing the detection, completion, and retention rate. The secondary objective was to check the usefulness of the 3D rendering and size measurement function in each patient’s clinical situation. We defined usefulness as any help in decision making on future management by additional information acquired through these two functions.
Statistical analysis
This is a prospective, non-blinded, single-arm study to assess the feasibility of newly developed capsule endoscopy. No sample size calculation was performed. We estimated that 15 subjects would allow reasonable assessment of the general safety and feasibility of the newly developed capsule endoscope.
Research involving human participants and Informed consent
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Results
Patients’ characteristics and bowel preparations
From June 2019 to January 2020, 31 patients (men 20, women 11; mean age 44.5, range 16–84) were enrolled in this study (Table 1). Indications for the study were obscure gastrointestinal bleeding, suspected Crohn’s disease, established Crohn’s disease, iron deficiency anemia, unexplained abdominal pain, suspected Behcet’s disease, and suspected juvenile polyposis (details are summarized in Supplementary Table S1). All patients received bowel preparation with purgatives, and most of them showed adequate quality of bowel preparation except eight patients with fair preparation and three patients with poor preparation (excellent 19.4%, good 45.2%, fair 25.8%, and poor 9.7%).
Feasibility of the new capsule endoscopy
Clinically significant lesions were detected in 20 subjects, most of which were erosions and ulcers (detection rate 64.5%). The overall completion rate was 77.4%, and a complete examination was not possible for seven subjects. In two patients with Crohn’s disease, the capsule was retained in the small bowel near the ulcer and inflammation site for a long time but was excreted the next day. For the other patients, the capsule was retained around the suspicious small bowel mass in one patient, and the small bowel transit time was significantly prolonged in four patients (one of them had total gastrectomy for gastric cancer). However, there was no capsule retention (retention rate 0%). Moreover, there were no technical defects such as battery shortages, failure of capsule activation, and failure to download capsule images, which prevented adequate visualization of the small bowel (Table 2).
Capsule images and usefulness of the new device functions (3D rendering & size estimation)
Figures 2, 3, and 4 show lesions and other small bowel structures detected in this study. In a patient with a subepithelial lesion (Fig. 2A), 3D rendering and size estimation gave more information on the lesion characteristics compared to the conventional 2D capsule images and was helpful in discriminating the mass from an innocent mucosal bulge. In this patient, the capsule was retained around this mass for a long time, and he was recommended for further evaluation, including deep enteroscopy, but he declined and was lost during follow-up. For the other patients, most detected lesions were hyperaemia or erosion/small ulcers (Fig. 4). Even though 3D reconstruction with size measurement gave more information about the characteristics of these lesions, they did not contribute significantly to clinical decision making and future management.
Discussion
Inability to control capsular movement is one of the major drawbacks of current capsule endoscopy. Due to this innate limitation, the characterization of small bowel lesions has been difficult in some cases (e.g., submucosal tumours), unlike conventional fibreoptic endoscopy, which can aerate, palpate, measure size, and acquire tissue. To obtain more information from limited image data from current capsule endoscopy, several efforts have been made, including three-dimensional reconstructions of the small bowel structure. There was a hardware approach using an infrared projector and a CMOS camera to construct a 3D structure, but the device failed commercialization due to size, power consumption, and packaging problems6,16,17. On the other hand, a software approach using the SfS technique yielded some promising results6,7. One study showed that 26.6% of 3D represented images made by this mathematical transformation presented enhanced visualization features compared to 2D conventional counterparts5. However, this function was helpful mainly for vascular lesions, which can be easily recognized and evaluated without the aid of 3D reconstruction. Image enhancement with 3D image representation software was not helpful for protruding lesions. In another study, the same image enhancement tool (SfS algorithm) was used to differentiate clinically significant masses from innocent mucosal bulges4. 3D reconstruction did not improve the performance of expert readers on the differentiation of masses from mucosal bulges, and there was limited use of this technique. Strictly speaking, there must be inherent limitations to the software approach. From the mathematical point of view, there is no information additional to that of the conventional 2D images, and only a change in visualization has been produced by the calculations of 2D image information. Therefore, previous studies showing the limitation of the software approach have consistently mentioned the need for hardware-enabled 3D reconstruction. Size estimation is also impossible by the software approach4,5.
In this study, we evaluated the feasibility of a newly developed stereo camera-based capsule endoscope, which enables the first hardware-enabled 3D capsule endoscopy. With the aid of dual image sensors and custom-developed algorithms, this device can objectively perform 3D reconstruction and size estimation of small bowel structures. We performed capsule endoscopy with this new device in 31 patients. There were no technical defects, including battery shortages, failure of capsule activation, and failure to download capsule images, and all capsule endoscopy procedures were uneventfully completed. Seven subjects showed incomplete examinations, and the overall study completion rate was 77.4%. The detection rate was 64.5%, and there was no capsule retention. This study showed values of the detection rate, completion rate and retention rate (64.5%, 77.4%, and 0%, respectively) that were comparable to those in previously published data with conventional capsule endoscopy (60–63%, 80–83%, and 1–2%, respectively)18,19,20,21,22. Even though the function of 3D reconstruction and size estimation was fascinating, these functions did not contribute significantly to the clinical management of most patients in this study. However, in one patient with a protruding small bowel mass, 3D reconstruction and size measurement added more information to characterize the lesion and were helpful in decision making.
Our feasibility study has several limitations. The small number of patients mostly consisting of subjects with inflammatory lesions was not enough to show the usefulness of the new function of capsule endoscopy. In addition, the accuracy of size measurement and the shape of the 3D rendered structure need to be confirmed in human data by surgery or deep enteroscopy. However, we were able to demonstrate the safety and technical feasibility of this newly developed capsule endoscope.
In conclusion, the newly developed 3D capsule endoscope MC4000 (IntroMedic® Co., Seoul, South Korea) was safe and feasible for small bowel evaluation and showed at least a similar performance as conventional capsule endoscopy. The clinical implications of the newly added fascinating features of 3D reconstruction and size measurement need to be validated in a larger number of patients. Particularly, subepithelial tumour characterization is expected to be the most promising part of the clinical applications of 3D capsule endoscopy.
References
Iddan, G., Meron, G., Glukhovsky, A. & Swain, P. Wireless capsule endoscopy. Nature 405, 417–417, https://doi.org/10.1038/35013140 (2000).
Koprowski, R. Overview of technical solutions and assessment of clinical usefulness of capsule endoscopy. Biomed. Eng. online 14, 111, https://doi.org/10.1186/s12938-015-0108-3 (2015).
Kwack, W. G. & Lim, Y. J. Current Status and Research into Overcoming Limitations of Capsule Endoscopy. Clin. Endosc. 49, 8–15, https://doi.org/10.5946/ce.2016.49.1.8 (2016).
Rondonotti, E. et al. Utility of 3-dimensional image reconstruction in the diagnosis of small-bowel masses in capsule endoscopy (with video). Gastrointest. endoscopy 80, 642–651, https://doi.org/10.1016/j.gie.2014.04.057 (2014).
Koulaouzidis, A. et al. Three-dimensional representation software as image enhancement tool in small-bowel capsule endoscopy: a feasibility study. Dig. Liver Dis. 45, 909–914, https://doi.org/10.1016/j.dld.2013.05.013 (2013).
Koulaouzidis, A. & Karargyris, A. Three-dimensional image reconstruction in capsule endoscopy. World J. gastroenterology 18, 4086–4090, https://doi.org/10.3748/wjg.v18.i31.4086 (2012).
Karargyris, A., Rondonotti, E., Mandelli, G. & Koulaouzidis, A. Evaluation of 4 three-dimensional representation algorithms in capsule endoscopy images. World J. gastroenterology 19, 8028–8033, https://doi.org/10.3748/wjg.v19.i44.8028 (2013).
Park, M.-G., Yoon, J. H. & Hwang, Y. In Patch-Based Techniques in Medical Imaging. (eds Wenjia Bai et al.) 48–56 (Springer International Publishing).
Mussetto, A. et al. MiroCam™ capsule for obscure gastrointestinal bleeding: A prospective, single centre experience. Digestive Liver Dis. 45, 124–128, https://doi.org/10.1016/j.dld.2012.08.016 (2013).
Mei, X. et al. On building an accurate stereo matching system on graphics hardware. 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, pp. 467–474 (2011).
Hirschmuller, H. Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341, https://doi.org/10.1109/TPAMI.2007.1166 (2008).
Hawkes, E., Keen, T., Patel, P. & Rahman, I. PTH-024 Novel capsule endoscope with 3D reconstruction and lesions size calculation: first study with MiroCam MC4000. Gut 68, A25–A25, https://doi.org/10.1136/gutjnl-2019-BSGAbstracts.49 (2019).
Rondonotti, E. et al. Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Technical Review. Endoscopy 50, 423–446, https://doi.org/10.1055/a-0576-0566 (2018).
Kim, S. H., Yang, D.-H. & Kim, J. S. Current Status of Interpretation of Small Bowel Capsule Endoscopy. Clin. endoscopy 51, 329–333, https://doi.org/10.5946/ce.2018.095 (2018).
Committee, A. T. et al. Wireless capsule endoscopy. Gastrointest. endoscopy 78, 805–815, https://doi.org/10.1016/j.gie.2013.06.026 (2013).
Kolar, A. et al. A System for an Accurate 3D Reconstruction in Video Endoscopy Capsule. EURASIP J. Embedded Syst. 2009, 716317, https://doi.org/10.1155/2009/716317 (2009).
Kolar, A., Romain, O., Ayoub, J., Viateur, S. & Granado, B. Prototype of Video Endoscopic Capsule With 3-D Imaging Capabilities. IEEE Trans. Biomed. Circuits Syst. 4, 239–249, https://doi.org/10.1109/TBCAS.2010.2049265 (2010).
Rondonotti, E. et al. Complications, limitations, and failures of capsule endoscopy: a review of 733 cases. Gastrointestinal endoscopy 62, 712–716; quiz 752, 754, https://doi.org/10.1016/j.gie.2005.05.002 (2005).
Liao, Z., Gao, R., Xu, C. & Li, Z. S. Indications and detection, completion, and retention rates of small-bowel capsule endoscopy: a systematic review. Gastrointest. endoscopy 71, 280–286, https://doi.org/10.1016/j.gie.2009.09.031 (2010).
Lim, Y. J. et al. Indications for Detection, Completion, and Retention Rates of Small Bowel Capsule Endoscopy Based on the 10-Year Data from the Korean Capsule Endoscopy Registry. Clin. Endosc. 48, 399–404, https://doi.org/10.5946/ce.2015.48.5.399 (2015).
Lee, H. S. et al. Outcomes and Management Strategies for Capsule Retention: A Korean Capsule Endoscopy Nationwide Database Registry Study. Dig. Dis. Sci. https://doi.org/10.1007/s10620-019-05659-7 (2019).
Nemeth, A., Wurm Johansson, G., Nielsen, J., Thorlacius, H. & Toth, E. Capsule retention related to small bowel capsule endoscopy: a large European single-center 10-year clinical experience. U Eur. Gastroenterol. J. 5, 677–686, https://doi.org/10.1177/2050640616675219 (2017).
Acknowledgements
This research was supported by a grant from the Korean Health Technology R & D project through the Korean Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (Grant Number: HI19C0665).
Author information
Authors and Affiliations
Contributions
S.-J.N. collected and analysed the data and wrote the paper. Y.J.L. conceived and designed the study, collected the data and revised the manuscript. J.H.N. collected the data and performed the analysis. H.S.L. contributed to the designing of the study and performed the data analysis. Y.H. created the new algorithms used in this study and performed the data interpretation and critical revision for the technical aspects of the developed device. J.P. contributed to the designing of the study and performed data analysis. H.J.C. performed data analysis and interpretation.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Nam, SJ., Lim, Y.J., Nam, J.H. et al. 3D reconstruction of small bowel lesions using stereo camera-based capsule endoscopy. Sci Rep 10, 6025 (2020). https://doi.org/10.1038/s41598-020-62935-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-020-62935-7
This article is cited by
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.