Abstract
A wide-field endoscope that is sensitive to fluorescence can be used as an adjunct to conventional white light endoscopy by detecting multiple molecular targets concurrently. We aim to demonstrate a flexible fiber-coupled accessory that can pass forward through the instrument channel of standard medical endoscopes for clinical use to collect fluorescence images. A miniature scan mirror with reflector dimensions of 1.30 × 0.45 mm2 was designed, fabricated, and placed distal to collimated excitation beams at λex = 488, 660, and 785 nm. The mirror was driven at resonance for wide angular deflections in the X and Y-axes. A large image field-of-view (FOV) was generated in real time. The optomechanical components were packaged in a rigid distal tip with dimensions of 2.6 mm diameter and 12 mm length. The scan mirror was driven at 27.6 and 9.04 kHz in the fast (X) and slow (Y) axes, respectively, using a square wave with 50% duty cycle at 60 Vpp to collect fluorescence images at 10 frames per sec. Maximum total divergence angles of ± 27.4° and ± 22.8° were generated to achieve a FOV of 10.4 and 8.4 mm, respectively, at a working distance of 10 mm. Multiplexed fluorescence images were collected in vivo from the rectum of live mice using 3 fluorescently-labeled peptides that bind to unique cell surface targets. The fluorescence images collected were separated into 3 channels. Target-to-background ratios of 2.6, 3.1, and 3.9 were measured. This instrument demonstrates potential for broad clinical use to detect heterogeneous diseases in hollow organs.
Similar content being viewed by others
Introduction
Improved methods are needed to detect diseases that originate from the mucosal surface of hollow organs, such as colon1,2,3. Endoscopes can be inserted into natural body orifices to access this highly metabolically active layer of tissue that serves as the origin of many human disorders4,5,6. Images can be collected in vivo in real time with high spatial resolution to identify and localize abnormal regions. White light illumination is currently used to identify pre-malignant masses, such as polyps, but has limited effectiveness to identify lesions that are flat and subtle in appearance7,8,9. Fluorescence can be used to label ligands, such as peptides, and provide high contrast to distinguish regions of pathology from surrounding areas of normal tissues. The optical spectrum provides a wide range of wavelengths (500–900 nm) that can be separated into different channels used to perform multiplexed detection of unique molecular targets10,11,12. A number of fluorophores, such as fluorescein and indocyanine green (ICG), are FDA-approved for clinical administration, and can be used to label ligands that bind specifically to biomarkers of disease13. Medical endoscopes have working channels that are typically several millimeters in diameter, and can pass instruments for tissue resection4,5,6. A flexible, fiber-coupled accessory can pass through this channel if the dimensions are sufficiently small.
The rigid distal tip of conventional medical endoscopes contains a complex arrangement of objective lenses to achieve high spatial resolution with a large image field-of-view (FOV)6. Barrel distortion is introduced by increasing the magnification in the center versus that in the periphery to achieve a maximum divergence angle up to 140° (± 70°) or more. Achromats are used to correct for chromatic dispersion so that light over the visible spectrum (400–700 nm) can focus on a detector. The pixel number in the sensor is typically much greater than the number of fibers in a flexible optical bundle. Alternatively, images can be generated over a large FOV using a much simpler strategy by deflecting a low numerical aperture (NA) collimated beam at wide angles. A miniature scan mirror placed distal to the collimating optics can deflect a diffraction-limited beam over an arbitrarily large FOV. The excitation beam passes through the center of the objective where sensitivity to aberrations is minimal. In this configuration, the FOV is limited only by the mechanical scan angle of the reflector. The instrument can be scaled down in diameter without loss of resolution and fiber-coupled for use as an accessory with medical endoscopes. Simple fabrication processes can be developed with potential for mass manufacture at low cost.
Micro-Electro-Mechanical Systems (MEMS) methods were used to design and fabricate miniature scan mirrors for use in the distal end of flexible fiber-coupled instruments14,15,16. Small dimensions allow for placement in the post-objective position, and allows for the distal tip to be scaled down in size. These devices were batch fabricated using simple processes on a single silicon wafer for mass manufacture17. By comparison, conventional endoscopes use bulky optics that are large in dimension and require multiple focusing elements. Here, we aimed to scale down the dimensions of a wide-field accessory to < 2.6 mm diameter and < 12 mm length. This small size allowed for forward passage through the instrument channel of standard medical endoscopes. A side-view geometry provides direct beam delivery to the mucosal surface without requiring bending of the distal tip. Multiplexed fluorescence images were collected using excitation at λex = 488, 660, and 785 nm to demonstrate use of a broad range of wavelengths in the visible and NIR spectrum. In vivo images were collected from adenomas that arise spontaneously in a pre-clinical model of colorectal cancer. Disease heterogeneity was addressed by detecting 3 molecular targets concurrently.
Results
Imaging system
A schematic is shown of the imaging system used to support the wide-field accessory to collect fluorescence images in 3 channels, Fig. 1. The excitation beams were combined using a custom wavelength division multiplexer (WDM) for delivery through a single mode fiber (SMF1). A second single-mode fiber (SMF2) was centered in the wide-field accessory using a ferrule. The accessory was coupled to the excitation fiber SMF1 using an FC/APC connector. The excitation beams exciting SMF2 were collimated using an aspheric (L1) and achromatic (L2) lens arranged in tandem. A miniature mirror (M) deflected the beams at a 90° angle, and performed wide angular scanning about the X and Y axes. A concave lens (L3) provided additional angular divergence, and served as a window to protect the accessory components. Laser power at λex = 488, 660, and 785 nm was transmitted out of the accessory with 22%, 16%, and 9% efficiency, respectively. During imaging, laser power between 3 and 5 mW was incident on the tissue. Visible and NIR fluorescence was collected by L3, and descanned by M. Prisms P1 and P2 with dimensions of 0.4 mm × 0.3 mm (hypotenuse × legs) were used to increase the effective area for fluorescence collection and delivery into 2 multi-mode fibers (MMF1 and MMF2). Lens L4 collimated the fluorescence beams for separation into 3 spectrally distinct channels using a set of dichroic mirrors DM1-DM3, long pass filters LPF1-LPF3, and lenses L5-L7 for detection by photomultiplier tubes PMT1-PMT3.
Imaging system. Schematic is shown for multiplexed detection of fluorescence images using λex = 488, 660, and 785 nm by the wide-field accessory. Details are provided in text. Key: amp amplifier, DM dichroic mirror, L lens, LPF low-pass filter, M mirror, MMF multi-mode fiber, P prism, PMT photomultiplier, SMF single-mode fiber, WDM wavelength division multiplexer.
Optical design of wide-field accessory
A simple optical design was used to deliver a diffraction-limited collimated beam for rapid deflection at wide angles in the X and Y axes by mirror M to achieve a large image FOV, Fig. 2. Ray trace simulations were performed to determine the FOV over a range of working distances WD = 0 to 50 mm. Commercially available lenses were used, including an aspheric (L1, 1.0 mm diameter, f = 0.60 mm, NA = 0.83, n = 1.84), achromatic doublet (L2, 1.0 mm diameter, f = 1.5 mm, NA = 0.33, n = 1.62 and 1.84), and concave (L3, 1.0 mm diameter, f = − 4 mm, n = 1.48). The optimal center-to-center distance between L1 and L2 to collimate the excitation beams was 1.19 mm. The optics were chosen to fit within the physical constraints of the accessory that could pass through the working channel of a standard medical endoscope. The resulting NA was 0.00015, and the beam diameter was 28.9 µm at WD = 10 mm. With the addition of L3, the maximum optical scan angles increased from ± 27.30° and ± 28.03° to ± 32.75° and ± 33.40° in the X and Y axes, respectively. This optical design allowed for a broad range of fluorescence wavelengths to be collected over the visible and NIR spectrum (λem = 500–900 nm). Multiplexed imaging could be performed with minimal chromatic aberrations over a large image FOV. In the simulations, a FOVx and FOVy of 11.25 and 9.12 mm were achieved at WD = 10 mm, Table S1. Ray trace simulations were performed to characterize chromatic aberrations at 488, 660, and 785 nm, Table S2, resolution at different positions in the image FOV, Table S3, and collection efficiency as a function of scan angle, Fig. S3.
Optical design. Ray trace simulations show the excitation and emission beam paths. (a) An aspheric and achromat lens collimate the excitation beam exiting SMF2 with minimal chromatic dispersion (side view). A miniature scan mirror (M) deflects the collimated beam at wide angles. A concave lens further increases the deflection angles. (b) Fluorescence is collected over a broad spectral bandwidth (500–900 nm) using a pair of prisms that are coupled to multi-mode fibers (MMF) located along either side of the accessory (top view).
Scan mirror
A compact 2D scan mirror (M) was designed with dimensions to fit in a distal rigid tip of a wide-field accessory with 2.6 mm diameter and 12 mm length. A reflector with dimensions of 1.30 × 0.45 mm was mounted on a 2.02 × 2.5 mm gimbal to minimize cross-talk between the X and Y axes. Finite element analysis was performed using COMSOL software. Eigenmodes EM7 = 13,992 Hz and EM2 = 4667.9 Hz, respectively, were identified for maximum X and Y axes tilt motions, Fig. 3a,b. The resonance frequency in the X-axis was determined by 2 inner torsional springs arranged in-line along the reflector length, Fig. 3c. Regions of increased stress are shown in pseudo-color. The resonance frequency in the Y-axis was determined by 4 outer torsional springs arranged in 2 symmetric sets on either side of the reflector, Fig. 3d. The dimensions of the springs are shown, Fig. 3e,f.
Spring design. CAD drawing shows areas of simulated active scan mirror tilt motion in the (a) X and (b) Y-axes at eigenmodes EM7 = 13,992 and EM2 = 4667.9 Hz, respectively (oblique view). Angular deflections of scan mirror and support from the backside islands are shown (side view). The color map shows the total displacement magnitude (pale green to dark blue corresponds to zero to max amplitude). Mechanical stress is shown for the (c) 2 inner torsional springs arranged in-line along the reflector length, and for (d) symmetric sets of 4 outer torsional springs. The maximum stresses applied to the springs are 1.69 and 1.07 GPa for the inner and outer springs, respectively, which are lower than the tensile strength of silicon. The dimensions of the inner (e) and outer (f) torsional springs are shown.
A CAD drawing is shown of the front-view of the scan mirror, Fig. 4a. A photo of the fabricated device is shown, Fig. 4b. The frequency response in the X and Y axes was generated using a down-sweep (high-to-low frequencies) and up-sweep (low-to-high frequencies), Fig. 4c,d. A time interval of ~ 1 s was employed in between frequency transitions to provide stable scan mirror motions at each drive frequency and to maximize the scan angles at resonance. Optical scan angles of ± 27.4° and ± 22.8° were measured in the X and Y axes, respectively. The drive frequencies measured 2ω agreed with the simulation results for ω.
Scan mirror design. (a) A CAD layout of the 2D scan mirror (M) is shown. The locations of the device layer (DL), backside islands (BI), and backside substrate (BS) are identified. A reflector with dimensions of 1.30 × 0.45 mm is mounted on a 2.02 × 2.5 mm gimbal that rotates about the X and Y axes to deflect the beams over wide angles. (b) Photo is shown of a fabricated scan mirror. (c,d) The frequency response of the scan mirror is shown using down-sweep (high-to-low frequencies) and up-sweep (low-to-high frequencies). Maximum optical scan angles of ± 27.4° and ± 22.8° were achieved in the X and Y axes using a square wave drive signal at 27.75 and 9.15 kHz, respectively, with 50% duty cycle at 60 Vpp during down-sweep of frequencies.
Packaging of wide-field accessory
The wide-field accessory was packaged in a stainless-steel outer tube with a 2.6 mm OD, 2.3 mm ID, and 12 mm length. A CAD drawing of the rigid distal tip shows the spatial arrangement of the internal components, Fig. 5. A photo of the flexible fiber-coupled wide-field accessory is shown after completion of packaging, Fig. 6a. This instrument passed forward seamlessly through the working channel of a standard medical endoscope, Fig. 6b. Delivery of excitation at λex = 488, 660, and 785 nm separately, Fig. 6c–e, and concurrently, Fig. 6f, shows a large image FOV.
Packaging strategy. CAD drawing shows the arrangement of individual optical and mechanical components in the rigid distal tip of the wide-field accessory.
Endoscope accessory. (a) The rigid distal tip of the flexible fiber-coupled wide-field accessory has dimensions of 2.6 mm diameter and 12 mm length. (b) The accessory passes forward easily through the 3.7 mm diameter instrument channel of a standard medical endoscope (Olympus, #GIF-1TH190). Photos show the scan mirror deflecting the excitation beams in a Lissajous pattern for λex = (c) 488, (d) 660, and (e) 785 nm at a working distance WD = 20 mm. (f) Concurrent excitation of 3 wavelengths is shown in the merged image.
Image characterization
The optical scan angle of the miniature mirror was confirmed by collecting fluorescence images from a grid target consisting of black dots spaced 1 mm apart at a working distance WD = 10 mm. The target was stained with fluorescence dyes, including FITC, Cy5.5, and IRDye800, for excitation with λex = 488, 660, and 785 nm, respectively. Fluorescence images were collected using the 3 excitation beams separately, Fig. S1a–c, and all together, Fig. S1d. A measured FOV of 10.4 × 8.4 mm2 was supported by detection of 10 and 9 black dots in the X and Y directions, respectively. Lateral resolution was characterized by measuring the full-width-at-half-maximum (FWHM) of the point spread function (PSF) in the X and Y axes at WD ranging between 0 and 50 mm, Fig. S2a,b. A FWHM of 307.7 µm, 198.8 µm, and 185.6 µm and 159.1 µm, 168.2 µm, and 168.3 µm was measured at WD = 10 mm in the X and Y axes, respectively, for λex = 488, 660, and 785 nm, respectively.
Multiplexed wide-field imaging in vivo
Multiplexed fluorescence images were collected in vivo from CPC;Apc mice that have been genetically engineered to spontaneously develop colonic adenomas, Fig. 7a. Fluorescently-labeled peptides, including KCC*-FITC, QRH*-Cy5, and KSP*-IRDye800, Fig. S4a–c, were topically administered to a region of prolapsed rectum. The fluorophores were chosen with absorption spectra that match the laser excitation wavelengths at λex = 488, 660, and 785 nm with minimal overlap of fluorescence emission spectra, Fig. S4d,e. The peptide ligands bind specifically to Prdx1, EGFR, and ErbB2, respectively. These molecular targets are overexpressed in pre-malignant (adenoma) but not normal colonic mucosa. Fluorescence images collected at WD = 10 mm showed strong intensity from the adenoma (arrow) by comparison with adjacent normal mucosa, Fig. 7b–d (Supplementary Videos 1–3). A signal-to-noise ratio (S/N) of 14.76, 13.04, and 12.49 was measured at λex = 488 nm, 660 nm, and 785 nm, respectively. Target-to-background (T/B) ratios of 2.6, 3.1, and 3.9, respectively, were measured. The merged image shows fluorescence from all 3 excitation wavelengths concurrently, Fig. 7e (Supplementary Video 4).
In vivo imaging. (a) Photo is shown of an ~ 4 mm diameter adenoma located adjacent to normal mucosa in the prolapsed rectum of a live CPC;Apc mouse. Fluorescence images were collected at a working distance of WD = 10 mm using excitation λex = (b) 488, (c) 660, and (d) 785 nm to detect fluorescently-labeled peptides KCC*-FITC, QRH*-Cy5, and KSP*-IRDye800, specific for Prdx1, EGFR, and ErbB2, respectively. (Supplementary Videos 1–3) Target-to-background (T/B) ratios of 2.6, 3.1, and 3.9, respectively, were measured. (e) The merged image shows use of all 3 excitation wavelengths concurrently. Images were captured at 10 fps. (Supplementary Video 4).
Discussion
Here, we demonstrate a flexible fiber-coupled wide-field accessory with 2.6 mm diameter and 12 mm length distal tip to collect fluorescence images in multiple spectral bands. These dimensions allow the instrument to pass through the working channel of standard medical endoscopes to collect fluorescence images with high contrast over large mucosal surface areas. A simple, scalable optical design was used to deliver a diffraction-limited collimated beam. Optics were designed to collect fluorescence over a broad spectral bandwidth (λex = 500–900 nm) with minimal chromatic dispersion. A miniature scan mirror was placed in the post-objective position to rapidly deflect the beams at maximum total divergence angles of ± 27.4° and ± 22.8° in the X and Y axes, respectively. Multiplexed fluorescence images were collected using excitation at λex = 488, 660, and 785 nm. In vivo images were collected from a spontaneous adenoma in the rectum of a live mouse using fluorescently-labeled peptides specific for Prdx1, EGFR, and ErbB2, respectively. T/B ratios of 2.6, 3.1, and 3.9, respectively, were measured. Detection of 3 molecular targets concurrently was demonstrated to address disease heterogeneity. While experimental validation was performed using a pre-clinical model, clinical translation is expected to involve similar peptide concentrations, laser powers, signal levels, and target-to-background ratios.
Standard medical endoscopes are used clinically to rapidly visualize large mucosal surface areas in hollow organs, such as colon. White light illumination is effective to localize pre-malignant lesions that are polypoid in morphology. Pre-malignant lesions that are either flat or subtle in appearance can be identified using fluorescently-labeled ligands that bind to molecular targets specific for disease18. A flexible fiber-coupled accessory can be used to collect fluorescence images for use as an adjunct to the conventional white light images. The instrument channel in standard medical endoscopes is typically only a few millimeters in diameter. Also, endoscopes have a large bending angle at the entrance that can obstruct passage of the rigid distal tip4,5,6. A miniature scan mirror placed in the post-objective position can steer a collimated excitation beam with wide angular deflections. Images with a very large FOV can be generated in a miniature instrument package. A side-view geometry allows for the accessory to deliver the excitation beams incident to the mucosal surface with only minimal maneuvering of the distal tip of the medical endoscope. A flexible fiber instrument that can pass forward through this channel may have broad clinical applications for disease detection in hollow organs.
Previously, a multi-modal scanning fiber endoscope (mmSFE) was clinically demonstrated for early detection of esophageal cancer19,20. This front-view accessory had rigid tip dimensions of 2.4 mm diameter and 9 mm length. Excitation at λex = 638 and 785 nm was delivered concurrently through a single mode optical fiber to collect multiplexed wide-field fluorescence images in 2 channels. A piezo tube actuator was used to generate a spiral scan pattern at a maximum total divergence angle of ~ 70° (± 35°). 6 multi-mode fibers were arranged around the instrument periphery to collect fluorescence. The compact distal tip dimensions allowed for forward passage through the working channel of a standard upper endoscope. Subsequently, instrument dimensions were scaled down for use in the bile ducts to image indeterminant strictures. The rigid tip dimensions were 1.2 mm diameter and 8 mm length with a similar maximum divergence angle of ~ 70°21. 2 multi-mode fibers were mounted on either side of the objective to collect fluorescence, and the accessory was passed through the working channel of a standard side-view duodenoscope. By comparison, our accessory has much simpler packaging requirements with comparable performance. The scan mechanism was placed in the post-objective position so that the incident beam was incident along the main optical axis to provide greater alignment tolerance.
Other wide-field endoscope designs have been demonstrated for in vivo imaging. A spectrally-encoded instrument employed a spectral-domain interferometry technique22. This endoscope was equipped with a diffraction grating to spectrally diverge a broadband laser beam from a 350 μm diameter single-mode optical fiber on the tissue surface along the X-axis. To form 2D images, a galvanometric mirror was employed to scan in the Y-axis. This system achieved a lateral resolution of 80 μm. Additionally, depth information was acquired using spectral-domain interferometry, resulting in an axial resolution of 135 μm at a depth of 2.8 mm. Using a different approach, a commercial video colonoscope was adapted with a custom illumination module consisting of a halogen lamp and a 635 nm laser diode23. A custom imaging head was mounted on the proximal end of the colonoscope, and housed cameras, optical components, and filters. This system was used in human subjects to collect real-time images of white light reflectance and NIR fluorescence from a c-Met peptide concurrently to improve polyp detection during colonoscopy. Furthermore, a side-view optical configuration using a MEMS scanner was previously developed for a confocal endomicroscope24.
The imaging performance of wide-field accessory can be improved in several ways. The inner and outer torsional springs can be optimized to increase the deflection angles of the scan mirror. The optical design allows for deflection angles up to ± 19.99° and ± 13.98° in the X and Y axes, respectively, without loss of excitation power. Optical scan angles can reach ± 32.75° and ± 33.40° in the X and Y-axes, respectively, and are comparable with that of the mmSFE. Additional sets of prisms can be introduced around the perimeter of the objective lens to direct the collected fluorescence into multi-mode fibers for increased signal. Also, a more precise alignment of the center-to-center distance between the aspheric and achromatic lenses can improve collimation of the excitation beams and hence image resolution over the range of working distances. The excitation beams can be better collimated to improve image resolution. This can be achieved with a more precise alignment of the center-to-center distance between the aspheric and achromatic lenses and by adjusting the spacing to focus the beam in the middle of the desired range of working distances from 0 to 50 mm. Laser systems and wide-field accessories can be constructed separately for visible and NIR imaging to improve efficiency for light delivery. These systems can be designed to precisely match the core diameters and operating wavelengths of the single-mode fibers. Moreover, additional fluorescence detection channels can be added to further address molecular heterogeneity of mucosal lesions. FITC, Cy5, and IRDye800 were chosen to provide emission spectra that span the full range of visible and NIR wavelengths (λem = 500–900 nm). The separation between emission peaks is sufficient to add additional fluorophores.
Methods
Imaging system
Fiber-coupled solid-state lasers (Coherent #OBIS 488-40 LS FP, TOPTICA #iBEAM-SMART-660-S, and Coherent #OBIS 785-70 LX FP) were used to deliver excitation at λex = 488, 660, and 785 nm, respectively. The beams were coupled via FC/APC connectors to a custom wavelength division multiplexer (WDM, Thorlabs, #CDI11086). The combined beams were delivered via a single-mode fiber (SMF1, Thorlabs, #630HP) with core/cladding dimensions of 4/125 µm, operating wavelengths of λ = 600–770 nm, and a multi-mode cut-off wavelength of λc = 570 ± 30 nm. The detachable wide-field accessory contained a separate single-mode fiber (SMF2, Thorlabs, #S405-XP) with core/cladding dimensions of 3/125 µm, operating wavelengths of λ = 400–680 nm, and a multi-mode cut-off wavelength at λc = 380 ± 20 nm terminated with a ferrule (Thorlabs, #30127A3). The excitation beams from SMF1 were coupled to the accessory using an FC/APC connector.
Multi-mode fibers (MMF1 and MMF2, Thorlabs, #FP200ERT) with core/cladding dimensions of 200/225 µm were used to transmit fluorescence, and were terminated with FC/PC connectors. Fluorescence exiting the MMFs was collimated and focused by lenses L4-L6 (Thorlabs, #F950FC-A) and L7 (Thorlabs, #F240FC-780). The beams were directed into 3 channels using a set of dichroic mirrors DM1 (IDEX-HS, #FF605-Di02), DM2 (IDEX-HS, #FF757-Di01), and DM3 (IDEX-HS, #FF925-Di01). DM1 transmits and reflects in spectral bands 612–950 and 350–596 nm, respectively. DM2 transmits and reflects from 768–1100 and 450–746 nm, respectively. DM3 transmits and reflects from 943.5–1600 and 350–906.5 nm, respectively. Long-pass filter LPF1 (IDEX-HS, #LP02-488RU-25) transmits from 494–1100 nm, LPF2 (IDEX-HS, #LP02-664RU-25) transmits from 673 to 1498 nm, and LPF3 (IDEX-HS, #VLP01-785–12.5) transmits from 789 to 1300 nm. Using this scheme, fluorescence was separated into 3 channels for delivery to photomultiplier tube (PMT) detectors PMT1 (Hamamatsu, #H7422-40) and PMT2-PMT3 (Hamamatsu, #H7422-50).
The fluorescence signal was augmented using current amplifiers (Edmund Optics, #59-179), and digitized using a DAQ board (National Instruments, #PCI-6115) at 10 MSa/s. The current amplifier, PMT gain, and laser power were tuned by visually inspecting the images, and these parameters were adjusted until high signal-to-noise (S/N) was achieved. Custom software (National Instruments, LabVIEW 2021) was developed to generate the drive signal to the scan mirror M and to perform real-time data acquisition and image reconstruction.
Optical design of wide-field accessory
A simple optical design was developed to scale down the dimensions of the fiber-coupled wide-field accessory for forward passage through the instrument channel of a standard medical endoscope. A single mode fiber (SMF2) was used to minimize the spot size of the collimated excitation beams. Ray trace simulations were performed using ZEMAX 2013 to identify an aspheric (L1, Edmund Optics, #65-273) and achromatic (L2, Edmund Optics, #65-564) lens for arrangement in tandem to collimate the excitation beams, and to collect fluorescence over a broad spectral bandwidth (λem = 500–900 nm) with minimal chromatic dispersion. A miniature scan mirror (M) deflects the beams at 90°. Simulations were performed to determine the image FOV over a wide range of angles. A custom concave lens (L3, Tower Optical Corp) was added to provide additional angular divergence for increased image FOV. Fluorescence was collected by L3, and descanned by M. Two pairs of right angle triangular prisms (P1 and P2, Edmund Optics, #66-770) were used to optimize fluorescence collection.
Scan mirror
A compact 2D scan mirror was designed based on the principle of parametric resonance to perform beam scanning in the X and Y axes25. The desired drive signal using this mechanism has a frequency near 2ω0/n (ω0 is the natural frequency of each resonance mode, n is an integer ≥ 1). Electrostatic comb-drive actuators were used to produce wide angular deflections by tuning the frequency, voltage, and sweep direction of the drive signal. A finite element model (FEM) was developed using COMSOL software to identify the resonance frequencies with desired mode shapes to minimize interference from unwanted parasitic vibrations that can distort the image. These extraneous motions result from mechanical and capacitive coupling of either super or sub-harmonic frequencies near the drive frequency.
The natural frequency of each mode was separated using a spacing of ∆ω ≥ 0.05ω. This guideline was based on our previous experience with parametric resonance scanners26,27,28. A pair of inner torsional springs were used to deflect the gimbal frame that supports the reflector at angles larger than that of the comb-drives actuating in the Y-axis. This design maintained a substantial overlap of the comb fingers and accommodates large actuation torques throughout mirror operation even at large angles. The springs were designed to generate the desired resonance frequencies for wide angle scanning while avoiding unwanted, higher-order resonance modes. The electrostatically actuated scan mirror was designed and fabricated using MEMS processes. The device surface, including the reflector, was coated with a ~ 70 nm layer of aluminum using a blanket evaporation process to enhance reflectivity in the visible and NIR spectral regimes29.
The frequency response of the scanner in the X and Y axes was measured using a position sensing detector (PSD, On-Track Photonics, #PSM 2-10). The optical window of the accessory was positioned in front of the PSD for testing. Optical scan angles at resonance frequencies were measured by driving each axis separately. The frequency response was evaluated using a down-sweep (high-to-low) and up-sweep (low-to-high) in the drive signal to determine the best direction to achieve resonance. These results were used to select the best set of parameters to produce a large range of angular deflections with minimal parasitic vibrations. The frequencies required to achieve optimal Lissajous scanning in the XY plane at 10 frames per sec (fps) were identified using custom MATLAB software (MathWorks).
Packaging of wide-field accessory
The rigid distal tip of the accessory was assembled using a stainless-steel outer tube. The scan mirror was packaged in a sealed cavity to mitigate variations in ambient temperature and pressure, and to provide a stable environment. A MEMS holder was 3D-printed using ABS-like plastic (acrylonitrile butadiene styrene) with resolution 63.5 µm (XY) and 203 µm (Z) (Proto Labs, MicroFine resin). The scanner was mounted at a 45° angle, and packaged inside the wide-field accessory in a side-view configuration. Wires (42 AWG, MWS, #B4421241) with 63 µm diameter were looped around electrode anchors to deliver the drive signal. The achromatic lens (L2) was placed at the end of the holder first followed by the aspheric lens (L1). SMF2 was coupled to the accessory using a fiber ferrule (1.0 mm OD, 126 µm ID, 6.4 mm long, Oz Optics, #FER-1.0-126-ZR-6.4-NS), attached to L1, and was protected by a polymer tube (Nordson MEDICAL, #Pebax 72D). Two pairs of prisms were glued prior to assembly on the 3D-printed holder. After alignment, one side of the prism was attached directly to the MMF. The position of the optics was fixed using UV glue (Norland, #NOA61). A stainless-steel end tube with dimensions of 2.0 and 1.4 mm OD and ID and 5 mm length, respectively, was inserted inside the proximal side of the outer tube to prevent deformation at the adhesive joint and to isolate the scan mirror from unwanted mechanical vibrations.
Image characterization
After the scan mirror was packaged in the wide-field accessory, the maximum deflection angles were confirmed by imaging a 2D grid target consisting of black dots with 1 × 1 mm spacing. Fluorophores, including FITC (Sigma-Aldrich, #F6377-100G), Cy5.5 (Lumiprobe, 67020), and IRDye800 (LiCor Biosciences, # 929-80050), were used to stain the target. The lateral resolution of each excitation beam was measured with the scan mirror placed in the neutral position using a beam profiler (Thorlabs, #BP209-VIS). A full-width-at-half-maximum (FWHM) was calculated over a range of working distances 0–50 mm. Lateral resolution was characterized by measuring the FWHM of the PSF in the X and Y axes. The spatial intensity profile of the excitation beam was measured using the beam profiler at WD ranging from 0 to 50 mm. This range spans the typical values used in vivo during endoscopy.
Ethical approval
All experimental procedures were performed in accordance with relevant guidelines and regulations of the University of Michigan. The mice were housed in pathogen-free conditions and supplied water ad libitum under controlled conditions of humidity (50 ± 10%), light (12/12 h light/dark cycle) and temperature (25 °C) per guidelines of the Unit for Laboratory Animal Medicine (ULAM). In vivo imaging studies were performed with approval of the University of Michigan Committee on the Use and Care of Animals (UCUCA). All methods are reported in accordance with ARRIVE guidelines (https://arriveguidelines.org).
In vivo imaging
CPC;Apc mice were genetically engineered using a Cre recombinase under the control of a Cdx2 promoter (CDX2P-9.5NLS-Cre) and a floxed allele of the APC gene30. This Cre-regulated somatic mutation in one Apc allele resulted in spontaneous development of adenomas in the distal colon. Anesthesia was induced and maintained via a nose cone with inhaled isoflurane mixed with oxygen at concentrations between 2 and 4% at a flow rate of ∼0.5 L/min. Fluorescently-labeled peptides, including KCC*-FITC, QRH*-Cy5, and KSP*-IRDye800, at concentrations of 1, 10, and 1 mM were mixed and sprayed onto the adenoma and adjacent normal mucosa. These ligands are specific for molecular targets Prdx1, EGFR, and ErbB2, respectively31,32,33. Spectral properties for absorption and emission were measured using a spectrophotometer (Thermo Fisher Scientific, Nanodrop #2000c) and a spectrometer (Ocean Insight, #USB2000 +), respectively. After 10 min for incubation, unbound peptides were washed away using phosphate buffered saline (PBS, Thermo Fisher Scientific, #10,010,023). The wide-field accessory was fixed on a linear stage. The distal tip was positioned at WD = 10 mm. Excitation at λex = 488, 660, and 785 nm was delivered concurrently, and wide-field fluorescence images were collected in 3 channels at the same time. This technique mitigated motion artifact from respiratory movements and colon peristalsis.
Image processing
Wide-field fluorescence images were reconstructed using the trajectory of the scan mirror in a Lissajous pattern34. Any phase discrepancies between the drive signal and the scan mirror motion were corrected in real-time using an auto phase correction algorithm35,36. The phase was swept in a step size of 0.286° over a range of 2.86° in the X and Y axes to identify the largest variance in neighboring pixel intensities to deblur the image. Grayscale images of fluorescence collected in 3 channels at λex = 488, 660, and 785 nm were pseudo-colored in blue, green, and red, respectively, to produce a merged image in real-time. All images were mapped to 400 × 400 pixels and collected at 10 fps. Images and videos were saved in BMP and AVI formats, respectively, and were post-processed using Fiji (ImageJ2) software by adjusting the brightness level and applying a median filter to reduce noise and enhance image quality. The target-to-background (T/B) was calculated as the ratio of mean intensities in the target region versus that in the surrounding boundary of equivalent area. The Otsu and the Markov random field (MRF) algorithms were used to classify pixels as either target or background by maximizing the variance between these two regions37,38. Pixel intensities were iteratively updated based on the difference between the intensity and the mean and standard deviation of the two classes. All algorithms were performed using custom MATLAB software (MathWorks).
Data availability
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Schoultz, I. & Keita, Å. V. The intestinal barrier and current techniques for the assessment of gut permeability. Cells 9, 1909 (2020).
Vancamelbeke, M. & Vermeire, S. The intestinal barrier: A fundamental role in health and disease. Expert. Rev. Gastroenterol. Hepatol. 11, 821–834 (2017).
Haber, A. L. et al. A single-cell survey of the small intestinal epithelium. Nature 551, 333–339 (2017).
Kohli, D. R. & Baillie, J. How endoscopes work. In Clinical Gastrointestinal Endoscopy (eds Kohli, D. R. & Baillie, J.) 24–31 (Elsevier, 2019).
Achord, J. L. & Muthusamy, V. R. The history of gastrointestinal endoscopy. In Clinical Gastrointestinal Endoscopy (eds Achord, J. L. & Muthusamy, V. R.) 2–11 (Elsevier, 2019).
Maitland, K. & Wang, T. D. Endoscopy. In Biomedical Technology and Devices Handbook 2nd edn (eds Moore, J. & Maitland, D.) 217–245 (Taylor and Francis, 2013).
Hong, S. W. & Byeon, J. S. Endoscopic diagnosis and treatment of early colorectal cancer. Intest. Res. 20, 281–290 (2022).
Lahiff, C. & East, J. E. Endoscopic approach to polyp recognition. Frontline Gastroenterol. 8, 98–103 (2017).
Kim, N. H. et al. Miss rate of colorectal neoplastic polyps and risk factors for missed polyps in consecutive colonoscopies. Intest. Res. 15, 411–418 (2017).
Andreou, C., Weissleder, R. & Kircher, M. F. Multiplexed imaging in oncology. Nat. Biomed. Eng. 6, 527–540 (2022).
Heinzmann, K., Carter, L. M., Lewis, J. S. & Aboagye, E. O. Multiplexed imaging for diagnosis and therapy. Nat. Biomed. Eng. 1, 697–713 (2017).
Wang, Y. W., Reder, N. P., Kang, S., Glaser, A. K. & Liu, J. T. C. Multiplexed optical imaging of tumor-directed nanoparticles: A review of imaging systems and approaches. Nanotheranostics 1, 369–388 (2017).
Luo, S., Zhang, E., Su, Y., Cheng, T. & Shi, C. A review of NIR dyes in cancer targeting and imaging. Biomaterials 32, 7127–7138 (2011).
Algamili, A. S. et al. A review of actuation and sensing mechanisms in MEMS-based sensor devices. Nanoscale Res. Lett. 16, 1–21 (2021).
Kaur, M., Lane, P. M. & Menon, C. Scanning and actuation techniques for cantilever-based fiber optic endoscopic scanners—A review. Sensors 21, 251 (2021).
Zhou, G., Lim, Z. H., Qi, Y. & Zhou, G. Single-pixel MEMS imaging systems. Micromachines 11, 219 (2020).
Dao, D. V., Nakamura, K., Bui, T. T. & Sugiyama, S. Micro/nano-mechanical sensors and actuators based on SOI-MEMS technology. Adv. Nat. Sci. Nanosci. Nanotechnol. 1, 013001 (2010).
Joshi, B. P. et al. Multimodal video colonoscope for targeted wide-field detection of nonpolypoid colorectal neoplasia. Gastroenterology 150, 1084–1086 (2016).
Chen, J. et al. Multiplexed endoscopic imaging of Barrett’s neoplasia using targeted fluorescent heptapeptides in a phase 1 proof-of-concept study. Gut 70, 1010–1013 (2021).
Chen, J. et al. Detection of Barrett’s neoplasia with a near-infrared fluorescent heterodimeric peptide. Endoscopy 54, 1198–1204 (2022).
Chang, T. S. et al. Flexible fiber cholangioscope for detection of near-infrared fluorescence. VideoGIE 8, 110–112 (2022).
Yelin, D. et al. Spectral-domain spectrally-encoded endoscopy. Opt. Express 15(5), 2432–2444 (2007).
Burggraaf, J. et al. Detection of colorectal polyps in humans using an intravenously administered fluorescent peptide targeted against c-Met. Nat. Med. 21(8), 955–961 (2015).
Duan, X. et al. Visualizing epithelial expression of EGFR in vivo with distal scanning side-viewing confocal endomicroscope. Sci. Rep. 6, 37315 (2016).
Turner, K. L. et al. Five parametric resonances in a microelectromechanical system. Nature 396, 149–152 (1998).
Li, H. et al. Integrated monolithic 3D MEMS scanner for switchable real-time vertical/horizontal cross-sectional imaging in dual axes confocal endomicroscope. Opt. Express 24, 2145–2155 (2016).
Li, H. et al. Large-displacement vertical electrostatic microactuator dynamics using duty-cycled softening/stiffening parametric resonance. J. Micromech. Microeng. 28, 351–361 (2019).
Li, H., Oldham, K. R. & Wang, T. D. 3 degree-of-freedom resonant scanner with full-circumferential range and large out-of-plane displacement. Opt. Express 27, 16296–16307 (2019).
Rakić, A. D. Algorithm for the determination of intrinsic optical constants of metal films: Application to aluminum. Appl. Optics 34, 4755–4767 (1995).
Hinoi, T. et al. Mouse model of colonic adenoma-carcinoma progression based on somatic Apc inactivation. Cancer Res. 67, 9721–9730 (2007).
Jaiswal, S. et al. Membrane bound peroxiredoxin-1 serves as a biomarker for in vivo detection of sessile serrated adenomas. Antioxid. Redox Signal. 36, 39–56 (2022).
Zhou, J. et al. EGFR overexpressed in colonic neoplasia can be detected on wide-field endoscopic imaging. Clin. Transl. Gastroenterol. 6, e102 (2015).
Joshi, B. P. et al. Design and synthesis of near-infrared peptide for in vivo molecular imaging of HER2. Bioconjugate Chem. 27, 481–494 (2016).
Li, G. et al. Ultra-compact microsystems-based confocal endomicroscope. IEEE Trans. Med. Imaging 39, 2406–2414 (2020).
Lee, M. et al. Confocal laser endomicroscope with distal MEMS scanner for real-time histopathology. Sci. Rep. 12, 20155 (2022).
Birla, M. et al. Image processing metrics for phase identification of a multiaxis MEMS scanner used in single pixel imaging. IEEE/ASME Trans. Mechatron. 26, 1445–1454 (2020).
Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9, 62–66 (1979).
Besag, J. Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B 36, 192–225 (1974).
Acknowledgements
This work was supported in part by the National Institutes of Health U01 EB028235 (TDW, KRO), R01 CA249851 (TDW), and the Jiangsu Industrial Technology Research Institute (JITRI).
Author information
Authors and Affiliations
Contributions
G.L., H.L., X.D. and T.D.W. designed the technology. G.L., M.L., T.-S.C., K.R.O. and T.D.W. conceived and designed the experiments. G.L., M.L., T.-S.C., J.Y., S.J. and S.F. performed the experiments. G.L., M.L. and T.-S.C. contributed to image and data analysis. M.L., T.-S.C. and T.D.W. wrote the manuscript. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Supplementary Video 1.
Supplementary Video 2.
Supplementary Video 3.
Supplementary Video 4.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, G., Lee, M., Chang, TS. et al. Wide-field endoscope accessory for multiplexed fluorescence imaging. Sci Rep 13, 19527 (2023). https://doi.org/10.1038/s41598-023-45955-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-45955-x
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
