Brain tumors are the most common solid tumors in children and are the leading cause of pediatric cancer mortality1. Extent of resection has been correlated with improved outcome in many types of pediatric and adult brain tumors2,3,4,5,6,7,8,9,10,11. However, maximal safe resection is often limited by an inability to distinguish tumor from normal brain. To address this problem, a variety of intraoperative imaging modalities have been investigated, including magnetic resonance imaging (MRI)12,13,14,15, wide-field fluorescence7, high-resolution fluorescence microscopy (confocal and multiphoton imaging)16, and optical coherence tomography (OCT)17,18,19,20,21.

Intraoperative MRI (iMRI) allows for the reliable identification of macroscopic residual tumor and has been shown to increase the extent of resection in glioma surgery12,13,14,15, however, iMRI has several limitations. iMRI systems are expensive and require the use of specialized non-ferromagnetic surgical equipment. The spatial resolution of MRI precludes the identification of small regions of residual tumor. Furthermore, the acquisition of iMRI is time consuming and requires a complete cessation of surgery for image acquisition.

Wide-field fluorescence guided brain tumor surgery, most notably with 5-aminolevulinic acid (5-ALA), has been shown to improve extent of resection for patients with glioblastoma and is easily incorporated into a standard surgical workflow7. However, wide-field fluorescence imaging relies on the presence of a disrupted blood brain barrier, and in the case of 5-ALA, specific biochemical alterations that are not present in all tumors22,23. Therefore, the utility of wide-field fluorescence guided brain tumor surgery outside of glioblastoma has been limited, especially in low grade glial tumors and pediatric tumors that may have an intact blood brain barrier24,25,26,27. Confocal microscopy allows for the imaging of 5-ALA fluorescence in some low-grade gliomas16. Additionally, label-free confocal and multiphoton microscopy systems are able to resolve individual axons and ex vivo brain tumor margins28,29,30,31,32. However the requirement for tissue contact, limited field of view, and limited depth of penetration inherent to confocal and multiphoton microscopy may limit their practical utility33.

OCT allows for rapid, wide-field, and label-free in vivo brain imaging with microscopic resolution and up to two millimeters of tissue penetration34,35,36. Additional features that make OCT particularly appealing for intraoperative use include the absence of photobleaching seen with fluorescence-based modalities, a safe low-energy infrared light source, and the ability to integrate with the existing optics of a standard neurosurgical operating microscope. Several studies have demonstrated the feasibility of building OCT systems into neurosurgical operating microscopes and endoscopes and the ability to successfully differentiate tumor from normal brain based on signal attenuation and the presence of gross structural features within the tumor18,19,20,21,37. Further work has established OCT attenuation thresholds that may allow for the rapid differentiation of tumor from cerebral white matter, though attenuation alone was not able to differentiate tumors from grey matter38,39. While promising, the effective resolution and image quality in these studies were significantly degraded by the presence of speckle noise that is intrinsic to OCT imaging, as well as to all coherent imaging methodologies, and arises from multiple scattering events within the light coherence time in a turbid sample, such as biological tissue.

In vivo neuroimaging studies with optical coherence microscopy (OCM), a closely related modality that utilizes high-magnification objective lenses in conjunction with OCT, indicate the image quality that can be obtained when speckle noise is eliminated from OCT imaging. The high magnification of OCM systems allows for the utilization of spatial averaging to eliminate speckle noise while still maintaining micrometer-scale resolution. After speckle removal, OCM is capable of in vivo imaging of individual axons and pyramidal cell bodies as well as the laminar architecture of cortical layers in mice and rats40,41,42. However, the narrow field of view, the need for contact of the imaging objective with the sample, and shallow depth of field, necessitate complex scanning of the image focal plane and extreme stabilization of the sample, thus limiting practical intraoperative use of OCM.

In this paper, we introduce a novel method for label-free in vivo neuroimaging and tumor margin detection using Speckle-Modulating OCT (SM-OCT)43. SM-OCT is able to effectively remove speckle noise by acquiring and averaging an unlimited number of frames with uncorrelated speckle patterns. Modulating the speckle pattern is achieved by introducing local variations in the phases of the incoming light, for example by placing a ground glass diffuser in the optical path (Fig. 1a) and rotating it so that each frame is acquired with the light passing through different locations of the diffuser. SM-OCT allows for a significant reduction of speckle noise within OCT images, effectively increasing the ability of standard OCT systems to visualize fine anatomical structures, so that it approaches the optical limit defined by the objective lens, while maintaining the depth of field and the field of view. Unlike SM-OCT, previously described speckle reduction methods, such as spatial44 and angular45 compounding are constrained in the amount of uncorrelated speckle patterns that they can acquire, which limits speckle noise removal. Various digital image processing methods46,47,48,49 have also been described for speckle noise reduction, however, these methods merely reduce the appearance of the noise, rather than reveal information that was lost due to it. A detailed description of SM-OCT, which includes comparisons to other speckle reduction methods as well as an analysis of speckle removal as a function of the number of averaged frames can be found in a previous publication43.

In this manuscript, we demonstrate that SM-OCT is capable of high-resolution structural imaging of the mouse brain, which is unprecedented for wide-field in vivo imaging. Specifically, individual white matter fascicles of the cingulum bundle, corpus callosum and alveolus of the hippocampus are clearly visible across a wide field of view using only the endogenous contrast of neural tissue. Fine structures of the brain including myelinated axons and cortical layers are also clearly visualized. in vivo SM-OCT imaging of mice with orthotopic glioblastoma xenografts clearly and precisely identifies tumor margins. Finally, when applied to ex vivo human samples, SM-OCT allows for label-free delineation of the brain tumor margin of a low-grade glioma. Our work suggests that fast, wide field of view, high-resolution in vivo optical imaging using SM-OCT has the potential to significantly advance the care of neurosurgical patients and expand the capabilities of small animal neuroimaging.


SM-OCT in vivo Neuroimaging

Anesthetized BL6 mice with glass cranial windows underwent wide-field brain imaging with OCT and SM-OCT (Figs 1 and 2). Images were acquired using a commercial OCT (Telesto, ThorLabs, Newton, NJ) which was adapted to perform speckle-modulation (SM-OCT)43 for the removal of speckle noise. The spectral domain OCT system, with a center wavelength of 1300 nm and 170 nm bandwidth, provides an axial resolution of 3.7 μm in tissue and a lateral resolution of 9 μm (FWHM). OCT imaging penetrates brain tissue to a depth of 1.5 mm and can resolve the gross structure of the hippocampus and corpus callosum, however speckle noise precludes visualization of further structural detail (Figs 1b, 2a,d and S2). SM-OCT significantly reduces speckle noise, revealing the fine anatomical structure of the brain including cortical layers (Fig. 1) and white matter fascicles (Fig. 2).

Figure 1
figure 1

SM-OCT imaging of mouse brain in vivo reveals cortical layers. (a) The sample arm the SM-OCT system, L1 is the main lens of the OCT, the diffuser is rotated in the focal plane, which is relayed by two lenses, L2, in a 4f configuration. (b) OCT B-scan of mouse cortex. (c) SM-OCT B-scan of mouse cortex, showing the cortical layers, which are revealed by removing the speckle noise. (d) Histology of mouse brain (image credit: Allen Institute)62, showing the corresponding cortical layers to SM-OCT.

Figure 2
figure 2

SM-OCT reveals white matter fascicles in mice in vivo. (a) OCT axial view of mouse cortex, depth is shown as yellow dashed line in (d). (b) SM-OCT image of the region showed in (a). The removal of speckle reveals white matter structures, including the cingulum bundle. (c) The white matter structures shown by SM-OCT can be enhanced by image processing. (d) OCT coronal view of mouse cortex, the location is shown as yellow dashed line in (a). SM-OCT image of region shown in (d), revealing the white matter structures in high-resolution. (f) A close-up axial view of the cingulum bundle. (g,h) close-up coronal views of white matter structures of various sizes including the tracts of the cingulum bundle and very small unnamed fascicles. (i) close-up coronal view with manual segmentation of several fibers of cingulum bundle (blue) and very small unnamed fascicle (yellow).

Mammalian cortex has four to six distinct layers that are differentiated both histologically and functionally. SM-OCT allows for clear visualization of the architecture of the cortical layers (Fig. 1c). The underlying signal characteristics of mouse brain imaged with SM-OCT demonstrate significant noise reduction compared to the OCT image, with distinct variability between the signal intensity among the different cortical layers allowing for their visualization (Supplementary Fig. 1). The signal differences between layers are consistent with prior literature from OCM and are explained by differences in the density of cells and neuritic processes between layers, with more cell-body dense layers such as layer two being darker and more axon dense layers such as layer one being brighter40,41,42.

Small individual white matter fascicles of the cingulum bundle, corpus callosum, and alveolus of the hippocampus are clearly visualized in vivo for the first time with optical imaging using SM-OCT (Fig. 2b,e–h, Supplementary Fig. 2b, Movies 13). These fascicles are approximately 40–80 um in diameter (Fig. 2f,g), and cannot be resolved using OCT (Supplementary Fig. 2a, Movie 3). To our knowledge, these structures have not been directly visualized in situ in live animals prior to this study. Three-dimensional imaging of these structures allows for them to be visualized in multiple planes, for example individual fascicles of the cingulum bundle are seen as long tubes in the axial plane (Fig. 2b,f, Supplementary Fig. 2b), and as circles in the coronal cross section (Fig. 2e,g,h and Supplementary Fig. S2d). These structures may also be manually segmented (Fig. 2i).

Closer examination of coronal plane SM-OCT images reveals even smaller structures, whose size (6–18 μm) and location are consistent with individual myelinated axons or very small fascicles consisting of a small number of myelinated axons (Fig. 2h). These structures can also be seen coursing from the medial cortex into the cingulum bundle in the axial plane (Supplementary Movie 2). These structures cannot be seen with OCT due to speckle noise (Supplementary Fig. 2c and Movie 3). As a wide-field in vivo neuroimaging platform, SM-OCT can resolve a wide range of tissue features from cortical layers to small white matter fascicles up to 1.5 mm deep within tissue using only endogenous contrast.

Unenhanced SM-OCT neuroimaging allows for exquisite in vivo imaging of the murine brain. However, SM-OCT data also lends itself to digital enhancement. To achieve this, we applied vessel shadow removal, anisotropic diffusion denoising50, and contrast enhancement to the SM-OCT volumes (Fig. 2c and Supplementary Movie 4). This enhancement allows improved visualization of the axonal and white matter architecture by enhancing edges and contrast.

High-resolution in vivo imaging of orthotopic murine xenograft brain tumors

We next applied SM-OCT to an orthotopic glioblastoma model. Nude mice were implanted with U87 glioblastoma cells and prepared with glass cranial windows. The mice were imaged with SM-OCT on post-implantation day seven (Fig. 3 and Supplementary Movies 57). The wide field of view achievable with SM-OCT allows for the entire tumor and surrounding brain tissue to be imaged. SM-OCT clearly differentiates tumor from normal brain based on signal intensity (Fig. 3). The undisturbed regions of the brain are rich in highly reflective myelin and appear bright, while the tumor appears dark because it lacks myelinated structures and includes mainly cell bodies that are significantly less reflective40,41,42. The high resolution of SM-OCT allows for precise delineation of the brain tumor margin to within approximately 10 μm, including visualization of the finger-like projections of the tumor growing along axonal projections (Fig. 3e). In the OCT images (Supplementary Fig. 3), the tumor margin is less visible due to speckle noise.

Figure 3
figure 3

SM-OCT reveals high-resolution features of tumor margin in vivo. (a) SM-OCT ortho-slice of the tumor volume, showing the different sections in three dimensions. (b) SM-OCT axial view of mouse cortex with a GBM tumor, depth is shown as yellow dashed line in (c,d). (c,d) SM-OCT coronal and sagittal views, respectively, showing the tumor margin, the locations are shown as yellow dashed lines in (b). (e) A close-up view of the tumor margin in (d), showing the finger-like invasion of the into the surrounding brain tissue.

Imaging of human brain ex vivo

Given the promising results of SM-OCT in murine neuroimaging, we investigated whether similar results could be obtained in human brain tissue. A fresh, unfixed section of the inferior temporal gyrus from a temporal lobectomy was taken directly from the operating room to our laboratory for imaging with OCT and SM-OCT. Similar to in vivo murine imaging, OCT was able to image human brain tissue to a depth of greater than 1.5 mm. However, speckle noise in the OCT image hinders the ability to resolve cortical layers or axonal processes (Fig. 4a). SM-OCT clearly resolves the first three cortical layers as well as individual myelinated axonal processes, primarily in layers one and three (Fig. 4b,d,e). Axons exist in all cortical layers. However, since SM-OCT is dependent on reflected light, axons oriented perpendicular to the incident beam, such as those in layers one and three provide significantly greater signal, allowing them to be resolved with this method.

Figure 4
figure 4

SM-OCT of ex vivo human brain sample reveals cortical layers and axons. (a,b) OCT and SM-OCT B-scans of cortex. The SM-OCT image reveals cortical layers and myelinated axonal projections. (c) Corresponding histology (image credit: Allen Institute)63. (d,e) A close-up view of the myelinated axons shown in (b). The contrast in e is enhanced to highlight the myelinated axons.

Imaging of human low-grade glioma margins ex vivo

Encouraged by the capabilities demonstrated by SM-OCT with in vivo murine and ex vivo human brain imaging, we sought to apply SM-OCT to the substantial clinical problem of tumor margin detection in low-grade gliomas. A child with a low-grade glioma and epileptic foci extending beyond the tumor underwent a planned supratotal resection of the tumor and adjacent epileptic regions. A fresh, unfixed sample that contained the presumed tumor margin based on stereotactic image guidance was taken directly from the operating room to our laboratory for imaging with SM-OCT and was subsequently fixed for histological processing. Alignment of the sample was preserved throughout imaging and histologic processing. Histologic sections were aligned with b-scans from the three-dimensional SM-OCT volume based on vessel anatomy (Fig. 5a–c). The tumor tissue in the SM-OCT image demonstrates lower intensity signal compared to the brain tissue as well as a loss of normal structures, such as layer one axons, seen in the normal brain (Fig. 5a,b). The subtle tumor margin seen in the histologic sample (Fig. 5a) is recapitulated in the SM-OCT image (Fig. 5b) and rendered quite clear with by stretching the brightness and contrast of the SM-OCT image (Fig. 5c). When the same sample is visualized from an axial/top down plane, following the flattening of the volume, several important features are apparent. Individual myelinated axonal processes in layer one of the healthy brain are clearly visualized on the left (Fig. 5d–f), while on the right side of the image, layer one axons are absent and have been replaced by the tumor (Fig. 5d,f). Thus, the presence or absence of layer one axons provides an additional means by which SM-OCT can resolve the tumor margin of this low-grade glioma with high spatial resolution across a field of view of several millimeters.

Figure 5
figure 5

ex vivo human LGG tumor margin visualized with histology and SM-OCT. (a) Histology shows the difference in cell density and arrangement in the brain (sparse and organized) and tumor (dense and disorganized) regions, the tumor margin can be roughly estimated and is shown by the blue dashed line. (b) SM-OCT B-scan of the brain sample, corresponding to (a), showing the tumor as having reduced signal intensity compared to the normal brain, likely due to the reduced optical scattering of the non-myelinated tumor cells. (c) The image in (b) after contrast-enhancement, emphasizing the difference in signal intensities between the tumor and brain regions. (d) SM-OCT axial view (en face) of the tumor margin, demonstrating invasion of layer 1 of the cortex by the glioma. The image shows the axonal projections of layer 1 at the left of the tumor margin and the lack of them at the region of the tumor. (e) A close-up view on the axonal projections of layer 1 of cortex. The thickness of the measured myelinated axons is limited by the optical resolution of the OCT. (f) A close-up view on the tumor margin, highlighted by the lack of axons and lower signal intensity in the tumor.


In this proof-of concept paper we demonstrate that by significantly reducing speckle noise, SM-OCT can address fundamental shortcomings of current in vivo neuroimaging modalities used for small animal neuroscience and intraoperative tumor margin detection. Specifically, SM-OCT can resolve brain tumor margins and neural structures ranging from individual axons to white matter fascicles and cortical layers across a wide field of view of several millimeters without the use of exogenous contrast agents. To our knowledge, there are no other imaging modalities that currently occupy this crucial space in in vivo imaging, and accordingly SM-OCT neuroimaging lends itself to a myriad of scientific and clinical uses.

SM-OCT could allow for serial in vivo imaging in murine models of neurologic disease such as Alzheimer’s disease, epilepsy, and stroke. A major advantage of serial imaging is that it allows for the disease process within a single mouse to be tracked over time, in response to treatments, and across different stages of disease. The resolution provided by SM-OCT may yield new insights into neurologic disease processes, as current murine studies that utilize serial in vivo imaging rely on MRI or PET, which have inherent resolution limitations. The design of these studies would need to be mindful of the limitations of SM-OCT, primarily the limited depth of penetration of 1–1.5 mm and a relative difficulty in imaging anisotropic structures oriented parallel to the incident beam due to their reduced reflectivity.

The ability of SM-OCT to resolve brain tumor margins with high resolution has exciting applications for intraoperative use. There are currently no intraoperative imaging modalities capable of differentiating certain tumor types, such as low-grade gliomas, from normal brain, and the ability of SM-OCT to accomplish this without the use of exogenous contrast is likely to be of significant value to neurosurgeons and patients with neurosurgical diseases. While this is only demonstrated in a single patient sample in this study due to the rarity of intact human low grade glioma samples spanning the tumor margin, we plan on exploring this topic further in future work. The SM-OCT setup described in this paper is simple, robust, and can likely be integrated with the optics of a standard operating microscope with relatively minor modifications. Alternatively, SM-OCT could be used on ex vivo samples taken during an operation, similarly to what was done in this paper, to provide information comparable to a frozen section, the current gold standard for intraoperative tissue diagnosis.

The implementation of SM-OCT in this manuscript is based on a commercial OCT system with the addition of a moving diffuser and two relay lenses to create uncorrelated speckle patterns, that when averaged, are able to significantly reduce speckle noise. This demonstrates that SM-OCT is a relatively low-cost and simple modification to existing OCT systems. The implementation of SM-OCT herein utilizes a ground glass rotating diffuser to scramble the phase of the incoming light, although it is very likely that this effect can be achieved by other optical components (for example, a spatial light modulator). Following the basic properties of frame compounding, the reduction of speckle noise in SM-OCT is proportional to the square root of the number of averaged frames43,51. In the current implementation, the additional frame acquisition which is needed to reduce speckle noise and reveal fine details, extends the total imaging time, however, as we have demonstrated, we do not expect the averaging requirement to limit SM-OCT imaging of brain tissue. It is clear that the benefits of the additional detail outweigh the extended acquisition time, especially for brain tissue which is static in its nature. As faster OCT systems are continuously being developed, we expect that the additional acquisition time would not limit other applications which could be more time sensitive. Furthermore, systems such as interleaved OCT52, could achieve image compounding without extending the acquisition time53.

OCT technology continues to develop rapidly and future iterations of SM-OCT will likely be able to take advantage of advances made across the field. For example, the SM-OCT system used in this paper can scan up to a relatively fast rate of 91khz. However, there are currently swept source OCT systems that are capable of scanning at rates more than an order of magnitude faster54. These systems could achieve volumetric video-rate imaging and have the potential to enable surgeries that are done under continuous OCT visualization55. Additionally, swept source systems often utilize higher powered light sources that could increase the imaging depth in tissue35. Polarization sensitive OCT has been utilized in ex vivo analysis of thin slices of brain tissue and provides high-resolution visualization of white matter structures owing to their variable directionality56,57,58. The modification of SM-OCT for swept source and polarization sensitive OCT systems and other varieties of OCT beyond simple spectral domain OCT represents an interesting future area of investigation.

In future experiments, we plan to utilize SM-OCT in the operating room to further investigate the role of this technology for intraoperative use. The ability to resolve cortical layer architecture and patterns of axonal arborization may allow for uses beyond tumor margin detection, such as identification of areas of cortical dysplasia to improve the accuracy of epilepsy surgery. The human tissue experiments presented in this paper attempted to mirror intraoperative conditions as much as possible by utilizing fresh, unfixed tissues taken immediately from the operating room to the lab for imaging. As such, we are hopeful that the results presented in this paper will be reproduced in future intraoperative experiments.


Cranial window preparation and orthotopic brain tumor animal models

The animal experiments were carried out in accordance with Stanford University Institutional Animal Care and Use Committee guidelines under APLAC protocols 26548 and 27499. Female athymic nude mice (Foxn1nu−/nu, Charles River Labs) were anesthetized with ketamine/xylazine (100/10 mg/kg, intramuscular injection) and placed in a stereotactic frame. The scalp and underlying soft tissue over the parietal cortex were removed bilaterally. A drill was used to create a rectangular cranial window centered on the midsagittal suture that extended from the bregma to the lambdoid sutures. U87 human xenografts were implanted (10 μL of 1 × 106 cells/ml cell suspension). Following implantation, a 5.5 mm × 7.5 mm × 3 mm glass cranial window was glued to the bone surrounding the cranial window with cyanoacrylate. Female BL6 mice were prepared using the same protocol, but without the injection of tumor cells.

Imaging system

All OCT images were acquired using a commercial OCT which was adapted to perform speckle-modulation (SM-OCT)43 for the removal of speckle noise. The commercial OCT was a spectral-domain system (Telesto, ThorLabs, Newton, NJ), with a center wavelength of 1300 nm and 170 nm bandwidth, which provides an axial resolution of 3.7 μm in tissue. The spectrometer of the OCT acquires 1024 samples for each A-scan at a rate of 28 kHz. Speckle modulation was implemented by inserting a rotating ground glass diffuser at a conjugate image plane. The main lens of the OCT provides a lateral resolution of 9 μm (FWHM) and depth of field (DOF) of 270 μm in water (LSM03, ThorLabs, Newton, NJ). The ground glass diffuser was custom made by lapping a 3 mm thick glass window with AR coating at the wavelength range of the OCT (ThorLabs, Newton, NJ). The window was ground by lapping with 5 μm aluminum oxide grit (Universal Photonics, Central Islip, NY) for 15 minutes. The diffuser was rotated using an electromechanical mount (RSC-103, Pacific Laser Equipment, Santa Ana, CA). The optic axis was aligned with a radial offset relative to the center of the diffuser such that rotation ensured changing speckle patterns in the incident beam. The image plane was replicated by two lenses (LSM02, ThorLabs, Newton, NJ) in a 4 F configuration. Traditional OCT volumes were acquired in addition to SM-OCT volumes, by removing the diffuser from the optical axis.

Imaging protocol

BL6 mice were imaged at least seven days after the implantation of the cranial window. Nude mice were imaged seven days after tumor-cell and window implantation. The mice were anesthetized with 2% isoflurane and placed in a stereotactic frame.

SM-OCT and OCT 3D volumes were acquired with 6 μm lateral spacing and 18 to 40 averages with an A-scan rate of 28 kHz. For SM-OCT acquisition, the diffuser was rotated at a tangential speed of ~2 mm/s, which created non-correlated speckle patterns. 2D B-scans (Figs 1 and 4) were acquired with 4 μm lateral spacing and 100 averages with an A-scan rate of 28 kHz.

OCT and SM-OCT post-processing

All processing and analysis were performed with Matlab (MathWorks, Natick, MA). Visualization of volumes was done using ImageJ59.

The structure of the tissues was obtained by a conventional OCT reconstruction algorithm with custom dispersion compensation60,61. SM-OCT provides frames with uncorrelated speckle patterns, which were intensity-averaged in linear-scale. The images and volumes were uniformly re-sampled to obtain a cube-shaped voxel with 4 μm sides.

The volume of mouse brain (Fig. 2c, Supplementary Movie 4) was further post-processed to enhance the visibility of the white-matter fascicles. First, the low signal of the blood vessels and their shadows was increased to the mid-range of the signal-scale. This was done by applying a compensating gain to the axial (en face) image so that the average signal along the depth dimension is uniform along the field of view. Next, three-dimensional anisotropic diffusion50 was applied to the volume. Last, the contrast was enhanced using histogram equalization.

Histology images

Histology images of mouse and human brain in Figs 1 and 4, respectively, were obtained from the Allen Institute website. The image of mouse cortical layers is found in:

The image of human cortical layers is found in:

The human tumor margin sample in Fig. 5 was fixed in formalin, paraffin embedded and stained with hematoxylin and eosin with care taken to preserve the orientation in which the sample was imaged with OCT. The tissue slides were then imaged with a bright-field microscope. All human tissue was obtained following IRB exemption for this study and was performed under the auspices of the existing Stanford Brain Bank IRB protocol.