Conventional methods for intraoperative histopathologic diagnosis are labour- and time-intensive, and may delay decision-making during brain-tumour surgery. Stimulated Raman scattering (SRS) microscopy, a label-free optical process, has been shown to rapidly detect brain-tumour infiltration in fresh, unprocessed human tissues. Here, we demonstrate the first application of SRS microscopy in the operating room using a portable fibre-laser-based microscope and unprocessed specimens from 101 neurosurgical patients. We also introduce an image-processing method—stimulated Raman histology (SRH)—that leverages SRS images to create virtual haematoxylin-and-eosin-stained slides, revealing essential diagnostic features. In a simulation of intraoperative pathologic consultation in 30 patients, we found a remarkable concordance of SRH and conventional histology for predicting diagnosis (Cohen’s kappa, κ > 0.89), with accuracy exceeding 92%. We also built and validated a multilayer perceptron based on quantified SRH image attributes that predicts brain-tumour subtype with 90% accuracy. Our findings provide insight into how SRH can now be used to improve the surgical care of brain-tumour patients.
The optimal surgical management of brain tumours varies widely depending on histologic subtype. Although some tumours of the central nervous system (CNS) have a distinct gross appearance, others are difficult to differentiate. Consequently, the importance of intraoperative histopathologic diagnosis in brain-tumour surgery has been recognized for over 85 years 1 .
Existing intraoperative histologic techniques, including frozen sectioning and cytologic preparations, require skilled technicians and clinicians working in surgical pathology laboratories to produce and interpret slides 2 . However, the number of centres where brain-tumour surgery is performed exceeds the number of board-certified neuropathologists, eliminating the possibility for expert intraoperative consultation in many cases. Even in the most advanced, well-staffed hospitals, turnaround time for intraoperative pathology reporting may delay clinical decision-making during surgery, highlighting the need for an improved system for intraoperative histopathology.
The ideal system for intraoperative histopathology would deliver rapid, standardized and accurate diagnostic images to assist in surgical decision-making. Improved access to intraoperative histologic data would enable examination of clinically relevant histologic variations within a tumour and the assessment of the resection cavity for residual tumour. In addition, given that the percentage of tumour removed at the time of surgery is a major prognostic factor for brain-tumour patients 3 , intraoperative techniques to accurately identify residual tumour are essential.
The development of stimulated Raman scattering (SRS) microscopy in 2008 created the possibility of rapid, label-free, high-resolution microscopic imaging of unprocessed tissue specimens
. Although SRS has been shown to reveal key diagnostic histologic features in brain-tumour specimens
However, leveraging advances in fibre-laser technology 8 , we have engineered a clinical SRS microscope, allowing us to execute SRS microscopy in a patient care setting. Light guidance by the optical core of the fibre and the unique polarization-maintaining implementation of the laser source have enabled service-free operation in our operating room for over a year. The system also includes improved noise cancellation electronics for the suppression of high relative intensity noise, one of the major challenges of executing fibre-laser-based SRS microscopy.
Using this system, we show that SRS microscopy can serve as an effective, streamlined alternative to traditional histologic methods, eliminating the need to transfer specimens out of the operating room to a pathology laboratory for sectioning, mounting, dyeing and interpretation. Moreover, because tissue preparation for SRS microscopy is minimal, key tissue architectural details commonly lost in smear preparations and cytologic features often obscured in frozen sections are preserved. We also report a unique method for SRS image processing that simulates haematoxylin and eosin (H&E) staining, called stimulated Raman histology (SRH), which highlights key histoarchitectural features of brain tumours and enables diagnosis in near-perfect agreement with conventional H&E-based techniques. Finally, we demonstrate how a supervised machine-learning approach, based on quantified SRH image attributes, effectively differentiates among diagnostic classes of brain tumours. Our study demonstrates that SRH may provide an automated, standardized method for intraoperative histopathology that can be leveraged to improve the surgical care of brain tumours in the future.
Engineering a clinical SRS microscope
To eliminate reliance on optical hardware incompatible with the execution of SRS microscopy in an operating room, we created a fully integrated imaging system with five major components: (1) a fibre-coupled microscope with a motorized stage; (2) a dual-wavelength fibre-laser module; (3) a laser control module; (4) a microscope control module; and (5) a computer for image acquisition, display and processing. The entire system is mounted in a portable, self-contained clinical cart, utilizes a standard wall plug and does not require water-cooling (Fig. 1a).
The dual-wavelength fibre-laser is based on the fact that the difference frequency of the two major fibre gain media, erbium and ytterbium, overlaps with the high wavenumber region of Raman spectra. As previously described 8 , the two synchronized narrow-band laser pulse trains required for SRS imaging are generated by narrow-band filtering of a broad-band super-continuum derived from a single fibre-oscillator and, subsequently, amplification in the respective gain media (Fig. 1b).
For clinical implementation, we developed an all-fibre system based on polarization-maintaining components, which greatly improved stability over the previous non-polarization-maintaining system. The system described here was stable throughout transcontinental shipping (from California to Michigan), and continuous, service-free, long-term (>1 yr) operation in a clinical environment, without the need for realignment. To enable high-speed diagnostic-quality imaging (1 megapixel in 2 seconds per wavelength) with a signal-to-noise ratio comparable to what can be achieved with solid-state lasers, we scaled the laser output power to approximately 120 mW for the fixed wavelength 790 nm pump beam and approximately 150 mW for the tunable Stokes beam over the entire tuning range from 1,010 to 1,040 nm at 40 MHz repetition rate and 2 ps transform-limited pulse duration. We also developed fully custom laser controller electronics to tightly control the many settings of this multi-stage laser system based on a micro-controller. Once assembled, we determined that the SRS microscope had a lateral resolution of 360 nm (full width of half maximum) and axial resolution of 1.8 μm (Supplementary Fig. 1).
Although development of an all-fibre system was necessary for clinical implementation of SRS, relative intensity noise intrinsic to fibre lasers vastly degrades SRS image quality (Fig. 1c). To improve image quality, we developed a noise-cancellation scheme based on auto-balanced detection 8 , in which a portion of the laser beam is sampled to provide a measure of the laser noise that can then be subtracted in real time. Here, we demonstrate that we can achieve approximately 25-fold improvement in the signal-to-noise ratio in a clinical setting, without the need for adjustment, which is essential for revealing microscopic tissue architecture (Fig. 1d).
Processing of clinical SRS images
Histologic images of fresh, unstained surgical specimens are created with the clinical SRS microscope by mapping two Raman shifts: 2,845 cm−1, which corresponds to CH2 bonds that are abundant in lipids (Fig. 2a), and 2,930 cm−1, which corresponds to CH3 bonds that predominate in proteins and DNA (Fig. 2b). Assigning a subtracted CH3–CH2 image (Fig. 2c) to a blue channel and assigning the CH2 image to the green channel results in an image with contrast that is suitable for brain-tumour detection (Fig. 2d)
. However, given the ultimate goal of creating an imaging system that produces histologic images that are familiar to clinicians
Detection of diagnostic histologic features with SRH
We assessed the ability of SRH to reveal the diagnostic features required to detect and classify tumours of the CNS by imaging fresh surgical specimens from 101 neurosurgical patients (Supplementary Table 1) via an institutional review board (IRB)-approved protocol (University of Michigan IRB HUM00083059). Like conventional H&E images, SRH images reveal the cellular and architectural features that permit differentiation of non-lesional (Fig. 3a–c) and lesional (Fig. 3d–i) tissues. When imaged with SRH, architecturally normal brain tissue from anterior temporal lobectomy patients (patients 6, 11 and 93) demonstrates neurons with angular cell bodies containing lipofuscin granules (Fig. 3a) and lipid-rich axons that appear as white linear structures (Fig. 3a,b). Non-neoplastic reactive changes, including gliosis (Fig. 3b) and macrophage infiltration (Fig. 3c), that may complicate intraoperative diagnosis are also readily visualized with SRH. Key differences in cellularity, vascular pattern and nuclear architecture that distinguish low-grade (Fig. 3d; patient 3) from high-grade (Fig. 3e,f; patient 21) gliomas are apparent as well. Notably, SRH suggests that the perinuclear halos of oligodendroglioma cells (Fig. 3d), not typically seen on frozen sections and thought to be an artefact of fixation 13 , are reflective of abundant protein-rich tumour cell cytoplasm. In addition, by highlighting the protein-rich basement membrane of blood vessels, SRH is well-suited for highlighting microvascular proliferation in high-grade glioma (Fig. 3f; patient 37).
SRH also reveals the histoarchitectural features that enable diagnosis of tumours of non-glial origin (Fig. 3g–i), including the whorled architecture of meningiomas (Fig. 3g; patient 26), the discohesive monomorphic cells of lymphoma (Fig. 3h; patient 31) and the glandular architecture, large epithelioid cells and sharp borders of metastatic adenocarcinoma (Fig. 3i; patient 57). SRH is also capable of visualizing morphologic features that are essential in differentiating the three most common paediatric posterior fossa tumours—juvenile pilocytic astrocytoma, medulloblastoma and ependymoma—each of which have divergent goals for surgical management 14 . In pilocytic astrocytomas, SRH detects piloid (hair-like) architecture and Rosenthal fibres, which appear dark on SRH due to their high protein content (Supplementary Fig. 2a; patient 98). SRH also reveals the markedly hypercellular, small, round, blue cell appearance and rosettes in medulloblastoma (Supplementary Fig. 2b; patient 101), as well as the monomorphic round-to-oval cells forming perivascular pseudorosettes in ependymoma (Supplementary Fig. 2c; patient 87).
Detection of intratumoural heterogeneity with SRH
Gliomas often harbour histologic heterogeneity, which complicates diagnosis and treatment selection. Heterogeneity is particularly common in low-grade gliomas suspected of having undergone malignant progression and demonstration of anaplastic transformation is essential for making a diagnosis. SRH was successful in detecting heterogeneity of tumour grade within a specimen collected from a patient with a recurrent oligodendroglioma of the right frontal cortex. In that specimen, SRH revealed both low-grade architecture and areas of high-grade architecture characterized by hypercellular, anaplastic and mitotically active tumour (Fig. 4a; patient 41).
In other tumours, such as mixed glioneuronal tumours, histologic heterogeneity is a necessary criterion for diagnosis: although any single histopathologic sample may reveal glial or neuronal architecture, the identification of both is necessary for diagnosis. In a patient with suspected ganglioglioma, a glioneuronal tumour, intraoperative SRH images of a superficial specimen (Fig. 4b; patient 96) revealed clustered dysplastic neurons, while a deep specimen revealed hypercellular piloid glial architecture. Consequently, by providing a rapid means of imaging multiple specimens, SRH reveals intratumoural heterogeneity needed to establish clinically relevant variations in both grade and histoarchitecture during surgery.
Quantitative evaluation of SRH-based diagnosis
Given its ability to reveal diagnostic histologic features, we hypothesized that SRH could provide an alternative to existing methods of intraoperative diagnosis. To test this hypothesis, we imaged specimens from 30 neurosurgical patients where intraoperative diagnosis was rendered using routine frozen sectioning or cytological techniques (Supplementary Table 1; patients 72–101). Adjacent portions of the same specimens were utilized for both routine histology and SRH.
To simulate the practice of intraoperative histologic diagnosis, a computer-based survey was created, in which three board-certified neuropathologists (K.A.M., S.R. and M.S.), each practising at different institutions, were presented with SRH or routine (smear and/or frozen) images (Fig. 5), along with a brief clinical history regarding the patient’s age group (child or adult), lesion location and relevant past medical history. The neuropathologists responded with an intraoperative diagnosis for each case the way they would in their own clinical practices. Responses were graded based on: (1) whether tissue was classified as lesional or non-lesional; (2) for lesional tissues, whether they had a glial or non-glial origin; and (3) whether the response contained the same amount of diagnostic information (lesional status, grade, histologic subtype) as the official clinical intraoperative diagnosis.
Assessing the pathologists’ diagnostic performance when utilizing SRH versus clinical frozen sections revealed near-perfect concordance (Cohen’s kappa) between the two histologic methods for distinguishing lesional and non-lesional tissues (κ = 0.84–1.00) (Table 1) and for distinguishing lesions of glial origin from non-glial origin (κ = 0.93–1.00) (Table 1). There was also near-perfect concordance between the two modalities in predicting the final diagnosis (κ = 0.89–0.92) (Table 1). Reliability among reviewers and concordance between SRH and standard H&E-based techniques for predicting diagnosis was also nearly perfect (κ = 0.89–0.92). Notably, with SRH, the pathologists were highly accurate in distinguishing lesional from non-lesional tissues (98%) and glial from non-glial tumours (100%), and in predicting diagnosis (92.2%). These findings suggest that pathologists’ ability to derive histopathologic diagnoses from SRH images is both accurate and highly concordant with traditional histologic methods.
Although both methods were highly accurate in predicting diagnosis, six of the SRH-based diagnostic discrepancies occurred in the classification of glial tumours (Table 1, Fig. 5c and Supplementary Fig. 3a). In three separate instances, pathologists were able to correctly identify a specimen as being glioma, but did not provide a specific grade. Two specimens classified as ‘glioma’ with SRH were classified as ‘high-grade glioma’ with H&E-based techniques. High-grade features in gliomas include: significant nuclear atypia, mitotic activity, microvascular proliferation and necrosis. Assessment of nuclear atypia and mitotic figures is subjective and requires ample expertise based on review of hundreds of cases to set up a threshold of ‘normal’ versus atypical morphology in a specimen. Given the subtle difference in appearance of nuclear architecture in H&E and SRH, pathologists may have been more conservative in terms of rendering atypical and mitotic attributions to tumour cells with SRH.
Differences in tissue preparation between conventional techniques (that is, sectioning) and SRH (that is, gentle squash) result in differences in the appearance of vascular architecture. Microvascular proliferation is defined as intraluminal endothelial proliferation (several layers of endothelial cells in a given vessel) and is essential in grading gliomas at the time of intraoperative consultation. This can be easier to observe when tissue is sectioned and analysed in two dimensions (Supplementary Fig. 3a). In contrast, although SRH is able to highlight basement membranes nicely, in some cases, it does not reveal the classic architectural features of microvascular proliferation (Supplementary Fig. 3c).
Undersampling from specimens may have also contributed to the discrepancies observed. In three survey items, pathologists misdiagnosed ependymoma as ‘pilocytic astrocytoma’ or gave a more general description of the tumour as ‘low-grade glioma’ using SRH images (Supplementary Fig. 3a). Ependymomas and pilocytic astrocytomas may have similar nuclear morphology of monotonous elongated nuclei embedded in a background composed of thin glial processes (piloid-like). In the absence of obvious perivascular pseudorosettes, ependymal rosettes or hyalinized vessels, which were not obvious in the survey items, and may be unevenly distributed throughout a tumour, it is understandable that an ependymoma could be misclassified as a pilocytic astrocytoma. Given the concordance of SRH images with traditional H&E images in our patients, we hypothesize that these errors might have been avoided if larger specimens were provided to reviewers.
Machine learning-based tissue diagnosis
Intraoperative image data that are most useful for clinical decision-making are rapidly obtained and accurate. Interpretation of histopathologic images by pathologists is labour- and time-intensive and prone to inter-observer variability. Consequently, a system rapidly delivering prompt, consistent and accurate tissue diagnoses would be greatly helpful during brain-tumour surgery. Although we have previously shown that tumour infiltration can be predicted by quantitative analysis of tissue attributes in SRS images 6 , we hypothesized that more robust computational processing would be required to predict tumour diagnostic class.
We employed a machine-learning process called a multilayer perceptron (MLP) for diagnostic prediction because it is easy to iterate, easy to verify and efficient with current computational power. To create the MLP, we incorporated 12,879, 400 μm × 400 μm SRH FOVs from our series of 101 patients. We used WND-CHRM, an open-source image classification programme that calculates 2,919 image attributes for machine learning 15 to assign quantified attributes to each FOV. Normalized quantified image attributes were fed into the MLP for training, iterating until the difference between the predicted and observed diagnoses was minimized (see Methods).
To test the accuracy of the MLP, we used a leave-one-out approach, wherein the training set contained all FOVs except those from the patient being tested. This method maximizes the size of the training set and eliminates possible correlation between samples in the training and test sets. The MLP makes predictions on an individual FOV level, yielding probabilities that a given FOV belongs to one of the four diagnostic classes: non-lesional, low-grade glial, high-grade glial or non-glial tumour (including metastases, meningioma, lymphoma and medulloblastoma) (Fig. 6a). The four diagnostic classes were selected because they provide critical information for informing decision-making during brain-tumour surgery.
Given the histoarchitectural heterogeneity of CNS tumours and the fact that some specimens may contain a mixture of normal and lesional FOVs, we judged the diagnostic accuracy of the MLP based on the most common or modal-predicted diagnostic class of FOVs within each specimen (Fig. 6b). For example, although the specimen from patient 87 exhibited some features of all diagnostic classes in various SRH FOVs (Fig. 6a), the MLP assigned the low-grade glial category as the highest probability diagnosis in most of the FOVs (Fig. 6b), resulting in the correct classification of this specimen as a low-grade glial tumour.
To evaluate the MLP in a test set of cases read by multiple pathologists, we applied the leave-one-out approach on each of the 30 cases included in the survey administered to pathologists, as described above in ‘Quantitative evaluation of SRH-based diagnosis’. Based on modal diagnosis, the MLP accurately differentiated lesional from non-lesional specimens with 100% accuracy (Fig. 7a). In addition, the diagnostic capacity of the MLP for classifying individual FOVs as lesional or non-lesional was excellent, with 94.1% specificity and 94.5% sensitivity (area under curve = 0.984; Supplementary Fig. 4). Among lesional specimens, the MLP differentiated glial from non-glial specimens with 90% accuracy at the sample level (Fig. 7b). The modal diagnostic class predicted by the MLP was 90% accurate in predicting the diagnostic class rendered by pathologists in the setting of our survey (Fig. 7c).
The cases misclassified by the MLP included a minimally hypercellular specimen with few Rosenthal fibres from a pilocytic astrocytoma (patient 84), which was classified as non-lesional rather than low-grade glioma. In this specimen, many of the FOVs resemble normal glial tissue (Supplementary Fig. 5a). Another misclassified specimen from a patient with leptomeningeal metastatic carcinoma (patient 72) contained only two FOVs containing tumour (Supplementary Fig. 5b). The glioblastoma specimen from patient 82 (Supplementary Fig. 5c), misclassified as a non-glial tumour by the MLP, contained protein-rich structural elements that resembled the histoarchitecture of metastatic tumours imaged with SRH (Supplementary Fig. 5d; patient 85). Despite these errors, the accuracy and overall ability of the MLP in automated detection of lesional status and diagnostic category provides proof-of-principle for how the MLP could be used for automated diagnostic predictions.
Accurate intraoperative tissue diagnosis is essential during brain-tumour surgery. Surgeons and pathologists rely on trusted techniques such as frozen sectioning and smear preparations that are reliable, but prone to artefacts that limit interpretation and may delay surgery. A simplified standardized method for intraoperative histology would create the opportunity to use intraoperative histology to ensure more efficient, comprehensive sampling of tissue within and surrounding a tumour. By ensuring high-quality tissue is sampled during surgery, SRH raises the yield on testing biopsies for molecular markers (for example, isocitrate dehydrogenase and α thalassemia/mental retardation syndrome X-linked mutation, 1p19q co-deletion, O-6-methylguanine-DNA methyl-transferase and telomerase reverse transcriptase-promoter alteration) that are increasingly important in rendering final diagnosis. Here, we report the first demonstration of SRS microscopy in a clinical setting and show how it can be used to rapidly create histologic images from fresh specimens with diagnostic value comparable to conventional techniques.
Fluorescence-guided surgery 16 , mass spectrometry 17 , Raman spectroscopy 18 , coherent anti-Stokes Raman scattering microscopy 19,20 and optical coherence 21 tomography, which exploit histologic and biochemical differences between tumour-infiltrated and normal tissues, have been proposed as methods for guiding excision of brain and other types of tumours 22,23 . To date, however, no microscopic imaging modality tested in a clinical setting has been successful in rapidly creating diagnostic-quality images to inform intraoperative decision-making. Here, we show that by leveraging advances in optics and fibre-laser engineering, it is possible to create an SRS microscope that is easy to operate, durable and compatible with a patient care environment, which rapidly provides diagnostic histopathologic images.
SRH is well-suited for integration into the existing workflow for brain-tumour surgery. A surgical instrument that can simultaneously collect biopsies for SRH and be tracked by a stereotactic navigational system would enable the linkage of histologic and positional information in a single display, as previously suggested 24 . Integration of SRH and surgical navigation would create the possibility of verifying that maximal safe cytoreduction has been executed throughout a surgical cavity. In situations where tumour is detected by SRH but cannot be safely removed, it might be possible to serve as a way to better focus the delivery of adjuvant therapies.
As medical data become increasingly computer-based, the opportunity to acquire virtual histologic sections via SRH creates numerous opportunities. For example, in many clinical settings where brain-tumour surgery is carried out, neuropathology services are not available. Currently there are 785 board-certified neuropathologists serving the approximately 1,400 hospitals performing brain-tumour surgery in the United States (Supplementary Table 2). A networked SRS microscope, like the prototype introduced here, streamlines both sample preparation and imaging and creates the possibility of connecting expert neuropathologists to surgeons—either within the same hospital or in another part of the world—to deliver precise intraoperative diagnosis during surgery.
Computer-aided diagnosis may ultimately reduce the inter-reader variability inherent in pathologic diagnosis and might provide guidance in settings where an expert neuropathologist is not available. Our results and the work of others suggest that machine-learning algorithms can be used to detect and diagnose brain tumours. Previous work in computer-aided diagnosis in neuropathology has shown promise in differentiating diagnostic entities in formalin-fixed, paraffin-embedded, H&E-stained whole slide images 25,26 . The ideal computer-aided diagnostic system for intraoperative histology would reliably predict diagnosis in small fresh tissue samples. The classifier reported here is capable of distinguishing lesional from non-lesional tissue samples and in predicting diagnostic class based on pooled tile data. In the future, we anticipate that a machine-learning approach will be capable of finer diagnostic classification. We also hypothesize that the accuracy of diagnostic classifiers might also be improved via (1) exploring alternative neural network configurations and systems for convolution; (2) employing feature-based classification; (3) utilizing support vector machines or statistical modelling approaches; and (4) applying rules for data interpretation that account for demographic factors and medical history.
SRS microscopy can now be utilized to provide rapid intraoperative assessment of tissue architecture in a clinical setting with minimal disruption to the surgical workflow. SRH images may ultimately be used to render diagnosis in brain-tumour specimens with a high degree of accuracy and near-perfect concordance with standard intraoperative histologic techniques. Prospective, randomized clinical studies will be necessary to validate these results and define how SRH can be used to expedite clinical decision-making and improve the care of brain-tumour patients.
The inclusion criteria for this study were as follows: (1) males and females; (2) subjects undergoing brain-tumour resection at the University of Michigan Health System (UMHS); (3) subjects (or designee) able to provide informed consent; and (4) subjects in which there was excess tumour tissue beyond what was needed for routine diagnosis. The sample size was estimated at 100 patients to ensure adequate representation of all major tumour types for analysis and based on the design of previous studies comparing SRS and H&E. The central goals of this study were: (1) to build and verify the first clinical SRS microscope; (2) to judge SRH as a means of providing diagnostic histopathologic images; and (3) to determine if machine learning could accurately classify SRH images of fresh human brain-tumour specimens. We began by collecting biopsies (N = 125) from neurosurgical patients undergoing tumour resection (N = 98) or anterior temporal lobectomy (N = 3). Each specimen was imaged immediately after removal with SRS microscopy. A trained neuropathologist (S.C.-P.) then classified each biopsy based on World Health Organisation diagnostic criteria 13 . We then quantified the correlation between SRH and H&E tissue imaging through a survey administered to neuropathologists (K.A.M., S.H.R. and M.S.). To quantify the SRH images, we utilized WND-CHRM, which assigns 2,919 attributes to each image. We then used the quantified image attributes to build and train an MLP to classify the images based on diagnostic class. Diagnostic predictions were rendered based on the diagnostic class predicted most commonly by the MLP for FOVs in a given specimen.
Tissue collection and imaging
All tissues were collected in the context of a University of Michigan Medical School IRB-approved protocol from patients who provided informed consent (IRB HUM0000083059). Tissues in excess of what was needed for diagnosis were eligible for imaging. In a subset of patients where the frozen section was large enough to split (patients 72–101), half of the specimen was routed for SRH imaging and the other half of the specimen became the tissue for clinical frozen section diagnosis.
To image tissue with the clinical SRS microscope, a small (approximately 3 mm thick) portion of fresh tissue was placed on a standard uncoated glass slide in the centre of a small piece of two-sided tape and flattened to a thickness of 120 μm in a manner similar to a standard squash preparation. Normal saline (50 μl) was applied to the tissue, then a coverslip was applied to the tissue and adhered to the slide, creating a chamber for imaging. This slide was then placed on a motorized stage and focused using standard transmission light microscopy. Using custom scripts in μ-Manager software and ImageJ software, two-channel (2,845 and 2,930 cm−1) images were obtained in a mosaic fashion.
Our prototype system is built on an Olympus microscope body, and we developed a fully custom beam-scanning unit that seamlessly integrates the laser source through fibre delivery. We also developed control electronics for both the laser and the microscope. Our custom imaging software is based on the open-source microscopy platform μ-Manager. The imaging system appears as a ‘camera’, allowing us to leverage all the automated microscopy features provided by the μ-Manager environment to enable multi-colour mosaic imaging.
Virtual H&E colouring
Generating a virtual H&E image from the 2,845 and 2,930 cm−1 images acquired from the SRS microscope utilizes a simple linear colour-mapping of each channel. After channel subtraction and flattening (described in ‘Image acquisition and stitching’), a linear colour remapping is applied to both the 2,845 and the 2,930 cm−1 channel. The 2,845 cm−1 image, a greyscale image, is linearly mapped such that a strong signal in the 2,930 cm−1 image maps to an eosin-like reddish-pink colour instead of white. A similar linear mapping is applied to the 2,930 cm−1 image with a haematoxylin-like dark-blue–violet colour mapped to a strong signal. Finally, these two layers are linearly added together to result in the final virtual-coloured H&E image.
The exact colours for the H&E conversion were selected by a linear optimization based on a collection of true H&E-stained slides created by the UMHS Department of Pathology. An initial seed colour was chosen at random for both H&E conversions. The previously described linear colour-mapping and addition process was completed with these initial seed colours. The ensuing image was hand-segregated into a cytoplasmic and nuclear portion. These portions were compared with the true H&E images and a cytoplasmic and nuclear hue difference between generated false-coloured H&E and true H&E was elucidated. The H&E seed colours were modified by these respective hue differences and the process was repeated until the difference between generated and true images was less than 1% different by hue.
Image acquisition and stitching
The procedure for generating a virtual-coloured H&E image from the SRS microscope consists of six discrete steps:
A mosaic acquisition script is started on the control computer that acquires an (N × N) series of 1,024 × 1,024 pixel images from a pre-loaded tissue sample. These images are acquired at the 2,845 and 2,930 cm−1 Raman shifts and saved as individual two-channel FOVs to a pre-specified folder.
The two-channel image is duplicated and a Gaussian blur is applied to the duplicated image. The original two-channel image is then divided by the Gaussian blur to remove artefacts of acquisition and tissue preparation.
The 2,845 cm−1 channel is subtracted from the 2,930 cm−1 channel in each FOV.
New FOVs are created with the 2,845 cm−1 channel and the 2,930 cm−1 minus 2,845 cm−1 channel.
The virtual-colour H&E script (described in ‘Virtual H&E colouring’) is run to create an H&E version of the subtracted and flattened tile.
The original tile is stitched as previously described 27 . The user is presented with an option to re-stitch with different stitching parameters if the initial stitch produces an unacceptable image. On successful stitching, a layout file is generated from the terminal positions of the individual tiles in the stitched image.
The virtual-colour H&E images are stitched using the layout file generated in step 6, a significantly faster process than re-computing the stitching offsets and merges from scratch.
A computer-based survey consisting of 30 patients was developed and given to blinded neuropathologists (K.A.M., S.H.R. and M.S.), who were presented with standard frozen H&E images and SRH images. All cases included in the survey were judged to have SRH and conventional H&E preparations that contained the essential architectural features required for diagnosis. Each image was accompanied by a short clinical history that included age group, sex and presenting symptom(s). Survey responses were recorded automatically by the survey software. The intraoperative frozen and final pathologic diagnoses determined by standard clinical protocol employed by the UMHS Department of Pathology were also recorded. The survey responses were scored for accuracy on four levels: (1) for all specimens, whether tissue was lesional versus non-lesional; (2) for lesional tissues, whether the origin was glial or non-glial; (3) for glial tumours, whether the tumour was low- or high-grade; and (4) for all tumours, the predicted diagnosis. Responses were considered concordant if accuracy scores were equal. The maximum possible score for each case was determined by the clinical frozen section diagnosis. For each case, the following diagnoses were used for statistical analysis: UMHS frozen section diagnosis, survey frozen section diagnosis and survey SRH diagnosis.
For each pathologist, we calculated Cohen’s kappa 28 for SRH versus H&E for lesion versus no lesion and for glioma versus no glioma. This provides information on how well SRH and H&E agree.
Cohen’s kappa was also calculated for final diagnosis from SRH versus truth (clinical frozen section diagnosis) and for H&E versus truth (clinical frozen section diagnosis), where final diagnosis was one of eleven categories, which tells us how well each pathologist was able to detect the truth from either SRH or H&E. Lastly, we calculated the reliability among the three pathologists (Fleiss’ kappa 29 ) for SRH lesion versus no lesion, SRH glioma versus no glioma, H&E lesion versus no lesion and for H&E glioma versus no glioma. R software (version 3.3.0; http://www.r-project.org) was used for all statistical analyses.
No distributional assumptions are necessary for the kappa statistic. The only assumption is that the data are categorical and that SRH and H&E are measured on the same data, which they are. There is no estimate of variance for groups.
Generation of the MLP
The MLP was programmed with two software libraries: Theano and Keras. Theano (http://deeplearning.net/software/theano/index.html) is a high-performance low-level mathematical expression evaluator used to train the MLP. Keras (http://keras.io) is a high-level Python framework that serves as a wrapper for Theano, allowing rapid iteration and testing of different MLP configurations.
The MLP is designed as a fully connected, 1,024-unit, one hidden layer, neural network. It comprises eight sequential layers in the following order: (1) dense input layer with uniform initialization; (2) hyperbolic tangent activation layer; (3) dropout layer with dropout probability 0.2; (4) dense hidden layer with uniform initialization; (5) hyperbolic tangent activation layer; (6) dropout layer with dropout probability 0.2; (7) dense output layer with uniform initialization; and (8) a softmax activation layer corresponding to the number of classifications (Supplementary Fig. 6).
Training of the MLP was performed using a training set that was exclusive from the survey test set. Loss was calculated using the multiclass log-loss strategy. The selected optimizer was the ‘Adam’ optimizer. The optimizer’s parameters were as follows: learning rate = 0.001, beta_1 = 0.9, beta_2 = 0.999 and epsilon = 1 × 10−8.
Image processing and analysis by the MLP
The process to convert a raw SRH image to a probability vector for each of the diagnoses is as follows:
Use Fiji software (http://fiji.sc) to subtract the CH2 layer from the CH3 layer and flatten the image as described in ‘Tissue collection and imaging.’
Use Fiji software to split the two-channel image into a separate CH2 layer and a CH3–CH2 layer.
For each of the previous tiles, create four duplications of the tile with 90° rotations (‘rotamers’).
Use WND-CHRM (http://scfbm.biomedcentral.com/articles/10.1186/1751-0473-3-13) to generate signature files for each of the tiles from the previous step.
Normalize the signature files such that all of the feature values are uniformly and linearly mapped to the range (−1.0, 1.0).
(CH2). For each of the tiles that correspond to CH2-channel tiles, run the MLP as described above.
(CH2). Gather all of the rotamers for a given tile and average (arithmetic mean) the prediction values from them to create one consolidated diagnosis-probability vector for a given CH2-channel tile.
Repeat steps 6–7 for the CH3–CH2 channel.
For a given tile, compare the CH2 channel and the CH3–CH2 channel and discard the diagnosis-probability vector for the tile that has a lower maximal probability value.
For a case-by-case diagnosis, group all of the tiles for a case, remove any tile that does not have a diagnosis probability of >0.25, and diagnose the case with the most prevalent (mode) diagnosis among the set of tiles.
MLP evaluation with the leave-one-out approach
To test the diagnostic accuracy of the MLP, we used a leave-one-out approach for the 30 patients that were used in the survey administered to neuropathologists. For each of the 30 patients used to evaluate the MLP, all FOVs (N) from that patient were placed in the test set. The training set was composed of the 12,879−N remaining FOVs. The 12,879 FOVs were screened by a neuropathologist to ensure they were representative of the diagnosis they were assigned to. FOVs were classified as non-lesional, pilocytic astrocytoma, ependymoma, oligodendroglioma, low-grade diffuse astrocytoma, anaplastic oligodendroglioma, anaplastic astrocytoma, glioblastoma, meningioma, lymphoma, metastatic tumour and medulloblastoma.
The MLP was trained for 25 iterations, with the following 26 iteration weights recorded to use for validation of the test set. The test set was fed into each of these 26 weights with the resulting probabilities of each of the 12 diagnostic classes averaged to create a final probability for each diagnosis for each FOV. The 12 diagnoses were condensed to four classes (non-lesional, low-grade glial, high-grade glial and non-glial) to achieve diagnostic predictions. The low-grade glial category included FOVs classified as pilocytic astrocytoma, ependymoma, oligodendroglioma and low-grade diffuse astrocytoma. The high-grade glial category included FOVs classified as anaplastic oligodendroglioma, anaplastic astrocytoma and glioblastoma. The non-glial category included FOVs classified as meningioma, lymphoma, metastatic tumour and medulloblastoma.
Nationwide inpatient sample query
The Nationwide Inpatient Sample database, obtained from the Healthcare Cost and Utilization Project of the Agency for Healthcare Research and Quality, was queried for years 2010 and 2011. The Nationwide Inpatient Sample database for these years contains discharge data for all discharges from a sample of hospitals representing 20% of all nationwide discharges from non-federal hospitals using a stratified random sampling technique.
Brain-tumour resections or biopsies were identified using combinations of International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) diagnosis and treatment codes that were previously used for studies of adult tumours 30 , paediatric tumours 31 and pituitary tumours 32 . Primary tumour ICD-9-CM diagnosis codes used include 191.0–191.9, 225.0 and 237.5 and procedure codes used include 01.53, 01.59, 01.13 and 01.14. For other tumours, ICD-9-CM diagnosis codes used include 225.2, 192.1 and 237.6 for meningioma, 198.3 for metastases, 225.1 for vestibular schwannomas and 227.3 for pituitary tumours. Procedure codes used include 01.51, 01.13 and 01.14 for meningioma, 01.53, 01.59, 01.13 and 01.14 for metastases, 04.01 for vestibular schwannomas, and 07.62 and 07.65 for pituitary tumours.
The SRS microscopy system described in this paper is a prototype system that is intended for research-use only. It does not comply with international safety standards nor has it received approval or clearance from any government agency such as the US Food and Drug Administration.
The computer code used to generate the results of this study is available on reasonable request from the corresponding author, with the exception of proprietary portions of code used for the generation of the virtual H&E colour scheme.
All raw and processed-image data generated in this work, including the representative images provided in the manuscript, are available from the corresponding author on reasonable request.
How to cite this article: Orringer, D. A. et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat. Biomed. Eng. 1, 0027 (2017).
The authors would like to thank H. Wagner for manuscript editing. Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering (R01EB017254 to X.S.X. and D.A.O.), National Institute of Neurologic Disorders and Stroke (K08NS087118 to S.H.R.), the National Institutes of Health Director’s Transformative Research Award Program T-R01 (R01EB010244-01 to X.S.X.) and the National Cancer Institute of the National Institutes of Health (P30CA046592). This work was also supported by Fast Forward Medical Innovation, the University of Michigan—Michigan Translational Research and Commercialization for Life Sciences Program (U-M MTRAC) and the Michigan Institute for Clinical and Health Research (2UL1TR000433).
Supplementary figures and tables.