Introduction

The field of histopathology is undergoing significant transformation, thanks to advances in imaging and computation technology. For example, in 2017 Philips received FDA approval for digital pathology in primary diagnosis1,2,3, while the government of Kuwait approved Leica Biosystems digital pathology as its standard of care4. Computer-assisted image analysis (CAIA) of whole-slide images (WSI) is slated to play an important role in corroborating information across multiple laboratories and, ultimately, improving diagnosis accuracy and patient outcome. The current gold standard in histopathology involves the microscopic investigation of a surgical specimen by a trained pathologist. The tissue is processed in the form of a thin section placed on a cover glass. Because such tissue sections are transparent under visible light, standard protocols include a staining procedure. As such, the pathologist is not observing the tissue morphology itself, but rather, a stain distribution that correlates with morphology. The contrast and color balance varies not only with the pathologist preference [see, e.g., Fig. 1 in ref.5] but also with the specimen processing, staining procedure and illumination. Staining variability has been identified as a major obstacle in producing consistent results across multiple specimens when using machine-learning algorithms on stained tissues6,7,8. To take advantage of the digital data that are increasingly available from commercial WSI, numerical solutions have been proposed to correct the staining inconsistency in post-processing9,10,11.

Figure 1
figure 1

cSLIM optical setup. (a) A commercial microscope with phase contrast illumination and bright-field objective. (b) SLIM module placed at the microscope output port, equipped with color RGB camera. (c) Phase shifts imparted between scattered and incident light by the SLM in the Fourier plane of L1. (d) Four intensity frames obtained for each phase modulation by weighted sum of the red, green and blue channel images of the RGB camera (e) Spectra and (f) Temporal autocorrelation (real part) for RGB and grayscale imaging. IP, Image Plane, SLM, Spatial light Modulator.

Label-free microscopy techniques are promising for meeting this current challenge in pathology. By generating intrinsic contrast in images based on physical properties of the specimen, these methods minimize inter-observer variation and subjectivity in the evaluation. Their label-free nature also simplifies the process of designing supervised learning schemes for automated image analysis as consistent feature values are expected between training and deployment12,13,14. Furthermore, by relying on novel contrast mechanisms yet unfamiliar to traditional pathology, these methods can also elucidate new cancer biology and introduce new markers of prognosis15,16,17. A number of label-free methods have been proposed for histopathology in the past. These can be broadly grouped into two categories: vibrational spectroscopy methods [including Fourier Transform Infrared spectroscopy (FT-IR)14,18,19,20 and Raman spectroscopy (RS)21,22,23,24] and non-linear optical microscopy methods [including second-harmonic-generation microscopy (SHGM)15,25,26, endogenous two photon excited fluorescence (TPEF)26 and third-harmonic-generation microscopy (THGM)27]. Each technique in these groups brings with it unique features in terms of chemical specificity and physical information extracted. However, these methods also face specific challenges for clinical translation, as they can pose new requirements on sample preparation and optical instrumentation. For example, FT-IR yields orders of magnitude lower spatial resolution and throughput than conventional bright-field microscopy. It also requires special tissue preparation steps such as mounting of unlabeled tissue sections on BaF2 slides20. RS is also much slower than conventional microscopy. In addition, these two techniques have been applied mainly on unstained tissue sections. Since staining is an inevitable part of standard tissue assessment, this complicates the process of clinical translation. While some non-linear optical microscopy studies have been performed on stained tissues13,15,25, techniques in this group also have much lower throughput than conventional microscopy, require expensive ultrashort-pulsed lasers and are only able to generate contrast in certain structures (such as collagen fibers, for example, in SHGM)28. All these methods require special illumination and detection systems that are difficult to combine with bright-field microscopes currently present in pathology labs.

Quantitative phase imaging (QPI)29, is an emerging technology that yields intrinsic contrast by mapping the optical pathlength difference (OPD) across the tissue specimen. QPI has been successfully employed for a number of quantitative histopathology investigations30, including diagnosis of prostate cancer31, prediction of recurrence in patients after prostatectomy16,32, colon cancer screening33, breast cancer diagnosis34, Gleason grading of prostate cancer12, collagen fiber orientation measurement in breast tissue35, assessment of malignancy in bile ducts36 and early prediction of risk of colon cancer37. QPI systems can be deployed as add-on modules to conventional microscopes and provide similar resolution and throughput to those in traditional systems33,38. Thus, they have the potential to minimally disrupt the current clinical pipeline. However, almost all of the work done thus far using QPI has involved unstained tissue biopsies. These QPI methods would require a pathology lab to prepare an additional unstained tissue section which needs to be scanned separately if both traditional and new markers for disease are to be obtained. While Greenbaum et al. retrieved phase information from stained breast tissue histologies in their work39, they used it merely for refocusing bright-field images that were then reviewed by a pathologist. No quantitative phase based biomarkers were discussed in their results.

Here, we propose a technical solution for extracting intrinsic tissue morphology information from stained tissues sections while being unaffected by variability in tissue staining. We developed color spatial light interference microscopy (cSLIM), in which the image is overlapped with a reference wave to reveal a detailed map of the OPD across the tissue. The cSLIM image is produced by the refractive index of the tissue, which is an intrinsic morphological marker correlated with disease31. The cSLIM instrument is an advanced form of QPI. However, unlike previous QPI methods applied for diagnosis16,29,31,32,33,35,38, cSLIM operates on existing stained tissue slides and, in a single scan, produces both a bright-field map and a phase map for intrinsic contrast. Using specimens prepared under the standard protocol, cSLIM yields simultaneously the typical image that the pathologist is accustomed to (e.g., H&E, immunochemical stains, etc.) and a quantitative phase image, which provides new information, currently not available in bright-field images (e.g., collagen fiber orientation35). Importantly, the effects of the stain on the phase image can be removed completely by using a simple mathematical operation. The cSLIM throughput as a WSI scanner is comparable with that of commercial (bright-field) instruments (see Results and Discussion for a detailed comparison)33. We anticipate that computer algorithms (see, e.g.8,40,41,42), currently used on stained tissue images will translate, upon adjustments, with strong performance on our quantitative images that are free of stain variability.

To illustrate the potential of the new cSLIM technology for histopathology applications, we demonstrate its value for breast cancer prognosis. Breast cancer is a significant global health problem, being the second most commonly diagnosed cancer worldwide according to the latest World Health Organization (WHO) statistics43. Within the US, while rates of mortality have been consistently falling, rates of incidence have been on the rise with 266,120 new cases of invasive breast cancer expected in 201844. This indicates that an increasing number of new diagnostic and prognostic evaluations are being performed on patients by pathology labs nationwide. Robust methods for quantitating disease signatures in breast tissue, independent of stain variation, remain highly desirable. Furthermore, breast cancer is a heterogeneous disease. Due to variations in tumor type and, therefore, patient responses to different therapies, the current set of prognostic indicators do not provide sufficient information for all patients15. There is, thus, a need for novel biomarkers that inform objectively on disease aggressiveness to account for individual variation45,46. QPI-based biomarkers can be significant in this regard since they can be easily integrated into the current workflow and, with the advent of computational pathology, can be extracted rapidly and reproducibly13,16. We demonstrate that cSLIM reveals the alignment and orientation of tumor-adjacent collagen fibers, a histological marker with known prognostic value for breast cancer patients15. We classified patients based on the cSLIM-measured value of this marker and showed its predictive power through survival analysis.

This paper is organized as follows. We first introduce the cSLIM instrument and describe the optical problem of interferometry with an RGB camera. Next, we describe the normalization process that eliminates the stain-dependent signal from our cSLIM images. We further prove that clinical bio-markers are independent of tissue staining on these normalized images. Finally, we demonstrate the ability of our stained tissue analysis system to detect aligned collagen fiber-based prognostic markers. We compare the cSLIM derived values of these markers with patient survival data to demonstrate their prognostic value.

Results and Discussion

Our results consist of a new instrument, cSLIM, and clinical results on breast cancer prognosis. The cSLIM system, which yields for the first time simultaneous color bright-field and quantitative phase images on stained slides is described below.

cSLIM optical setup

The cSLIM optical setup is illustrated in Fig. 1. The optical train builds on the principle of spatial light interference microscopy (SLIM)47. The SLIM module (CellVista SLIM Pro, Phi Optics, Inc.) is placed at the output port of a commercial microscope. The conjugate image plane at the microscope output port is imaged onto the camera using a 4 f system formed by lenses L1 and L2. At the Fourier plane of lens L1 a spatial light modulator (SLM, Boulder Nonlinear Systems) is used to modulate the phase difference between the scattered and incident light. As illustrated in Fig. 1(c,d), four phase shifts are employed, φ = 0, π/2, π, 3π/2 rad, and intensity images are recorded by the camera for each shift. Combining the four intensity images47, we obtain the quantitative phase image, free of amplitude effects.

There are several significant modifications in the current cSLIM system with respect to previous reports47, as follows. First, in place of the phase contrast microscope objective used in SLIM we employ a 40x/0.75 NA bright-field objective. The use of a bright-field objective allows us to obtain typical bright-field images, i.e., H&E-stained histology images that are the mainstay of histopathology in the clinic. The annular condenser ring is retained from the phase contrast microscope. Second, the grayscale camera is replaced by an RGB camera (Carl Zeiss Axiocam MRc) which provides red, green, and blue channels. Finally, because we use an RGB camera, the detected phase is that of the sum of the autocorrelation functions for each spectral channel (see below). As a result, the calibration of the SLM and the physical meaning of the measured phase are different from the previous reports.

For each phase shift, we acquire three intensity frames corresponding to the red, green and blue channels of the camera: R(x, y; φ), G(x, y; φ) and B(x, y; φ), respectively. In each case, these channels are combined linearly to obtain an equivalent grayscale intensity image, as

$$I(x,y;\phi )=rR(x,y;\phi )+gG(x,y;\phi )+bB(x,y,\phi ),$$
(1)

where r, g, b are constants such that r + g + b = 1. Throughout the manuscript, we use r = 0.1, g = 0.6, b = 0.3. This weighting was determined empirically, as it determines the equivalent central wavelength, λc, at which the phase modulation by the SLM occurs. In order to describe the interference resulting from the superposition of the three spectral channels, we start with the equivalent spectrum for cSLIM imaging, namely

$${S}_{c}(\omega )={S}_{i}(\omega )[{S}_{R}(\omega )+{S}_{G}(\omega )+{S}_{B}(\omega )]$$
(2)

where, ω is the optical angular frequency, Si is the incident light spectrum, and SR, SG, and SB are the spectral responses of the red, green, and blue channels, respectively. These responses capture the effect of the RGB camera color filters as well as the weighting factors [r, g and b]. The wavelength spectra, Sc(λ) and Si(λ), are shown in Fig. 1(e), while the spectral responses of the three color channels have been plotted in Fig. S4(c) (see Supplementary Information). The temporal autocorrelation function of the equivalent source, Γc(τ), is obtained by taking the Fourier transform of Eq. (2),

$${\Gamma }_{{c}}(\tau )={\Gamma }_{i}(\tau ){\rm{\circ{v}}}[{\Gamma }_{{R}}(\tau )+{\Gamma }_{{G}}(\tau )+{\Gamma }_{{B}}(\tau )].$$
(3)

In Eq. (3) Γi(τ) is the autocorrelation of the illumination source and ΓR(τ), ΓG(τ), and ΓB(τ) are the autocorrelations corresponding to the spectra of the three color channels. The symbol denotes the convolution operation and τ refers to the time delay, conjugate variable to ω. A comparison between the autocorrelation functions (real part) calculated for the illumination source and the equivalent cSLIM source is shown in Fig. 1(f). The functions are plotted against distance d = , with c the speed of light in air.

It is apparent that the optical problem of interferometry using a color camera is equivalent to using three independent sources. In essence, although the complex fields from the three spectral channels do not interfere with one another, their autocorrelation functions add. As a result, the new, equivalent correlation function (Γc) is characterized by a new envelope (indicative of the coherence time, inversely proportional to the spectral width) and phase (describing the new central wavelength). The central wavelength detected in cSLIM was measured as λc = 558 nm while the incident white light was at λi = 589 nm. The coherence lengths, lc and li, measured from the two corresponding autocorrelation functions [Fig. 1(f)] were 3.24 μm and 2.26 μm, respectively (in air). We define the coherence length as the full-width half maximum (FWHM) of the envelope [dashed line in Fig. 1(f)] of the autocorrelation function.

The cSLIM system requires two calibration steps. First, the SLM was calibrated to ensure that the correct value of φ was used for each of the four frames, resulting in increments of \(\pi /2\) at the central wavelength λc47. This was performed by configuring the SLM in amplitude mode and measuring the amplitude modulation in I(x, y; φ) as a function of the SLM 8-bit grayscale input voltage47. The calibration curve for phase was then obtained by taking the Hilbert transform of the amplitude modulation curve [see Section S.1 of Supplementary Information]. Second, the SLIM phase reconstruction algorithm47 includes the attenuation term αpc, measured experimentally, which is the attenuation factor of the incident light compared to the scattered light in a phase contrast objective. Since bright-field objectives do not impart this attenuation, an equivalent attenuation αbf was introduced numerically in place of αpc in the cSLIM phase reconstruction. The correct value of αbf was determined by imaging an unstained tissue microarray (TMA) core using both phase contrast and bright-field objectives and tuning αbf until the correlation between the phase values from the two acquisitions was maximized [see Section S.2 in Supplementary Information]. The resulting value of αbf = 3.4 was used for all subsequent phase reconstructions. While this attenuation factor was obtained from a single unstained TMA core, the resulting normalized phase values are consistently similar between stained and unstained cores, indicating correct calibration (see section on Normalizing out the effects of staining).

The typical raw outputs generated by the cSLIM system are illustrated in Fig. 2. Results are shown for an H&E-stained tissue microarray at the slide [Fig. 2(a,d)], core [Fig. 2(b,e)] and cellular scales [Fig. 2(c,f)]. The raw phase, ϕ(x, y), and bright-field microscopy images of the whole slide are obtained in a single scan, i.e., the bright-field image is merely one frame of the cSLIM reconstruction. These capabilities of the cSLIM scanner are also illustrated in the Supplementary Video File (Video_S1.mov). Thus, a key feature of the cSLIM outputs is that standard histopathology and quantitative phase images are perfectly registered. This is significant in carrying out QPI studies with large cohorts, since independent pathologist evaluation can be done on the same tissue rather than a parallel section. This ensures that a pathologist’s diagnosis matches precisely with markers extracted in QPI without the need for duplicate scanning of tissue section before and after staining. Thus, information from both channels (phase and bright-field) can be combined for improving results of classification and segmentation problems for which algorithms related to both modalities have been published separately so far8,12,34,48.The scan time for a single core (approx. 1 mm2 area) was approx. 13 seconds, at a pixel ratio of 7.4 pixels/μm. If we compare this acquisition rate with that of the Zeiss Axio Scan.Z149, a commercial WSI instrument, our instrument is approximately 4.6 times slower (scanning a 225 mm2 area in 1100 sec vs. to 240 sec for the Zeiss Axio Scan.Z1, at the same sampling rate). This throughput is remarkable since cSLIM provides simultaneous access to both phase and bright-field images and gathers 4 times as much data per frame to perform phase reconstruction (as discussed above). Note that no commercial WSI provides stromal information for prognosis applications. Furthermore, compared to other label-free imaging modalities (for example FT-IR and SHGM) our system is significantly faster.

Figure 2
figure 2

cSLIM outputs obtained by scanning a whole tissue microarray slide. (ac) H&E-stained tissue bright-field images of whole slide, single core, and epithelial region within the core, respectively. (df) Raw phase maps of the whole slide, single core and epithelial region within the core, respectively.

Normalizing out the effects of staining

Absorption in stained tissue is expected cause further variation in the cSLIM equivalent spectrum [shown in Fig. 1(e)]. In addition, refractive index of stained tissue is expected to vary from that of unstained tissue because absorption and refractive index are related via the Kramers-Kronig relations50 (see Supplementary Information, Section S.4 for analysis of dispersion in tissue). As a result, phase maps extracted from stained and unstained tissue samples may differ. To quantify this difference, we imaged a TMA (TMA-1) of breast tissue biopsies before H&E staining using SLIM and after staining using cSLIM (see Materials and Methods for details about TMA-1). The raw phase maps ϕ(x, y) for one TMA core, before and after staining, are illustrated in Fig. 3(a,b), respectively. It is evident that staining causes a reduction in phase values as well as the image contrast. These effects are also illustrated in the histograms of the two phase images in Fig. 3(c) where the stained tissue histogram is noticeably narrower. To normalize these effects of staining we computed for each core the standard normal variable Z(x, y) from ϕ(x, y) using the equation

$$Z(x,y;{\lambda }_{c,i})=\frac{\varphi (x,y;{\lambda }_{c,i})-\mu }{\sigma },$$
(4)

where \(\varphi (x,y;{\lambda }_{c,i})=2\pi [n(x,y;{\lambda }_{c,i})-{n}_{0}]t/{\lambda }_{c,i}\) is the measured phase map, with λc,i the central wavelength of the detected spectrum, n(x, y; λc,i) the tissue refractive index map, n0 the refractive index of the medium surrounding the tissue and t the thickness of the tissue histology section. \(\mu ={\langle \varphi (x,y;{\lambda }_{c,i})\rangle }_{(x,y)}\) is the phase spatial average and \({\sigma }^{2}={\langle {\varphi }^{2}(x,y;{\lambda }_{c,i})\rangle }_{(x,y)}-{\mu }^{2}\) the phase spatial variance, and the operator \(\langle .\rangle \) refers to the expected value, computed over spatial coordinates (x, y). Both μ and σ were computed in the tissue region only, after removal of background pixels. Details about this computation, including removal of background pixels, are described in Section S.3 in Supplementary Information.

Figure 3
figure 3

Comparison of phase maps obtained from the TMA-1 before and after staining. (a,b) TMA core raw phase images before and after staining, respectively. (c) Histograms of the phase images in (a,b). The similarity of the two histograms is quantified by computing the Pearson’s correlation coefficient ρ between them. (d) and (e) Normalized maps for the same core before and after staining, respectively. (f) Histograms of the images in (d,e). The similarity of the two histograms is quantified by computing the Pearson’s correlation coefficient ρ between them. (g) Bar plots showing the mean value of Pearson’s correlation coefficient ρ between core histograms for raw and normalized phase maps. Error bars show the standard deviation over the 30 cores.

The results of this normalization procedure are shown in Fig. 3(d–f). The normalized phase maps are visually very similar between the stained and unstained tissue and the histograms seem to overlap almost perfectly. To quantify this similarity, the Pearson’s correlation coefficient ρ51 was computed between the stained and unstained tissue histograms, before and after phase normalization. As illustrated in Fig. 3(c,f), the ρ values improved significantly due to the normalization procedure, indicating a very high degree of correlation. The ρ values for a total of 30 cores (15 cancerous and 15 normal, selected randomly from TMA-1), before and after normalization, are summarized in Fig. 3(g). The bar heights represent mean values whereas the error bars represent the standard deviations over the 30 cores. The high correlations demonstrate that Z(x, y) maps are almost stain-independent, which offers excellent opportunities for quantitative pathology. Despite the stain variability associated with varying proportions of epithelium and stroma both within a core and across a population of cores, our normalization technique remains robust.

It can be deduced from Eq. (4) that division by the standard deviation removes the λc,i dependence of the phase map, minimizing the differences between unstained and stained tissue Z(x, y) images caused by different spectra. While dispersion in tissue (spatial variation of both wavelength and refractive index) is also expected to cause differences between these images, our detailed analysis of this phenomenon (included in Supplementary Information Section S.4) showed that these changes are small. The fact that a global normalization is effective in removing the stain-related effects also indicates that dispersion in tissue is not a dominant effect (see Supplementary Information).

Comparison of malignancy markers between stained and unstained tissue

Previous reports have shown that disease markers in quantitative phase images of unstained tissue can detect malignancy in different organs31,33,34. To demonstrate that these markers are preserved in stained tissue cores imaged by cSLIM, we assembled Z(x, y) maps both before and after staining for the 30 cores in TMA-1 selected in the previous section. Each core belonged to a different case/patient with known diagnosis results through pathologist evaluation. All malignant cases were diagnosed as infiltrating ductal carcinoma (IDC). From the Z(x, y) maps, we developed a supervised learning method for breast cancer diagnosis that relied on three types of features: the median gland or epithelial compartment (EC) curvature 〈C〉, the median of mean scattering length within an EC 〈ls〉, and the median texture vector for the EC 〈T34. We used the pathologist’s diagnosis for each EC as the ground truth for training (see Materials and Methodss for details). Details of the feature extraction, training and validation steps have also been included in Materials and Methods. The biological meaning of these three types of features and how each differs between benign and malignant tissue has already been discussed in ref.34.

Figure 4 illustrates the diagnosis results obtained for stained and unstained tissue. Figure 4(a,b,d,e) compare Z(x, y) for benign and malignant ECs before and after staining. Once again, the images are very similar between the two cases, proving the efficiency of our normalization procedure. In cSLIM, morphological details of these ECs are also available for traditional histopathological assessment through bright-field images [Fig. 4(g,h)]. To test whether features derived from these Z(x, y) maps can detect malignancy in breast tissue, 3-fold cross-validation was performed, consisting of three trials. ECs from all cores were pooled and divided into three equal sets and in each trial two sets were used for training and one set for validation. Figure 4(g,h) compare the values of the three features between unstained and stained ECs, for one training set. Since texture feature 〈T〉 is multidimensional (Materials and Methods), it is represented by its first principal component in the plot. Our results show that the feature values have a similar distribution in the unstained versus stained cases, for both normal and diseased tissue.

Figure 4
figure 4

Comparison of diagnosis results between stained and unstained tissue using Z(x, y) maps. (a,b) Z(x, y) images of an unstained and stained malignant EC, respectively (c) H&E-stained bright-field image of the same malignant EC. (d,e) Z(x, y) images of an unstained and stained benign EC, respectively. (f) H&E-stained bright-field image of the same benign EC. (g,h) Separation of benign and malignant ECs in training feature space for stained and unstained tissue, respectively. (i) ROC curves for 3-fold cross-validation for classifying benign and malignant ECs.

In each case, the probability scores for all ECs, generated by a linear discriminant analysis (LDA) classifier in all three trials, were pooled together to generate a receiver operating characteristic (ROC) curve for the cross-validation (see Materials and Methods). As shown in Fig. 4(i), similar area under the curve (AUC) was measured for analysis on both stained and unstained tissue using Z(x, y) maps, indicating that markers of malignancy remain preserved in stained tissue normalized phase maps. While the AUCs are similar, they are not identical. This can be attributed to the fact that the tissue morphology itself (while similar) is not identical between the two experiments since the process of removing the coverslip from the TMA slide and staining it results in some physical changes to the tissue biopsies.

Comparison of collagen fiber orientation between stained and unstained tissue

The prognostic value of tumor associated collagen signatures (TACS) in breast tissue has been demonstrated in a number of studies15,52. Traditionally, these markers have been measured using SHGM which provides chemical specificity to collagen. However, in SHGM the tumor edge is difficult to identify due to the absence of second harmonic signals in centrosymmetric structures and the acquisition speed is lower than in full-field microscopy due to the point-scanning geometry35. Our group demonstrated in ref.35, by comparing QPI and SHGM images of the same stromal tissue, that QPI allows detection of collagen fibers within breast biopsies. While phase imaging itself does not have the specificity to collagen, collagen fibers dominate the connective tissue in the breast and their morphology can be detected through analysis of phase images as shown in ref.35. Here, we demonstrate that the measured collagen fiber angle θ is similar between stained and unstained tissue, after stain normalization. θ is the relative angle between a collagen fiber near an EC edge and the tangent to the nearest point on the edge itself [Fig. 5(c)].

Figure 5
figure 5

Comparison of relative collagen fiber angle θ between stained and unstained tissue biopsies. (a) and (b) Bright-field images of malignant and benign ECs, respectively. (c) Illustration of relative fiber angle θ. (d,e) Orientations of collagen fibers in stained tissue, in the vicinity of malignant and benign ECs, respectively. Fiber orientations are shown as green lines while the EC edge is marked in orange. (f) Normalized histogram of θ measured for all ECs within the stained tissue dataset (15 malignant and 15 benign cores). The p-value for separation between the two classes was determined using the two-sample Student’s T-test. (g) and (h) Orientation of collagen fibers in unstained tissue in the vicinity of malignant and benign ECs, respectively. Fiber orientations are shown as green lines while the EC edge is marked in orange. (i) Normalized histogram of θ measured for all ECs within the unstained tissue dataset (15 malignant and 15 benign cores). The p-value for separation between the two classes was determined using the two-sample Student’s T-test.

For the 30 cases selected from TMA-1 (used in the previous two sections), we measured θ in the Z(x, y) maps of both stained and unstained tissue using an open source MATLAB-based fiber measurement tool called CurveAlign13 (see Materials and Methods for details and parameter specifications). All fibers within a distance of 63 μm from the EC edge were considered. This is within the range of the typical intercellular signaling distance reported in literature13,53. ECs in all cores were segmented out before computation of θ so that their cellular structures did not interfere with the process of collagen fiber extraction (see Materials and Methods for details on EC segmentation).

Figure 5 compares the obtained results between unstained versus stained tissue biopsies. Figure 5(a,b) show the bright-field images of malignant and benign ECs whereas Fig. 5(c,d,f,g) illustrate the fiber orientation in their vicinity. As evident from these images, values of θ are on average larger for malignant tissue than for benign. Furthermore, the orientation measured on stained tissue qualitatively matches that measured on the unstained. Figure 5(e,h) show the histograms of θ measured for all tumor-adjacent fibers across the 30 core dataset. Once again malignant ECs show a greater probability of forming larger angles with their adjacent fibers. These measurements agree with previous results in literature where it was demonstrated that larger values of θ are associated with more aggressive disease and that aligned collagen fibers, oriented perpendicularly to tumor edge, facilitate local invasion15. The results here are significant because collagen-fiber-based parameters show similar values between stained and unstained normalized phase maps. Next we show that prognostic markers, based on stromal fibers, can be evaluated using cSLIM on tissues from cancer patients with different outcomes.

Breast cancer prognosis using cSLIM – measurement of TACS-3

Using the fiber extraction and orientation measurement demonstrated in the previous section, here we measure the prognostic marker TACS-315. TACS-3 refers to the presence of aligned collagen fibers that terminate at the tumor edge at high (near perpendicular) values of relative angle θ. In Bredfeldt et al.13, TACS-3 was measured using an automated, supervised learning scheme within a TMA imaged using SHGM. Using cSLIM, we imaged the same H&E-stained TMA (TMA-2) and extracted Z(x, y) maps for each core. We then used the fiber analysis method described earlier13 to demonstrate that cSLIM can successfully detect TACS-3.

TMA-2 comprised 196 cases (1 core per case) of IDC with disease-free survival (DFS) and disease-specific survival (DSS) information available for each case15. Using open-source MATLAB-based fiber quantification tools CT-FIRE13,54,55 and CurveAlign, features were computed for each fiber that was within a distance of 100 μm from the tumor edge (see Materials and Methods for details on fiber extraction and feature computation). Once again, this distance was chosen to be comparable with the typical intercellular signaling distances reported in literature53. The features extracted from each fiber were then combined to generate a feature vector for the core. Three core-level features were found to be most informative in distinguishing between TACS-3 positive and negative patients: mean of ε (mean nearest fiber alignment), mean of l (nearest distance of fiber from tumor edge), and skewness of θ. These features are listed in the table of Fig. 6(a). For detailed feature description see Materials and Methods and refs13,54.

Figure 6
figure 6

Results of survival analysis for TACS-3 positive and TACS-3 negative groups, classified based on fiber features measured from Z(x, y) maps obtained by cSLIM. (a) Fiber features found to be most informative in classifying cores as positive or negative. μp and μn refer to the mean of these features for the groups classified as positive and negative, respectively. (b) Results of the Univariate Cox proportional hazard regression with TACS-3 status as variable. (c) Kaplan-Meier estimate of survival function for DSS and DFS. Vertical tic marks represent right-censoring events. The number of cores classified as TACS-3 negative were 52 and those classified as TACS-3 positive were 144.

Three-dimensional feature vectors for 10 cores marked as TACS-3 positive and 10 cores marked as TACS-3 negative (based on pathologist consensus15) were used as predictors for training a linear Support Vector Machine (SVM) classifier. The classifier was then used to classify all 196 cores as either TACS-3 positive or TACS-3 negative. Figure 6(a) shows the difference in the feature means of groups classified as TACS-3 positive and TACS-3 negative. According to these mean values, cores classified as TACS-3 positive have a higher probability of containing aligned fibers (high ε) that terminate at or near the tumor edge (low l). Furthermore, these cores have θ histograms that are more positively skewed, reflecting an asymmetry due to more instances of high θ. These measurements agree with the pathologist definition of TACS-3, indicating successful classification.

Survival analysis was carried out to test whether patients classified as TACS-3 positive had significantly worse outcomes than those deemed TACS-3 negative. Figure 6 summarizes these results. Univariate Cox proportional hazard regression56 and Kaplan-Meier estimates56 were used to compare survival between TACS-3 positive and TACS-3 negative cases. As shown in Fig. 6(b), TACS-3 positive patients had hazard ratios56 of greater than 2 and p-values < 0.05, representing a statistically significant chance of worse disease-specific survival (DSS) and disease-free survival (DFS) outcomes (see Materials and Methods for definitions of DSS and DFS). This trend is also evident in the Kaplan-Meier estimate of the DFS and DSS survival functions [Fig. 6(c)] where TACS-3 positive patients show significantly higher frequency of events. The p-values in this case were computed using the log-rank test57.

Our results are significant because in addition to quantitating the TACS-3 marker, the cSLIM system provides high-throughput acquisition, operates on existing stained tissues, and provides simultaneous bright-field color images. The instrument, thus, allows the assessment of traditional prognostic markers (e.g. tumor grade and molecular subtype) as well as new prognostic markers (such as TACS-3) in a single scan. This advantage is obtained at the expense of some additional instrument optics with no additional constraints on sample preparation.

Summary and Conclusion

In summary, we presented cSLIM, a tissue imaging modality that provides stain-independent, quantitative markers of disease while simultaneously providing traditional histopathology images. Our instrument facilitates clinical translation of our disease markers by posing no new sample preparation requirements and providing traditional and novel markers in a single acquisition. These advantages are obtained by simply upgrading an existing microscope to a phase-sensitive instrument with an optics module at the output port. Because cSLIM provides full-field diffraction-limited imaging with visible light, it results in much higher resolution and throughput with relatively inexpensive equipment, compared to other label-free methods (vibrational spectroscopy and non-linear microscopy). We demonstrated that cSLIM normalized phase maps are stain-independent by comparing the results of phase imaging of tissue before and after staining. We also demonstrated that disease markers, previously found to be effective for unstained tissue biopsies, are extendable to stained tissues.

Building on these promising results we then demonstrated what is the main application of the cSLIM system for histopathology - extracting stromal collagen-based prognostic markers. By providing convenient access to these markers our instrument can potentially help pathologists predict disease aggressiveness in patients for whom other more traditional disease markers fail. In the past prognostic markers related to collagen fiber alignment have been measured using SHGM. Their measurement using cSLIM provides advantages of higher throughput, higher signal in cellular structures and lower additional instrumentation costs compared with SHGM. While some past works have used machine learning to extract prognostic information directly from bright-field images of H&E-stained stromal tissue58,59,60, these approaches do not measure alignment metrics of individual collagen fibers. This is due to the difficulty in extracting this information in H&E-stained tissue images, which is why SHGM has been used for this purpose instead. In addition, as with all analysis on stained tissue, such bright-field approaches face the challenge of signal variability due to changes in staining intensity.

The simultaneous acquisition of tissue refractive index and stain-absorbance related information also opens many possibilities in terms of automated image analysis. Deep-learning algorithms are increasingly being applied for image analysis on bright-field H&E images6,61. With cSLIM, these approaches can be combined with phase-imaging related algorithms discussed in this work and others12,34. These multimodal approaches can be especially useful in difficult cases for pathologists, where complementary information is needed rapidly and objectively.

While we focused here on breast diagnosis and prognosis, it is evident that cSLIM can contribute to the pathology of other diseases. Furthermore, imaging tissues stained with other agents besides H&E is also feasible with cSLIM without modifications. For example, immunochemical stains often contain less dye, which reduces the effect on the phase values. For these reasons, we anticipate that cSLIM will have a low threshold for adoption at large scales.

Recently, it has been shown that deep learning can be used to predict H&E images from unlabeled tissues62. It would be interesting to study in the future the reverse process, i.e., whether, in our case, SLIM images can be predicted from H&E stained tissues. While the accuracy of this process is expected to be limited, given that microenvironment structures (e.g., collagen fibers) are visible with high contrast in SLIM but do not stain well in H&E, finding these limits may prove valuable.

Materials and Methods

Tissue microarrays

TMA-1 was purchased unstained from US Biomax (Serial #BR1002) and comprised cases of IDC and normal benign tissue. The TMA was obtained from the manufacturer with all human subject identifiers removed. Neither the authors nor their institutions were involved in the tissue collection. Details regarding this TMA have already been reported in ref.34. Case-by-case diagnosis was provided by the manufacturer’s board-certified pathologist through examination of both H&E-stained tissue and immunohistochemical (IHC) markers, both on parallel sections of tissue. After acquiring unstained tissue phase images, the TMA was H&E-stained for cSLIM imaging using standard protocols63. Before staining, the coverslip was removed from the slide and post-staining the slide was re-coverslipped using the same mounting medium as before (Permount). As discussed in Results and Discussion, 15 cases of IDC and 15 cases of normal tissue were randomly selected from the TMA for the studies done in this paper. One core per case was available. Each biopsy core had a diameter of 1 mm and a thickness of 5 μm. For the IDC cases, a second board-certified pathologist also marked any benign regions within the tissue cores which were then excluded from the analysis. In this way, diagnosis of each EC within each core (Benign or Malignant) was available.

TMA-2 was used in previous studies by Bredfeldt et al.13 and Conklin et al.15. Details regarding patient profiles, tissue processing and core selection have been described in ref.15. The dataset used from the TMA consisted of 196 cores (1 core per patient) and patients were followed up for a median time of 6.2 years, ranging from 1–223 months in order to determine patient outcomes. DSS and DFS information was available for each patient. DSS was defined as the time from diagnosis to death from breast cancer or date of last follow up evaluation. DFS was defined as the time from date of diagnosis to the first date of recurrence. TMA-2 was H&E-stained using standard protocols63, allowing for simultaneous acquisition of both normalized phase and bright-field images using the cSLIM system. All patient information was de-identified before the TMA was obtained and used in the current study.

Epithelial tissue segmentation for feature extraction

For computation of epithelial features during supervised learning as well as for measurement of relative fiber orientation, knowledge of the EC boundary is required. For TMA-1, the EC boundaries were annotated in all the cores manually using the region-of-interest tool in ImageJ by using the H&E-stained tissue bright-field images as a guide. Consistent criteria were used during annotation– groups of epithelial cells bounded by stroma on all side were considered a single EC. Other tissue components were considered part of the EC if surrounded on all sides by epithelial cells34. While for these studies, manual delineation of epithelial tissue was performed before feature extraction, automated segmentation of epithelial tissue is feasible on our images (as shown below for TMA-2) given that both phase and color imaging based algorithms have been developed for the purpose12,48.

For TMA-2 we used the same EC segmentation masks as in ref.13. These masks were generated through an automated segmentation algorithm, relying on the supervised learning scheme discussed in13. We registered the bright-field images from cSLIM with those from the original study by using Speed-Up Robust Features (SURF)64. The Z(x, y) maps were, thus, registered with the EC segmentation masks as well.

Classification scheme for cancer diagnosis

The supervised classification of benign versus malignant lesions (Fig. 4 in Results and Discussion) is based on the procedure we reported in ref.34. We apply this procedure to our Z(x, y) images, in three steps: feature extraction, training, and validation.

During feature extraction, first maps of the EC curvature C, mean scattering length ls and texture vector T were extracted for each core within our datasets. The EC curvature C refers to the extrinsic curvature of a two-dimensional plane (in this case a benign or malignant EC) and can be construed as the magnitude of the rate of change of a vector tangent to the EC perimeter. We used an open-source MATLAB code to measure C for each annotated EC65. The mean scattering length ls is the length-scale over which a single scattering event happens on average and can be computed from tissue phase images using the scattering-phase theorem66. The texture vector T consists of frequencies of elements known as ‘textons’ within the vicinity of a pixel in the image. Textons have been shown to be effective measures of the unique texture surrounding a pixel12,67,68. To separate benign and malignant lesion in this paper, contrary to our previous work where 50 textons were trained (resulting in a 50 dimensional vector T)34, we found that 30 textons were sufficient due to the smaller size of the dataset. This number was obtained iteratively by measuring the cross-validation AUC (discussed below) while increasing the number of textons and stopping at the point where no improvement in AUC was noticed, to avoid overfitting. For feature extraction, all other parameters were identical to those used in ref.34.

After pixel-wise computation of these features, the median over each EC was calculated for each feature [〈C〉, 〈ls〉, 〈T〉], using the EC masks obtained through manual segmentation (see section on Epithelial tissue segmentation for feature extraction). These features were then concatenated to generate an overall (32 dimensional) feature vector for each EC. Using pathologist diagnosis result for each EC as the class label (benign or malignant) and its overall feature vector as the predictor, an LDA classifier was trained. During validation, feature vectors for unknown ECs were input to the classifier which generated likelihood scores for the output classes. During validation, the overall data set was partitioned into three equal partitions. Three validation trials were performed (3-fold cross validation)69. In each trial, two partitions were used for training and the remaining one for validation. The classifier performance was measured using ROC curve analysis. Likelihood scores for each EC, generated by the classifier from the three validation trials, were pooled together70 to generate an overall ROC curve [Fig. 4(i)] and the AUC was used as a metric for classifier accuracy.

Fiber orientation extraction on TMA-1 using CurveAlign

In Results and Discussion, we compared θ, the relative angle between the orientation of a collagen fiber and the tangent to the nearest point on the tumor edge [depicted in Fig. 5(c)], between benign and malignant cases. The results were extracted using an open source MATLAB-based tool called CurveAlign, algorithmic details of which have already been described in a number of publications13,54,71,72. For our analysis, we chose the Curvelet Transform (CT) based fiber analysis method within CurveAlign. This method uses the curvelets provided by curvelet transformation73 of the image to represent the edges of collagen fibers, without segmenting the individual fibers. From computation of these curvelets, thus, scale, location and relative orientation of each fiber can be calculated54. TIFF files that contained masks of the ECs (obtained through manual annotation) were used in the ‘Boundary Method’ field within CurveAlign. The fraction of coefficients to keep, during curvelet transform computation, was set at 0.005 and the distance from the tumor edge, up to which fibers are analyzed, was set to 100 pixels or approx. 63 μm. Before extraction of θ, ECs were segmented out from all the core images so that the cellular structures within them did not interfere with the process of fiber extraction during curvelet transformation.

TACS-3 measurement on TMA-2 using CT-FIRE and CurveAlign

As described in Results and Discussion, for detecting the TACS-3 prognostic marker on cSLIM Z(x, y) maps the same general method as used in ref.13 was employed. We summarize that analysis method here. First, the ECs within each core were segmented out using its corresponding segmentation mask (see section on Epithelial tissue segmentation for feature extraction). This step was carried out to ensure that during subsequent fiber segmentation, cell edges did not interfere. CT-FIRE was then used to segment out all fibers within the Z(x, y) map of each core. Default parameters, as outlined in the CT-FIRE manual55, were used except for the parameters labelled “thresh_im2” and “s_xlinkbox” for which values were set to 30 and 5 respectively. These fiber segmentation maps, along with EC segmentation masks, were then input to CurveAlign for extraction of fiber features. Features were computed for all fibers that were within a distance of 100 μm from the tumor edge. In CurveAlign, the “CT-FIRE Fibers” fiber analysis method was chosen and the “TIFF Boundary” was chosen as the boundary method. CurveAlign extracts a total of 34 fiber features as part of its standard computation. These features are related to the fiber curvature, width, length, density, alignment, proximity to epithelium and relative angle to epithelial boundary13. As discussed in the main text, core-level statistics derived from 3 features (related to alignment, proximity to epithelium and relative angle) were found to be the best predictors of DSS and DFS. The first of these features was the mean nearest alignment ε, defined as the mean of the alignment of a fiber to its nearest 2, 4, 8 and 16 fibers. The algorithm for measuring the alignment of a fiber to its nearest neighbors has already been described in ref.13. The second feature used was the distance of each fiber to the nearest EC boundary, denoted l. The final feature was the relative fiber angle θ. Means of ε and l and skewness of θ, over all fibers in each core, were used as predictors for SVM training (Results and Discussion). For survival analysis, the Kaplan-Meier estimate was computed using an open-source MATLAB code74.