Virtual histological staining of unlabeled autopsy tissue

Traditional histochemical staining of post-mortem samples often confronts inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, and such chemical staining procedures covering large tissue areas demand substantial labor, cost and time. Here, we demonstrate virtual staining of autopsy tissue using a trained neural network to rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images, matching hematoxylin and eosin (H&E) stained versions of the same samples. The trained model can effectively accentuate nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining fails to provide consistent staining quality. This virtual autopsy staining technique provides a rapid and resource-efficient solution to generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.


Introduction
Autopsy, also referred to as post-mortem examination, is a critical medical procedure that entails a comprehensive examination of a deceased individual's body 1 . The autopsy process mainly involves a gross examination, which encompasses a naked eye-based evaluation of the body and its organs at a macroscopic scale, often followed by multiple organ sampling for microscopic histological examination. This examination provides crucial information by offering insights into cellular and molecular changes that occurred in the deceased tissue, which can further aid in determining the cause and manner of death 2 , enabling a more comprehensive study of the disease progression 2 and evaluating the effectiveness of any medical treatments 3 .
However, histochemical staining of autopsy samples presents several challenges. One of the primary challenges is the staining artifacts that frequently occur in formalin-fixed, paraffin-embedded (FFPE) autopsy tissue sections due to delayed fixation. Fixation is a foundational step in the study of pathology and prevents the degradation of tissue and tissue components, ensuring that their features can be preserved and observed anatomically and microscopically following tissue sectioning 4 . Owing to the inherent delay of post-mortem processing, the tissue of a deceased body remains unfixed for a considerable duration, which results in an extended period of autolysis-a process of selfdigestion that occurs in cells and tissues after death or when they are not properly preserved. This process causes various physical and chemical changes in the cellular and tissue structures, such as morphology distortion, vacuolation, and cytoplasmic basophilia 5 , impeding the chemical binding between the staining dyes and biomolecules in the tissues.
As a result, various staining artifacts, including poor nuclear contrast and color fading in cytoplasmic-extracellular tissue staining, are introduced compared to staining of tissue samples with no processing delays, such as biopsy samples 6,7 . Furthermore, since the tissue samples extracted during an autopsy are often very large, their fixation time is prolonged compared to small tissue fragments, considering formalin's slow tissue penetration rate (<1 mm/hour 8 ).
This leads to further delays in the fixation of the inner regions of the extracted tissue samples, resulting in pronounced autolysis in these autopsy regions and, consequently, a further decline in the staining quality 9 . Together, these staining artifacts negatively affect the pathological interpretation of autopsy specimens, reducing the reliability of the tissue examination compared to non-autolytic tissue staining 5 .
In addition to these staining challenges, the current workflow for histochemical staining is costly, time-consuming, and labor-intensive, as it demands complex sample processing procedures carried out by skilled technicians 10,11 . The challenge of meeting these demands -in terms of reagents, laboratory infrastructure, and professional labor -becomes overwhelming, especially during global health crises such as COVID-19, when a marked increase in fatalities 3 intensifies the need for rapid and accurate autopsy sample analysis. Such an increased need for autopsy analyses and the shortage of related resources can cause severe delays in post-mortem processing and histochemical staining procedures, further compromising the staining quality and complicating the image interpretation.
With the rapid advancement of artificial intelligence in recent years, deep learning technologies have introduced transformative opportunities to biomedical research and clinical diagnostics [12][13][14] . One notable application of this is the use of deep learning for virtual histological staining of label-free tissue samples 10,15,16 . In this technique, a deep neural network (DNN) is trained to computationally transform microscopic images of unlabeled tissue sections into their histologically stained counterparts, constituting a promising solution to circumvent the challenges associated with traditional histochemical staining. This deep learning-based virtual staining technique has been extensively explored by multiple research groups and successfully applied to generate a range of histological stains, such as H&E 10,17-29 , Masson's trichrome staining 10,18,20 and immunohistochemical staining 30,31 . These previous works utilized images from various label-free microscopy modalities, including autofluorescence microscopy 10,20,[23][24][25]30 , quantitative phase imaging 18,32 , photoacoustic microscopy 27,29,33 and reflectance confocal microscopy 26 , among others 17,19,21,22,28,31,34-36 . However, these earlier studies have primarily focused on standard biopsy samples, and there has been no virtual staining study on autopsy samples and other large specimens, which often demonstrate suboptimal staining quality with traditional histochemical approaches due to delayed fixation and autolysis.
Here, we report the first demonstration of virtual staining of label-free autopsy tissue samples. As depicted in Fig.   1(a), we use a convolutional DNN to digitally transform autofluorescence images of label-free lung tissue sections into their corresponding brightfield H&E stained versions, effectively circumventing autolysis-induced staining artifacts inherent in traditional histochemical staining processes. To successfully achieve virtual staining of autopsy tissue and mitigate the aforementioned challenges, we created a novel and data-efficient deep learning framework, termed RegiStain, which incorporates concurrent training of an image registration neural network along with a virtual staining network to progressively facilitate the accurate image registration between the virtual staining network's output and the histology ground truth. Utilizing a structurally-conditioned generative adversarial network (GAN) scheme [37][38][39] , the RegiStain framework was trained using autofluorescence images of well-preserved, unstained autopsy tissue areas (obtained before the COVID-19 pandemic) as the network input and their well-stained "select" H&E histology images as the ground truth (corresponding to well-preserved tissue regions), as illustrated in Fig. 1(b), left.
After its training and validation using >0.7 TB of microscopic image data, as illustrated in Fig. 1(b), right, this virtual staining network, despite being trained solely using well-fixated tissue samples, can perform rapid and accurate virtual H&E staining of label-free lung tissue sections that experienced severe autolysis due to delayed fixation, including those from COVID-19-induced pneumonia autopsy samples. Our virtually stained tissue images exhibit a remarkable improvement in staining quality compared to standard histochemical staining by effectively highlighting nuclear, cytoplasmic, and extracellular features, which were not clearly visible using traditional histology, indicating the model's resilience to accommodate unseen variations in tissue fixation quality. Furthermore, we quantitatively evaluated the staining quality of our virtual H&E autopsy model by comparing the virtually stained tissue images with their well-stained histochemical counterparts, selected only from well-preserved autopsy tissue areas. These 4 quantitative comparisons involved digital image analysis algorithms as well as score-based evaluations by boardcertified pathologists, overall revealing no statistically significant differences between the virtual and histochemical staining results for these well-preserved autopsy samples.
This post-mortem virtual histology staining technique can substantially save time, reagents, and professional labor, which would be particularly valuable in demanding scenarios such as global health crises, where rapid escalation in the number of cases necessitates efficient and swift examination techniques. We also envision that our investigations can be extended beyond the realm of autopsies to encompass the staining of necrotic tissue, which poses histochemical staining challenges similar to those presented by autolytic tissue. histochemical staining (top) requires chemical sample processing procedures performed by histotechnologists, which are time-consuming, labor-intensive, costly, and prone to potential staining failures caused by tissue autolysis in autopsy samples. In contrast, a deep learning-based virtual staining neural network (bottom) can be used to perform rapid, costeffective, and accurate virtual staining of unlabeled autopsy tissue sections based on their autofluorescence microscopy images, which can provide high-quality staining results even in autolytic tissue areas where the histochemical staining fails. b, In RegiStain, we employ a novel training strategy where the training of the virtual staining network and the image registration network are mutually optimized; the virtual staining network is efficiently trained to learn the intrinsic mapping between the tissue autofluorescence texture and the H&E stained image texture. This training was performed exclusively using well-preserved autopsy samples (collected before , and once the training was completed, the virtual staining network model was used to successfully stain autopsy samples that experienced severe autolysis, obtained from COVID-19 as well as non-COVID-19 cadavers.

Training of the autopsy virtual staining network using RegiStain framework
To train our DNN model for virtual staining of unlabeled autopsy samples, we obtained paired image data, both preand post-histochemical H&E staining. These paired images must be accurately aligned in space so that the network can learn the effective image transformation functions between the autofluorescence and brightfield imaging modalities. For this purpose, we collected unlabeled lung tissue samples from eight cadavers diagnosed with pneumonia (prior to the COVID-19 pandemic) and captured microscopic autofluorescence images of these tissue sections using two standard fluorescence filter cubes, DAPI and TxRed. Following label-free imaging, these autopsy tissue samples underwent the standard process of H&E histochemical staining and digitization, which resulted in whole-slide brightfield microscopic images of H&E-stained tissue samples (serving as ground truth targets), paired with their autofluorescence counterparts (serving as network inputs). These whole-slide image (WSI) pairs further underwent an affine transformation-based image registration, followed by division into smaller image patches, each with a size of 3248×3248 pixels. The resulting paired image patches then underwent a final round of affine registration, ultimately resulting in a training dataset of 16,159 paired microscopic image patches.
However, the affine transformation-based registration performed at this image patch level is not sufficient to address all the local spatial mismatch between these image pairs, which can be attributed to the unavoidable morphological deformations of tissue samples induced by the histochemical H&E staining process, as well as the optical aberrations associated with different microscopic imaging systems. To mitigate these issues, a much finer image registration process using elastic registration algorithms can be employed to achieve pixel-level alignment between the paired images. This step is typically utilized in training image data preparation for supervised learning of virtual staining models for biopsy tissue samples 10,30,40 . However, due to its iterative and intense computational nature, such an elastic registration process is generally highly time-consuming and requires substantial data storage. Moreover, because each autopsy slide's sample area is much larger than a typical biopsy slide, performing fine registration on the entire autopsy dataset using conventional elastic registration algorithms becomes impractical. In our specific training task, such a fine registration process would normally take months to complete and require more than 1 terabyte of data storage.
As an efficient and highly accurate solution to train our autopsy virtual staining network, we created the RegiStain framework, which integrates the image registration process with the training of the virtual staining generator model. This significantly reduces the time required for elastic image registration while simultaneously enhancing the precision of image registration, thereby enabling the efficient use of our massively large autopsy image dataset within a reasonable training time. As illustrated in Fig. 2, this framework consists of three distinct convolutional neural networks (CNNs): the virtual staining generator network (G), the discriminator network (D), and the image registration analysis network (R). Specifically, the G and D networks form a structurally-conditioned GAN module, with the former performing virtual staining by transforming autofluorescence microscopic images of label-free tissue into H&E equivalent brightfield images. Network R compares the virtually stained images outputted by network G and their histochemically stained corresponding ground truth, coarsely registered using affine transformation-based registration steps, and it rapidly outputs a displacement vector field (DVF) that characterizes the pixel-wise relative displacement between the two images. Compared to elastic image registration methods that are based on iterative multi-scale image cross correlations 10,20,30,40 , network R substantially shortens the time required for the training image registration process 41 . In each batch of the learning, the resulting DVF is further fed into a spatial transformation module, which consequently aligns the histochemically stained ground truth images based on the DVF to ensure precise registration with the output of the network G (see the Methods); this process dynamically corrects and aligns the training image targets for the network G. As the training progresses, these networks improve their respective capabilities, which can be attributed to frequently alternating iterations between the training of the networks. Specifically, as the network R gradually refines its capability to execute the image registration analysis, the network G, within the GAN module, concurrently improves its learning of the image transformation function, generating images increasingly similar to their ground truths. This, in turn, further aids the network R's learning process. Therefore, the networks G and R evolve symbiotically during the learning process, each benefiting from the other's iterative improvements. Moreover, the loss functions for training the networks G and R are uniquely tailored to focus on distinct aspects of the image features, i.e., color intensity and morphological structures for G and R, respectively, enforcing desired performance in their respective tasks (see the Method section for details). This reciprocal and symbiotic iterative enhancement leads to an optimal equilibrium, wherein the network R can accurately estimate the DVF between a virtually stained image and its histochemically stained ground truth, and the network G exhibits proficient virtual staining capability.
It is also worth highlighting that our RegiStain framework is designed as a plug-and-play system. This means none of the networks (G, D, and R) would need a particular initialization or pre-training process, rendering the RegiStain framework highly applicable for rapid, efficient training using large amounts of autopsy tissue images. As a result of this RegiStain framework, the total training time required for our autopsy tissue data (~730 gigabytes), including all the precise image registration steps dynamically implemented through network R, is drastically shortened to ~60 hours, which would normally take months using, e.g., iterative elastic image registration methods running on the same computing hardware (see the Methods section). Upon the completion of training, only network G is used in the testing/inference phase, forming the autopsy virtual staining DNN model that promptly infers virtually stained H&E images based on the autofluorescence images of label-free autopsy samples.

Virtual staining results of unlabeled autopsy tissue sections
Following the training phase, we evaluated the performance of our trained DNN model (network G) for autopsy virtual staining of lung tissue. This evaluation utilized ten new autopsy sample slides, all of which underwent the same autofluorescence image acquisition, histochemical staining, and digitalization process as their counterparts in the training set. These ten autopsy slides originated from ten different patients (never seen by the network before), with three of them diagnosed with COVID-19-induced pneumonia and the remaining seven diagnosed with infectious pneumonia, but not related to COVID-19. Due to the frequent occurrence of autolysis in tissues extracted post-mortem, all these testing slides presented staining issues and artifacts of different degrees. These staining artifacts ranged from inadequate labeling of the cell nuclei (due to weak binding with hematoxylin) to insufficient coloring of cytoplasmic and extracellular regions (due to weak binding with eosin). Given the suboptimal staining quality in these regions, they could not be effectively employed as references (ground truth) for evaluating the quality of our virtual staining 8 results. Consequently, it became imperative to distinguish between fields-of-view (FOVs) with high staining quality and those with staining artifacts before embarking on further steps of this evaluation process.
To achieve this, we established an image feature-based staining artifact identification process. To begin with, we divided all ten WSIs of the autopsy test samples into ~2000 FOVs, each corresponding to an area of 1.3 × 1.3 mm 2 (8000 × 8000 pixels). Following that, we performed a quantitative assessment of these FOVs using two metrics: (1) the area percentage of stained nuclei within the tissue region, and (2) the average intensity of adjacent cytoplasmicextracellular regions; both of these metrics were calculated for each one of these 2000 histochemically stained FOVs.
Based on these quantitative scores, these FOVs were split into two distinct categories: well-stained areas and poorlystained areas. Supplementary Fig. S1 illustrates the workflow of this quantification process, and additional details can be found in the Methods section. These analyses revealed that autolytic tissues were predominantly found in four autopsy slides, exhibiting low-quality staining in ~60-80% of the total tissue area. The remaining six slides exhibited preserved histology for ~95% of their total tissue areas. Notably, three of the four pneumonia samples suffering from severe autolysis-related artifacts were obtained from COVID-19-positive cadavers. This finding correlates with the fact that the average time interval from death to autopsy for COVID-19 pneumonia samples in our study was significantly longer (~20 days), where the increased delay in fixation resulted in a much higher degree of autolysis in the tissue. We provide more detailed information on these autopsy slides in Supplementary Table 1, which includes the time elapsed from death to autopsy for each test case and their evaluation results in terms of the percentages of tissue areas exhibiting staining artifacts for each autopsy slide. Taken together, ~26% of all the test sample FOVs from the ten autopsy slides exhibited staining artifacts in their histochemical H&E staining results.
After this initial quality screening of 2000 unique autopsy FOVs, next, we inspected the virtual staining results for the test FOVs that exhibit histochemical staining artifacts. A comparative visualization of these results is provided in Fig.   3, using an autopsy sample slide that exhibited staining artifacts in ~62% of its total area. Figures 3(a), (

b), and (c)
provide an overview of the autofluorescence, virtually stained and histochemically stained H&E WSIs of the sample, respectively. From the histochemically stained H&E images of these regions shown in Fig. 3(d4), (e4), and (f4), it is clear that a significant number of nuclei appear faded due to inadequate staining. On the other hand, virtual H&E staining results are significantly better, as shown in Fig. 3(d3), (e3), and (f3), which reveal that the nuclear details and the structures of various cell types, including neutrophils, lymphocytes, and macrophages, are substantially enhanced, greatly surpassing the level of clarity offered by their histochemically stained counterparts. An additional exemplary WSI from another autopsy case, which exhibits under-staining issues at the cytoplasmic and extracellular regions in their histochemically stained results, is also shown in Fig. 4, demonstrating similar findings. As showcased in Fig. 4(d4), (e4), and (f4), the cytoplasmic and extracellular areas within these local regions present staining artifacts revealing faded colors, while their virtually stained counterparts shown in Fig. 4(d3), (e3) and (f3) provided considerably enhanced color intensity at the same locations. Overall, the virtually stained H&E images generated by our autopsy virtual staining network exhibit a much superior staining quality compared to their histochemically stained counterparts, demonstrating the effectiveness of our framework in mitigating the staining challenges observed for autopsy tissue slides.  These DAPI and TxRed autofluorescence image pairs serve as the inputs to our autopsy virtual staining network. (d3), (e3), and (f3) are the virtual H&E staining results corresponding to the same regions of (d1), (e1) and (f1) (or (d2), (e2) and (f2)), which are digitally generated by our virtual staining network based on (d1) and (d2), (e1) and (e2), and (f1) and (f2), respectively. (d4), (e4), and (f4) are the histochemical H&E staining results corresponding to the same regions of (d3), (e3), and (f3), which exhibit under-staining artifacts in their cytoplasmic and extracellular regions. In addition to these staining artifact-related analyses reported in Figs. 3-4, we show in Fig. 5 additional examples that depict the efficacy of the virtual H&E staining on well-preserved tissue regions characterized by high-quality histochemical staining; each of these regions is from a distinct autopsy sample slide. A notable degree of visual similarity and structural alignment can be discerned between the virtually stained and histochemically stained H&E images (corresponding to well-preserved FOVs), thereby providing additional evidence that underscores the effectiveness of our virtual staining model. Importantly, even though our virtual staining neural network model was trained solely using autopsy samples from non-COVID-19 patients, it consistently demonstrated high performance when applied to previously unseen samples from COVID-19 patients. This consistency suggests that the mapping function between the autofluorescence texture of label-free autopsy tissue to histochemically stained H&E image texture, learned by our virtual staining network model, can accommodate potential variations unseen during the training phase.

Quantitative evaluation of autopsy virtual staining results based on digital image analysis
In addition to the visual comparisons summarized earlier, we also exploited digital image analysis algorithms to evaluate the image quality of our autopsy virtual staining results in a quantitative manner. For this analysis, we first excluded all the test sample FOVs that were found to exhibit staining artifacts in their histochemical staining results through our previous identification process, which account for ~26% of all the test sample FOVs. Within the remaining sample FOVs that have well-stained ground truth (corresponding to well-preserved autopsy tissue regions), we then randomly selected a set of 100 FOVs, where 70 FOVs were from the seven non-COVID-19 pneumonia lung tissue samples, and 30 FOVs were from the three COVID-19-induced pneumonia lung tissue samples. The virtual staining results corresponding to these 100 FOVs were then compared with their histochemically stained counterparts using the following four metrics: (1) the structural similarity index (SSIM) 42 , (2) the peak signal-to-noise ratio (PSNR), (3) the nuclear count distribution, and (4) the nuclear size distribution. See the Methods section and Supplementary Fig.   S4 for further details on this quantification.
The results of this analysis are summarized in Fig. 6, represented by box plots that show the statistical distribution of the aforementioned metrics quantified across all the test sample FOVs. Our results reveal that the calculated SSIM and PSNR values consistently exceed 0.8 and 20, respectively, presenting a high degree of structural similarity between the virtual H&E and histochemically stained H&E images. Moreover, the quantification of the cell nuclei (in terms of their number and average size) in the virtually stained H&E images exhibits a good agreement with the box plots derived from the corresponding histochemically stained H&E images (see Fig. 6). This is further substantiated by the calculated P values based on a paired t-test (two-tailed), which indicates that there is no statistically significant difference (P > 0.05) between the virtually stained and histochemically stained H&E images in terms of the distribution of nuclei number and size. All these quantitative comparisons provide compelling evidence for the match between the virtually stained H&E images produced by our staining network model and their well-stained histochemical counterparts (from the well-preserved tissue regions).

Score-based evaluation of autopsy virtual staining results by board-certified pathologists
In addition to our quantitative assessment of autopsy virtual staining results using digital image analysis summarized in Fig. 6, we also performed a score-based quantitative evaluation by four board-certified pathologists. This was conducted using the same set of 100 test sample FOVs that have well-stained histochemical ground truth corresponding to well-preserved tissue regions. In this analysis, we randomly mixed the virtually stained and the histochemically stained H&E images corresponding to these 100 test FOVs, forming a set of 200 images. We also randomly flipped, rotated, and shuffled these images to ensure their sequence and orientation were random; these randomized images were then sent to four board-certified pathologists for their quantitative evaluation. The pathologists were requested to independently assess the staining quality of each image from four different aspects: (1) the existence of staining artifacts, (2) extracellular detail, (3) cytoplasmic detail, and (4) nuclear detail. For each of these categories, they assigned every image (each corresponding to a well-preserved unique tissue FOV) a score between 1 and 4, where 4 indicates "perfect" results, 3 represents "very good", 2 stands for "acceptable", and 1 is for "unacceptable" quality. In addition to this blinded stain quality evaluation, they were asked to assign a cellularity score to each image, also on a scale of 1 to 4. Here, 1 indicates a low cell count, 2 stands for a fair number of cells, 3 represents a substantial number of cells, and 4 means an extremely high cell count. Importantly, during this evaluation process, the pathologists were not provided with any prior information about the origin of each image, namely, whether it was created by our virtual staining technique or the conventional histochemical H&E staining process to ensure an unbiased comparison between the virtually and histochemically stained images.
The results of this score-based quantitative evaluation by board-certified pathologists are reported in Fig. 7. Figure   7(a) presents the staining quality scores of virtually stained images and histochemically stained images corresponding to only well-preserved tissue regions. The mean and standard deviation values of each reported metric were calculated across all the 100 test FOVs and across all four pathologists. These results present close statistical distributions between the staining quality scores obtained for the virtually stained images and the histochemically stained results, demonstrating the consistency of our virtual staining model. A similar conclusion can be derived from the cellularity assessment summarized in Fig. 7(b). As desired, the cellularity distributions corresponding to the virtually stained images and the histochemically stained images consistently exhibit a good match with each other. We also conducted a paired t-test (two-tailed) for each metric of the staining quality evaluation as well as the cellularity assessment per pathologist. The results of these statistical analyses are collated in Supplementary Fig. S5, along with individual illustrations of staining quality scores for each evaluation metric and illustrations of cellularity assessment per pathologist. These statistical analyses of the staining quality scores, presented in Supplementary Fig. S5(a), revealed that for the metrics related to "free of staining artifacts" and "extracellular detail", three out of the four pathologists did not find the histochemically stained H&E images statistically significantly superior to the virtual H&E staining results. Similarly, in terms of the quantitative metrics related to "cytoplasmic detail" and "nuclear detail", half of the pathologists did not consider the histochemically stained H&E images to significantly outperform their virtually stained counterparts. Moreover, the statistical analysis of the cellularity assessment (Supplementary Fig.   S5(b)) indicated that no pathologist found a statistically significant difference in the cellularity distributions between the virtually and histochemically stained H&E images. These findings further substantiate the reliability and accuracy of our virtual H&E staining model in producing high-quality results.
We should emphasize that the conclusions of these statistical analyses, revealing a comparable staining quality between the virtually stained and histochemically stained H&E images, are solely based on the FOVs selected from the well-preserved tissue regions. However, when factoring in the remaining ~26% of the testing image dataset that presented tissue preservation-related staining artifacts (see e.g., Figs. 3-4), our virtual staining method, in general, exhibits a consistent and superior staining quality compared to histochemical H&E staining of autopsy tissue samples.

Discussion
Challenges associated with the histochemical staining of autolytic tissue, as highlighted in this work, are commonly known to pathologists, and they can significantly impede accurate histological evaluations. In particular, autolyzed tissue shows increased eosinophilia due to the loss of normal basophilia imparted by ribonucleic acid (RNA) in the cytoplasm and increased binding of eosin to denatured intracytoplasmic proteins 43 , thus adversely affecting the normal staining of these tissues by H&E. Although most of the histological feature changes observed through microscopic evaluation directly result from cellular alterations in autolytic tissue, it is important to note that various anomalies arise from less effective chemical staining processes. For instance, a change of tissue pH caused by cell lysis and their intracytoplasmic content spillage into the extracellular space can reduce stain avidity; hence even cells that maintain their membrane integrity (such as certain white blood cells) may appear pale and lack nuclear contrast. Encouragingly, our autopsy virtual staining technique successfully stained these cells, enabling practitioners and researchers to effectively visualize them in digitally stained slides. To be more specific, after conducting a comprehensive analysis of the particular tissue components, which exhibited enhancements in our virtual staining model, we determined that significant improvements were observed in white blood cells, predominantly lymphocytes, and to a lesser extent, neutrophils, and macrophages; see for example Fig. 3. This finding aligns well with prior research that highlighted the inherent resilience of immune cells to hypoxic changes when compared to other cell types, as well as their relative durability under autolytic conditions 44 .
A key element behind the success of our autopsy virtual staining technique lies in the high-quality training dataset we secured. Through an algorithm-based filtering and manual screening process, the training image pairs of our dataset did not harbor any histochemical staining issues prevalent in autolytic tissue, and we also eliminated other problems, including image defocus, tissue damage or detached areas that randomly occur during the histochemical staining process. To achieve this, an initial training dataset was constructed based on the algorithm depicted in Supplementary   Fig. S1, where the training image pairs were selected only from the well-preserved tissue regions with high staining quality of nuclear, cytoplasmic, and extracellular features. Then, all the autofluorescence images in the training dataset (network input) were transformed into their H&E stained counterparts (network output) using an initially trained virtual staining network (which is not the final one -only used for pre-screening of training data) and then compared with their corresponding histochemically stained images (network target). Those output and target image pairs that fall below a PSNR of 15 or an SSIM of 0.6 were further filtered out (rejected) to avoid any mismatch between each training image pair caused by e.g., potential tissue damage, tissue folding, or image defocus problems. The remaining image pairs in the training dataset were finally reviewed by the authors to exclude any remaining artifacts in the training images. These rigorous steps ensured that the image pairs used for the training of RegiStain framework were of high quality and not contaminated by artifacts that might impair the training.
Another pivotal factor in the success of our autopsy virtual staining technique is the RegiStain framework itself. As mentioned earlier, this innovative framework utilizes a symbiotic iterative collaboration between an image registration analysis network (R) and the virtual staining network (G), which significantly enhances the efficiency of training image registration and the learning of the virtual staining network, while also eliminating the need for constructing a pixel-level registered paired image dataset that often requires extensive computational resources involving e.g., iterative elastic registration algorithms. Importantly, RegiStain enables a coarsely registered training image dataset to be sufficient for training a high-performance virtual staining network model, reducing the whole training process to a couple of days instead of months. To shed more light on this point, we further conducted an ablation study to ascertain the influence and necessity of incorporating an image registration analysis network (R) for the development of a competitive virtual staining model. In this comparative analysis and ablation study, we again performed the same training process, except that network R was removed from the training framework. The results of this training are exemplified in Supplementary Fig. S6, which clearly reveals a significant performance degradation in the outputs of the virtual staining network G when network R was not part of the training framework. In particular, due to the absence of fine image registration in the training, the virtually stained H&E images produced by this comparison model showed noticeable staining artifacts, such as hallucinations in the nuclear features and over-staining of cytoplasmic and extracellular regions. A subsequent algorithm-based quantitative analysis (Supplementary Fig. S7) further confirmed these observations: the average SSIM and PSNR values between the virtually and histochemically stained H&E images (corresponding to well-preserved tissue regions) without the involvement of network R were only 0.78 and 18.25, respectively, which are lower than the results of 0.82 and 20.32 obtained when network R was employed.
Moreover, in the virtual staining results without using network R, there was a statistically significant difference in the distribution of the number of cell nuclei per FOV and the average nuclei size between the virtually stained images and their ground truth corresponding to well-preserved tissue regions (P < 0.05, using a two-tailed paired t-test); on the other hand, this difference was not statistically significant when the network R was incorporated during the RegiStain based training. All these findings underscore the crucial role that the image registration network plays in the training process of the virtual staining network.
The demonstration of our autopsy virtual staining technique opens up new possibilities for future studies that involve precise quantification, classification, and spatial analysis of e.g., white blood cells in inadequately fixed tissues, which were previously unattainable. Furthermore, from the perspective of histology, microscopic morphological changes associated with tissue autolysis closely resemble, if not mirror, those observed in areas undergoing necrosis. Necrosis refers to non-programmed cell death that occurs after irreversible tissue injury. It is a progressive process within living tissue, primarily caused by the release of lysosomal enzymes from infiltrating leukocytes, which are the key players in inflammatory processes. Under optical microscopy, necrosis and autolysis exhibit similar morphological alterations, except that autolysis occurs without the presence of inflammatory cell infiltrates. Future explorations could involve using our virtual staining technology to enhance the characterization of necrosis by focusing on the accurate imaging of the immune cells involved.
In conclusion, our autopsy virtual staining technique is accurate, rapid, and cost-effective, and it presents a superior alternative to conventional histological staining of large tissue sections obtained from cadavers. This advancement equips medical personnel and researchers with a powerful AI-based tool to improve their evaluation and characterization of tissue samples procured during autopsies. In addition to further improving the virtual staining performance of our network model, future research will focus on testing the applicability and efficiency of our methodology on tissue sections containing necrotic areas.

Sample preparation and standard histochemical H&E staining
The unlabeled lung autopsy tissue blocks used for this study were sourced from existing deidentified specimens, collected before this work, from the UCLA Translational Pathology Core Laboratory (TPCL) under UCLA IRB 18-001029. These specimens were originally acquired from 15 non-COVID-19 patients with pneumonia (8 for training and 7 for testing) and 3 COVID-19 pneumonia patients (all of them reserved for blind testing). The tissue blocks were then cut into ~4 μm thin sections, deparaffinized, and mounted onto standard glass slides, resulting in 18 tissue section slides, each corresponding to a unique patient. After their autofluorescence images were captured, these unlabeled lung autopsy tissue sections were processed by UCLA TPCL and the Department of Anatomic Pathology of Cedars-Sinai Medical Center (Los Angeles, CA, USA) for standard histochemical H&E staining.

Image data acquisition
The autofluorescence images of the unlabeled autopsy tissue slides were acquired using a Leica DMI8 microscope with a 40×/0.95 NA objective lens (Leica HC PL APO 40×/0.95 DRY), controlled using Leica LAS X microscopy automation software. Two fluorescence filter cubes, DAPI (Semrock OSFI3-DAPI5060C, EX377/50 nm EM 447/60 nm) and TxRed (Semrock OSFI3-TXRED-4040C, EX 562/40 nm EM 624/40 nm), were used to capture the autofluorescence images at different excitation-emission wavelengths. Each image was captured with a scientific complementary metal-oxide-semiconductor (sCMOS) image sensor (Leica DFC 9000 GTC) with an exposure time of ∼100 ms for the DAPI channel and ∼300 ms for the TxRed channel. Following the standard histochemical H&E staining procedure, the stained tissue slides were then digitized by a brightfield slide scanner (Leica Biosystems Aperio AT2).

Training dataset preparation
To train the autopsy virtual staining network model in a supervised manner, it was crucial to create co-registered image pairs comprising autofluorescence images (used as network input) and their histochemically stained H&E counterparts (used as network target, corresponding to well-preserved tissue regions only). To achieve this, we implemented a twostep registration workflow. At the first step, we performed a rigid registration step at the WSI level by computing the maximum cross-correlation coefficient between the autofluorescence WSI and its corresponding histology WSI, estimating the relative rotation angle and shift distance between the two WSIs. These parameters were applied to a spatial transformation of the histochemically stained H&E WSI, which better aligned it with its autofluorescence counterpart. In the second step, we entailed a finer registration process at the image patch level. This involves dividing these coarsely matched WSI pairs into smaller FOV pairs, each with a dimension of 3248×3248 pixels (~528×528 μm 2 ), and then performing a multi-modal affine image registration 45 to correct the shifts, size alterations, and rotation between the histology image FOVs and their autofluorescence counterparts. After this step, the registered histological image FOVs (each with 3248×3248 pixels) were further center-cropped to 2048×2048 pixels (~333×333 μm 2 ) to eliminate potential artifacts at the image edges. As a result, 16,159 paired sample image FOVs of 2048×2048 pixels were generated to form the training dataset of the virtual staining network, and 10% of the data was randomly selected and separated as the validation dataset. In each epoch of the training process, these paired image FOVs were further divided into even smaller patches of 256×256 pixels, normalized to have a distribution with zero mean and unit variance, and augmented through random flipping and rotation.

Statistical analysis
A paired t-test (two-tailed) was performed to compare the cell nuclei quantification results determined by digital image 20 analysis algorithms, including the number of nuclei per FOV and the average size of nuclei, between virtually stained and histochemically stained H&E images to validate whether these metrics have the same mean value for both virtually and histochemically stained images. As for the staining quality comparison between the virtually and histochemically stained images evaluated by the pathologists, a paired t-test (two-tailed) was performed to determine whether there is no significant difference between the histochemically stained H&E images and their virtually stained counterparts for each staining quality metric and per pathologist. In addition, a two-tailed paired t-test was performed to determine whether the cellularity level of the virtually and histochemically stained images have the same mean value. For all the tests, a P value of <0.05 was considered to indicate a statistically significant difference. All the analyses were performed using IBM SPSS Statistics v29.0.

Other implementation details
For the pathologist-involved blind quantitative evaluation reported in Figs. 7 and Supplementary Fig. S5, we used an online image-sharing platform (https://www.pathozoom.com/) for the pathologists to view and comment on the test sample FOVs. All the image preprocessing conducted in this work was executed in MATLAB, version R2022b