The goal when imaging bioprocesses with optical microscopy is to acquire the most spatiotemporal information with the least invasiveness. Deep neural networks have substantially improved optical microscopy, including image super-resolution and restoration, but still have substantial potential for artifacts. In this study, we developed rationalized deep learning (rDL) for structured illumination microscopy and lattice light sheet microscopy (LLSM) by incorporating prior knowledge of illumination patterns and, thereby, rationally guiding the network to denoise raw images. Here we demonstrate that rDL structured illumination microscopy eliminates spectral bias-induced resolution degradation and reduces model uncertainty by five-fold, improving the super-resolution information by more than ten-fold over other computational approaches. Moreover, rDL applied to LLSM enables self-supervised training by using the spatial or temporal continuity of noisy data itself, yielding results similar to those of supervised methods. We demonstrate the utility of rDL by imaging the rapid kinetics of motile cilia, nucleolar protein condensation during light-sensitive mitosis and long-term interactions between membranous and membrane-less organelles.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Light-sheets and smart microscopy, an exciting future is dawning
Communications Biology Open Access 09 May 2023
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Get just this article for as long as you need it
Prices may be subject to local taxes which are calculated during checkout
The published BioSR dataset was used to train the rDL models for the biological structures of ER, MT, actin and CCPs. The training data for the rDL models of other biological structures were also uploaded into the figshare repository of the BioSR dataset, which is publicly accessible via https://doi.org/10.6084/m9.figshare.13264793. Source data are provided with this paper.
The source codes of rDL SIM, several representative pre-trained models as well as some example images for testing are publicly accessible via https://github.com/qc17-THU/rDL-SIM.
Moore, A. S. et al. Actin cables and comet tails organize mitochondrial networks in mitosis. Nature 591, 659–664 (2021).
Liao, Y.-C. et al. RNA granules hitchhike on lysosomes for long-distance transport, using annexin A11 as a molecular tether. Cell 179, 147–164 (2019).
Phillips, M. J. & Voeltz, G. K. Structure and function of ER membrane contact sites with other organelles. Nat. Rev. Mol. Cell Biol. 17, 69–82 (2016).
Prinz, W. A., Toulmay, A. & Balla, T. The functional universe of membrane contact sites. Nat. Rev. Mol. Cell Biol. 21, 7–24 (2020).
Schermelleh, L. et al. Super-resolution microscopy demystified. Nat. Cell Biol. 21, 72–84 (2019).
Wu, Y. & Shroff, H. Faster, sharper, and deeper: structured illumination microscopy for biological imaging. Nat. Methods 15, 1011–1019 (2018).
Huang, X. et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nat. Biotechnol. 36, 451–459 (2018).
Zhao, W. et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy. Nat. Biotechnol. 40, 606–617 (2022).
de Haan, K., Rivenson, Y., Wu, Y. & Ozcan, A. Deep-learning-based image reconstruction and enhancement in optical microscopy. Proc. IEEE 108, 30–50 (2019).
Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).
Rahaman, N. et al. On the spectral bias of neural networks. Proc. of the 36th International Conference on Machine Learning. 5301–5310 (PMLR, 2019).
Guo, Y. et al. Visualizing intracellular organelle and cytoskeletal interactions at nanoscale resolution on millisecond timescales. Cell 175, 1430–1442 (2018).
Shah, Z. H. et al. Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images. Photonics Res. 9, B168–B181 (2021).
Dabov, K., Foi, A., Katkovnik, V. & Egiazarian, K. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16, 2080–2095 (2007).
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
Qiao, C. et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat. Methods 18, 194–202 (2021).
Kiepas, A., Voorand, E., Mubaid, F., Siegel, P. M. & Brown, C. M. Optimizing live-cell fluorescence imaging conditions to minimize phototoxicity. J. Cell Sci. 133, jcs242834 (2020).
Li, D. et al. Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 349, aab3500 (2015).
Ma, Y., Li, D., Smith, Z. J., Li, D. & Chu, K. Structured illumination microscopy with interleaved reconstruction (SIMILR). J. Biophotonics 11, e201700090 (2018).
Oltean, A., Schaffer, A. J., Bayly, P. V. & Brody, S. L. Quantifying ciliary dynamics during assembly reveals stepwise waveform maturation in airway cells. Am. J. Respir. Cell Mol. Biol. 59, 511–522 (2018).
Jing, J. C., Chen, J. J., Chou, L., Wong, B. J. F. & Chen, Z. Visualization and detection of ciliary beating pattern and frequency in the upper airway using phase resolved doppler optical coherence tomography. Sci Rep. 7, 8522 (2017).
Chien, A. et al. Dynamics of the IFT machinery at the ciliary tip. eLife 6, e28606 (2017).
Ott, C. & Lippincott‐Schwartz, J. Visualization of live primary cilia dynamics using fluorescence microscopy. Curr. Protoc. Cell Biol. Chapter 4, Unit 4.26 (2012).
Du, M. & Chen, Z. J. DNA-induced liquid phase condensation of cGAS activates innate immune signaling. Science 361, 704–709 (2018).
Lehtinen, J. et al. Noise2noise: learning image restoration without clean data. Preprint at https://arxiv.org/abs/1803.04189 (2018).
Sengupta, P. et al. ER trapping reveals Golgi enzymes continually revisit the ER through a recycling pathway that controls Golgi organization. Proc. Natl Acad. Sci. USA 112, E6752–E6761 (2015).
Chen, B. C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).
Lu, L., Ladinsky, M. S. & Kirchhausen, T. Cisternal organization of the endoplasmic reticulum during mitosis. Mol. Biol. Cell 20, 3471–3480 (2009).
Kendall, A. & Gal, Y. What uncertainties do we need in Bayesian deep learning for computer vision? in Advances in Neural Information Processing Systems 30. https://papers.nips.cc/paper/2017/hash/2650d6089a6d640c5e85b2b88265dc2b-Abstract.html (NIPS, 2017).
Liu, T.-L. et al. Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. Science 360, eaaq1392 (2018).
Ji, N. Adaptive optical fluorescence microscopy. Nat. Methods 14, 374–380 (2017).
Valm, A. M. et al. Applying systems-level spectral imaging and analysis to reveal the organelle interactome. Nature 546, 162–167 (2017).
Lubeck, E. & Cai, L. Single-cell systems biology by super-resolution imaging and combinatorial labeling. Nat. Methods 9, 743–748 (2012).
Gustafsson, M. G. et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J. 94, 4957–4970 (2008).
Muller, M., Monkemoller, V., Hennig, S., Hubner, W. & Huser, T. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ. Nat. Commun. 7, 10980 (2016).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778 (2016).
Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. Inception-v4, Inception-ResNet and the impact of residual connections on learning. https://ojs.aaai.org/index.php/AAAI/article/view/11231. Thirty-First AAAI Conference on Artificial Intelligence (2017).
Hu, J., Shen, L. & Sun, G. Squeeze-and-excitation networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition. 7132–7141 (2018).
Zhang, Y. et al. Image super-resolution using very deep residual channel attention networks. Proc. of the European Conference on Computer Vision (ECCV) 286–301 (2018).
Liu, S. et al. sCMOS noise-correction algorithm for microscopy images. Nat. Methods 14, 760–761 (2017).
Descloux, A., Grussmayer, K. S. & Radenovic, A. Parameter-free image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918–924 (2019).
Cleveland, W. S. LOWESS: a program for smoothing scatterplots by robust locally weighted regression. Am. Stat. 35, 54 (1981).
Qian, H., Sheetz, M. P. & Elson, E. L. Single particle tracking. Analysis of diffusion and flow in two-dimensional systems. Biophys. J. 60, 910–921 (1991).
Ba, Q., Raghavan, G., Kiselyov, K. & Yang, G. Whole-cell scale dynamic organization of lysosomes revealed by spatial statistical analysis. Cell Rep. 23, 3591–3606 (2018).
Grimm, J. B. et al. A general method to improve fluorophores for live-cell and single-molecule microscopy. Nat. Methods 12, 244–250 (2015).
Zhao, H. et al. Fibrogranular materials function as organizers to ensure the fidelity of multiciliary assembly. Nat. Commun. 12, 1273 (2021).
Delgehyr, N. et al. Ependymal cell differentiation, from monociliated to multiciliated cells. Methods Cell. Biol. 127, 19–35 (2015).
Sanjana, N. E. et al. A transcription activator-like effector toolbox for genome engineering. Nat. Protoc. 7, 171–192 (2012).
The authors thank T. Kirchhausen for the donor plasmids used for genome editing and help in generating the genome-edited cell lines. This work was supported by grants from the National Natural Science Foundation of China (31827802, 32125024, 31970659 and 62088102); the Ministry of Science and Technology (2021YFA1300303 and 2020AA0105500); the Chinese Academy of Sciences (ZDBS-LY-SM004 and XDA16021401); the Collaborative Research Fund of the Chinese Institute for Brain Research, Beijing (2021-NKX-XM-03); the Tencent Foundation through the XPLORER PRIZE; the Shuimu Tsinghua Scholar Program; and the Youth Innovation Promotion Association of the Chinese Academy of Sciences (2020094).
Dong Li and C.Q. have pending patent applications on the presented frameworks. The other authors declare no competing interests.
Peer review information
Nature Biotechnology thanks the anonymous reviewers for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended Data Fig. 1 Illustration of the spectral bias effect in deep learning super-resolution (DLSR) task.
a, Concepts (upper row) and typical results (lower row) of wide-field, DFCAN-SIM, rDL SIM and conventional SIM imaging. b, Typical DFCAN-SIM images of F-actin output at 4000, 24000, 52000, 100000, 200000, and 400,000 training iterations, respectively. c, The evolutions of the training loss and the resolution of the intermediate outputs during the training process of the DFCAN-SIM model of F-actin dataset (averaged from n = 20 test images). The curves shows that there exists a significant resolution gap between DFCAN-SIM (red dashed line) and GT-SIM (black dashed line) images, which implies DLSR models suffer from severe spectral bias. d, The corresponding GT-SIM image of the same ROI with (b) for reference. e, Typical rDL denoised images (top left) and rDL SIM images (bottom right) of F-actin output at 4000, 24000, 52000, 100000, 200000, and 400,000 training iterations, respectively. f, The evolutions of the training loss and the resolution of the intermediate outputs during the training process of the rDL denoising model of F-actin dataset (averaged from n = 180 test images). Compared with DLSR task in b and c, the rDL denoising of raw SIM images is hardly impaired by the spectral bias problem, preventing the resolution degradation of rDL SIM image relative to the GT-SIM image, as illustrated in (a). g, The corresponding ground truth raw SIM image of the same ROI with (e) for reference. Scale bar, 3 μm (a, b, d, e, g), 0.5 μm (zoom-in regions of a, b, d, e, g). Gamma value, 0.7 for DFCAN-SIM images, rDL SIM images and GT-SIM images.
Extended Data Fig. 2 Generalization and Robustness validation of rDL SIM.
a, Typical SR images of F-actin, MTs and CCPs reconstructed by rDL SIM (top right corner) and CARE-SIM (bottom left corner) trained with only F-actin data. Zoom-in regions show the comparisons of raw SIM images, GT-SIM images and SR images reconstructed by conventional SIM (Conv. SIM), CARE-SIM, rDL SIM trained with F-actin data (orange) and rDL SIM trained with corresponding specimens (red). These results illustrate the strong generalization capability of rDL SIM. b, PSNR quantifications of CARE-SIM and rDL-SIM in terms of F-actin, MTs and CCPs, where both models are trained with only F-actin data. The PSNRs of Conv. SIM are shown for reference, n = 145. Center line, medians; limits, 75% and 25%; whiskers, the larger value between the largest data point and the 75th percentiles plus 1.5× the interquartile range (IQR), and the smaller value between the smallest data point and the 25th percentiles minus 1.5× the IQR; outliers, data points larger than the upper whisker or smaller than the lower whisker. c, Plots of amplitudes (left) and orientations (right) of the pattern wave vector for the 45,000-time-point ER data shown in Fig. 3h and Supplementary Video 8. The pattern parameters are estimated from the single time point of raw SIM images (gray lines), averaged 20 consecutive time points of raw SIM images (black lines), single time point of CARE denoised images (blue lines) and rDL denoised images (orange lines). d, Representative rDL SIM images of F-actin with the pattern initial phase shifted by -0.2π, -0.1π, 0, 0.1π and 0.2π, respectively, relative to that of GT-SIM. Conv. SIM and GT-SIM images are shown for reference. The PSNRs are labelled at the top right of all Conv. SIM images and rDL SIM images. e, PSNR quantifications of the rDL SIM images at different phase errors under different fluorescence intensity levels in the cases of F-actin, n = 50. f, Representative rDL SIM images of F-actin with spherical or coma aberration distorted PSF in MPE branch. Conv. SIM and GT-SIM images are shown for reference. Scale bar, 1μm (a), 0.5 μm (zoom-in regions in a), 1.5 μm (d and f), and 0.5 μm (zoom-in regions in d and f). Gamma value: 0.7 for all SIM images of F-actin.
Extended Data Fig. 3 Uncertainty comparison of DFCAN-SIM, DFGAN-SIM and rDL SIM models.
a, Representative SR images of F-actin produced by the three models of DFCAN-SIM, DFGAN-SIM and rDL SIM that are independently trained with the same dataset (upper), and the corresponding standard deviation (SD) maps (lower). b, The magnified regions of the white box in (a). c, The raw SIM image, Conv. SIM and GT-SIM images shown for reference. The arrows indicate the fine structures of F-actin that exhibit large reconstruction variation by both DFCAN- and DFGAN-SIM models, whereas that are stably reconstructed by rDL SIM with high consistency to the GT-SIM image. Scale bar, 1 μm (a), 0.3 μm (b, c). Gamma value, 0.7 for all SIM images of F-actin.
Extended Data Fig. 4 rDL SIM resolves the ring structure of CCPs and visualizes multiple IFT trains’ unusual behaviors in ependymal motile cilia.
a, Representative SR images of CCPs reconstructed by conventional (Conv.) SIM, CARE-SIM and rDL SIM, whose raw images were acquired at 18-fold lower fluorescence than those of GT-SIM. It shows that rDL-SIM provides high-fidelity ring-like structure and prevents the reconstruction artifacts. (Supplementary Video 4). b-d, Kymographs (left) and time-lapse images (right) show that, (b) IFT trains stopped halfway or paused for a while before reaching the ciliary base; (c) IFT trains reversed their direction of motion in the midway of the microtubule (MT) tracks; (d) Two IFT trains moving along the same MT track collided with each other, which remodeled them into three independent IFT trains. As described in Fig. 3d, the IFT trains are resolved into three groups that transported along the microtubule tracks residing at the left (green), middle (magenta), and right (yellow) sides of the ciliary axoneme. Scale bar, 5 μm (a) and 0.5 μm (zoom-in regions of a, b-d).
Extended Data Fig. 5 Interactions between endoplasmic reticulum (ER) and peroxisomes (POs) as seen by rDL GI-SIM.
a, Comparisons of single raw SIM image (left), rDL denoised raw image (middle) and rDL GI-SIM image (right) in the case of two-color live-cell imaging of ER and POs. Scale bar, 2μm. b, Time-lapse two-color images of ER and POs, and the corresponding dynamic characteristics of Mander’s overlap coefficient (MOC) for POs that are stably contacted with ER over time (top and middle rows) or that is freely moving without stable association with ER (bottom row). Scale bar, 0.5 μm. c, Distribution of the ER-PO MOCs’ standard deviation (S.D.) versus their mean values, n = 137. d, Time-lapse images of a tubular ER generation event by hitchhiking on a moving PO. Scale bar, 1 μm. Gamma value, 0.7 for rDL GI-SIM images of ER, and 0.8 for rDL GI-SIM images of POs.
Extended Data Fig. 6 Visualizing the behaviors and interactions of NPM1 and RPA49 during cell mitosis via rDL LLS-SIM.
a, 3-D rendering of three-color time-lapse images of fibrillar centre (FC, labeled by Halo-RPA49) and granular component (GC, labeled by mEmerald-NPM1), and chromosome (labeled by mCherry-H2B) at different stages of mitosis in a HeLa cell, knocking-in mEmerald-NPM1 and stably expressing mCherry-H2B and Halo-RPA49. b, Time-lapse max intensity projections of NPM1 and RPA49 showing that the dissociation of RNA polymerase I subunits (e.g., RPA49) from the innermost FC preceded the disassembly of the outermost GC during prophase to prometaphase. c, Time-lapse max intensity projections of NPM1 and RPA49 showing that small nucleolar-derived foci (NDFs) of NPM1 coalesced into giant ones (indicated by arrows in the second row), while the innermost RPA49 marked FC foci of large size could be partitioned into several independent small foci during telophase (arrows in the third row). Scale bar, 6 μm (a), 3 μm (b), 2 μm (c).
Extended Data Fig. 7 Temporally and Spatially interleaved self-supervised learning of rDL denoising networks.
a, Representative LLSM images of Golgi SiT-vesicles denoised by TiS-rDL self-supervised and rDL supervised learning models. Depth color-coded maximum intensity projections (MIPs) of input noisy raw data (left), the images stack denoised by the TiS-rDL model (middle) and by the supervised learning model (right). These results illustrate the performance of TiS-rDL denoising model is as good as the rDL model trained with the supervision of high-SNR images. b, c, Representative TiS-rDL denoised images of Golgi SiT-vesicles (b) and statistical analysis in terms of PSNR under the conditions of temporally down-sampled at different factors (c), n = 50. Center line, medians; limits, 75% and 25%; whiskers, maximum and minimum. Noisy raw images and images denoised by the supervised model are shown for reference. These results indicate that the decrease of temporal sampling rate gradually affects the output fidelity of the TiS-rDL method. d, e, Schematic of SiS-rDL denoising network training (d) and inference (e). f, g, Representative outputs (f) and PSNR comparisons (g) of SiS-rDL denoising models with and without gap-amending regularization (GAR) in terms of ER and Mito in mitotic cells at different axial sampling intervals of 50, 100, 200, 300, 400, 500, and 600 nm, n = 31 for ER, 42 for Mito. Center line, medians; limits, 75% and 25%; whiskers, 95% and 5%. These results show that GAR effectively improves the performance of SiS-rDL models; The PSNR of SiS-rDL denoised images decreases by less than 4% before the axial sampling rate is lower than Nyquist criterium, that is, 300 nm for LLSM, which suggests the general applicability of the SiS-rDL method for 3D imaging. Scale bar, 2 μm (a, b), 5 μm (f), 3 μm (zoom-in regions of f).
Extended Data Fig. 8 Three-color rDL LLSM images of ER, H2B, and Golgi apparatus at different stages of mitosis.
Three-color 3-D rendering (left panel) and single slices of X-Y view (right panel) showing the remodeling process of Golgi apparatus, as well as their interactions with ER at different stages of mitosis (A-H). Scale bar, 10 μm.
Extended Data Fig. 9 3-D rendering of three-color rDL LLSM images of ER, H2B, and mitochondria (Mito) at different stages of mitosis.
For each stage of mitosis, images of volume rendering (first row), single slices of X-Y view (second row), color-coded in z axis surface rendering of Mito in X-Y view (third row) and X-Z view (forth row), respectively, are shown to visualize the segregation process of Mito and their interaction dynamics with ER during mitosis. Scale bar, 8 μm.
Extended Data Fig. 10 Sustained live imaging enabled by rationalized deep learning microscopy.
a, Flowchart of the rDL SIM denoising and reconstruction algorithm. Scale bar, 1 μm. b, Synopsis of rDL imaging modalities, including (i) rDL TIRF-SIM, rDL GI-SIM, rDL 3D-SIM, and rDL LLS-SIM that utilize the physical model of SIM to guide the network training and inference processes; (ii) rDL-TiS LLSM and rDL-SiS LLSM, which utilize the spatial/temporal continuity of acquired biological data to implement self-supervised denoising, yielding comparable results to the supervised methods. The rDL methods enable investigations into the fine spatial details, rapid kinetics and long-time dynamics of a variety of bioprocesses, showing great promise for shedding light on diverse biological phenomena.
Glossary of Abbreviations, Supplementary Notes 1–3, Supplementary Figs. 1–12, Supplementary Tables 1–4, Captions for Supplementary Videos 1–17 and Supplementary References
Supplementary Video 1
Spatial and temporal resolution degradation of rDL SIM versus Hessian-SIM. SIM images reconstructed by Hessian-SIM (μ = 150, σ = 1) (left panel) and rDL SIM (middle panel) of CCPs in a SUM159 cell for 149 timepoints at 1-second intervals. The corresponding GT-SIM images (right panel) are shown for reference. The trajectory of a CCP is delineated to illustrate the temporal degradation effect by Hessian-SIM.
Supplementary Video 2
Comparison of raw SIM images denoising by deep learning algorithms with and without physical model rationalization. Top left: raw SIM images acquired at the illumination patterns of 3-phase × 3-orienation, which are the inputs of the rDL and CARE denoising DNNs. Top right: the raw SIM images denoised with CARE network without physical model rationalization. Bottom left: the raw SIM images denoised with rDL network rationalized with SIM’s physical model. Bottom right: the raw SIM images acquired at high SNR condition serving as the GT for reference
Supplementary Video 3
The coordinated remodeling dynamics of F-actin and myosin-IIA over hour-long settling process after dropping a U2OS cell onto the coverslip. rDL TIRF-SIM records the adhesion and spreading dynamics of a U2OS cell co-expressing mEmerald-Lifeact (green) and mCherry-myosin-IIA (magenta) for 749 timepoints at 5-second intervals.
Supplementary Video 4
Long-term rDL TIRF-SIM imaging and tracking of CCPs. Formation, maturation and internalization of endogenous clathrin-EGFP in a live SUM159 cell for 5,500 timepoints with a time duration of 45.8 minutes at 1-second intervals.
Supplementary Video 5
Ciliary beating dynamics by rDL GI-SIM of ultra-high spatiotemporal resolution. Ciliary beating behaviors recorded at the spatiotemporal resolution of 97 nm and 684 Hz for ~60,000 frames without notable phototoxicity and photobleaching effects in live mEPCs.
Supplementary Video 6
The transportation dynamics of the IFT trains along cilia by two-color rDL GI-SIM. The spatiotemporal coordination and remodeling of the IFT trains in live mEPCs expressing EGFP-IFT81 (red) and stained by LysoTracker (blue) as seen by rDL GI-SIM for 1,946 timepoints at 0.09-second intervals.
Supplementary Video 7
A set of examples of IFT train transportation and remodeling dynamics along the three groups of MT tracks in the axoneme of live mEPCs. Right: two-color rDL GI-SIM imaging of live mEPCs expressing EGFP-IFT81 (red) and stained by LysoTracker (blue). Left: kymography of the IFT trains indicated by the arrows in right. The IFT trains are resolved into three groups that transported along the MT tracks residing at the left (green), middle (magenta) and right (yellow) sides of the ciliary axoneme.
Supplementary Video 8
Dynamic interactions between ER and POs. Two-color rDL GI-SIM imaging of ER–PO contacts at high spatiotemporal resolution of 97 nm and 182 Hz for long duration of 45,000 timepoints in a COS-7 cell expressing mEmerald-KDEL (blue) and mCherry-SKL (orange).
Supplementary Video 9
Dynamic interactions between ER and the condensates of cGAS-DNA. Two-color rDL GI-SIM imaging of HEK293T cell stably expressing cGAS-Halo and mEmerald-KDEL for 1,000 timepoints at 1-second intervals, revealing the phase separation process of cGAS-DNA condensates growing progressively and their interaction with the surrounding ER tubules.
Supplementary Video 10
Examples of cGAS-DNA condensates stably associated with ER. Two-color rDL GI-SIM imaging at 1-second intervals in a second HEK293T cell stably expressing cGAS-Halo and mEmerald-KDEL, revealing that cGAS-DNA condensates are tightly associated with adjacent ER tubules or even corralled by ER polygons over time.
Supplementary Video 11
Dynamic interactions between ER and cGAS-DNA in another HEK293T cell. Long-time rDL GI-SIM for 2,000 timepoints at 1-second intervals in another HEK293T cell stably expressing cGAS-Halo and mEmerald-KDEL, revealing that cGAS-DNA condensates traffic over a long range and sometimes changes the direction of their movement, during which the ER keeps contacting with the moving cGAS-DNA condensates.
Supplementary Video 12
Volumetric dynamics of mitochondrial cristae by rDL 3D-SIM. Long-time rDL 3D-SIM imaging of a COS-7 cell expressing PHB2-mEmerald for 448 timepoints at 3-second intervals.
Supplementary Video 13
Disassemble and reassemble process of nucleoli over the entire mitosis as seen by rDL LLS-SIM. Three-color volumetric SR imaging of NPM1, RPA49 and chromosomes at the speed of 12 seconds per three-color volume for 748 timepoints from a HeLa cell of knocking-in mEmerald-NPM1 and stably expressing mCherry-H2B and Halo-RPA49, revealing the spatiotemporally coordinated multi-phase separation dynamics of nucleoli during mitosis.
Supplementary Video 14
Ultra-fast whole-cell imaging of the dynamic behaviors of SiT-Golgi vesicles in 3D color-coded representation as seen by rDL LLSM. Volumetric imaging at 10 Hz (1,000 z-slices per second) reveals the ultra-dynamics of SiT-Golgi vesicles across the entire intracellular space of an R1 mammalian ependymal cell expressing SiT-Halo-JF642 over 10,000 timepoints.
Supplementary Video 15
The remodeling and segregating dynamics of Golgi apparatus during mitosis. Three-color rDL LLSM imaging for 1,000 timepoints at 6-second intervals in a HeLa cell stably expressing calnexin-mEmerald (ER in gray), H2B-mCherry (chromosome in green) and SiT-Halo (Golgi in yellow).
Supplementary Video 16
Dynamic interactions between ER and mitochondria during mitosis. Three-color rDL LLSM imaging for 890 timepoints at 6-second intervals in a HeLa cell stably expressing calnexin-mEmerald (ER in gray), H2B-mCherry (chromosome in green) and Mito-Halo (mitochondria in blue).
Supplementary Video 17
Self-aggregation and deaggregation dynamics of lysosomes (Lyso) during mitosis. Three-color rDL LLSM imaging for 879 timepoints at 6-second intervals in a HeLa cell stably expressing calnexin-mEmerald (ER in gray), H2B-mCherry (chromosome in green) and Lamp1-Halo (Lyso in orange).
Source Data Fig. 1
Statistical Source Data
Source Data Fig. 2
Statistical Source Data
Source Data Fig. 3
Statistical Source Data
Source Data Fig. 5
Statistical Source Data
Source Data Fig. 6
Statistical Source Data
Source Data Extended Data Fig. 1
Statistical Source Data
Source Data Extended Data Fig. 2
Statistical Source Data
Source Data Extended Data Fig. 5
Statistical Source Data
Source Data Extended Data Fig. 7
Statistical Source Data
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Qiao, C., Li, D., Liu, Y. et al. Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes. Nat Biotechnol 41, 367–377 (2023). https://doi.org/10.1038/s41587-022-01471-3
This article is cited by
Light-sheets and smart microscopy, an exciting future is dawning
Communications Biology (2023)
ERnet: a tool for the semantic segmentation and quantitative analysis of endoplasmic reticulum topology
Nature Methods (2023)