Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy



To increase the temporal resolution and maximal imaging time of super-resolution (SR) microscopy, we have developed a deconvolution algorithm for structured illumination microscopy based on Hessian matrixes (Hessian-SIM). It uses the continuity of biological structures in multiple dimensions as a priori knowledge to guide image reconstruction and attains artifact-minimized SR images with less than 10% of the photon dose used by conventional SIM while substantially outperforming current algorithms at low signal intensities. Hessian-SIM enables rapid imaging of moving vesicles or loops in the endoplasmic reticulum without motion artifacts and with a spatiotemporal resolution of 88 nm and 188 Hz. Its high sensitivity allows the use of sub-millisecond excitation pulses followed by dark recovery times to reduce photobleaching of fluorescent proteins, enabling hour-long time-lapse SR imaging of actin filaments in live cells. Finally, we observed the structural dynamics of mitochondrial cristae and structures that, to our knowledge, have not been observed previously, such as enlarged fusion pores during vesicle exocytosis.

  • Subscribe to Nature Biotechnology for full access:



Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.


  1. 1.

    Far-field optical nanoscopy. Science 316, 1153–1158 (2007).

  2. 2.

    , & Super-resolution fluorescence microscopy. Annu. Rev. Biochem. 78, 993–1016 (2009).

  3. 3.

    , & A guide to super-resolution fluorescence microscopy. J. Cell Biol. 190, 165–175 (2010).

  4. 4.

    , & Visualizing cell structure and function with point-localization superresolution imaging. Dev. Cell 23, 1092–1102 (2012).

  5. 5.

    Editorial. Artifacts of light. Nat. Methods 10, 1135 (2013).

  6. 6.

    et al. Long time-lapse nanoscopy with spontaneously blinking membrane probes. Nat. Biotechnol. 35, 773–780 (2017).

  7. 7.

    et al. Long-term live-cell STED nanoscopy of primary and cultured cells with the plasma membrane HIDE probe DiI-SiR. Angew. Chem. Int. Edn Engl. 56, 10408–10412 (2017).

  8. 8.

    , , , & Real-time light-driven dynamics of the fluorescence emission in single green fluorescent protein molecules. Proc. Natl. Acad. Sci. USA 97, 7237–7242 (2000).

  9. 9.

    et al. Analysis of red-fluorescent proteins provides insight into dark-state conversion and photodegradation. Biophys. J. 101, 961–969 (2011).

  10. 10.

    et al. Video-rate nanoscopy using sCMOS camera-specific single-molecule localization algorithms. Nat. Methods 10, 653–658 (2013).

  11. 11.

    et al. Ultrafast, temporally stochastic STED nanoscopy of millisecond dynamics. Nat. Methods 12, 827–830 (2015).

  12. 12.

    et al. Fast live simultaneous multiwavelength four-dimensional optical microscopy. Proc. Natl. Acad. Sci. USA 107, 16016–16022 (2010).

  13. 13.

    et al. Advanced imaging. Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 349, aab3500 (2015).

  14. 14.

    Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J. Microsc. 198, 82–87 (2000).

  15. 15.

    et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J. 94, 4957–4970 (2008).

  16. 16.

    et al. Instant super-resolution imaging in live cells and embryos via analog image processing. Nat. Methods 10, 1122–1126 (2013).

  17. 17.

    & Ultrafast superresolution fluorescence imaging with spinning disk confocal microscope optics. Mol. Biol. Cell 26, 1743–1751 (2015).

  18. 18.

    et al. Fast structured illumination microscopy using rolling shutter cameras. Meas. Sci. Technol. 27, 055401 (2016).

  19. 19.

    et al. Increased spatiotemporal resolution reveals highly dynamic dense tubular matrices in the peripheral ER. Science 354, aaf3928 (2016).

  20. 20.

    et al. Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy. Proc. Natl. Acad. Sci. USA 110, 21000–21005 (2013).

  21. 21.

    , & Structured illumination microscopy: artefact analysis and reduction utilizing a parameter optimization approach. J. Microsc. 216, 165–174 (2004).

  22. 22.

    et al. Comment on “Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics”. Science 352, 527 (2016).

  23. 23.

    et al. Image reconstruction for structured-illumination microscopy with low signal level. Opt. Express 22, 8687–8702 (2014).

  24. 24.

    , & Optimal 2D-SIM reconstruction by two filtering steps with Richardson-Lucy deconvolution. Sci. Rep. 6, 37149 (2016).

  25. 25.

    , , , & Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ. Nat. Commun. 7, 10980 (2016).

  26. 26.

    et al. Strategic and practical guidelines for successful structured illumination microscopy. Nat. Protoc. 12, 988–1010 (2017).

  27. 27.

    & Theory of cross-correlation analysis of PIV images. Appl. Sci. Res. 49, 191–215 (1992).

  28. 28.

    , , & Iterative CBCT reconstruction using Hessian penalty. Phys. Med. Biol. 60, 1965–1987 (2015).

  29. 29.

    et al. Fluorogenic probes for live-cell imaging of the cytoskeleton. Nat. Methods 11, 731–733 (2014).

  30. 30.

    , , , & Stroboscopic illumination using light-emitting diodes reduces phototoxicity in fluorescence cell imaging. Biotechniques 41, 191–197 (2006).

  31. 31.

    , & Mechanisms underlying phasic and sustained secretion in chromaffin cells from mouse adrenal slices. Neuron 23, 607–615 (1999).

  32. 32.

    Calcium control of neurotransmitter release. Cold Spring Harb. Perspect. Biol. 4, a011353 (2012).

  33. 33.

    & Amperometric detection of quantal secretion from patch-clamped rat pancreatic beta-cells. J. Biol. Chem. 271, 270–277 (1996).

  34. 34.

    , , & Release of small transmitters through kiss-and-run fusion pores in rat pancreatic beta cells. Cell Metab. 4, 283–290 (2006).

  35. 35.

    et al. Diacylglycerol guides the hopping of clathrin-coated pits along microtubules for exo-endocytosis coupling. Dev. Cell 35, 120–130 (2015).

  36. 36.

    et al. Delay between fusion pore opening and peptide release from large dense-core vesicles in neuroendocrine cells. Neuron 33, 287–299 (2002).

  37. 37.

    & Beginning of exocytosis captured by rapid-freezing of Limulus amebocytes. J. Cell Biol. 90, 40–54 (1981).

  38. 38.

    & Super-resolution microscopy of mitochondria. Curr. Opin. Chem. Biol. 20, 9–15 (2014).

  39. 39.

    et al. Super-resolution fluorescence imaging of organelles in live cells with photoswitchable membrane probes. Proc. Natl. Acad. Sci. USA 109, 13978–13983 (2012).

  40. 40.

    et al. SIMcheck: a toolbox for successful super-resolution structured illumination microscopy. Sci. Rep. 5, 15915 (2015).

  41. 41.

    et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).

  42. 42.

    , , , & Nanoscopy in a living mouse brain. Science 335, 551 (2012).

  43. 43.

    Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl. Acad. Sci. USA 102, 13081–13086 (2005).

  44. 44.

    et al. Imaging Ca2+ nanosparks in heart with a new targeted biosensor. Circ. Res. 114, 412–420 (2014).

Download references


We thank B.-C. Chen and C. Shan for commenting on the optics and biological experiments and B.-C. Chen, T. Maritzen, and P. Cheng for reading the manuscript and providing suggestions. The work was supported by grants from the National Science and Technology Major Project Program (2016YFA0500400), National Natural Science Foundation of China (31327901, 31521062, 31570839, 31428004, 61375018, 61672253 and 91750203), the Major State Basic Research Program of China (2013CB531200), and Beijing Natural Science Foundation (L172003).

Author information

Author notes

    • Xiaoshuai Huang
    • , Junchao Fan
    •  & Liuju Li

    These authors contributed equally to this work.


  1. State Key Laboratory of Membrane Biology, Beijing Key Laboratory of Cardiometabolic Molecular Medicine, Institute of Molecular Medicine, Peking University, Beijing, China.

    • Xiaoshuai Huang
    • , Liuju Li
    • , Lisi Wei
    • , Yanmei Liu
    •  & Liangyi Chen
  2. Key Laboratory of Image Processing and Intelligent Control of Ministry of Education of China, School of Automation, Huazhong University of Science and Technology, Wuhan, China.

    • Junchao Fan
    • , Haosen Liu
    •  & Shan Tan
  3. School of Electronics Engineering and Computer Science, Peking University, Beijing, China.

    • Runlong Wu
    •  & Yunfeng Zhang
  4. School of Software and Microelectronics, Peking University, Beijing, China.

    • Yi Wu
  5. School of Mathematical Sciences, Peking University, Beijing, China.

    • Heng Mao
  6. College of Engineering, Department of Biomedical Engineering, Peking University, Beijing, China.

    • Amit Lal
    •  & Peng Xi
  7. ColdSpring Science Corporation, Beijing, China.

    • Liqiang Tang


  1. Search for Xiaoshuai Huang in:

  2. Search for Junchao Fan in:

  3. Search for Liuju Li in:

  4. Search for Haosen Liu in:

  5. Search for Runlong Wu in:

  6. Search for Yi Wu in:

  7. Search for Lisi Wei in:

  8. Search for Heng Mao in:

  9. Search for Amit Lal in:

  10. Search for Peng Xi in:

  11. Search for Liqiang Tang in:

  12. Search for Yunfeng Zhang in:

  13. Search for Yanmei Liu in:

  14. Search for Shan Tan in:

  15. Search for Liangyi Chen in:


L.C. and S.T. conceived and supervised the research; X.H. designed and built the optical system; J.F. developed the reconstruction algorithm; X.H. and L.L. performed the experiments; R.W. wrote the control software under the supervision of Y.Z. and H.M.; A.L., L.T., Y.W., H.L., Y.L., and L.W. helped with the optics, SIM reconstruction, SLM pattern generation, algorithm and biological experiments, respectively; P.X. proposed the idea of 'rolling' SIM; X.H., J.F., and L.L. analyzed the data and prepared the figures; and L.C. and S.T. wrote the paper. All of the authors participated in discussions and data interpretation.

Competing interests

L.T. works at ColdSpring Science Corporation.

Corresponding authors

Correspondence to Shan Tan or Liangyi Chen.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–18, Supplementary Notes 1–3, Supplementary tables 1–6

  2. 2.

    Life Sciences Reporting Summary

Zip files

  1. 1.

    Supplementary Code 1

    Hessian-SIM Software package, relevant scripts and demo data.


  1. 1.

    97 Hz TIRF-SIM imaging of actin dynamics in a HUVEC for 6,800 consecutive time points.

    The HUVEC labeled with Lifeact-EGFP was imaged for 6,800 consecutive time points at 97 Hz. Scale bar: 2 μm.

  2. 2.

    49 Hz dual-color imaging of actin and tubulin dynamics in a HUVEC.

    The HUVEC labeled with SiR-tubulin (magenta) and LifeactEGFP (green) was imaged continuously at 49 Hz. Scale bar: 2 μm.

  3. 3.

    49 Hz dual-color imaging of EB3 and tubulin dynamics in an INS-1 cell.

    The INS-1 cell labeled with SiR-tubulin (magenta) and EB3-EGFP (green) was imaged continuously at 49 Hz. Scale bar: 2 μm.

  4. 4.

    Actin dynamics in HUVECs imaged at 10 Hz with different exposures.

    Conventional SIM and Hessian-SIM compared at 10 Hz with 7ms and 0.2ms exposure time for 3,000 consecutive time points. Scale bar: 2 μm.

  5. 5.

    Actin dynamics in HUVECs imaged at 1 Hz with different exposures.

    Conventional SIM and Hessian-SIM compared at 1 Hz with 7ms and 0.2ms exposure time for 3,000 consecutive time points. Scale bar: 2 μm.

  6. 6.

    EB3 dynamics in an INS-1 cell imaged at 188 Hz.

    INS-1 cell labeled with EB3-EGFP was imaged at 188 Hz with for 3,000 consecutive time points. Scale bar: 2 μm.

  7. 7.

    EB3 dynamics in an INS-1 cell imaged at 1 Hz with 0.2-ms exposure time per frame.

    The INS-1 cell labeled with EB3-EGFP was imaged at 1 Hz for 3,000 consecutive time points. Scale bar: 2 μm.

  8. 8.

    Resolving and tracking of rapid movements of small ER loops with 188 Hz Hessian-SIM.

    The HEK293 cell was labeled with KDEL-EGFP, and imaged at 188 Hz for 6,800 consecutive time points. Scale bars: the first one is 200 nm and the later one is 2 μm.

  9. 9.

    Continuous imaging of vesicle fusion labeled by VAMP2- pHluorin in an INS-1 cell for 10 min.

    The INS-1 cell was labeled with VAMP2-pHluorin, and stimulated with glucose and KCl. Fusion events were detected for more than 10 min at 97 Hz frame rate (~540,000 consecutive raw images). Videos of three different durations (2-12, 280-290, and 519-529 s) were shown on the top, middle, and bottom, respectively. Scale bar: 2 μm.

  10. 10.

    Vesicle fusion events with “ring” and “no ring” structures.

    The INS-1 cell was labeled with VAMP2-pHluorin. Scale bar: 200 nm.

  11. 11.

    Live mitochondria imaged under Hessian-SIM for 800 frames.

    The mitochondria in the COS-7 cell was labeled with MitoTracker Green, and imaged under 0.5-ms exposure with an initial illumination intensity of ~18 W/cm2 light intensity (which increased by 0.05% during each SIM image). Scale bar: 2 μm.

  12. 12.

    The fusion of two live mitochondria into one mitochondrion.

    The COS-7 cell was labeled with MitoTracker Green. The boxed region in the left is magnified and shown on the right. Scale bars: 1 μm (left) and 0.2 μm (right).

  13. 13.

    The fission of a mitochondrion into two mitochondria.

    The COS-7 cell was labeled with MitoTracker Green. The boxed region in the left is magnified and shown on the right. Scale bars: 1 μm (left) and 0.5 μm (right).

  14. 14.

    Intra-mitochondrial cristae re-organization in which two cristae structures merged into one.

    The COS-7 cell was labeled with PHB2-mScarlet and MitoTracker Green. Boxed regions in the left are magnified and shown on the right. Scale bars: 1 μm (left), 0.5 μm (center) and 0.2 μm (right).

  15. 15.

    97 Hz 2D-Hessian-SIM imaging of cytosolic actin dynamics in a HUVEC for 2,000 consecutive time points.

    The HUVEC labeled with Lifeact-EGFP was imaged for 2,000 consecutive time points at 97 Hz. Scale bar: 2 μm.