Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

sCMOS noise-correction algorithm for microscopy images

Nature Methods volume 14, pages 760–761 (2017)Cite this article

Subjects

To the Editor:

Scientific complementary metal-oxide semiconductor (sCMOS) cameras are rapidly gaining popularity in the biological sciences. The sCMOS sensor provides significant advances in imaging speed, sensitivity and field of view over traditional detectors such as charge-coupled devices (CCD) or electron-multiplying CCDs (EMCCD)1,2. However, this sensor introduces pixel-dependent noise; each pixel has its own noise statistics—primarily offset, gain and variance. Left uncorrected, this sCMOS-specific noise generates imaging artifacts and biases in quantification3. A suite of algorithms was developed to characterize this noise in each pixel and incorporate the noise statistics in the likelihood function for single-molecule localization3. However, these algorithms work exclusively on images with point objects such as in single-particle tracking or single-molecule-switching nanoscopy. No general algorithm that works on conventional microscopy images exists. We developed such an algorithm that dramatically reduces sCMOS noise from microscopy images with arbitrary structures. We show that our new method corrects pixel-dependent noise in fluorescence microscopy using an sCMOS sensor, and this allows the sensor's performance to approach that of an ideal camera.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Concept and results of noise-correction algorithm for sCMOS camera.

References

  1. von Diezmann, A., Shechtman, Y. & Moerner, W.E. Chem. Rev. 117, 7244–7275 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Huang, Z.-L. et al. Opt. Express 19, 19156–19168 (2011).

    Article  PubMed  Google Scholar 

  3. Huang, F. et al. Nat. Methods 10, 653–658 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Goodman, J.W. Introduction to Fourier Optics (Roberts & Company, 2005).

    Google Scholar 

  5. Liu, S., Kromann, E.B., Krueger, W.D., Bewersdorf, J. & Lidke, K.A. Opt. Express 21, 29462–29487 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank C. Pellizzari for discussion on algorithm development; D.A. Miller, K.F. Ziegler and P.M. Ivey for helping with the project and for their suggestions on the manuscript. D.M.S. was supported by an NSF grant (1146944-IOS). S.L., M.J.M. and F.H. were supported by grants from the NIH (R35 GM119785) and DARPA (D16AP00093).

Author information

Authors and Affiliations

  1. Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA

    Sheng Liu, Michael J Mlodzianoski & Fang Huang

  2. School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA

    Zhenhua Hu

  3. Department of Biological Sciences, Purdue University, West Lafayette, Indiana, USA

    Yuan Ren, Kristi McElmurry & Daniel M Suter

  4. Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, Indiana, USA

    Daniel M Suter & Fang Huang

  5. Bindley Bioscience Center, Purdue University, West Lafayette, Indiana, USA

    Daniel M Suter

  6. Birck Nanotechnology Center, Purdue University, West Lafayette, Indiana, USA

    Daniel M Suter

  7. Purdue Institute of Inflammation, Immunology and Infectious Disease, Purdue University, West Lafayette, Indiana, USA

    Fang Huang

Authors
  1. Sheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J Mlodzianoski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenhua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kristi McElmurry
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniel M Suter
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang Huang.

Ethics declarations

Competing interests

S.L. and F.H. are co-inventors on a patent application related in part to the material presented here.

Integrated supplementary information

Supplementary Figure 1 Diagram of the noise correction algorithm

Starting with a raw sCMOS frame, sequential steps include, pre-correction using offset and gain pixel maps, calculating negative log-likelihood using variance and gain pixel maps and noise contribution in Fourier space and iterative update to minimize the pixel-wise sum of the two quantities (See Supplementary Notes 2–7)

Supplementary Figure 2 Temporal fluctuation comparison of fluorescence microscopy images.

Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) Temporal standard deviation (STD) map over 400 sCMOS frames (pre-corrected by gain and offset). The colormap scale is from min (STD of 2.3, black) to max (STD of 12.3) in units of effective photon count. (B) Temporal STD map over 400 NCS (noise correction for sCMOS camera) frames. The terms sCMOS and NCS frame will be used throughout the supplemental figures. (C) Zoom in regions i and ii from A and B show pixels with high variance are effectively removed after NCS. (D) The pixel intensity traces of selected pixels from cropped regions i and ii over 50 frames. For the pixels with high readout noise (pixel 1 and 3), the value fluctuation decreases significantly after NCS, while for the pixels with low readout noise (pixel 2 and 4), the pixel value fluctuation remains the same. And the mean pixel values stay the same in both high and low readout noise cases before and after noise correction.

Supplementary Figure 3 Pixel fluctuation comparison before and after noise correction at low photon levels.

End-binding protein 3 in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) A single sCMOS frame pre-corrected for gain and offset for comparison purpose with an exposure time of 10 ms and at time point t = 0 s. (B) Time series of selected regions in A from sCMOS frames and the corresponding NCS frames showing the significant reduction of sCMOS noise while maintaining the underlying signal.

Supplementary Figure 4 Resolution comparison using both experimental data and simulated data.

(A) 100 nm yellow-green fluorescent bead images from sCMOS camera and NCS corrected images. To cancel readout noise in sCMOS frames for a fair comparison between the sCMOS frames and NCS frames, images were averaged over 200 frames for both cases. The intensity profiles were generated by averaging over the vertical dimension of each bead image and fitted with a Gaussian function to extract their widths, σsCMOS and σNCS. (B) Simulated bead images based on the parameters in Supplementary Note 14. The simulated bead images were averaged over 20 frames from sCMOS and NCS frames. From both experimental data and simulated data, the σNCS is slightly larger than σsCMOS, resulting in 5.5 nm and 4 nm decrease in resolution, a small decrease is potentially negligible compared with the diffraction limit of approximately 250 nm.

Supplementary Figure 5 Comparison of NCS result using OTF weighted and noise only masks.

Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) Temporal standard deviation map over 400 sCMOS frames, NCS frames with OTF weighted mask and noise only mask respectively. The colormap scale is from min (STD of 2.3, dark red) to max (STD of 12.3, white) in units of effective photon count. (B) Average of a sequence of sCMOS and NCS frames as in A and their corresponding amplitudes in Fourier space. Average images were used to cancel pixel dependent noise for fair comparison of sCMOS and NCS frames. (C) Radial average of the amplitude in Fourier space from the sCMOS and NCS frames.

Supplementary Figure 6 Comparison of NCS algorithm with low pass filter.

In order to illustrate the fundamental differences between low pass filters and the NCS algorithm, the simulated bead data uses a simulated high variance map (3000∼6000 ADU2). (A) sCMOS frame. (B) NCS frame. (C) The sCMOS frame blurred by a 2D Gaussian kernel with a sigma equal to 1 pixel. (D) The sCMOS frame after a low pass filter with a cutoff frequency equal to the OTF radius. The cutout region in each image is the 2× zoom of the region above the white box. It shows that both the Gaussian blur and the OTF filter cannot effectively remove the sCMOS noise (yellow boxes and yellow arrows), while the NCS algorithm can significantly reduce the sCMOS noise fluctuations. Furthermore, the Gaussian blur method also reduces the resolution of the original image (red circles) (Supplementary Note 12).

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–6, Supplementary Methods and Supplementary Notes 1–15 (PDF 8395 kb)

Supplementary Software

NCS software package (ZIP 15641 kb)

Supplementary Video 1

Time series and pixel fluctuation of sCMOS and NCS frames - EB3 in COS-7 cells End-binding protein 3 in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. The top panel shows the time series of the sCMOS and NCS frames of the experimental data. The bottom traces show the pixel value fluctuations of the circled regions in sCMOS and NCS frames. (AVI 78450 kb)

Supplementary Video 2

Time series of raw sCMOS and NCS frames – SiR-actin in Aplysia bag neuron cell F-actin in peripheral domain and transition zone of Aplysia bag cell neuronal growth cones tagged with SiR-actin were imaged on a conventional wide-field fluorescence microscope. The movie shows the time series of the raw sCMOS (without offset and gain correction) and NCS frames of the experimental data. (AVI 65351 kb)

Life Sciences Reporting Summary

Reporting Summary (PDF 67 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Mlodzianoski, M., Hu, Z. et al. sCMOS noise-correction algorithm for microscopy images. Nat Methods 14, 760–761 (2017). https://doi.org/10.1038/nmeth.4379

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.4379

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing