Analysis | Published:

Quantitative evaluation of software packages for single-molecule localization microscopy

Nature Methods volume 12, pages 717724 (2015) | Download Citation

Abstract

The quality of super-resolution images obtained by single-molecule localization microscopy (SMLM) depends largely on the software used to detect and accurately localize point sources. In this work, we focus on the computational aspects of super-resolution microscopy and present a comprehensive evaluation of localization software packages. Our philosophy is to evaluate each package as a whole, thus maintaining the integrity of the software. We prepared synthetic data that represent three-dimensional structures modeled after biological components, taking excitation parameters, noise sources, point-spread functions and pixelation into account. We then asked developers to run their software on our data; most responded favorably, allowing us to present a broad picture of the methods available. We evaluated their results using quantitative and user-interpretable criteria: detection rate, accuracy, quality of image reconstruction, resolution, software usability and computational resources. These metrics reflect the various tradeoffs of SMLM software packages and help users to choose the software that fits their needs.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    et al. Imaging intracellular fluorescent proteins at nanometer resolution. Science 313, 1642–1645 (2006).

  2. 2.

    , & Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods 3, 793–795 (2006).

  3. 3.

    , & Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys. J. 91, 4258–4272 (2006).

  4. 4.

    & Plasmonics meets far-field optical nanoscopy. ACS Nano 6, 4580–4584 (2012).

  5. 5.

    , & Super-resolution surface mapping using the trajectories of molecular probes. Nat. Commun. 2, 515 (2011).

  6. 6.

    , & A starter kit for point-localization super-resolution imaging. Curr. Opin. Chem. Biol. 15, 813–821 (2011).

  7. 7.

    , & A guide to super-resolution fluorescence microscopy. J. Cell Biol. 190, 165–175 (2010).

  8. 8.

    Localization microscopy coming of age: from concepts to biological impact. J. Cell Sci. 126, 3505–3513 (2013).

  9. 9.

    New directions in single-molecule imaging and analysis. Proc. Natl. Acad. Sci. 104, 12596–12602 (2007).

  10. 10.

    & Fluorophore localization algorithms for super-resolution microscopy. Nat. Methods 11, 267–279 (2014).

  11. 11.

    & Art and artifacts in single-molecule localization microscopy: beyond attractive images. Nat. Methods 11, 235–238 (2014).

  12. 12.

    , , , & Measuring localization performance of super-resolution algorithms on very active samples. Opt. Express 19, 7020–7033 (2011).

  13. 13.

    , , & 3-D PSF fitting for fluorescence microscopy: implementation and localization application. J. Microsc. 249, 13–25 (2013).

  14. 14.

    , , & SimpleSTORM: a fast, self-calibrating reconstruction algorithm for localization microscopy. Histochem. Cell Biol. 141, 613–627 (2014).

  15. 15.

    , , , & ThunderSTORM: a comprehensive ImageJ plugin for PALM and STORM data analysis and super-resolution imaging. Bioinformatics 30, 2389–2390 (2014).

  16. 16.

    , , & High density 3D localization microscopy using sparse support recovery. Opt. Express 22, 31263–31276 (2014).

  17. 17.

    , , , & Localization-based super-resolution microscopy with an sCMOS camera part III: camera embedded data processing significantly reduces the challenges of massive data handling. Opt. Lett. 38, 1769–1771 (2013).

  18. 18.

    et al. Localization events-based sample drift correction for localization microscopy with redundant cross-correlation algorithm. Opt. Express 22, 15982–15991 (2014).

  19. 19.

    , , & Localisation microscopy with quantum dots using non-negative matrix factorisation. Opt. Express 22, 24594–24605 (2014).

  20. 20.

    & The lateral and axial localization uncertainty in super-resolution light microscopy. ChemPhysChem 15, 664–670 (2014).

  21. 21.

    et al. Measuring image resolution in optical nanoscopy. Nat. Methods 10, 557–562 (2013).

  22. 22.

    , & Statistical deconvolution for superresolution fluorescence microscopy. Biophys. J. 102, 2391–2400 (2012).

  23. 23.

    , , , & Quantitative study of single molecule location estimation techniques. Opt. Express 17, 23352–23373 (2009).

  24. 24.

    , & Precise nanometer localization analysis for individual fluorescent probes. Biophys. J. 82, 2775–2783 (2002).

  25. 25.

    , & Localization accuracy in single-molecule microscopy. Biophys. J. 86, 1185–1200 (2004).

  26. 26.

    et al. High throughput 3D super-resolution microscopy reveals Caulobacter crescentus in vivo Z-ring organization. Proc. Natl. Acad. Sci. USA 111, 4566–4571 (2014).

  27. 27.

    , & Visualization of localization microscopy data. Microsc. Microanal. 16, 64–72 (2010).

  28. 28.

    , & A high-density 3D localization algorithm for stochastic optical reconstruction microscopy. Opt. Nanoscopy 1, 1–10 (2012).

  29. 29.

    , , & Fast and efficient molecule detection in localization-based super-resolution microscopy by parallel adaptive histogram equalization. ACS Nano 7, 5207–5214 (2013).

  30. 30.

    , , & Faster STORM using compressed sensing. Nat. Methods 9, 721–723 (2012).

  31. 31.

    , & D.A.O.S.T.O.R.M.: an algorithm for high-density super-resolution microscopy. Nat. Methods 8, 279–280 (2011).

  32. 32.

    et al. in Focus on Microscopy (FOM2013) (Maastricht, the Netherlands, 2013).

  33. 33.

    et al. FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data. Sci. Rep. 4, 4577 (2014).

  34. 34.

    et al. in Proceedings of the 10th International Conference on Sampling Theory and Applications (SAMPTA) (Bremen, Germany, 2013).

  35. 35.

    , , , & in 2011 International Conference on Field Programmable Logic and Applications (FPL) 1–5 (2011).

  36. 36.

    & Image analysis with rapid and accurate two-dimensional Gaussian fitting. Langmuir 25, 8152–8160 (2009).

  37. 37.

    , , & Fast, single-molecule localization that achieves theoretically minimum uncertainty. Nat. Methods 7, 373–375 (2010).

  38. 38.

    & GraspJ: an open source, real-time analysis package for super-resolution imaging. Opt. Nanoscopy 1, 11 (2012).

  39. 39.

    , , & Fast compressed sensing analysis for super-resolution imaging using L1-homotopy. Opt. Express 21, 28583–28596 (2013).

  40. 40.

    , & Fast maximum likelihood algorithm for localization of fluorescent molecules. Opt. Lett. 37, 413–415 (2012).

  41. 41.

    et al. Ultra-fast, high-precision image analysis for localization-based super resolution microscopy. Opt. Express 18, 11867–11876 (2010).

  42. 42.

    , , & Fast and precise algorithm based on maximum radial symmetry for single molecule localization. Opt. Lett. 37, 2481–2483 (2012).

  43. 43.

    & Investigating intracellular dynamics of FtsZ cytoskeleton with photoactivation single-molecule tracking. Biophys. J. 95, 2009–2016 (2008).

  44. 44.

    et al. Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure. Proc. Natl. Acad. Sci. USA 106, 3125–3130 (2009).

  45. 45.

    et al. QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ. Nat. Methods 7, 339–340 (2010).

  46. 46.

    Rapid, accurate particle tracking by calculation of radial symmetry centers. Nat. Methods 9, 724–726 (2012).

  47. 47.

    et al. Patch-based non-local functional for denoising fluorescence microscopy image sequences. IEEE Trans. Med. Imaging 29, 29 (2010).

  48. 48.

    Localization of a fluorescent source without numerical fitting. Opt. Express 16, 18714–18724 (2008).

  49. 49.

    , , , & Real-time analysis and visualization for single-molecule based super-resolution microscopy. PLoS ONE 8, e62918 (2013).

  50. 50.

    , , & in Focus on Microscopy (FOM2013) (Maastricht, the Netherlands, 2013).

  51. 51.

    & The Handbook of Single-Molecule Biophysics (Springer, 2009).

  52. 52.

    , , & Simultaneous multiple-emitter fitting for single molecule super-resolution imaging. Biomed. Opt. Express 2, 1377–1393 (2011).

  53. 53.

    , , & PALMER: a method capable of parallel localization of multiple emitters for high-density localization microscopy. Opt. Express 20, 16039–16049 (2012).

  54. 54.

    , & A high-density 3D localization algorithm for stochastic optical reconstruction microscopy. Opt. Nanoscopy 1, 6 (2012).

  55. 55.

    et al. Bayesian localization microscopy reveals nanoscale podosome dynamics. Nat. Methods 9, 195–200 (2012).

  56. 56.

    Photon Transfer (SPIE Publications, 2007).

  57. 57.

    , , & Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 319, 810–813 (2008).

  58. 58.

    et al. Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function. Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).

  59. 59.

    et al. Three-dimensional sub-100 nm resolution fluorescence microscopy of thick samples. Nat. Methods 5, 527–529 (2008).

  60. 60.

    , , & Improved single particle localization accuracy with dual objective multifocal plane microscopy. Opt. Express 17, 6881–6898 (2009).

  61. 61.

    , , & Optimized localization analysis for single-molecule tracking and super-resolution microscopy. Nat. Methods 7, 377–381 (2010).

  62. 62.

    , , & in Fourth Single Molecule Localisation Microscopy Symposium (SMLMS′14) (London, UK, 2014).

  63. 63.

    et al. Video-rate nanoscopy using sCMOS camera-specific single-molecule localization algorithms. Nat. Methods 10, 653–658 (2013).

  64. 64.

    et al. Precisely and accurately localizing single emitters in fluorescence microscopy. Nat. Methods 11, 253–266 (2014).

  65. 65.

    , , & Fourier ring correlation as a resolution criterion for super-resolution microscopy. J. Struct. Biol. 183, 363–367 (2013).

  66. 66.

    & Optimal determination of particle orientation, absolute hand, and contrast loss in single-particle electron cryomicroscopy. J. Mol. Biol. 333, 721–745 (2003).

  67. 67.

    , & A call for bioimaging software usability. Nat. Methods 9, 666–670 (2012).

Download references

Acknowledgements

We thank N. Olivier for providing the experimental data and R. Nieuwenhuizen for his technical assistance in running the Fourier ring correlation. We thank also P. Thévenaz for critical reading and for his assistance in writing the manuscript. We thank the participants in the ISBI 2013 localization microscopy challenge: S. Anthony, S. Andersson, T. Ashley, D. Baddeley, K. Bennett, J. Boulanger, N. Brede, L. Dai, L. Fiaschi, F. Gruell, G. Hagen, R. Henriques, A. Herbert, S. Holden, E. Hoogendoorn, B. Huang, Z.-L. Huang, A. Kechkar, K. Kim, M. Kirchgessner, U. Koethe, P. Krizek, M. Lakadamyali, Y. Li, K. Lidke, R. McGorty, L. Muresan, R. Parthasarathy, B. Rieger, H. Rouault, M. Sauer, J.-B. Sibarita, I. Smal, A. Small, S. Stahlheber, Y. Tang, Y. Wang, S. Watanabe, S. Wolter, J.C. Ye and C. Zimmer. This work was supported by the Biomedical Imaging Group, the School of Engineering at the Ecole Polytechnique Fédérale de Lausanne, the European Research Council (ERC) FUN-SP project (267439), the ERC Starting Grant PALMassembly (243016) and the Eurobioimaging Project (WP11).

Author information

Affiliations

  1. Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.

    • Daniel Sage
    • , Hagai Kirshner
    •  & Michael Unser
  2. Center for Genomic Regulation, Barcelona, Spain.

    • Thomas Pengo
  3. Howard Hughes Medical Institute, University of California (UCSF), San Francisco, California, USA.

    • Nico Stuurman
  4. Department of Cellular and Molecular Pharmacology, UCSF, San Francisco, California, USA.

    • Nico Stuurman
  5. Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.

    • Junhong Min
  6. Laboratory of Experimental Biophysics, EPFL, Lausanne, Switzerland.

    • Suliana Manley

Authors

  1. Search for Daniel Sage in:

  2. Search for Hagai Kirshner in:

  3. Search for Thomas Pengo in:

  4. Search for Nico Stuurman in:

  5. Search for Junhong Min in:

  6. Search for Suliana Manley in:

  7. Search for Michael Unser in:

Contributions

D.S., H.K., T.P., J.M. and N.S. conceived the project. D.S. developed the project and organized the challenge with contribution from all authors. D.S. and H.K. wrote the code for the simulated data and analyzed the results. S.M. and M.U. directed the project. D.S. and H.K. wrote the manuscript with input from all authors.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Daniel Sage.

Integrated supplementary information

Supplementary information

PDF files

  1. 1.

    Combined PDF

    Supplementary Figures 1–3 and Supplementary Notes 1–3

Zip files

  1. 1.

    Supplementary Data 1

    Visual results, comparison to ground-truth

  2. 2.

    Supplementary Software 1

    Java software to compare two sets of localization

Excel files

  1. 1.

    Supplementary Data 2

    Numerical results, values, grades, and ranking

Videos

  1. 1.

    Supplementary Video 1

    HD1 100 frames of the contest dataset HD1

  2. 2.

    Supplementary Video 2

    HD2 100 frames of the contest dataset HD2

  3. 3.

    Supplementary Video 3

    HD3 100 frames of the contest dataset HD3

  4. 4.

    Supplementary Video 4

    LS1 100 frames of the contest dataset LS1

  5. 5.

    Supplementary Video 5

    LS2 100 frames of the contest dataset LS2

  6. 6.

    Supplementary Video 6

    LS3 100 frames of the contest dataset LS3

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nmeth.3442

Further reading