Abstract
Localization microscopy is a superresolution imaging technique that relies on the spatial and temporal separation of blinking fluorescent emitters. These blinking events can be individually localized with a precision significantly smaller than the classical diffraction limit. This subdiffraction localization precision is theoretically bounded by the number of photons emitted per molecule and by the sensor noise. These parameters can be estimated from the raw images. Alternatively, the resolution can be estimated from a rendered image of the localizations. Here, we show how the rendering of localization datasets can influence the resolution estimation based on decorrelation analysis. We demonstrate that a modified histogram rendering, termed bilinear histogram, circumvents the biases introduced by Gaussian or standard histogram rendering. We propose a parameterfree processing pipeline and show that the resolution estimation becomes a function of the localization density and the localization precision, on both simulated and stateoftheart experimental datasets.
Similar content being viewed by others
Introduction
In 2019 we proposed a novel method to estimate resolution using decorrelation analysis on a single image^{1}. A partial phase autocorrelation for a series of filtered images determines the highest spatial frequency with sufficiently high signal to noise ratio. The method has now been tested for more than a year across imaging modalities using the open source software (https://github.com/Ades91/ImDecorr). We received overall positive comments from specialists ranging from twophoton microscopy to structured illumination imaging and beyond. Initially, we presented decorrelation analysis on the singlemolecule localization microscopy symposium (SMLMS), looking for feedback from this community that relies heavily on image processing. Unlike other superresolution methods, most SMLM software does not directly output an image, but a set of localizations that need to be rendered for visualization. This adds another level of complexity to the interpretation of the images. Interactive discussions around increasing localization precision in optimized SMLM (due to new developments in experiments^{2,3,4} and SMLM software^{5}) and the resulting demands on localization density prompted us to investigate in greater detail the best practice of using our algorithm.
Ideally, the estimated resolution should not change when choosing a different method to render the dataset. Nevertheless, commonly used rendering choices (fixed and localizationuncertaintybased Gaussian rendering) are implying additional assumptions about the underlying localization statistics, which can impact the resolution estimate^{6}. In our publication, we rendered all the localizations as a Gaussian with a standard deviation equal to their respective localization uncertainty. The estimation of the localization uncertainty is a nontrivial task and depends on many parameters such as camera calibration or the noise model used^{7,8}. In addition, there exist localization softwares using e.g., machine learning that do not routinely return localization estimates^{9}. In our recent addendum^{6}, we advise, based on simulations, to choose histogram rendering to construct input images for decorrelation analysis. We show that the use of fixed Gaussian rendering is not optimal, as it is likely to bias the resolution estimate and does not provide any benefits compared to histogram rendering. We also discussed the challenges of using histogram rendering in conjunction with our resolution estimation algorithm.
Here, we focus on diverse experimental data, pinpointing the limits of SMLM and decorrelation analysis and proposing a workflow for accurate resolution estimation. We present a modified histogram method for SMLM dataset rendering that is compatible with resolution estimation using decorrelation analysis. The proposed bilinear histogram rendering method does not require the knowledge of the localization uncertainty estimate or artificial jittering of the localizations and minimizes the rounding error of naïve histogram rendering. We demonstrate that the modified histogram rendering is able to convey the localization information into the image accurately. Using experimental data and simulations, we show how our resolution estimate depends on the rendering pixel size and localization density. We find that our method expects a localization density of about 1–4×10^{4} loc. per µm^{2} to work reliably with experimental data.
Results
Histogram and bilinear histogram rendering
Let us consider an SMLM dataset of N localizations with positions \(\left[{x}_{n},{y}_{n}\right]\). One way to render the data without making additional assumptions is to plot them as a histogram, which is usually expressed as
where \(\left\lfloor {x}_{n}\right\rfloor\) denotes the \(x\) position of the \({n}^{{th}}\) localization event floored to the nearest integer multiple of the chosen pixel size. This rounding operation is problematic as it introduces a roundoff error, which can be detrimental to the resolution estimation, especially if the pixel size is on the order of the localization uncertainty. In order to alleviate this effect, we propose to render the data using a modified histogram expression
where \(*\) denotes the convolution operation of a discrete Dirac distribution with the 2 × 2 matrix \({w}_{n}\). Compared to Eq. 1, each localization is spread on the four nearest pixels and the weights are given by
where \(\otimes\) denotes the tensor product. This rendering approach is equivalent to a linear (or bilinear for the 2D case) interpolation and is also equivalent to a 2D average shifted histogram with an infinitesimal shift^{10}. The weight distribution of three localizations is illustrated in Fig. 1a. Figure 1b and c show the rendering of the same dataset (central region of cSir phalloidin dataset presented in Fig. 2) using standard histogram rendering (b) and bilinear histogram rendering (c). We see that the standard histogram image looks more pixelated (due to the rounding operation) while the bilinear histogram looks noticeably smoother.
This rendering method will be referred to as bilinear histogram rendering. We note that this rendering approach is not new^{11} but it is the first time, to our knowledge, that it is used in the context of resolution estimation. With this approach, the information about the position of the localization is conveyed into the image and it is, to our knowledge, the most appropriate rendering choice for resolution estimation based on decorrelation analysis^{1}. The resolution estimation assumes that the image is expressed as the sum of a signal (exhibiting spatial correlations) and uncorrelated noise. The estimation relies on the computation of crosscorrelation coefficients \({d}\) between the highpassed version of the input image and its lowpass filtered Fourier normalized version. Normalizing the Fourier transform balances the contributions of signal and noise. The lowpass filter consists in an ideal lowpass filter characterized by its cutoff frequency \(r\), which allows to compute the crosscorrelation coefficient as a function of \(r\). With decreasing r, first noise contributions are gradually removed while preserving the bandwidthlimited signal. The asymmetry (noise rejection but signal preservation) is mathematically translated into an increase in the correlation. Eventually, the lowpass filter will start to remove the signal which reduces the correlation. The presence of a local maximum in the function \(d\left(r,\sigma \right)\) indicates the spatial frequency of best noise rejection and signal preservation. The highpass filter is an inverted Gaussian shaped filter of standard deviation \(\sigma\). We have demonstrated that the image resolution can be estimated from the function \(d\left(r,\sigma \right)\) by finding, for each value of \({{\sigma }}_{i}\), the position \({r}_{i}\) of the local maxima. The image cutoff is then defined as the largest \({r}_{i}\).
In the case of SMLM data, the resulting resolution estimate depends on the density of localizations, the homogeneity of the labeling, the filtering of high uncertainty localizations and the accuracy of the drift correction. A Matlab implementation of the bilinear histogram method used in this work is made publicly available at https://github.com/Ades91/ImDecorr [https://doi.org/10.5281/zenodo.4655984].
The only remaining open parameter for rendering is the pixel size. If it is set too small, the localizations might not exhibit any spatial correlations as gaps will appear between localizations, which is likely to result in an underestimated resolution. If the pixel size is set too large, our resolution estimation will possibly be close to twice the pixel size, which means that the Nyquist sampling criterion is not fulfilled. Similarly, if the localization density is too low, such that no continuous structures can form, the algorithm will also underestimate the resolution (see Supplementary Information, Note and Figs. 1, 2 and 3 for a comparison between standard and bilinear histogram and Gaussian rendering).
Experimental results
In order to demonstrate the applicability and advantages of the proposed bilinear histogram rendering method for resolution estimation of SMLM, we applied the discussed methodology on several SMLM datasets, including (d)STORM, DNAPAINT and multicolor STORM. Although most journals mandate or encourage data sharing, comprehensive and uptodate database(s) of localization microscopy are still rare. In our opinion, they are an invaluable asset for tool developers and the SMLM community.
Experimental results: ShareLoc
We first applied our method on several single channel (d)STORM datasets publicly available at https://shareloc.xyz/. We selected five different proteins from different cellular structures acquired by three different groups (see Table 1 below for details).
Figure 2a–e shows the five selected data sets rendered at their highest localization density using bilinear histogram rendering with a pixel size of 5 nm. The resolution indicated corresponds to the smallest resolution estimated from all the tested pixel size. For comparison purposes, a fieldofview of 12 × 12 µm^{2} is chosen for all data sets. Figure 2f–j shows a 2.5 × 2.5 µm^{2} zoomin of Fig. 2a–e, indicated by the white squares. Figure 2k shows the resolution estimate as a function of the pixel size at their highest localization density. We see that for large pixel size, the resolution evolves linearly at twice the pixel size (sampling limited). As the pixel size decreases, the resolution estimate reaches a minimum depending on the data set and then rises again (sparsity limited). The minimum of each curve corresponds to the resolution indicated in Fig. 2a–e. Figure 2l shows the estimated resolution as a function of the number of localizations included in the analysis. For each sample point, we computed the resolution as a function of the pixel size and retained only the smallest estimate. We see that the number of localizations required to reach a stable resolution estimate depends on the structure. However, all the curves exhibit the same trend of an underestimated resolution that converges to a plateau once a certain number of localizations are exceeded.
Figure 2m shows the same curves as Fig. 2l but with the number of localizations normalized to the area covered by the structure. The sample area is estimated directly from the image rendered at the highest localization density with a pixel size of 5 nm and estimated as the number of pixels with a value greater than 0.5 multiplied by the area of a single pixel. We see from the proposed normalization that the five images have similar localization density as well as a relatively similar threshold for the convergence of the resolution estimate of about 1–4 × 10^{4} loc. per µm^{2}.
We note that the estimated sample area and, therefore the localization density can strongly vary depending on the chosen pixel size and threshold (5 nm and 0.5 in this manuscript; both values were found, based on visual inspection, to produce an adequate estimate of the sample area for all the datasets presented). To be able to compare the localization density from different experiments, it is mandatory to use the same pixel size and threshold.
Experimental results: DNAPAINT
We also applied our method to DNAPAINT data of microtubules in COS7 cells (courtesy of F. Schueder and R. Jungmann, reported localization precision of ~5.5 nm^{12}). The dataset considered has a reported localization uncertainty of about 8 nm.
Figure 3a–c show the bilinear histogram rendering of three randomly selected fieldsofview. Figure 3d shows the estimated resolution as a function of the pixel size. Figure 3e shows the estimated resolution as a function of the localization density. For each sample point, the resolution is estimated as a function of the pixel size and the smallest resolution is retained. We observe that a localization density smaller than about 10^{4} loc. per µm^{2} is not sufficient for our method to output a resolution estimate. Past this threshold, we see that the resolution gradually improves with the localization density and stabilizes at around 3–4×10^{4} loc. per µm^{2}. These numbers are consistent with the results obtained with the shareloc data. Finally, Fig. 3f shows a zoomin of Fig. 3c. The ability to visually resolve the microtubule hollowness confirms the high resolution predicted by our algorithm.
Experimental results: Multicolor STORM
Finally, we applied our method to multicolor STORM/DNAPAINT data (courtesy of A. Jimenez and C. Leterrier, reported localization precision of ~4 nm, Fig. 6a of ref. ^{13}). Bilinear histogram rendering of a 12 × 12 µm^{2} selected ROI is shown in Fig. 4a. We show in the inset of Fig. 4a, a zoomin of the tubulin channel where the microtubule hollowness is also clearly visible.
Figure 4b shows the resolution as a function of the pixel size for all the channels. Figure 4c shows the resolution estimate as a function of the localization density. Again, we observe that a localization density of about 10^{4} loc./µm^{2} is required to get a first estimate. As we include more localizations, the resolution gradually improves until it reaches a plateau at about 2–4 × 10^{4} loc. per µm^{2}.
Discussion
As previously shown^{1}, localizationuncertaintybased Gaussian rendering can also provide a reasonable resolution estimate, but is making additional assumptions about the localization statistics and requires correct estimation of the individual localization uncertainty^{1}. On the other hand, using a constant Gaussian kernel can bias the resolution estimate and should not be used with decorrelation analysis^{6}. Bilinear histogram rendering is the method of choice since it uses the smallest amount of information about the localizations, their position. It also alleviates the rendering rounding error compared to standard histogram rendering, thus requiring a significantly lower number of localizations enabling reliable resolution estimation for state of the art SMLM data.
Here, we have shown how the resolution is a function of the rendering pixel size for bilinear histogram rendering. To find the optimal pixel size (balance between sampling and density), we advise to compute the resolution as a function of the pixel size and retain the smallest resolution achieved. To assess the labeling density, we computed the resolution as a function of the number of localizations included in the rendering. Since the optimal pixel size depends on the number of localizations included, we advise to again compute the resolution as a function of the pixel size and retain the smallest resolution achieved. The presence of a plateau is a reliable indicator of sufficient labeling density. The average runtime for the estimation of the resolution of a single experimental dataset over a fieldofview of 12 × 12 µm is about 30 seconds (see Supplementary Information, Note 4). Using a variety of experimental datasets, we have shown that a localization density of about 1–4 × 10^{4} loc. per µm^{2} was required for the resolution estimation to converge. A Matlab implementation of the bilinear histogram method used in this work as well as a basic script for processing of localization data is publicly available at https://github.com/Ades91/ImDecorr^{12}.
Methods
The localization dataset is loaded into Matlab. The x and y localization positions are converted to nanometers. All the localizations outside of the specified fieldofview are then filtered. To estimate the localization density, we have to estimate the sample surface. To estimate the sample surface, we render all the localizations using bilinear histogram rendering with a pixel size of 5 nm. The resulting image is then binarized with a threshold of 0.5. The surface is given as the number of pixels with a value >0.5 times the area of a single pixel. The localization density is then given by the number of localizations included in the rendering divided by the sample area. Then for each localization density and each pixel size, we render the localization and estimate the image resolution using decorrelation analysis^{1}.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
The shareloc data are publicly available at https://shareloc.xyz/#/. Data requests for DNA paint datasets must be addressed to Florian Shueder and Ralf Jungmann. Datasets for multicolor STORM are publicly available https://figshare.com/articles/dataset/Source_Data_for_Figure_6_of_Jimenez_et_al_Methods_2020/12279917.
Code availability
All the codes used in this manuscript are publicly available at https://github.com/Ades91/ImDecorr.
References
Descloux, A. C., Grussmayer, K. S. & Radenovic, A. Parameterfree image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918–924 (2019).
Coelho, S. et al. Ultraprecise singlemolecule localization microscopy enables in situ distance measurements in intact cells. Sci. Adv. 6, eaay8271 (2020).
Cnossen, J. et al. Localization microscopy at doubled precision with patterned illumination. Nat. Methods 17, 59–63 (2020).
Gwosch, K. C. et al. MINFLUX nanoscopy delivers 3D multicolor nanometer resolution in cells. Nat. Methods 17, 217–224 (2020).
Li, Y. et al. Realtime 3D singlemolecule localization using experimental point spread functions. Nat. Methods 15, 367–369 (2018).
Descloux, A., Grußmayer, K.S. & Radenovic, A. Addendum: parameterfree image resolution estimation based on decorrelation analysis. Nat. Methods 1061–1063 (2020).
Mortensen, K. I., Churchman, L. S., Spudich, J. A. & Flyvbjerg, H. Optimized localization analysis for singlemolecule tracking and superresolution microscopy. Nat. Methods 7, 377–381 (2010).
Huang, F. et al. Videorate nanoscopy using sCMOS camera–specific singlemolecule localization algorithms. Nat. Methods 10, 653–658 (2013).
Ouyang, W., Aristov, A., Lelek, M., Hao, X. & Zimmer, C. Deep learning massively accelerates superresolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018).
Scott, D. W. Averaged shifted histograms: effective nonparametric density estimators in several dimensions. Ann. Stat, 13 1024–1040 (1985).
Wolter, S. et al. Real‐time computation of subdiffraction‐resolution fluorescence images. J. Microsc. 237, 12–22 (2010).
Schueder, Florian et al. “An order of magnitude faster DNAPAINT imaging by optimized sequence design and buffer conditions”. Nat. Methods 16, 1101–1104 (2019).
Angélique, Jimenez, Friedl, Karoline & Leterrier, Christophe About samples, giving examples: optimized single molecule localization microscopy. Methods 174, 100–114 (2020).
Acknowledgements
We thank Christian Sieben and Jean Comtet for the helpful and critical reading of our manuscript. We also thank Jonas Ries for bringing the motivation to propose a modified rendering method compatible with our resolution estimation. We thank Wei Ouyang, Orestis Faklaris, Markus Sauer, Christophe Leterrier, Florian Schueder and Ralf Jungmann for providing the experimental datasets. Color of line plots were taken from Matlab file exchange: Beautiful and distinguishable line colors by Jonathan C. Lansey. This project has been partly funded by the European Union’s Horizon 2020 research and innovation program via grant 686271/SEFRI 16.0047. [750528]. A.D. and A.R. acknowledge the support from Zeiss IDEAS center and EPFL open science fund.
Author information
Authors and Affiliations
Contributions
A.C.D. initiated the work, wrote the algorithms and processed the data. A.C.D., K.S.G. and A.R. wrote the manuscript and selected the experimental data.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Descloux, A.C., Grußmayer, K.S. & Radenovic, A. Parameterfree rendering of singlemolecule localization microscopy data for parameterfree resolution estimation. Commun Biol 4, 550 (2021). https://doi.org/10.1038/s42003021020861
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s42003021020861
This article is cited by

Fluorescence nanoscopy at the sub10 nm scale
Biophysical Reviews (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.