Addendum: Parameter-free image resolution estimation based on decorrelation analysis

Addendum to: Nature Methods https://doi.org/10.1038/s41592-019-0515-7, published online 26 August 2019.

Recently we reported a new method for the estimation of the resolution of light microscopy images. Our goal was to demonstrate that that method is applicable to any kind of microscopy images, including single-molecule localization microscopy (SMLM). Unlike other super-resolution methods, single-molecule localization datasets must be rendered in order to be visualized. Ideally, the resolution estimate should not depend on the method chosen to render the data. In practice, however, this is not the case and commonly used rendering methods implicitly make additional assumptions about the underlying localization statistics, which in turn can have an impact on the estimated resolution obtained using decorrelation analysis.

In our publication, we rendered all the localizations as a Gaussian with a standard deviation equal to their respective localization uncertainty. We acknowledge that the results we presented may not apply if another rendering method is used. Here we discuss the impact on our resolution estimation method of three standard approaches for SMLM dataset rendering: histogram rendering, fixed Gaussian rendering and localization-uncertainty-based Gaussian rendering.

Localization-uncertainty-based Gaussian rendering

Localization-uncertainty-based Gaussian rendering is the method we used in the original publication. We have shown, using simulations and experimental data (see Fig. 4 and Supplementary Results 7 of the original article) that this was a valid rendering choice for localization datasets and that we were able to get a meaningful resolution estimate. However, the estimation of the localization uncertainty itself is not a trivial task and depends on several parameters—for example, a proper camera calibration to estimate photon counts. Ideally, the resolution estimation should not depend on external parameters and should be contained in the localizations themselves. Hence, we discuss the impact of two alternative standard rendering methods on our resolution estimation method that only consider the localizations’ locations: fixed Gaussian and histogram rendering.

Fixed Gaussian rendering

Fixed Gaussian rendering consists in displaying each localization as a Gaussian with a small fixed standard deviation. While being a fast and perfectly valid choice in terms of visualization, this method is implicitly making a strong assumption concerning the data. If we could experimentally repeat the localization procedure of a single emitter an infinite number of times and render the data as a histogram, the result would approximate a Gaussian-like distribution with a spatial spread equal to the localization precision, centered around the true position of the fluorophore. In contrast, when we use a Gaussian to render a single localization event, it will be centered around the measured position. We thus make the strong assumption that the measured position is the true position of the fluorophore and that the localizations are spread according to the chosen standard deviation. This assumption is also made by the localization-uncertainty-based Gaussian rendering, albeit with a physically meaningful localization spread.

Let us consider an SMLM dataset of N localizations with positions p_n = [x_n, y_n]. The rendered image using a fixed Gaussian as a kernel is, in Fourier space:

$$I_G\left( {{\boldsymbol{k}},\sigma } \right) = \mathop {\sum}\limits_n^N {e^{i{\boldsymbol{kp}}_n}G\left( {{\boldsymbol{k}},\sigma } \right) = F\left( {\boldsymbol{k}} \right)G\left( {{\boldsymbol{k}},\sigma } \right)}$$

where k is the reciprocal space of p, $F\left( k \right) = \mathop {\sum}\limits_n^N {e^{i{\boldsymbol{kp}}_n}}$is the Fourier transform of the Dirac distribution and G(k, σ) is the Fourier transform of a Gaussian distribution of standard deviation σ. The object spectrum is simply weighted by a Gaussian. From an image-processing point of view, this corresponds to a filtering of Fourier space and is a direct manipulation of the frequency content of the image. The estimated resolution of I_G(k, σ) is therefore a function of both the object spectrum F(k) and the (arbitrary) Gaussian filter size σ. If the localization density is too low, the object spectrum will not overcome the Gaussian contribution, which will result in a biased resolution estimate dominated by the Gaussian spread. Even for higher localization density, the estimated resolution will be a function of both the object spectrum and the chosen Gaussian size (see Simulations section below and ref. ¹).

Another open parameter is the pixel size of the rendered image. As long as the pixel size is smaller than the Gaussian spread (appropriate sampling of the Gaussians), the resolution estimate is independent of the pixel size. However, if the pixel size is larger than the Gaussian spread, the Gaussian will be well approximated by a Dirac distribution and the image resolution will behave as if it was rendered as a histogram.

Histogram rendering

One way to render the data without making additional assumptions is to plot them as a histogram, which is usually expressed as

$$I\left[ {x,y} \right] = \mathop {\sum}\limits_n^N {\delta \left[ {x - x_n,y - y_n} \right]}$$

where x_n denotes the x position of the nth localization event floored to the nearest integer multiple of the chosen pixel size. However, this rounding operation introduces a roundoff error whose amplitude depends on the chosen pixel size. This roundoff error is minimal when the pixel size is set smaller than the localization uncertainty. However, a small pixel size also implies that the localizations might not exhibit any spatial correlations, as gaps will appear between localizations, which is likely to result in an underestimated resolution or even no resolution estimate at all (no local maxima produced in the decorrelation curves). Similarly, if the localization density is too low, such that no continuous structures form, the algorithm will also underestimate the resolution. We refer to this case as sparsity limited. Conversely, if the pixel size is set too large, our resolution estimate might be very close to twice the pixel size, which means that the Nyquist sampling criterion is not fulfilled (Fig. 1). We refer to this case as sampling limited.

Fig. 1: Resolution as a function of rendering method.

Ground truth localization spread, 10 nm. a, Resolution estimation for histogram (magenta curve) and Gaussian rendering (green curve, rendering pixel size of 2 nm, rendering Gaussian s.d. K of 5 nm). The dashed black line indicates the ground truth (GT) resolution estimated directly from the image. b, Resolution estimate using the fixed Gaussian rendering method as a function of rendering Gaussian size K (2–14 nm) and number of localizations (1 × 10² to 5 × 10⁶). c, Resolution of histogram rendering as a function of the pixel size.

A consequence of the countervailing effects—small roundoff error but sparse signal at small pixel size, denser signal but large round-off error at larger pixel size—is that obtaining a resolution estimate using histogram rendering requires a substantial localization density to produce spatial correlations at a small enough pixel size. A solution based on a variation of histogram rendering that minimizes the roundoff error is in preparation and will be subject of a follow-up publication.

Localization density

As already stated above, the labeling and localization density is as important as the localization precision when considering the resolution. This fact is well known and has previously been discussed by others^2,3.

To assess whether a given SMLM experiment has sufficiently sampled the fluorescent emitters’ distribution, we propose to estimate the resolution as a function of the number of localizations included in the rendering. Since the resolution, especially for histogram rendering, is also a function of the pixel size (too large, sampling limited; too small, sparsity limited), the resolution must be estimated as a function of pixel size for each localization subset and the lowest resolution must be retained. As more localizations are included, the resolution estimate should evolve and stabilize after a certain number of frames. The presence of a plateau is the sign that the addition of more localizations does not significantly affect the resolution. We note that this procedure is not restricted to our method and can also be used, for example, with Fourier ring correlation (see Fig. 4 of the original article).

Simulations: resolution as a function of the rendering method

We now illustrate how the use of fixed Gaussian rendering is intrinsically problematic with our resolution estimation approach. This discussion applies to both direct Gaussian rendering of the localizations with a fixed standard deviation and histogram rendering followed by Gaussian blurring.

We performed simulations of lines with decreasing spacing (30 to 2 nm in steps of 2 nm, line width of 2 nm; see Supplementary Results 7 and Supplementary Results Fig. 7.1 of the original article for an example of similar structures). To restrict the discussion to the question of rendering, we directly simulate the localization procedure and do not discuss how different SMLM analysis software packages might influence the results. A detailed comparison of the performance of localization software can be found in refs. ^4,5, and it is the task of the user to select the appropriate software.

The localizations are simulated by randomly selecting an emitter and adding to its position a normally distributed random shift with a standard deviation of σ_gt = 10 nm. This simulation therefore does not consider many experimental effects such as drift, background localizations or bleaching and is merely presented to provide some qualitative understanding of the important consequences of the rendering choice for resolution estimation of localization microscopy data. Figure 1a shows the evolution of the resolution estimate using histogram rendering and Gaussian rendering for simulated localization data, along with the theoretical input resolution (~22 nm = 2.2σ_gt). We observe two distinct behaviors depending on the rendering scheme. For histogram rendering, the resolution slowly decreases as the localization density increases and stabilizes at about 110% of the ground truth resolution, which is consistent with previous simulations (see Supplementary Results 1 of the original article). For Gaussian rendering and for a low number of localizations, the resolution is initially given by the size of the rendering kernel. The resolution estimate slowly increases as we increase the number of localizations. At a certain number of localizations that depends on the rendering kernel, the resolution starts to increase. This is due to the competition between the object spectrum F(k) and the fixed Gaussian G(k, σ) over which of them will produce the strongest spatial correlations.

The non-trivial interaction between the F(k) and G(k, σ) contributions is illustrated in Fig. 1b. We show how fixed Gaussian rendering will influence the evolution of the resolution estimate as a function of the number of localizations included in the analysis. We observe two regimes that correspond to different ratios of the rendering kernel K versus the actual localization uncertainty σ_gt. For K < σ_gt/2 (kernel size smaller than half the localization uncertainty), the size of the Gaussian kernel is large enough to produce spatial correlations that will be picked up by the algorithm, predicting an overestimated resolution for low numbers of localizations. We see, however, that the localization density required to reach the plateau varies greatly and that the final resolution still depends on the input kernel size. The plateau was not even reached for K = 2 and 3 nm, despite the very high number of localizations. All the simulated fixed Gaussian images were rendered at a pixel size of 2 nm to ensure sufficient sampling of the Gaussians, even at the smallest rendering kernel of 2 nm.

For K > σ_gt/2 (kernel size larger than half the localization uncertainty), the resolution is obviously underestimated, again due to the convolution of the ground truth resolution with the Gaussian rendering size, effectively blurring the image.

Finally, Fig. 1c shows the evolution of the resolution estimate of the histogram-rendered image as a function of the pixel size. We see three distinct regimes. For a pixel size that is too large, the resolution increases linearly with the pixel size (the sampling-limited case). For the regime of appropriate pixel sizes, we reach a plateau of minimal resolution estimate (sometimes as narrow as a single value). If the pixel size is too small, the localizations are spread and do not correlate spatially (the sparsity-limited case). We also note that if the number of localizations is too low, the resolution estimate will not reach its theoretical minimum.