Abstract
Over the last decades, superresolution techniques have revolutionized the field of fluorescence microscopy. Among them, interferometric or 4Pi microscopy methods exhibit supreme resolving power in the axial dimension. Combined with singlemolecule detection/localization and adaptive optics, current 4Pi microscopy methods enabled 10â€“15â€‰nm isotropic 3D resolution throughout whole cells. However, further improving the achieved 3D resolution poses challenges arising from the complexity of singlemolecule emission patterns generated by these coherent singlemolecule imaging systems. These complex emission patterns render a large portion of information carrying photons unusable. Here, we introduce a localization algorithm that achieves the theoretical precision limit for a 4Pi based singlemolecule switching nanoscopy (4PiSMSN) system, and demonstrate improvements in localization precision, accuracy as well as stability comparing with stateoftheart 4PiSMSN methods.
Similar content being viewed by others
Introduction
The achievable resolution of a farfield fluorescence microscope was constrained by the diffraction limit of light, approximately 200â€“300â€‰nm in the lateral direction and 500â€“700â€‰nm in the axial direction. Over the last decades, substantial efforts were made to overcome this resolution limit. Based on confocal and widefield microscopy geometries respectively, 4Pi (type AC)^{1,2,3} and I^{n}M^{4,5,6} methods use coherent illumination and/or detection based on two opposing objectives to improve the resolution by 3â€“7 fold in the axial dimension^{7}. Combining coherent detection and the stochastic switching of single molecules with highprecision localization, 4Pi (or interferometric) based singlemolecule switching nanoscopy (SMSN) techniques, such as iPALM^{8,9} and 4PiSMSN^{10}, allow another 5â€“10 fold improvement in the axial resolution^{8,11} in comparison with conventional 3D SMSN. Further incorporating adaptive optics and interferometryspecific algorithm design, W4PiSMSN^{12} allowed highresolution reconstruction of the whole mammalian cell (up to ~9â€‰Î¼m) without deterioration of resolution throughout the imaging depth.
The resolution enhancing capacity of the interferometric systems comes from their complex point spread function (PSF) patterns, thereafter, referred as 4PiPSF. One of its distinct features is the rapid intensity modulation within the pattern center along the depth (axial) direction which, in turn, improves the axial resolution when combined with confocal, stimulated emission depletion microscopy and singlemolecule localization microscopy methods. Aside from the center peak, the complexities of 4PiPSF majorly arise from the features such as rings and lobes at its periphery. Depending on the axial position of the single emitter, these features contains up to 80% of the entire 4PiPSF energy (Methods). Theoretical precision calculations based on the Fisher information theory suggested that better localization precision can be obtained in both axial and lateral directions^{11,13} when compared with other incoherent dualobjective systems^{14}. To date, these complex features remain largely unexplored when localizing singlemolecule emission patterns from 4Pi systems, due to the challenge in modeling this complex pattern with high accuracy. An accurate 4PiPSF model must be able to take into account of the static imperfections of the interferometric singlemolecule imaging system, such as aberrations and transmission variances in both interferometric arms, partial coherence due to the broadband emission spectra and the dynamic changes of the system, such as the temperaturedependent changes of the interferometric cavity length.
To overcome those difficulties, we introduce a method based on coherent pupil functions to allow accurate modeling of 4PiPSFs and develop an localization algorithm to extract the position information content at the theoretical information limit while, at the same time, dynamically compensates the temperatureinduced cavity drift. The confluence of these new analytical methods improves both localization precision and bias in all three dimensions compared with existing 4PiSMSN systems^{10,12} and at the same time extends imaging volume in the axial direction.
Results
4PiPSF modeling with coherent phaseretrieved pupils
The hallmark of a 4PiPSF generated by an interferometric microscopy system is the distinct multilobe PSF in both axial and lateral dimensions. Currently, extracting singlemolecule axial positions from the center lobes (including center peak intensity and intensities falls within a empirically defined ring region) has been the major focus in 4Pi singlemolecule localization analysis^{8,10,12}, discarding or underutilizing substantial amount of informationcarrying photons among the lateral side lobes. At the same time, when pinpointing molecular centers at the lateral dimension (i.e., localization in x, y) the interferometric features of the 4PiPSFs have been, to a large extent, ignored^{8,12}. As a result, no lateral resolution enhancement is reported comparing with conventional PSFs^{8,12}.
To allow extracting information contained within the complex interferometric PSF, we describe here a PSF model based on two coherent pupil functions for 4Pi singlemolecule imaging systems^{10,12}. This method allowed us to create a realistic model of the interferometric emission patterns taking into account of the independent wave front distortions from the two interference arms, the wavelengthdependent coherence modulation and the spontaneous interferometric path length drift.
Let \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\) be a pair of pupil functions of the 4Pi system, representing the wave fields at the pupil planes of the upper and lower interference paths respectively (Fig.Â 1). The two pupil functions, retrieved independently through a phase retrieval (PR) algorithm^{15,16,17}, allowed us to incorporate aberrations for both interference paths. Interferometric PSFs (termed as 4PiPSF) are generated from the superposition of the wave fields, \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\). Four superposed wave fields with different polarizations and phase shifts can be written as,
Here, we assume \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\) are independent against polarization directions, where \(\varphi _{{\mathrm{sp}}} = 2\pi \delta L(n_{\mathrm{s}}  n_{\mathrm{p}})/\lambda\), with \(n_{\mathrm{s}}\) and \(n_{\mathrm{p}}\) the refractive indices of s and ppolarized light in quartz, and \(\delta L\) the quartz thickness difference between the two interference paths (Supplementary Note 1). \(\varphi _0\) is the phase difference between the two interference arms when the single emitter is in the common focus of the two objectives and thereafter we refer \(\varphi _0\) as the cavity phase. The defocus term \(D\left( z \right)\) equals to \({\mathrm{exp}}(ik_zz)\), with \(k_z\) being the \(z\) (axial) component of the wave vector and \(z\) is the relative axial position of the single emitter (axial distance to the common focus of the two objective lenses). A factor \(I_{\mathrm{t}}\), referred as the intensity ratio, was introduced to account for the transmission efficiency difference between the two interference paths. The value of \(I_{\mathrm{t}}\) was estimated for each experiment (Supplementary NotesÂ 1â€“2, Supplementary Figs.Â 1â€“2).
We can then write out the coherent PSF, \(\mu _{{\mathrm{I}}m}\), in terms of Fourier transform of each superposed wave field,
where subscript I indicates interference and Eq.Â 2 assumes that the wave fields \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\) are perfectly coherent. However, because of the finite spectral width of the emission filter, the estimated coherence length of the emission light is small, ~7.5â€‰Âµm (with an emission filter of 700/50â€‰nm, center wavelength/band width). Therefore, a slight change of the optical path length difference (OPD) between the two interference arms will result in a moderate reduction of the modulation depthâ€”the peak to valley contrast in an interferometric PSF and the degree of this reduction is wavelength dependent. To this end, we assume \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\) are partially coherent and the incoherent part will produce a conventional incoherent PSF described as \(\mu _{\mathrm{W}}\),
with subscript W indicates widefield in the sense of conventional microscope. Therefore, the final PSF is a combination of the interferometric PSF and the conventional PSF with an empirical factor \(a \in [0,1]\), describing the coherence strength (Supplementary NoteÂ 2 and Supplementary Fig.Â 2),
where \(\mu _0\) represents the normalized PSF (sum of the PSF intensities in s or ppolarization equals to 1). By adding a total photon count of \(I\) and a background of \(b\), the PR4PiPSF model is,
which includes quadruple PSFs for each emitter (Fig.Â 1). Here all \(I_m\) and \(b_m\) were considered as independent to account for the difference in transmission efficiency between the two polarizations and the emission paths after the beam splitter^{12}. We found that PR4PiPSFs can produce relatively uniform resolutions within a large \(a\) range (0.5â€“1), an attractive feature for thicker specimens, where \(a\) is depth dependent (Supplementary Fig.Â 2 and Supplementary NoteÂ 2).
As a demonstration of the accuracy of the PR4PiPSF model, we tested it with the experimental PSFs obtained by imaging 40â€‰nm beads attached on the coverslip surface. We pinpointed 3D positions of isolated beads with various photon counts and compared with the position readout from a highprecision piezo stage (Â±0.5â€‰nm closeloop) (Fig.Â 2, Supplementary Figs.Â 3â€“5). We found that comparing with ideal4PiPSF model (where \(h_{\mathrm{A}}\) and \(h_{\mathrm{B}}\) are unaberrated pupil functions, with \(I_{\mathrm{t}}\) and \(a\) equal to 1, Supplementary Fig.Â 3), PR4PiPSF model based localization results in comparable localization accuracy (bias) in the lateral dimension and improved localization accuracy in the axial dimension (Supplementary Fig.Â 4). When comparing with contrastbased method (referring to the method using incoherent PSF models for lateral localization and PSFâ€™s central moment for axial localization^{12}), both ideal and PR4PiPSF models show improved localization accuracies in both lateral and axial directions (Supplementary Fig.Â 4). We also notice that the localization deviations (bias) caused by contrastbased method are bead specific (Fig.Â 2, Supplementary Figs.Â 4â€“5, all bead data were collected from the same sample), especially in the lateral dimension, making it difficult to perform post acquisition correction due to the random nature of such deviations.
Information content within a 4PiPSF
Given the ability to generate a realistic 4PiPSF model, we are able to quantitatively investigate the information content within the 4PiPSF pattern recorded on our microscope using Fisher information matrix^{18,19}. By quantifying information content pixel by pixel within the 4PiPSF (as similarly shown previously^{11} for ideal PSFs), we found previous 4PiSMSN localization analysis either discarded the additional information from interferometric detection (as for lateral localization, Supplementary NoteÂ 3 includes derivation of a general case for such information gain) or incorporated only part of it (as for the axial localization that uses 0th and/or 3rd central moments of the PSFs, Supplementary Figs.Â 6â€“7). Incorporating a comprehensive and accurate PSF model allows us to further improve localization accuracy and precision in three dimensions.
To quantify the information content of 4PiPSFs, we calculated the Fisher information matrix, which quantifies the lower bound of the localization precision for an unbiased estimator: the CramÃ©râ€“Rao lower bound (CRLB)^{19},
where F is the Fisher information matrix, \(\theta\) is a vector of estimation parameters, i denotes the index of each parameter.
Incorporating Poisson noise and the readout noise from an sCMOS camera, a numerical calculation of each element in Fisher information matrix is^{20},
where q is the pixel index and m is the index of the quadruple 4PiPSFs, the factor\(\gamma\) equals to \(\sigma ^2/g^2\), where \(\sigma ^2\) is the variance of the readout noise per pixel and g is the gain of each pixel.
In comparison with incoherent dualobjective PSFs^{14}, we observed the predicted information in a 4PiPSF increase significantly along the axial dimension (17â€“160 folds at 1000 total emitted photon and 10 background photons) and a relatively large information gain at the outoffocus regions along the lateral dimension (5â€“10 folds at 1000 total emitted photon and 10 background photon) (Supplementary Figs.Â 8â€“9). Comparing to an ideal4PiPSF (Supplementary Fig.Â 3), the information contained in a realistic PSF model as PR4PiPSF, is slightly lower (Supplementary Fig.Â 9).
This observed informationcontent boost predicts and quantifies the improvement in the achievable localization precisions in all three dimensions by using 4PiPSFs. Existing 4Pi localization methods obtain the lateral position estimation by either merging the multichannel 4PiPSFs into incoherent dualobjective PSFs^{8,9,12} or modifying the PSFs with a Gaussian mask^{10} and applying a Gaussian or Airy disk PSF model on the merged or modified PSFs (Supplementary TableÂ 1). These methods result in reduction of information since the interference pattern is not fully considered leading to up to twofold worse localization precisions when compared with the information limit, of which the deterioration is increasingly pronounced when emitters are located away from the common focus of the two objective lenses (Supplementary Figs.Â 6â€“7). As for axial localization, previously, position estimations were extracted from intensity contrast calculated from PSFs at different detection channels and such calculation often involves summing over all or partial pixel values (e.g., using the central moments, Supplementary TableÂ 1) within the PSF of each channel. We found that these methods perform well within a small distance (Â±300â€‰nm) from the common focus region but result in partial information loss, where the achieved precision (uncertainty value) and bias of these methods increase rapidly with the axial position of a single emitter (Supplementary Figs.Â 6â€“7).
In order to utilize the complete information contained within the interferometric PSF pattern and minimize localization biases, we combined our coherent pupil based 4PiPSFs model with a maximum likelihood estimator with the appropriate noise model^{20} (sCMOS noise model in our case) for three dimensional localization of 4PiPSF patterns (termed as PR4Pi algorithm, Supplementary NotesÂ 4â€“5, Supplementary TableÂ 2). We demonstrated in the following sections that the PR4Pi algorithm achieve the theoretical precision limit (defined by CRLB) of our 4PiSMSN system in both axial and lateral dimensions.
As a demonstration of PR4Pi localization algorithm, first we tested the algorithm through a set of simulated 4PiPSFs. While the localization precisions using conventional intensitycontrast based methods varies drastically in different axial position, we found PR4Pi algorithm provides localization precisions consistently approaching the theoretical information limit calculated by CRLB. A detailed characterization of PR4Pi algorithm and other existing 4Pi localization algorithms are shown in Supplementary Fig.Â 6 and Supplementary TableÂ 1. For fair comparison with existing 4Pi algorithms based on Gaussian mask for lateral and central moments for axial localizations^{10}, we generate 4PiPSFs without astigmatism modification (Supplementary Figs.Â 6, 8, 10, astigmatism modification^{9,12} were used to reduce localization artifacts). We found that for lateral localization, Gaussian mask method performs worse than using incoherent PSFs (contrastbased method, Supplementary Fig.Â 6). And for axial localization, combining 0th and 3rd central moments^{10} achieves better precision at outoffocus region than using only the 0th moment (contrastbased method), but still fail to achieve the theoretical limit (CRLB) when considering equal split of photon energy through the four detected channels (five estimation parameters include: x, y, z, I, and b, Supplementary Fig.Â 6 and Supplementary NoteÂ 4). Interestingly, when considering independent intensity and background for each channel (11 estimation parameters), the CRLB for axial estimation increases, which results from an anticorrelation between estimations on intensity/background and axial positions with astigmatism modification (Supplementary NoteÂ 4 and Supplementary Fig.Â 7). Nonetheless, in both cases, PR4Pi algorithm achieves CRLB within the entire tested range (see Supplementary NoteÂ 6, Supplementary Figs.Â 11â€“15 for robustness test of PR4Pi algorithm on various conditions).
Further, we tested localizations on 40nm bead data with a photon range of 470 to 5700 per objective lens and a background range of 0â€“4 photons per pixel (see Supplementary Fig.Â 16 for theoretical precisions at various photon and background levels). Being consistent with our simulated results, PR4Pi pinpoints the 3D position of a single bead with improved precision (1â€“3.5 fold within 2Âµm axial range, quantified from localization results of 17 bead datasets) and bias than contrastbased method (Supplementary Figs.Â 17â€“18). We found the bias caused by inaccurate PSF model is rather random between different beads instead of consistent, and therefore impedes the possibility of calibrate a systematic bias for postanalysis correction (Fig.Â 2E and Supplementary Figs.Â 4â€“5). Agreeing with simulation results, PR4Pi with 11 estimation parameters (independent background and intensity estimations for each quadrant, Supplementary NoteÂ 4) worsens the axial localization precision at near focus region, while, PR4Pi with five estimation parameters (single background and single intensity estimations for all quadrants, Supplementary NoteÂ 4) worsens the axial localization bias at outoffocus region (Supplementary Figs.Â 17â€“18). Therefore, we propose to use PR4Pi with five parameters when the axial localization range is less than 1.2â€‰Âµm.
Continuous cavity phase measurement
Another key feature in 4Pi/interferometric systems is the cavity phase drift. The cavity phase describes the OPD between the two interference arms when the emitter is in focus at the common focal plane of the two objectives. The cavity phase, \(\varphi _0\), modulates with path length fluctuations and the changes of refractive indices of materials which photons travel through along the interference arms (~600â€‰mm for each arm in our system). In turn, cavity phase drift changes 4PiPSF from one pattern to another rapidly. We found the fluctuation of the room temperature (RT) lead to obvious drift in \(\varphi _0\) (Supplementary NoteÂ 7 and Supplementary Fig.Â 19). The cavity phase drift (i.e., \(\varphi _0\) drift) is different from sample drift. It is a unique challenge for interferometric systems. In comparison with commonly observed sample axial drift which would make infocused object out of focus, the cavity phase drift simply shifts the energy distribution between the four detected quadrants while an infocus emitter remains infocus. Here we examine the properties of cavity phase and propose a cavityphase driftcorrection method.
The cavity phase drift, left uncorrected, degrades the axial resolution achievable in a 4Pi systems and creates image artifacts in the final reconstruction. We found a 0.04â€‰rad drift per second will cause noticeable axialresolution deterioration in 10â€‰s (equivalent to ~18â€‰nm axial drift, Supplementary Fig.Â 20), mandating a correction per 10â€‰s to avoid resolution deterioration. This is especially challenging for the previous W4PiSMSN method as the number of emitters collected within 10â€‰s is not sufficient to correctly perform, ridgefinding, phase unwrapping and subsequent drift correction^{12}. In contrast, pupilbased method allows bypassing these problems by first, no longer relying on phase unwrapping to recover axial positions and second, automatically compensating the cavity phase drift during the regression step (Supplementary NoteÂ 4 and Supplementary Figs.Â 1, 21).
We are able to obtain reliable calibration of \(\varphi _0\) from each 10s data batch, so that a unique PR4PiPSF model for a specific segment of time was generated to account for the \(\varphi _0\) drift. This constricted the axialresolution deterioration within 5â€‰nm on average (Supplementary Fig.Â 20). In short, within a short segment of blinking dataset, total (or interference) phase \(\varphi\) and shape metric \(\sigma _s\) of each emitter were measured. Assuming linear dependences of \(\varphi\) with the axial position of the molecule as well as the shape metric \(\sigma _s\) within a 2Ï€ range, the cavity phase \(\varphi _0\) was estimated as the averaged interference phase \(\varphi\) at the common focus plane of the two objectives. For singlesection imaging, assuming a smooth change of the cavity phase during imaging, the estimated \(\varphi _0\) over time were interpolated with a smooth spline to eliminate abrupt transitions generated by estimation uncertainties and the interpolated values were incorporated into the PSF model during the regression step (Supplementary NoteÂ 4 and Supplementary Fig.Â 1).
For data acquired at multiple optical sections for large volumetric 3D imaging, the scanning of the specimen to image different optical sections in the axial direction also changes the cavity phase. Assuming the scanning step size \(d\) is a constant during the data acquisition, this additional phase can be calculated from,
where \(\lambda _0\) is the emission wavelength in air, and \(N_{{\mathrm{step}}}\) is the step number, which is indexed from 0. We would like to note that when the refractive index of the immersion media \(n_{{\mathrm{imm}}}\) matches the one of the sample media \(n_{{\mathrm{med}}}\), sample scanning in axial direction no longer causes cavity phase change. EquationÂ 8 allows us to quantitatively connect the cavity phases obtained at different optical sections and therefore, results in a continuous phase curve which improves the robustness of cavity phase calibration (Supplementary Fig.Â 1). Together, we are able to calibrate cavity phase based on short singlemolecule blinking dataset (~10â€‰s) with high consistency across multiple optical sections throughout a cell. We found that for both singlesection and multisection imaging the calibrated curves match well with the estimated \(\varphi _0\) and are robust to the estimation noise (Supplementary NoteÂ 4 and Supplementary Fig.Â 1).
We tested the reliability of our cavity phase measurement by localizing bead images at various \(z\) positions (Supplementary Fig.Â 22). An incorrect cavity phase can result in nearly 50% ghost images and when using an ideal4PiPSF model (Supplementary Fig.Â 3), in spite of cavity phase values, the smallest ghostimage percentage is nearly 20% of the entire localizations. In contrast, we found that using a correctly retrieved cavity phase together with PR4Pi algorithm, the percentage of ghost images is limited to 5â€“10% for both bead and cell imaging. For imaging in biological specimens, additional ghost reduction procedures further reduce it to ~2% (Supplementary NoteÂ 4, Supplementary Fig.Â 1 and Supplementary TableÂ 1).
Imaging of various cellular structures
Next, we applied the developed PR4Pi localization algorithm to resolve a variety of cellular structures and in vitro specimens using fluorescent labels with different photon budgets. For single opticalsection data, a total of 40â€“360 distinct PR4PiPSFs, each representing the concurrent state during a 10â€‰s imaging period, were generated for localization analysis (Supplementary Fig.Â 1). For multisection volumetric 4PiSMSN imaging, 120â€“320 distinct PR4PiPSFs (10â€‰s per PSF) were used to obtain 3â€“4 optical volumes spaced 500â€“800â€‰nm apart (Supplementary Fig.Â 1). These time dependent 4PiPSFs allow capturing the dynamic states of the interferometric system with high accuracy, and therefore pinpointing singlemolecule interferometric patterns with high precision and fidelity.
We imaged immunelabeled Nup98 protein at the top nucleus envelop in COS7 cells. From profiles of multiple nucleus pore complexes (NPC), we found that the PR4Pi improves the sharpness of resolved structure in both lateral and axial directions (Fig.Â 3). We observe this improvement is more obvious with increased axial distance from the focal plane (e.g., localizations with blue color, Fig.Â 3) especially along the lateral dimension. This improved precision as well as its dependence on the axial position of a single molecule is in agreement with our simulation results and experimental results based on beads (Fig.Â 2, Supplementary Figs.Â 4â€“7). Similarly, the superresolution reconstruction of immunelabeled TOM20 protein in COS7 cells also shows improved lateral sharpness of resolved clusters using PR4Pi (example profiles and quantifications are shown in Fig.Â 4). In comparison, contrastbased method shows increased blurriness in one of the lateral dimensions both above and below the focal plane (zâ€‰=â€‰0â€‰nm), which is caused by the lateral stretching of the incoherent astigmatism PSFs at outoffocus regions^{21} (validations of driftcorrection performance is shown in Supplementary Fig.Â 23). Importantly, PR4Pi also results in higher number of localizations at the outoffocus region because of its realistic modeling of the 4Pi emission patterns (~80% more localizations for Nup98 within the axial range of âˆ’700 to âˆ’300â€‰nm, blue color in Fig.Â 3).
To evaluate the performance of PR4Pi for low photon budget emitters such as photoswitchable/convertible proteins, we imaged purified mEos3.2 proteins on coverslips (Supplementary Fig.Â 24). The median photon count of localized mEos3.2 emission events is 358â€‰Â±â€‰140 (medianâ€‰Â±â€‰s.t.d.) per objective with a median background count of 1.6â€‰Â±â€‰0.5 photon (medianâ€‰Â±â€‰s.t.d.) per pixel. At such low photon condition, we found that even though most localized emitters are in focus, PR4Pi still shows improved localization precision in both axial and lateral dimensions (10% (nâ€‰=â€‰28) and 22% (nâ€‰=â€‰40), Supplementary Fig.Â 25, narrower profiles in lateral and axial direction, examples are shown in Supplementary Fig.Â 24). This suggests PR4Pi is robust for imaging singlemolecule probes with low photon budget.
To provide quantitative measurements on these resolved volumes, we measured the full width at half maximum (FWHM) from Gaussian fit of 30 to 70 profiles of isolated clusters from each of the reconstructed images (Supplementary Fig.Â 25). We found that PR4Pi localization of mEos3.2 on cover glass approaches the equivalent resolution predicted from the CRLB outputs of localized emission events. Assuming that each cluster represents a single mEos3.2 molecules with a diameter of ~5â€‰nm, well below the CRLB predicated resolution of PR4Pi results (xy: 27.7â€‰Â±â€‰5â€‰nm, z: 22.4â€‰Â±â€‰5.6â€‰nm, medianâ€‰Â±â€‰s.d., nâ€‰=â€‰88056 (number of localizations)), the measured FWHM values (xy: 29.7â€‰Â±â€‰3â€‰nm (nâ€‰=â€‰28), z: 24.7â€‰Â±â€‰2â€‰nm (nâ€‰=â€‰40), medianâ€‰Â±â€‰s.d.) of PR4Pi localization of mEos3.2 suggest that the achieved 3D resolution is mainly limited by the low photon count emitted by the fluorescent protein. For PR4Pi reconstructions of TOM20 and Nup98, we found the measured FWHMs are much larger than the predicated resolutions (FWHMs) from CRLB (Supplementary Fig.Â 25). This discrepancy can be caused by two potential reasons: each cluster may contain multiple single molecules and the linker size using primary and secondary antibodies is larger than the achievable resolution. As for the TOM20 reconstruction, assuming a linker size^{22} of ~17.5â€‰nm, we found that the FWHMs (xy: 17.1â€‰Â±â€‰3.3â€‰nm (nâ€‰=â€‰58), z: 16.1â€‰Â±â€‰2.5â€‰nm (nâ€‰=â€‰66), medianâ€‰Â±â€‰s.d.) of individual clusters from PR4Pi are close to the combined antibody size, suggesting that the 3D resolution of PR4Pi achieved resolving TOM20 clusters is mainly limited by the size of labeling antibodies. We further quantified the resolution of PR4Pi reconstructed images using the recently published method based on decorrelation analysis^{23} (Supplementary Fig.Â 26). The resolution is calculated by 2â€‰Ã—â€‰SMSN pixel size/maximum frequency^{23}, where SMSN pixel size is the pixel size of the SMSN reconstructed images. We found that the resolution obtained from decorrelation analysis is slightly smaller than the FWHMs from CRLB (Supplementary Fig.Â 25), which may be caused by the difference in resolution definitions between the two methods. For all specimens quantified with decorrelation analysis, axial cross sections achieved 3â€“5â€‰nm smaller resolution than that of the lateral cross sections (Supplementary Fig.Â 26).
Discussion
In this work, we developed a coherent, phaseretrieved pupil based singlemolecule localization algorithm for 4PiSMSN systems. We demonstrated PR4PiPSF allows us to accurately model the complex singlemolecule emission patterns generated from 4Pi/interferometric singlemolecule imaging systems and subsequently allows us to fully extract the information carried by the detected photons. Comparing with existing 4Pi singlemolecule localization algorithms, we demonstrated that PR4Pi allows further improvement on both precision and bias in all three dimensions. We demonstrated the performance of PR4Pi in a variety of cellular structures and in vitro specimens by resolving mitochondria networks and nuclear pore complexes in mammalian cells and individual fluorescent proteins in vitro.
The demonstrated ability to fully extract molecular position information generated by 4Pi singlemolecule systems could potentially enable highprecision localization with low photon budget (150â€“600 photons per emission per objective), a typical condition for live cell singlemolecule experiments with fluorescent proteins and high speed acquisition^{20,24}. In synergy with the rapid development of novel probes^{25,26,27}, sample preparation techniques^{28,29}, labeling methods^{30,31} and PSF engineering methods^{22,32}, we hope this approach allows efficient use of detected photons to overcome the practical barriers of achievable resolution in superresolution experiments in both fixed and living specimens.
Methods
Sample preparation
Before singlemolecule imaging, round coverslip (25mm diameter) containing immunestained COS7 cells was incubated with 200â€‰ÂµL bead dilution, containing 100â€‰nm crimson bead (customdesigned, Invitrogen) diluted to 1:10^{6} in PBS, for 10â€‰min. Then the coverslip was washed three times with PBS and placed on a custommade sample holder. Then 150â€‰ÂµL imaging buffer (10% (w/v) glucose in 50â€‰mM Tris (JT410902, VWR), 50â€‰mM NaCl (S271500, Fisher Scientific), 10â€‰mM MEA (M650025G, SigmaAldrich), 50â€‰mM BME (M314825ML, SigmaAldrich), 2â€‰mM COT (1389241â€‰G, SigmaAldrich), 2.5â€‰mM PCA (3758025GF, SigmaAldrich), and 50â€‰nM PCD (P827925UN, SigmaAldrich), pH 8.0) was added on top of the coverslip. Then a cleaned coverslip of the same size was carefully placed on top of it and the excessive buffer was removed with Kimwipes. The sample was sealed with twocomponent silicone sealant (Picodent Twinsil, Picodent, Germany).
For beads imaging, a cleaned round coverslip (25mm diameter) was incubated with 200â€‰ÂµL bead dilution, containing 40â€‰nm dark red bead (F8789, Invitrogen) diluted to 1:10^{7} in PBS, for 10â€‰min. Then the sample was rinsed with PBS, drained and placed on a custommade sample holder. Then 10â€‰ÂµL PBS was added on top of the coverslip and a second precleaned coverslip was placed on top and the sample was sealed with twocomponent silicone sealant (Picodent Twinsil, Picodent, Germany).
Immunofluorescence labeling of TOM20
COS7 cells (CRL1651, ATCC) were seeded on 25mm diameter coverslips (CSHPNo1.525, Bioscience Tools, San Diego, CA) 1â€“2 days before immunofluorescence labeling. Cells were first rinsed one time with prewarmed (at 37â€‰Â°C) phosphate buffered saline (PBS, 806552500â€‰ML, SigmaAldrich) and then fixed for 15â€‰min at RT with prewarmed (at 37â€‰Â°C) 3% paraformaldehyde (PFA, 15710, Electron Microscopy Sciences, Hatfield, PA) and 0.5% glutaraldehyde (GA, Electron Microscopy Sciences, 16019, Hatfield, PA) in PBS. Cells were then washed twice with PBS and treated for 10â€‰min in freshly prepared fluorescence quenching buffer (0.1% sodium borohydride (NaBH_{4}, 45288225â€‰G, SigmaAldrich) in PBS). After fluorescence quenching, cells were washed three times with PBS and treated for 10â€‰min with 10â€‰mM Tris (pH 7.3, JT410902, VWR). Cells were then rinsed three times with PBS and permeabilized with blocking buffer (3% bovine serum albumin (BSA, 001000162, Jackson ImmunoResearch) and 0.2% Triton X100 (X100, SigmaAldrich) in PBS) for 30â€‰min, gently rocking at RT. After blocking, cells were incubated with antiTOMM20 primary antibody (sc11415, Santa Cruz Biotechnology), diluted to 1:500 in 1% BSA and 0.2% Triton X100 in PBS, at 4â€‰Â°C for overnight. Cells were then washed three times each time for 10â€“15â€‰min with wash buffer (0.05% Triton X100 in PBS) and incubated with secondary antibody conjugated with Alexa Fluor 647 (A21245, Life Technologies, Grand Island, NY), diluted to 1:500 in 1% BSA and 0.2% Triton X100 in PBS, at RT for 4â€“5â€‰h. After incubation with secondary antibody, cells were washed three times each time for 10â€“15â€‰min with wash buffer. And then cells were postfixed with 4% PFA in PBS for 10â€‰min. After postfixation, cells were rinsed three times with PBS and stored in PBS at 4â€‰Â°C until they were imaged.
Immunofluorescence labeling of Nup98
COS7 cell were seeded on 25mm diameter coverslips 1â€“2 days before immunofluorescence labeling. Cells were first rinsed one time with prewarmed (at 37â€‰Â°C) PBS and then extracted for 60â€‰s at RT with prewarmed (at 37â€‰Â°C) 0.6% PFA and 0.1% GA and 0.25% Triton X100 in PBS. Cells were then fixed for 15â€‰min at RT with prewarmed (at 37â€‰Â°C) 3% PFA and 0.1% GA in PBS. After fixation, cells were washed twice with PBS and treated for 10â€‰min in freshly prepared fluorescence quenching buffer (0.1% NaBH_{4} in PBS). After fluorescence quenching, cells were washed three times with PBS and treated for 10â€‰min with 10â€‰mM Tris (pH 7.3). Cells were then rinsed three times with PBS and permeabilized with blocking buffer (3% BSA and 0.2% Triton X100 in PBS) for 30â€‰min, gently rocking at RT. After blocking, cells were incubated with antiNUP98 primary antibody (2598, Cell Signaling Technology), diluted to 1:200 in 1% BSA and 0.2% Triton X100 in PBS, at 4â€‰Â°C for overnight. Cells were then washed three times each time for 10â€“15â€‰min with wash buffer (0.05% Triton X100 in PBS) and incubated with secondary antibody conjugated with Alexa Fluor 647 (A21245, Life Technologies, Grand Island, NY), diluted to 1:200 in 1% BSA and 0.2% Triton X100 in PBS, at RT for 4â€“5â€‰h. After incubation with secondary antibody, cells were washed three times each time for 10â€“15â€‰min with wash buffer. And then cells were postfixed with 4% PFA in PBS for 10â€‰min. After postfixation, cells were rinsed three times with PBS and stored in PBS at 4â€‰Â°C until they were imaged.
Preparation of mEos3.2 on cover glass
First 25mm diameter coverslips were cleaned as follows: first sequentially sonicated in nanopure water, 100% ethanol (04355223, Fisher Scientific) and 1â€‰M potassium hydroxide (KOH, P250500, Fisher Scientific), each for 10â€“15â€‰min; then thoroughly rinsed six times with nanopure water, and dried on lens paper and stored in parafilm sealed 100mm petri dish. Then, cleaned coverslip was incubated with 1â€‰mg/mL biotinBSA (29130, Thermo Scientific) in nanopure water at RT for 5â€‰h. Coverslip was then washed three times with PBS and incubated with 0.1â€‰mg/mL streptavidin (858781MG, SigmaAldrich) in nanopure water at RT for 1â€‰h. Coverslip was then washed three times PBS and incubated with purified mEos3.2 (from Pollardâ€™s lab, Yale University), diluted to 1:1000 in 0.2â€‰mg/mL biotinBSA in nanopure water, at RT for 15â€‰min. After incubation with mEos3.2, coverslip was washed three times with PBS and stored in PBS at 4â€‰Â°C until it was imaged.
Data acquisition
All data were acquired on a custombuilt 4PiSMSN microscope, constructed from the previous design^{12}. In short, emitted photons were collected through two opposing highNA oil immersion objectives (Olympus UPLSAPO 100XO, 1.4 NA) and received by a sCMOS camera (OrcaFlash4.0v2, Hamamastu, Japan). One quadband dichroic/filter pair (FF01446/523/600/67725, Di01R405/488/561/63517.5Ã—24, Semrock) was installed immediately after each objective for reflection of excitation lasers and initial filtering of fluorescence emission.
The SMSN data were collected with the following procedure: (1) Apply astigmatism to each DM, amplitudes of 2 and âˆ’1.5 (unit: Î»/2Ï€) for the 5th Zernike polynomials (Wyant order)^{33} were used for the upper and lower DMs respectively. (2) Collect a stack of incoherent PSF images through the upper and lower objectives independently by imaging a bead on the bottom coverslip at z positions from âˆ’1 to 1â€‰Âµm, with a step size of 100â€‰nm, a frame rate of 1â€‰Hz and taking one frame per axial position. Those PSF data were used to generate phaseretrieved pupil functions of upper and lower emission paths. (3) Align the x, y, and z positions of both objectives by centering and focusing the beads images from those objectives, and then collect a stack of interference PSF images by imaging a bead on the bottom coverslip at z positions from âˆ’1 to 1â€‰Âµm, with a step size of 10â€‰nm, a frame rate of 1â€‰Hz and taking one frame per axial position. This PSF dataset was used for the estimation of the quartz induced phase shift \(\Delta \varphi _{{\mathrm{sp}}}\), emission wavelength \(\lambda\), intensity ratio \(I_{\mathrm{t}}\) and shape metric of infocus PSF \(\sigma _{s0}\) (Supplementary NotesÂ 1, 4). (4) Focus on a cell that took up greater than 70% of the field of view and collect 40 frames of cell images through only the lower objective at a frame rate of 10â€‰Hz. This cell data was used for quadrant alignment (Supplementary NoteÂ 4). Steps 1 to 4 were usually repeated for each sample to ensure a realistic modeling of the 4PiPSF (see phase retrieval section in Supplementary NoteÂ 4) and an accurate alignment of the four quadrants. 40â€‰nm dark red beads (F8789, Invitrogen) excited with a 642â€‰nm laser were used for Alexa Fluor 647 tagged samples and the fluorescence was filtered through an emission filter (ET700/75â€‰m, Chroma) installed on a filter wheel before the camera. And 40â€‰nm red beads (F8793, Invitrogen) excited with a 561â€‰nm laser were used for mEos3.2 sample and the fluorescence was filtered through an emission filter (FF01600/5225, Semrock) installed on a filter wheel before the camera.
For imaging of Alexa Fluor 647 tagged sample (TOM20 and Nup98): (5) Locate a region of interest and realign the objectives with a 642â€‰nm laser (2RUVFLP2000642B1R, MPB Communications Inc.) at an excitation intensity of 90â€‰Wâ€‰cm^{2}. (6) Collect blinking data with a laser intensity of 2â€“7â€‰kWâ€‰cm^{âˆ’2} under epi illumination. For imaging of mEos3.2: 5) briefly illuminate with a 405â€‰nm laser (DL405100, CrystaLaser) at 2â€‰Wâ€‰cm^{âˆ’2} to convert a subset of molecules to their red form. (7) Collect blinking data with a 561â€‰nm laser at 1â€‰kWâ€‰cm^{âˆ’2} and a 405â€‰nm laser at 2â€‰Wâ€‰cm^{âˆ’2} to maintain the density of emission events. For singleoptical section imaging, 20,000â€“180,000 frames were collected. For multioptical section imaging, the cell was scanned axially by moving the stage at a step size of 500â€“800â€‰nm from the bottom to the top of the cell for 10â€“20 cycles, 3â€“4 axial planes were imaged per cycle and 2000 frames were collected per axial plane in each cycle.
Bead data for testing PR4Pi algorithm were collected by imaging a bead on the bottom coverslip at \(z\) positions from âˆ’1 to 1â€‰Âµm, with a step size of 100â€‰nm, and taking 40 frames per axial position. The bead intensity of each dataset was adjusted by varying the excitation laser power and the exposure time.
Fisher information content of contrast method
Contrast method in this work refers to the algorithm developed in ref. ^{12} that extracts the axial positions using the Gaussianweighted 0th central moment (M0, Supplementary NoteÂ 4 and Supplementary TableÂ 1). In the lateral dimension, the Fisher information content of contrast method^{12} is equivalent to the one using astigmatism method on a dualobjective system^{14} (Supplementary Figs.Â 8â€“9). In the axial dimension, the contrast method operates on the center lobe of each PSF, which was extracted by multiplying the PSF with a 2D Gaussian at a width of one pixel. After multiplication, the remaining information for axial localization is 20â€“60%.
Statistics and reproducibility
For testing the performance of PR4Pi on bead imaging, 17 bead data were collected at various photon count and lateral positions sampled from the full field of view. For testing the performance of PR4Pi on cellular structures, at least three datasets (\(n\, \ge \,3\)) were analyzed for each type of demonstrated structures and similar performance was observed. Statistical analyses of the bead data and cellular structures (Supplementary Figs.Â 17, 18, 25) were analyzed using the function boxplot from MATLAB (MathWorks, Natick, MA), detailed number of measurements are given in the Main Text and the figure legends.
Reporting summary
Further information on research design is available in theÂ Nature Research Reporting Summary linked to this article.
Data availability
Data underlying the plots in Figs.Â 2bâ€“e, 3d,e and Fig.Â 4e, f are available as Excel files in Supplementary Data and via Figshare at https://doi.org/10.6084/m9.figshare.11987448.v1. Example test data for using PR4Pi algorithm are also available in theÂ Supplementary Software packages. Other data that support the findings of this study are available from the corresponding author upon request
Code availability
The analyzing algorithms for 4PiSMSN data are implemented as a MATLAB class, named SR4pi_demo, which provides localization methods using PR4Pi algorithm and contrast algorithm. An example script on using SR4pi_demo class is available inÂ Supplementary Software and further updates will be made available at https://github.com/HuanglabPurdue/PR4Pi.
Change history
17 February 2021
A Correction to this paper has been published: https://doi.org/10.1038/s4200302010203
References
Hell, S. & Stelzer, E. H. K. Properties of a 4Pi confocal fluorescence microscope. J. Opt. Soc. Am. A 9, 2159 (1992).
Gugel, H. et al. Cooperative 4Pi excitation and detection yields sevenfold sharper optical sections in livecell microscopy. Biophys. J. 87, 4146â€“4152 (2004).
Bewersdorf, J., Bennett, B. T. & Knight, K. L. H2AX chromatin structures and their response to DNA damage revealed by 4Pi microscopy. Proc. Natl Acad. Sci. USA. 103, 18137â€“18142 (2006).
Gustafsson, M. G. L., Agard, D. A. & Sedat, J. W. Sevenfold improvement of axial resolution in 3D widefield microscopy using two objective lenses. in ThreeDimensional Microscopy: Image 454 Acquisition and Processing II 2412, 147â€“156 (1995).
Gustafsson, M. G. L., Agard, D. A. & Sedat, J. W. I5M: 3D widefield light microscopy with better than 100 nm axial resolution. J. Microsc. 195, 10â€“16 (1999).
Shao, L. et al. I5S: Widefield light microscopy with 100nmscale resolution in three dimensions. Biophys. J. 94, 4971â€“4983 (2008).
Nagorni, M. & Hell, S. W. Coherent use of opposing lenses for axial resolution increase II Power and limitation of nonlinear image restoration. J. Opt. Soc. Am. A 18, 49 (2001).
Shtengel, G. et al. Interferometric fluorescent superresolution microscopy resolves 3D cellular ultrastructure. Proc. Natl Acad. Sci. USA. 106, 3125â€“3130 (2009).
Brown, T. A. et al. Superresolution fluorescence imaging of mitochondrial nucleoids reveals their spatial range, limits, and membrane interaction. Mol. Cell. Biol. 31, 4994â€“5010 (2011).
Aquino, D. et al. Twocolor nanoscopy of threedimensional volumes by 4Pi detection of stochastically switched fluorophores. Nat. Methods 8, 353â€“359 (2011).
v. Middendorff, C., Egner, A., Geisler, C., Hell, S. W. & SchÃ¶nle, A. Isotropic 3D nanoscopy based on single emitter switching. Opt. Express 16, 20774 (2008).
Huang, F. et al. Ultrahigh resolution 3D imaging of whole cells. Cell 166, 1028â€“1040 (2016).
Backlund, M. P., Shechtman, Y. & Walsworth, R. L. Fundamental precision bounds for threedimensional optical localization microscopy with poisson statistics. Phys. Rev. Lett. 121, 23904 (2018).
Xu, K., Babcock, H. P. & Zhuang, X. Dualobjective STORM reveals threedimensional filament organization in the actin cytoskeleton. Nat. Methods 9, 185â€“188 (2012).
Hanser, B. M., Gustafsson, M. G. L., Agard, D. A. & Sedat, J. W. Phaseretrieved pupil functions in widefield fluorescence microscopy. J. Microsc. 216, 32â€“48 (2004).
Liu, S., Kromann, E. B., Krueger, W. D., Bewersdorf, J. & Lidke, K. A. Three dimensional single molecule localization using a phase retrieved pupil function. Opt. Express 21, 29462 (2013).
Zhang, P. et al. Analyzing complex singlemolecule emission patterns with deep learning. Nat. Methods 15, 913â€“916 (2018).
Sengupta, S. K. & Kay, S. M. Fundamentals of Statistical Signal Processing: Estimation Theory. Technometrics 37 (Prentice Hall, 1995).
Ober, R. J., Ram, S. & Ward, E. S. Localization accuracy in singlemolecule microscopy. Biophys. J. 86, 1185â€“1200 (2004).
Huang, F. et al. Videorate nanoscopy using sCMOS cameraspecific singlemolecule localization algorithms. Nat. Methods 10, 653â€“8 (2013).
Von Diezmann, A., Shechtman, Y. & Moerner, W. E. Threedimensional localization of single molecules for superresolution imaging and singleparticle tracking. Chem. Rev. 117, 7244â€“7275 (2017).
Aristov, A., Lelandais, B., Rensen, E. & Zimmer, C. ZOLA3D allows flexible 3D localization microscopy over an adjustable axial range. Nat. Commun. 9, 2409 (2018).
Descloux, A., GruÃŸmayer, K. S. & Radenovic, A. Parameterfree image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918â€“924 (2019).
Marsh, R. J. et al. Artifactfree highdensity localization microscopy analysis. Nat. Methods 15, 689â€“692 (2018).
Grimm, J. B. et al. Bright photoactivatable fluorophores for singlemolecule imaging. Nat. Methods 13, 985â€“988 (2016).
Uno, S. N. et al. A spontaneously blinking fluorophore based on intramolecular spirocyclization for livecell superresolution imaging. Nat. Chem. 6, 681â€“689 (2014).
Takakura, H. et al. Long timelapse nanoscopy with spontaneously blinking membrane probes. Nat. Biotechnol. 35, 773â€“780 (2017).
Chen, F., Tillberg, P. W. & Boyden, E. S. Expansion micrsocopy. Science347, 543â€“548 (2015).
Chozinski, T. J. et al. Expansion microscopy with conventional antibodies and fluorescent proteins. Nat. Methods 13, 485â€“488 (2016).
Jungmann, R. et al. Multiplexed 3D cellular superresolution imaging with DNAPAINT and exchangePAINT. Nat. Methods 11, 313â€“318 (2014).
Pallikkuth, S. et al. Sequential superresolution imaging using DNA strand displacement. PLos ONE 13, e0203291 (2018).
Shechtman, Y., Weiss, L. E., Backer, A. S., Sahl, S. J. & Moerner, W. E. Precise threedimensional scanfree multipleparticle tracking over large axial ranges with tetrapod point spread functions. Nano Lett. 15, 4194â€“4199 (2015).
Wyant, J. C. & Creath, K. Basic wavefront aberration theory. Appl. Opt. Opt. Eng. Xl, 1â€“53 (1992).
Acknowledgements
We thank E. S. Allgeyer and G. Sirinakis from University of Cambridge and Y. Zhang from Yale University for valuable suggestions on construction of our 4PiSMSN microscope. We thank K. A. Lidke from University of New Mexico for initial implementation of interpolation based PSF generation on GPU. We thank H. M. Sun and T. D. Pollard from Yale University for providing purified mEos3.2 sample. We thank F. Xu for helpful discussions on algorithm development and suggestions to the manuscript. We thank D. A. Miller and C. Bi for suggestions on cell culture. S.L. and F.H. were supported by grants from NIH (R35 GM119785) and DARPA (D16AP00093).
Author information
Authors and Affiliations
Contributions
S.L. and F.H. developed the algorithm. S.L. performed the experiments and analyzed the data. S.L. and F.H. constructed the imaging systems. S.L. and F.H. wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisherâ€™s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the articleâ€™s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the articleâ€™s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Liu, S., Huang, F. Enhanced 4Pi singlemolecule localization microscopy with coherent pupil based localization. Commun Biol 3, 220 (2020). https://doi.org/10.1038/s4200302009082
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s4200302009082
This article is cited by

Deep learningdriven adaptive optics for singlemolecule localization microscopy
Nature Methods (2023)

Optimal precision and accuracy in 4PiSTORM using dynamic spline PSF models
Nature Methods (2022)

Implementation of a 4PiSMS superresolution microscope
Nature Protocols (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.