Accurate localization microscopy by intrinsic aberration calibration

A standard paradigm of localization microscopy involves extension from two to three dimensions by engineering information into emitter images, and approximation of errors resulting from the field dependence of optical aberrations. We invert this standard paradigm, introducing the concept of fully exploiting the latent information of intrinsic aberrations by comprehensive calibration of an ordinary microscope, enabling accurate localization of single emitters in three dimensions throughout an ultrawide and deep field. To complete the extraction of spatial information from microscale bodies ranging from imaging substrates to microsystem technologies, we introduce a synergistic concept of the rigid transformation of the positions of multiple emitters in three dimensions, improving precision, testing accuracy, and yielding measurements in six degrees of freedom. Our study illuminates the challenge of aberration effects in localization microscopy, redefines the challenge as an opportunity for accurate, precise, and complete localization, and elucidates the performance and reliability of a complex microelectromechanical system.


INTRODUCTION
Microscopic objects have structure and motion in three spatial dimensions and six degrees of freedom. Whereas classical implementations of optical microscopy resolve images in only two dimensions, recent advances enable localization of the positions of single emitters in all three dimensions 1 . Such measurements typically involve custom optics to encode aberrations that vary predictably as a function of position along the optical axis and decoding axial positions from the resulting lateral images. This engineering approach can improve some metrics of microscopes while degrading others, within theoretical limits 1,2 , and has practical limitations. Models of microscopes are imperfect and nontrivial to develop 1,3 , discouraging microscopists who focus on applications rather than instrumentation. Custom optics add complexity to the integration and alignment of microscope systems, degrade localization precision by reducing the transmission of signal photons to the imaging sensor, and degrade localization accuracy by increasing unpredictable errors from aberration effects [4][5][6][7] . For this latter reason, engineering approaches require at least estimation of localization errors, if not calibration to correct the errors. However, such analysis is uncommon in practice, resulting in a common discrepancy between precision and accuracy that can approach a factor of four orders of magnitude across a wide field 8 .
Many applications can benefit from precise and accurate localization of single emitters in three dimensions 2 . We consider two applications that bracket a wide range of experimental complexity, extending the scope of the measurement to tracking multiple emitters as indicators of the six degrees of freedom of microscale bodies. These two aspects of this complete measurement are synergistic, as multiple emitters improve localization precision and orientation precision by rigid transformations that combine information through the central limit theorem, while the rigidity and planarity of a microscale body enable tests of tracking accuracy 9-12 . Toward the simple end of the application range, the deposition of fluorescent particles on an imaging substrate -a microscale body that is ubiquitous in localization microscopy -allows calibration of aberration effects 5,6, 13,14 and correction of instrument drift [15][16][17] . The interest in performing localization microscopy within macroscopic volumes [18][19][20] introduces challenges of leveling samples and imaging them through focus. Toward the complex end of the range of applications, the coupling of microscale bodies within a microsystem controls the output of force and motion to perform work. This essential function of machines has diverse applications to optical traps 21 , colloidal motors 22 , tunable photonics 23,24 , reconfigurable metadevices 25 , materials characterization 26,27 , and even safety switches of extreme consequence 28 . The latter application is exemplary of microsystem technologies that integrate multiple parts, are nominally planar, and benefit from tracking in two dimensions and three degrees of freedom to elucidate their motion 9, 11,12,29,30 . However, measurements in three dimensions 31 and six degrees of freedom are much more informative 32 .
In the present study, we demonstrate that comprehensive calibration of the effects of intrinsic aberrations of an ordinary microscope enables precise and accurate tracking of single emitters in three dimensions throughout an ultrawide 6 and deep focal volume. This concept makes use of the latent information of intrinsic aberrations, avoids custom optics, maximizes signal photons, and preserves the intrinsic lateral extent of the point spread function 33 . We exploit intrinsic astigmatism and defocus among other aberrations to localize multiple emitters in three dimensions on an imaging substrate and on a complex microsystem 34,35 , extending the concept to measurements of motion in six degrees of freedom ( Figure 1). The development and application of our method are synergistic, as the microsystem functions both as a rotary microstage to rigorously test the accuracy of position data in three dimensions, and as a device under test with critical kinematics in six degrees of freedom to elucidate. Even for the slight aberrations that remain in a modern microscope after optical engineering to correct them, our method achieves axial precision of 25 nm and axial range of 10 µm, and lateral precision of 1 nm and lateral range of 250 µm, at a frequency of nearly 100 Hz. These performance metrics, and the ability to measure motion in six degrees of freedom with an ordinary optical microscope and simple localization analysis, distinguish our method from more specialized combinations of microscopy and interferometry (Supplementary Table 1) 31,[36][37][38][39] . Just as importantly, our method achieves accuracy that is commensurate with precision, by calibrating magnification 8, 40 and the field dependence of aberration effects that cause localization errors 4,8 . Most importantly, our study illuminates a fundamental problem -intrinsic aberrations deform imaging fields of surprisingly small extent, causing errors which can require widefield calibration and axial localization to achieve lateral accuracy that is truly better than the imaging resolution 4 . Our method provides not only a practical solution to this problem but also the opportunity to exploit intrinsic aberrations for localization microscopy in all three dimensions and six degrees of freedom. (a-c) Fluorescence micrographs showing images of a particle at positions of (a) 2 µm above, (b) near, and (c) 2 µm below best focus. The particle diameter is 1 μm and the resolution limit is 0.7 μm. Two aberration effects are apparent -symmetry variation from astigmatism and intensity variation from defocus. Dots indicate asymmetry in (a, c). Vertical positions correspond to white boxes in (d). (d) Schematic showing (red) fluorescent particles on part of a complex microsystem. We localize single particles in three dimensions and fit a rigid transformation to measure motion with six degrees of freedom -translations ∆ , ∆ , and ∆ , intrinsic rotation about the axis of rotation ⃗, nutation , and precession . White arrows indicate play due to clearances in the microsystem. (d) Lateral dimensions are nearly to scale. Vertical dimensions are not to scale.

RESULTS AND DISCUSSION
Overview of method. Whereas localization precision requires signal photons, localization accuracy requires microscope calibration. Random arrays of subresolution particles enable characterization of the point spread function and registration of localization data from different wavelengths 5,6, 13,14,41,42 . Regular arrays of subresolution apertures allow calibration of magnification and distortion 8 , and other aberration effects on localization 4,8 . Random arrays of molecular nanostructures provide reference positions to determine local magnification 43,44 .
However, no study has completely calibrated a localization microscope. We approach this closer than before by integrating information from two types of emitter arrays. We image fluorescent microparticles and subresolution apertures through focus ( where is the position of a pixel in the x direction, is the position of a pixel in the y direction, is the amplitude, ′ is the apparent position of an emitter in the x direction, 0 is the apparent position of an emitter in the y direction, 2 is the standard deviation in the x direction, 2 is the standard deviation in the y direction, ) is the correlation coefficient between the x and y directions, Axial dependence of aberration effects. We emphasize a critical result that is fundamentally problematic for super-resolution. Intrinsic aberrations affect apparent lateral positions, causing systematic errors that depend on axial position (Figure 2a) 4,8,45 . These errors approach the imaging resolution, rendering much smaller values of localization precision potentially meaningless or even misleading. To achieve lateral localization accuracy that is truly superior to the imaging resolution, both axial localization and complete calibration of the field dependences are potentially necessary.
Fortunately, intrinsic aberrations also encode axial information into emitter images, providing a latent capability for axial localization. , where 6 = / !8! 9 is the amplitude after normalization to its value in the image for which ) = ) : , with ) : set to the minimum value of |)|. Fits of bivariate Gaussian models to emitter images determine the (black data markers) parameter values, and (green lines) polynomials model the z dependence for (b, c) lateral correction, (d) determination of the axial position of best focus 3 , and (e-f) axial localization. Residual values indicate an uncertainty for each parameter. Values in the bottom panels are uncertainties of (b, c) apparent lateral position < ∆ ' % and < ∆ * % from the polynomial models, and (e-f) z position < from inversion of the polynomial models.
We develop this latent capability into a general and practical solution. For each calibration particle, empirical polynomials of high order model changes in ' and ', ∆ 0 ( ) = 0 ( ) − ′( 3 ) (2) and ∆ 0 ( of uncertainty in our study. We report uncertainties as 68 % coverage intervals, corresponding to ± one standard deviation or ± one standard error, depending on the context and accounting for a large number of replicate measurements, or we note otherwise. Lateral accuracy also depends on field curvature, lateral drift of the microscope system, and uncertainty of the independent variable for calibration of apparent lateral motion (Supplementary Note 1).
Away from 3 , intrinsic astigmatism causes asymmetry, and defocus decreases amplitude and increases width, of emitter images 46 . In a bivariate Gaussian fit, these effects manifest as we forgo any optical engineering or even careful alignment of our microscope system in a practical approach to extracting more information from the default data and analysis.
The bivariate Gaussian model approximates the image loci as ellipses with axes that, for our microscope, are at an angle of π/4 radians with respect to the x and y axes of the imaging sensor, and with eccentricity that varies with z position. Over an axial range of a few micrometers, ) has unique values with a nearly linear dependence on z position, due to intrinsic astigmatism. ,   show these representative locations. White data markers indicate the true lateral position of each particle, which we define as being at the z position of best focus, 3 , for each particle.
The selection and optimization of a widefield calibration function for ) 4 are nonobvious.
The purpose of this function is to accurately model astigmatic defocus throughout the field, on the basis of a discrete and finite sampling. However, axial dependences of ) 4    Microsystem tracking. We apply our measurement concept to track a complex microsystem ( Figure 6). In the process of localizing particles in three dimensions and using a rigid transformation to track motion in six degrees of freedom, the microsystem serves as a rotary microstage that enables rigorous tests of our method. For a particle constellation on a microscale body that moves with six degrees of freedom and rotates multiple times through the focal volume, periodic deviations from rigidity and planarity enable evaluation of the effects of the main components of uncertainty, as well as determination of the importance of field corrections for distortion and apparent lateral motion that are possible to apply or omit (Supplementary Note 3).
The microsystem consists of a rotational electrostatic actuator coupling through a ratchet mechanism to a ring gear, forming a drive motor that operates in an open loop 34,35 . The ring gear has 200 teeth that couple to a load gear with 80 teeth and a diameter of 328 µm (Figure 6a (Figure 8c), the small nutation (Figure 8b-d) causes the rigid transformations to be insensitive to this degree of freedom, so that most of the variability is within uncertainty. This is not a limitation of the method but is rather a consequence of the particular orientation of the load gear within the extrinsic reference frame of the imaging sensor. A different selection of reference frame could trade off these uncertainties against others.
These results further elucidate our method and provide insights into the kinematics of complex microsystems.  Reprinted, with permission, from Ref. 32 .
In conclusion, we introduce the concept of fully exploiting the intrinsic aberrations of an optical microscope to accurately localize single emitters in three dimensions through a deep and ultrawide field. Our approach is counterintuitive, as the tendency is to consider intrinsic aberrations as defects to reduce through optical engineering, which increases the complexity and cost of optics, or to tolerate by error analysis, which quantifies the degradation of measurement performance. We invert this perspective to reveal and apply the latent capability of an ordinary microscope for axial localization, lowering the barrier to entry of localization microscopy in three dimensions. This result is important, because we also show that lateral accuracy generally requires axial localization.
In this way, we elucidate and solve a fundamental problem of localization microscopy.
In the absence of optical engineering and in the presence of intrinsic aberrations, we develop a general and practical method for axial localization by Gaussian fitting. Several image parameters enable robust localization. For a constant emission intensity, an astigmatic defocus parameter yields useful precision and uniformity throughout a deep and ultrawide field. We elucidate the transition from local to widefield calibration, which is nonobvious due to the nonuniformity of the field of an ordinary microscope, even for field widths of less than ten wavelengths. We test several calibration functions to solve this fundamental problem, finding that Zernike polynomials model variations in astigmatic defocus and apparent position with the best accuracy, and characterize the utility of intrinsic aberrations of microscopes for our method.
In an application of our method, we introduce another concept of tracking emitters in three dimensions to measure the motion of a microscale body in six degrees of freedom. This analysis is analogous to tracking of point clouds at the macroscale, which is common and important.
Comparisons and combinations of the trajectories of single emitters and rigid transformations in three dimensions elucidate both the tracking method and the microsystem motion in six degrees of freedom. Our method is immediately applicable to the imaging of fiducial particles for the analytical leveling of imaging substrates, now in six degrees of freedom, complementing the analytical stabilization of instrument drift and characterization of aberration effects. As well, our method enables study of the motion of other microscale bodies.
We combine these methods of localization microscopy and rigid transformation and apply them to explore the motion of a complex microsystem. Our study reveals that nanoscale clearances between multiple parts in sliding contact not only degrade control of intentional motion but also cause unintentional motion in six degrees of freedom. Advancing practical measurements to study complex microsystems will help to fulfill their latent potential to perform reliably in applications that require multiradian rotations and other critical kinematics that are impossible to achieve by compliant mechanisms and impractical to measure by existing methods. Considering the importance of complex mechanical systems in the history of technology, it seems well worth the effort to understand and optimize their motion at small scales.

METHODS
Optical microscope. Our microscope has an ordinary combination of objective lens and tube lens, among other optics in their default configuration from the manufacturer. The microscope has an inverted stand, a scanning stage that translates the sample in the x and y directions, and a piezoelectric actuator that translates the objective lens in the z direction. The objective lens has air immersion, a working distance of 9.1 mm, a numerical aperture of 0.55, a nominal magnification of 50×, corrections for chromatic and flatness aberrations, and infinity correction. A light-emitting diode (LED) array and lens assembly yield nominal Kohler illumination of the sample. The emission spectrum of the LED is in Supplementary Figure 21. The apochromatic tube lens has a focal length of 165 mm and a working distance of 60 mm. The tube lens focuses images onto a complementary metal-oxide-semiconductor (CMOS) camera with 2048 pixels by 2048 pixels, each with an on-chip size of 6.5 μm by 6.5 µm. The camera operates at a sensor temperature of -10 °C by thermoelectric and water cooling. We calibrate the microscope for these parameters 8 .
The microscope records epifluorescence micrographs with a short-pass excitation filter with a transition at 628.0 nm, a dichroic mirror with a transition at 635.0 nm, and a long-pass emission filter with a transition at 634.5 nm. The microscope equilibrates for at least 1 h before use. calibrate the full field. A subset of particles forms images that differ significantly from the rest of the population of calibration particles, causing errors in the calibration data that are clearly systematic and not representative of the field dependence that we calibrate. Visual inspection confirms that these particles produce images with anomalous features, and we identify and cull such defective particles from the calibration data.

Tilt correction.
In an initial analysis that is necessary to understand and calibrate axial dependences, we measure and correct any tilt of the calibration substrate relative to the z axis by subtracting the plane of best fit from the surface of best focus 8 . This analytical leveling can replace the physical leveling of imaging substrates 8 , which is rare even as samples extending across ultrawide lateral fields are becoming common. Moreover, the common use of fluorescent particles as fiducials for drift correction, and the application of our method, present the opportunity for a complementary correction of tilt. shutter. In this mode, the camera triggers the LED illumination on when the entire sensor is exposing, rather than a rolling shutter for which pairs of pixel rows expose sequentially. A global shutter eliminates motion artifacts from a rolling shutter but introduces a delay between sequential images due to the readout time. For a global shutter, the imaging frequency is 1 / (τe + (nprp × 10 µs)), where τe is the exposure time and nprp is the number of pixel row pairs, which defines the readout extent and has a maximum value of 1024. A decrease of readout extent enables imaging frequencies extending into the kilohertz range 9 . There is a trade-off, however, as a decrease of readout extent also decreases the total number of signal photons from multiple emitters that contribute precision to a rigid transformation (Supplementary Figure 22). In our measurements,  Table 2). We fit polynomial models to data and calculate the coefficients of Zernike models using least-squares estimation with uniform weighting and the Levenberg-Marquardt algorithm.

Rigid transformations.
The iterative-closest-point algorithm 55 determines rigid transformations W X that map particle positions Y ⃗ Z between consecutive motion cycles [ and [ − 1, The axis-angle representation describes the rotations of the load gear. The direction of the eigenvector of the rotation matrix f Z = g T T T _ T T T _ T _ T _ T __ h X (10) with corresponding eigenvalue 1,

DATA AVAILABILITY
The data supporting the findings of this study are available from the corresponding author upon reasonable request.

Supplementary Note 1. Apparent lateral motion
Even for emitters on a planar sample that is normal to the optical axis, field curvature 2 causes apparent variation of axial position, resulting in errors of apparent lateral position. The field curvature for our imaging system is significant (Supplementary Figure 3), although lesser in magnitude than that resulting from an objective lens with a higher value of numerical aperture 2 . If a stationary reference is unavailable, then measurements of apparent lateral motion can include actual lateral motion due to stage drift. An available stationary reference can also exhibit apparent lateral motion with actual axial motion. Both issues can cause errors in the correction of apparent lateral motion. In this study, the calibration data is a combination of 21 data sets from sequential measurements, averaging any systematic effects of stage drift in each measurement. This challenge highlights the benefits of approaches to correct for stage drift that are insensitive to aberration effects, such as direct measurements of stage position. Depending on the requisite accuracy, drift correction by image analysis may be sufficient. Determining suitable reference objects that do not exhibit apparent lateral motion with axial motion is a subject of future work.

Supplementary
Our evaluation of accuracy in correcting apparent lateral motion takes reference values of z position from a piezoelectric actuator, omitting additional error from experimental uncertainty of z position. In an alternate analysis, we treat calibration particles as experimental particles and use the z positions from the combination of local and widefield calibration. The experimental uncertainty in z position (Figure 2f, Figure 5f) propagates through the calibration of apparent lateral motion, but increases the errors of lateral calibration (Figure 5d-e) by less than 0.1 nm for interpolation, and both increases and decreases error, depending on z position, by less than 0.1 nm for Zernike polynomials. An iterative process of determining axial position and correcting lateral position might further improve localization accuracy in all three spatial dimensions.
We could model and correct apparent lateral position as a function of ρ OE instead of z, but doing so provides little benefit and noise in ρ OE complicates the process, such as by requiring sorting of data to establish monotonicity of ρ OE before fitting a local calibration function. We can omit such particles from the calibration data, but this limits the polynomial order of the Zernike model or requires a larger axial range for the calibration data to encompass the full experimental range of ρ OE . In contrast, interpolant models require less data to calibrate the same axial range. This is because it is unnecessary to have calibration data spanning the entire lateral extent of the field for every experimental value of the astigmatic defocus parameter.

Supplementary Note 2. Rigid transformation model
Accuracy of rigid transformations and particle tracking A comparison of the residuals of the rigid transformation to the errors that we expect from the uncertainty of localizing single particles confirms the accuracy of the rigid transformation and our method of particle localization and tracking. In the x and y directions, the residuals of the rigid transformation are approximately equal to the sum of the random error from shot noise, pixelation, and background noise ( Supplementary Figure 15a-b, gray data), as well as the systematic error from apparent lateral motion (Supplementary Figure 16, red data), which includes any fitting errors from model mismatch. Even with slight deviations from rigidity in the z direction ( Supplementary  Figure 15c-e), the residuals are approximately equal to the sum in quadrature of the two main uncertainties in our method of axial localization, from local and widefield calibration. The former uncertainty includes the effects of shot noise, pixelation, and background noise. This agreement indicates the absence of additional errors that any photobleaching of the fluorescent particles would cause in axial localization by the parameter ρ OE . Errors from apparent lateral motion and widefield calibration of axial localization vary over the tracking range in z. Therefore, for this comparison (Supplementary Table 3), we consider mean values of root-mean-square error in the range of -1.5 µm < z < 2 µm, bounding the range of the experimental particles. Uncertainty of motion measurements in six degrees of freedom We evaluate the uncertainty of our motion measurements using Monte-Carlo simulations, propagating the total experimental localization uncertainties of single particles through the rigid transformation in three dimensions. This evaluation of uncertainty provides values that are specific to the particular motion in an experiment with a particular coordinate system and constellation of particles. We simulate the gear motion by applying the experimental transformations in series to the particle positions in the first micrograph of the measurement series, producing a series of particle positions that are identical to the experimental data in all aspects except for the effects of noise due to localization uncertainty. In the experimental data, this noise produces residual errors in the one-to-one mapping of particle positions from the rigid transformation (Supplementary Figure 15). We add comparable noise to the synthetic particle positions, drawing random values from normal distributions with means and variances corresponding to the means and variances of the experimental residuals for each particle. Additional variance of 8 % to 15 % is necessary to match the experimental residuals, possibly due in part to the systematic deviations from normality in the experimental data (Supplementary Figure 15). Representative data from one simulation are in Supplementary Figure 17 and Supplementary Table 4. We measure the synthetic motion in the presence of this additional noise by fitting rigid transformations, and we define measurement errors as the difference between this motion and the true motion in the absence of noise. Representative distributions of these errors are in Supplementary Figure 17. Finally, we pool the standard Any unintentional motion of the microscope system, which occurs in a common mode for all experimental particles, can produce translation relative to the imaging sensor that is consistent with rigid translation of the gear. However, such motion is not a result of the intentional operation of the microsystem, constituting another potential component of error for translations but not for rotations 4 . We estimate the error resulting from unintentional motion of the microscope system, by the apparent translations of the centroid of the particle constellation in the absence of intentional motion of the load gear. These components sum in quadrature, giving the total values in Table 1.

Distortion
In a previous study, we showed that localization measurements in the two lateral dimensions can manifest large errors due to nonuniform magnification across the lateral field, and that such distortion can vary with axial position 2 . In the present study, our objective lens has a relatively low value of numerical aperture of 0.55. We find that this lens has relatively low distortion, resulting in errors in the rigid transformation that are consistent with the errors that we expect from photon shot noise in combination with localization errors and the field dependence thereof (Supplementary Figure 15). Therefore, calibration of the mean value of image pixel size, which still deviates substantially from the nominal value, is sufficient in the present study.

Apparent lateral motion in microsystem tracking
Counterintuitively, the rigid transformation model achieves better accuracy without correcting for the apparent lateral motion that results from axial motion of each particle. This is because single motion cycles produce relatively small displacements in z of each particle, with a mean value of 78 nm (Supplementary Figure 16). For such displacements, the contribution of the apparent but erroneous lateral motion to the residuals of the rigid transformation (Supplementary Figure 15) is less than the uncertainty of correcting the apparent lateral motion (Figure 5d-e). Therefore, we omit this correction of lateral position for our specific application of microsystem tracking by a rigid transformation, although this correction remains generally necessary. involves the simplifying approximation of a uniform number of signal photons per unit area per unit time across the lateral imaging field. Across the maximum area of the field, the number of signal photons per unit time is equivalent to that of a constellation of 28 particles in our micrographs. Reducing the field size by reducing the readout extent increases the imaging frequency but decreases the total number of signal photons. This approximates how a smaller field limits the number of emitters that can contribute signal photons to a rigid transformation.