Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Multifocal imaging for precise, label-free tracking of fast biological processes in 3D

## Abstract

Many biological processes happen on a nano- to millimeter scale and within milliseconds. Established methods such as confocal microscopy are suitable for precise 3D recordings but lack the temporal or spatial resolution to resolve fast 3D processes and require labeled samples. Multifocal imaging (MFI) allows high-speed 3D imaging but is limited by the compromise between high spatial resolution and large field-of-view (FOV), and the requirement for bright fluorescent labels. Here, we provide an open-source 3D reconstruction algorithm for multi-focal images that allows using MFI for fast, precise, label-free tracking spherical and filamentous structures in a large FOV and across a high depth. We characterize fluid flow and flagellar beating of human and sea urchin sperm with a z-precision of 0.15 µm, in a volume of 240 × 260 × 21 µm, and at high speed (500 Hz). The sampling volume allowed to follow sperm trajectories while simultaneously recording their flagellar beat. Our MFI concept is cost-effective, can be easily implemented, and does not rely on object labeling, which renders it broadly applicable.

## Introduction

Life happens in three dimensions (3D). Organisms, cells, and subcellular compartments continuously undergo 3D movements. Many biological processes happen on the micrometer to millimeter scale within milliseconds: in one second, insects flap their wings 100–400 times1,2, microorganisms swim 0.5–50 body lengths3,4,5, cilia and flagella beat for up to 100 times6,7, and the cytoplasm of plants streams over a distance of 100 µm8. Although methods such as confocal and light-sheet microscopy allow precise 3D recordings, these techniques lack either the time or spatial resolution or are constrained to a small field-of-view (FOV) that is too narrow to follow fast biological processes in 3D. Rapid 3D movements can be studied by two complementary high-speed microscopy methods: digital holographic microscopy (DHM)5,9,10,11,12 and multifocal imaging (MFI)13,14,15,16.

DHM relies on the interference between two waves: a coherent reference wave and a wave resulting from the light scattered by the object. The 3D reconstruction of objects featuring both weak and strong scattering compartments, e.g., tail and head of sperm, is challenging because interference patterns of strong scattering objects conceal the patterns of weakly scattering objects. Additionally, DHM is very sensitive to noise and 3D reconstruction from DHM data requires extensive computations.

A simple alternative to DHM is MFI. MFI produces a 3D image stack of the specimen by recording different focal planes simultaneously. An MFI device, placed into the light path between microscope and camera, splits the light collected by the objective and projects multiple focal images of the sample onto distinct locations of the camera chip. However, this approach constrains the FOV13 and lowers the signal-to-noise ratio (SNR) when increasing the number of focal planes. Thus, state-of-the-art 3D tracking based on MFI requires bright fluorescent labels and is limited to either lower speeds, smaller sampled volumes, or lower precision13,15,16,17,18,19. Here, we develop dark-field-microscopy-based MFI for label-free high-speed imaging at a high SNR and across a large volume. We introduce a 3D reconstruction method for MFI that is based on inferring the z-position of an object based on its defocused image. This allowed tracking spheres and reconstructing filaments, such as flagella, at frequencies of 500 Hz, with submicrometer precision, in a FOV of up to 240 × 260 µm, and across a large depth of up to 21 µm.

## Results

### Assembling a broadly applicable MFI setup

We aim to establish an MFI system based on dark-field microscopy that can deliver high-contrast images without sample labeling. Generally, five alternative approaches to simultaneously acquire different focal planes are exploited for imaging: using (1) an optical grating13,18, (2) a deformable mirror20, (3) a dual-objective microscope21,22, (4) different optical path lengths14,16,23, and (5) lenses of different focal power15,17,19. Several approaches suffer from some disadvantages: optical gratings are wavelength-dependent and have low optical efficiency. Deformable mirrors are bulky, expensive, and highly wavelength-dependent, which prohibits their use for low-cost compact applications. Finally, dual-objective microscopes are expensive and complex—e.g., they require two tube lenses and cameras—and are limited to two focal planes only and require fluorescent labels. Therefore, we aim for an MFI system that combines approaches (4) and (5) and thus, can be accommodated in different experimental setups. We use a multifocal adapter that splits the light coming from the microscope into four light paths that are projected to different areas of the camera chip (Supplementary Fig. 1a, b). By inserting four lenses of different focal power into the individual light paths, four different focal planes of the specimen are obtained (Supplementary Fig. 1c and e). Using only four focal planes minimizes the loss of SNR due to light splitting and keeps the system low in complexity. We established an alignment procedure based on recording images of a calibration grid (see Methods) to precisely overlay the four focal plane images (Supplementary Fig. 2) and to correct for subtle magnification differences between the four focal planes (Supplementary Table 1, 2). Of note, the images of the calibration grid did not show any apparent comatic aberrations, spherical aberrations, field curvature, astigmatism, or image distortions (Supplementary Fig. 1c, 2a). Using the thin-lens approximation, we can predict the defocusing of a pattern in the setup based on the set of lenses and the magnification (Supplementary Fig. 1d and f). We assembled the multifocal adapter to study objects of submicrometer to millimeter size by changing the magnification (Supplementary Table 3). This flexibility enables studying fast-moving objects, ranging from subcellular organelles to whole animals.

### Extended depth-of-field imaging of fast-moving objects

Long-term imaging of a fast-moving object with high spatial and temporal resolution is a common, yet challenging goal in microscopy. Imaging at high resolution with objectives of high magnification and numerical aperture constrains the depth-of-field and the FOV, which increases the odds that the specimen exits the observation volume, thereby limiting the duration of image acquisition. This limitation can be overcome by combining images acquired at different depths using extended depth-of-field (EDOF) algorithms24. However, this technique is not suited for fast-moving objects if different focal planes are acquired sequentially. MFI allows employing EDOF algorithms for fast-moving objects (Fig. 1). We combine MFI (Fig. 1a) and EDOF (Fig. 1b) (see Methods) to study living specimens over a broad range of sizes: grooming Drosophila melanogaster (Supplementary Movie 1), foraging Hydra vulgaris (Supplementary Movie 2), crawling Amoeba proteus (Supplementary Movie 3), and beating human sperm (Supplementary Movie 4). In each of the four different focal planes, distinct structural features appear sharply (Fig. 1c). Using EDOF algorithms, these regions are assembled into a single sharp image that reveals fine structural features, such as intracellular compartments (Supplementary Movie 3, Fig. 1b). The extended imaging depth allows tracking objects over long time periods and with high speed, even when the sample moves along the z-axis (Supplementary Movies 14). Of note, the EDOF method also reveals a depth map (Fig. 1c) that, in combination with MFI, allows coarse 3D tracking at high speed.

### High-precision 3D tracking with MFI

MFI has been applied for 3D tracking13,15,16,17,18,19, but only with poor precision or a small sampling volume. The image of an object, i.e. the sharpness, size, and intensity of the image, depends on the object’s z-position relative to the focal plane. This dependence is used in “depth-from-defocus” methods to infer the z-position of an object based on the defocus of the object. Such methods have been applied to estimate the third dimension in physical optics25,26,27 and, for biological specimens, from 2D microscopy images28,29,30,31. To determine the z-position, a precise calibration of the relationship between the image of the object and the z-position is required. We combined the depth-from-defocus approach with MFI to achieve high-precision 3D tracking. We illustrate the power of this concept by reconstructing the 3D movement of spherical objects (latex beads) and filamentous structures (sperm flagella) using label-free imaging by dark-field microscopy and a multifocal adapter.

We next experimentally scrutinized the predicted z-precision. We recorded a new dataset of multifocal z-stacks from immobilized latex beads using a piezoelectric-driven objective, measured the radius of the bead images in the different focal planes, and inferred the z-positions of the beads using the relationship between radius and z-position (Fig. 2a) (see also Methods). Comparison of the inferred and piezo z-positions revealed a linear relationship (mean ± standard deviation of the slope: 0.98 ± 0.02, n = 5 beads). We determined the z-precision based on linear fits with a slope of unity (Fig. 2f); the predicted and experimentally determined z-precision closely matched across the entire depth (Fig. 2g).

To gauge the validity of our 3D-localization approach, we studied the stochastic Brownian motion of beads (n = 81) that were tracked for a mean duration of 3.3 s (Fig. 2h) and we determined the x, y, and z distributions of bead displacement (Fig. 2i). The distributions indicated that stochastic bead motion was isotropic and normal distributed for all spatial coordinates. From the distributions of bead displacement (Fig. 2i) we inferred the x-, y-, and z-precision of our 3D-localization approach based on the assumption that the distributions represent the positional variations resulting from diffusion superimposed on the positional variations resulting from the measurement precision (see Methods, Fig. 2j). We obtained a precision of 45, 37, and 154 nm in x, y, and z, respectively. Of note, the z-precision agrees with the z-precision predicted and measured during the calibration procedure (Fig. 2g). We conclude that MFI is suitable for precise 3D tracking in a large volume.

### High-precision 3D reconstruction of the flagellar beat with MFI

Eukaryotic cells such as sperm or green algae deploy motile, lash-like appendages protruding from the cell surface—called flagella. The 3D flagellar beat propels the cell on a 3D swimming path29,33. Because the beat frequency is high (up to 100 Hz)6,7, the precise reconstruction of flagellar beating at high temporal and spatial resolution is challenging. The fastest calibrated 3D reconstructions of human sperm achieved temporal resolutions of 100 frames per second (fps) with DHM34 and 180 fps using a piezo-driven objective35. This temporal resolution allows determining the beat frequency of human sperm that ranges, considering cell-to-cell variability, from 10 to 30 Hz36. However, this resolution is not sufficient to characterize higher harmonics of the beat6,37 or to track and characterize the first harmonic of fast-beating flagella, e.g., from Anguilla sperm (about 100 Hz)7. Detecting the second harmonic frequency (60 Hz) for a sperm beating at a fundamental frequency of 30 Hz will require, according to the Nyquist Shannon theorem, an acquisition rate of at least 120 fps. To determine the beat frequency of Anguilla sperm (about 100 Hz), an acquisition rate of at least 200 fps is required. Of note, the beat frequency of sperm is variable, rendering the determination of the flagellar beat frequency noisy. Thus, a temporal resolution of 120 fps would suffice only for regularly beating cells.

Recently, using holography, the flagellar beat of human sperm has been reconstructed with a high temporal resolution (2000 fps)38. However, the small spectral power compared to the first harmonic and the low number of beat cycles sampled during the short recording time (0.5 s) did not allow for a clear detection of the second harmonic. Moreover, the tracking precision was not evaluated and the sampled volume was very small (ca. 30 × 45 × 16 µm)38 compared to the flagellum length (50 µm), which compromises deriving the relation between flagellar beat and trajectory.

We aimed to establish MFI for reconstructing the 3D flagellar beat of human sperm at high speed (500 Hz), with high precision, and in a FOV that allows tracking both the flagellar beat and trajectory over a long time.

We used a magnification of only ×32, which decreased the optical resolution compared to previous methods, employing ×100 magnification17,38. However, this yielded a 58-times larger sampled volume of ca. 240 × 260 × 20 µm compared to a DHM-based method (30 × 45 × 16 µm)38 and an 80-times larger sampled volume compared to an MFI-based method (ca. 80 × 35 × 5.6 µm)17. We compensated for the lower optical resolution by establishing a precise reconstruction method similar to that for bead tracking.

To characterize the relationship between image and z-position of flagella, we acquired multifocal z-stacks of immotile sperm using the piezo-driven objective. Similar to the latex beads, the flagellum image displayed a characteristic widening (Fig. 3a) and an intensity decrease (Fig. 3b) when imaged out of focus.

We refined SpermQ, a software for analyzing motile cilia in 2D39, to incorporate the analysis of multifocal images (SpermQ-MF); SpermQ-MF determines flagellar z-positions based on the calibrated relationship between flagellar width, position along the flagellum, and z-distance to the respective plane. In each plane of multifocal images, SpermQ reconstructs the flagellum in 2D (x and y coordinates) with a precision of about 0.05 µm for the focused and about 0.1 µm for the defocused flagellum (Supplementary Fig. 3). To determine the z-precision, we inferred the z-position of an immotile sperm using SpermQ-MF in multifocal z-stacks recorded with the piezo-driven objective (Fig. 3c). Inferred and piezo z-positions showed a linear relationship with a slope of unity (Fig. 3c). From linear fits to the relationships of inferred and piezo z-position at different arc-length positions (n = 3 sperm) (Fig. 3c), we determined a z-precision of ~1.2 µm at the head and midpiece (arc length < 10 µm) and 0.5 µm at the principal piece (arc length > 10 µm) (Fig. 3d).

We next tested whether our approach is applicable to flagella of other species by applying the same procedure to sperm from the sea urchin Arbacia punctulata (Supplementary Fig. 4). Here, fits to determine the flagellar width at the neck region were compromised by the highly flexible neck and the high-intensity contrast between head and flagellum in sea urchin sperm. Thus, the z-precision at the flagellar neck region (arc length < 10 µm) was almost twofold less for sea urchin sperm (~2.1 µm, Supplementary Fig. 4d) than for human sperm (~1.2 µm, Fig. 3d). In the more distal part of the flagellum (arc lengths > 10 µm), sea urchin sperm were reconstructed with a better z-precision (~0.4 µm, Supplementary Fig. 4d) compared to human sperm (~0.5 µm, Fig. 3d).

In conclusion, our approach provides high temporal resolution (500 Hz), a large FOV of ca. 240 × 260 × 20 µm, and allows flagellar 3D reconstruction with a z-precision of down to ~0.4 µm, whereas for other methods the z-precision has not been reported17,38.

### Relating the flagellar beat pattern to the swimming trajectory of sperm

We next studied and compared the flagellar beat of free-swimming sperm from human and sea urchin using the calibrated MFI setup and SpermQ-MF. As proof of principle of the applicability of MFI for 3D tracking of sperm, we recorded the 3D flagellar beat of n = 7 human and n = 10 sea urchin sperm swimming in a 150-µm deep chamber. For human sperm, the tracking duration was limited by the FOV to about 2–3 s, i.e., the time it takes a sperm cell crossing the FOV (Fig. 4a, Supplementary Movie 5). Due to the precise localization of the flagellum, we resolved the 3D flagellar beat in detail (Fig. 4b, c). The kymographs revealed a flagellar beat wave that propagates in 3D along the flagellum (Fig. 4b).

Making use of the large FOV of our MFI system, we recorded the flagellar beat and swimming path over a time span that allows relating beat pattern and trajectory (Fig. 5, Supplementary Fig. 58). Trajectories of human sperm varied between cells and over time for single cells (Fig. 5a, Supplementary Fig. 6). For example, one sperm cell changed from a slightly curved to a straight trajectory (Supplementary Fig. 6a); another sperm cell changed swimming directions multiple times (Supplementary Fig. 6e). By contrast, sea urchin sperm swam on a regular, circular swimming path based on a curved beat envelope (Fig. 5b, Supplementary Fig. 78). Sea urchin sperm can spontaneously depart from circular swimming to produce a stereotypical motor response known as “turn-and-run”40. Making use of the large FOV of our method, we were able to record a sperm during such a motor response and correlate the swimming trajectory and the flagellar beat (Supplementary Fig. 9). The flagellar beat transitioned from an asymmetric beat during the “turn” to a symmetric beat during the “run” (Supplementary Fig. 9).

We further quantified the 3D trajectory and the 3D flagellar beat (Fig. 5c-l). Human sperm swam at lower speed than sea urchin sperm (Fig. 5c-d). Additionally, the flagellar shape of human sperm was characterized by a large out-of-plane component (Fig. 5a) while the flagellar shape of sea urchin sperm was rather planar (Fig. 5b). We quantified the out-of-plane component using the nonplanarity ratio29 (see also Methods). This ratio ranges from zero to one. A ratio of zero indicates a perfectly planar beat, whereas a ratio of unity indicates a flagellar beat that has no preferential plane. The ratios largely fluctuated over time, especially for human sperm (Fig. 5e): The mean standard deviation over time of the nonplanarity ratio was 0.17 ± 0.02 (mean ± standard deviation of n = 7 sperm) in human sperm and 0.08 ± 0.03 (mean ± standard deviation of n = 10 sperm) in sea urchin sperm. To exclude any bias by large fluctuations, we determined the median of the ratio over time as a reference for the nonplanarity per sperm cell (Fig. 5e, f). The median nonplanarity ratio was 0.26 ± 0.04 (mean ± standard deviation of n = 7 sperm) in human sperm and 0.17 ± 0.04 (mean ± standard deviation of n = 10 sperm) in sea urchin sperm (Fig. 5f), confirming a more planar beat of sea urchin compared to human sperm. In addition, the beat plane in human sperm rolled around the longitudinal sperm axis (Supplementary Movie 6), whereas no rolling was detected for sea urchin sperm (Fig. 5g, h). Our measurements of the nonplanarity ratio (0.26) and the rolling velocity (3.4 Hz, representing 21.3 rad s−1) agree with a previous analysis using phase-contrast microscopy29.

Sperm steering has been related to asymmetric flagellar beat patterns that can be described by superposition of three components: (1) a mean curvature component C0, (2) the curvature amplitude C1 of the bending wave oscillating with the fundamental beat frequency, and (3) the curvature amplitude of the second harmonic C26,37. To visualize higher harmonic components of the flagellar beat, we determined the frequency spectrum of the signed flagellar 3D curvatures (see Methods) for human and sea urchin sperm (Fig. 5i-j). The principal beat frequency was 22.6 ± 6.4 Hz for human and 50.2 ± 3.6 Hz (mean ± standard deviation) for sea urchin sperm (Fig. 5j), in line with other reports12,36,41. We observed higher harmonics in the spectrum of human and sea urchin sperm (Fig. 5i). The curvature components C0 and C1 were much more prominent in sea urchin compared to human sperm, whereas the second harmonic component C2 was only marginally different between sea urchin and human (Supplementary Fig. 10). We determined a second harmonic component of C2 = 0.006 ± 0.002 µm−1 (mean ± standard deviation) for human sperm. Notably, the widths of the frequency peaks were narrower for sea urchin than for human sperm (Fig. 5i, k), indicating that the beat frequency of human sperm is less constant than the beat frequency of sea urchin sperm. To confirm this observation, we analyzed the beat frequency of human and sea urchin sperm at different recording times, which revealed large variations in the beat frequency of human sperm over time but only small variations for sea urchin sperm (Fig. 5l). Such variations have not been reported in previous analysis, where a constant beat pattern with sharp frequency peaks was measured6,37. However, these measurements relied on tethering sperm at their head to establish sufficiently long observations for resolving the power spectrum. When tethered, human sperm are prevented from rolling—they are forced into a stable condition and their beat plane is aligned with the glass surface by hydrodynamic interactions42. In contrast, our technique, due to precise tracking in a large FOV, allows recording freely swimming human sperm under unconstraint conditions and over a sufficiently long time to detect variations in the flagellar beat.

### 3D particle imaging velocimetry to reconstruct fluid flow around a human sperm cell

The behavior of ciliated cells and the underlying physics is of great interest for cell biology and biotechnological applications. We are only beginning to understand how sperm manage to navigate in a highly complex environment like the female genital tract43 or how motile cilia synchronize their beat to produce fluid flow44. Key to understanding these phenomena are the hydrodynamic interactions of motile cilia with each other and their aqueous environment4,44. The fluid flow resulting from flagellar beating has been studied experimentally and theoretically. Experimental studies resolved the 2D flow pattern for different microorganisms such as Giardia lamblia45, Chlamydomonas reinhardtii, and Volvox46. The flow profile around human sperm has been reported in 2D47. The 3D flow has been inferred from the flagellar beat for human sperm48 or from numeric simulations of the flagellar beat near walls for sea urchin sperm42. However, the predicted 3D flow profiles have not been tested experimentally.

To visualize the 3D flow around human sperm with our MFI technique, we used latex beads whose relationship between radius and z-position had been previously calibrated. To follow the direction of the bead movement relative to the sperm flagellum, we tethered the sperm head to the glass surface.

Bead tracking revealed long-range hydrodynamic interactions between sperm and beads (Fig. 6a, b, Supplementary Movie 7). At large distances from the sperm cell, beads displayed a random Brownian motion (Fig. 6c). Near the cell, beads displayed a Brownian motion with a drift towards the sperm head (Fig. 6d) and away from the flagellar tip (Fig. 6e). Near the flagellum, beads described a fast 3D spiral-like movement resulting from the 3D flagellar beat (Fig. 6d-e).

We computed the 3D flow field around a human sperm based on a time-lapse recording of 40 s at 500 Hz, revealing 2,920,731 bead positions with a predicted median z-precision of 0.34 µm (prediction based on the approach shown in Fig. 2d; median used to exclude outliers in this calculation). The flow field shows that the fluid moves from a large distance towards the sperm cell and converges at the flagellum, where the flow speed steeply increases (Fig. 6f, g). This agrees with the flow profiles predicted by theory4,42,48. We observed a flow speed of up to 125 µm s−1 directly at the flagellum (Fig. 6g). Ishimoto et al. predict for the flow speed close to the flagellum of human sperm a value of 0.1 L/T, where L corresponds to the length of the flagellum and T to the duration of one beat cycle48. For a beat frequency of 25 Hz and a flagellar length of 50 µm, this corresponds to a flow speed of 125 µm s−1, in perfect agreement with our experimental results. For human sperm, fluid flow at the flagellum has been only measured in 2D, revealing a flow speed of up to 45 µm s−1 in low-viscosity medium47. The flow speed derived from projecting our data into 2D by averaging the speeds across z is in a similar range (Fig. 6f). Taken together, our 3D analysis confirms predictions based on mathematical modeling of the 3D fluid flow around sperm and highlights the importance of measuring flow in 3D to achieve accurate estimates of the flow speed. Finally, we unravel spiral flow patterns close to the sperm cell.

## Discussion

State-of-the-art MFI systems that employ four focal planes have been limited to a depth <6 µm and use high-magnification objectives (≥×60) to resolve the specimen, thereby constraining the FOV (≤80 × 35 µm)15,17. Although such a small FOV allows reconstructing the beat period of a freely swimming human sperm (flagellum length: 50 µm), it is impractical to relate the beat pattern to the swimming trajectory because cells quickly leave the FOV.

Here, we present a depth-from-defocus-based 3D reconstruction algorithm that allows to computationally achieve a z-precision in four-plane multifocal images as high as 0.15 µm. Thereby, we are able to use a lower magnification (×32) and a wider plane spacing (planes spanned 8.8 µm). Using this algorithm, we precisely reconstruct beads and flagella across a 21-µm depth, a FOV of up to 240 × 260 µm, and at a high volume-sampling rate (500 Hz). This represents a 3.5-fold increase of depth, a 22-fold increase of FOV, and a 2.5-fold increase of sampling speed compared to other MFI systems15,17. The sufficiently large FOV allows relating swimming trajectories and beat pattern of sperm without compromising precision and speed.

For sperm, depth-from-defocus methods have been applied to roughly estimate the z-position in single-plane microscopy, without considering the z-precision29,39. We demonstrate that the z-precision inferred from a single plane is not equal for different z-positions and poor when the object is located near focus. We have surmounted these shortcomings by using multiple focal planes to derive the z-position and show that this allows unambiguous determination of the z-position, improvement of the z-precision, equalization of the z-precision over a large depth, and a significant increase in depth compared to using only one focal plane. Of note, these improvements depend on a well-designed MFI setup. To establish a good tracking precision over a large depth range, the interplane distance needs to be chosen based on the relationship between object width and z-position for an individual plane image.

Our MFI system represents a cost-effective 3D imaging device that does not rely on any special equipment and uses a commercially available adapter that is compatible with most microscopes. The system is flexible: it allows customizing the focal distance between image planes using different adapter lenses and it can be adapted to object sizes ranging from nano- to millimeter using different objectives. Thus, our MFI system is affordable and accessible to a large scientific community. To allow broad accessibility and applicability of our computational methods, we have implemented the whole-image analysis pipelines presented in this study as plugins for the free, open-source software ImageJ (https://github.com/hansenjn/MultifocalImaging-AnalysisToolbox). Our software features a user interface and does not require any programming knowledge. Thus, we envision a broad applicability of these methods.

Alternative methods to derive depth information from a single-plane image use chromatic and spherical aberrations or astigmatism49,50,51,52,53,54,55. Like our method, these methods cannot resolve the z-position of objects that are stacked above each other in z or that are very close. Of note, to avoid any bias from this limitation, we implemented an algorithm that excludes objects with low-quality fits (see Methods). Although deriving the depth via aberrations or astigmatism allows deciding whether an object lies before or behind the focal plane from one image only, these methods suffer from other disadvantages. First, using chromatic aberrations for depth estimates, multiple colors have to be acquired at the same time, and the object shape in different colors is compared to determine the z-position. In contrast to depth-from-defocus methods, this is not applicable to fluorescent samples. Moreover, it will lower the SNR and image contrast similarly to splitting light into multiple paths in our MFI setup. Second, astigmatic imaging to derive depth information has been only established for single-molecule imaging and spherical particles52,54,55,56. 3D tracking of flagella by astigmatic imaging or spherical aberrations is challenging, because the flagellar orientation relative to the focal plane and, for spherical aberrations also the flagellar x, y-position, affect the flagellar image acquired by these techniques. Thus, extensive calibration of additional variables that take these effects into account would be required.

Previous studies of sperm’s flagellar 3D beat used light microscopy with a piezo-driven objective or digital holographic microscopy (DHM)34,35,38,57,58,59. The piezo-driven method allows recording up to 180 volumes s−1 35,59, which represents a third of the sampling speed achieved by our MFI method. Of note, the piezo-driven method sequentially acquires multiple focal planes and, thus, requires high-speed high-sensitivity cameras that can record at high frame rates. For example, previous reconstructions of the flagellar beat using the piezo-driven method required acquisition of 8000 images s−1 35,59. Cameras achieving such acquisition rates are very expensive, restricting the availability of this technique to few labs. DHM can provide the same temporal resolution as MFI and allows high-precision 3D particle imaging velocimetry (PIV) achieving a precision down to 10 nm for simple spherical objects60,61,62. However, such precision is not achievable for biological samples: for cells, precisions of about 0.5 µm have been reported63,64. There are more factors that limit DHM. First, DHM is challenging when imaging strong- and weak-scattering objects at the same time; therefore, by contrast to MFI, DHM would not be capable of reconstructing a “small” sperm cell close to a “large” egg. Similarly, tracking the fluid flow around sperm may be challenging with DHM. Second, DHM is not applicable to fluorescent objects, whereas, using different labels, MFI could reveal different objects or processes simultaneously13,15,16,17,18,19. For example, applying our 3D reconstruction method to multi-color MFI allows to simultaneously record the 3D flagellar beat and fluorescent indicators for subcellular signaling. Third, DHM requires complex and extensive computational methods, whereas MFI does not. A state-of-the-art DHM-based reconstruction method for flagella featured a high temporal resolution (2000 volumes s−1) but a small sampled volume of ca. 30 × 45 × 16 µm; the precision of the method was not reported38. Our technique for 3D reconstruction of human sperm combines a large sampled volume (ca. 240 × 260 × 20 µm), precision (≥0.4 µm), and high speed (500 Hz), which is superior to currently available alternative methods. Therefore, we could record higher harmonic frequencies in the flagellar beat of swimming sperm and follow the sperm trajectory upon a change in the flagellar beat pattern.

Present techniques allow studying the fluid flow surrounding flagellated microorganisms in 2D45,46,48. The 3D profile has previously been estimated by numerical simulations of the 3D flagellar beat of sperm42,48. Applying our technique for 3D particle imaging velocimetry in a proof-of-principle experiment allowed to record in 3D the fluid flow around a motile cilium, i.e. the human sperm flagellum. We reveal 3D spiral-shaped flow patterns around sperm. Moreover, we show that fluid flow transports molecules from afar to the flagellum. The flagellum acts as a propeller and a rudder, whose function is regulated by extracellular cues that act on intracellular signaling pathways4. Thus, it will be important to further investigate how the spiral-shaped flow pattern and the transport of fluid to the flagellum impact the sensing of chemical or mechanical cues by sperm during chemotactic or rheotactic navigation, respectively. Combining MFI and fluorescence microscopy13,15,16,17,21,22,23,65 allows simultaneously 3D tracking of the sperm flagellum and performing 3D particle imaging velocimetry using different fluorescent labels. This will reveal how a flow profile is generated by the 3D beat and how fluid flow varies in time as the beat progresses66.

Our method does not rely on a particular microscope configuration, which ensures broad applicability. We envisage three additional applications.

First, although we focused here on label-free samples using dark-field microscopy, our method is directly adaptable to fluorescently labeled samples. Dark-field microscopy with a standard condenser, as applied here, limits the application to moderate NA objectives and consequentially also limits the optical resolution. However, using oil-immersion dark-field condensers or fluorescence-based MFI, high NA objectives could be applied, improving optical resolution.

Second, our MFI-based filament reconstruction method could be applied to larger objects, for instance, to study rodent whisking during active vibrissal sensing67.

Third, we show that the concept of combining depth-from-defocus with multiple focal planes significantly improves depth estimates. Depth estimates are not only relevant for high-speed microscopy but also for research in 3D object detection by computer vision.

## Methods

### Species

Amoeba proteus and Hydra vulgaris were purchased freshly before the experiments (Lebendkulturen Helbig, Prien am Chiemsee, Germany). Drosophila melanogaster were adults (>2 days old). Human sperm cells were purified by a “swim-up” procedure68 using human tubular fluid (HTF) (in mM: 97.8 NaCl, 4.69 KCl, 0.2 MgSO4, 0.37 KH2PO4, 2.04 CaCl2, 0.33 Na-pyruvate, 21.4 lactic acid, 2.78 glucose, 21 HEPES, and 25 NaHCO3 adjusted to pH 7.3–7.4 with NaOH). Liquefied semen (0.5–1 ml) was slowly pipetted under 4 ml HTF using a Pasteur pipette in a 50 ml falcon tube. Motile sperm were allowed to swim out of the semen into the HTF for 60–90 min at 37 °C. The HTF buffer containing motile sperm was transferred to a 15 ml falcon tube and the tube was filled with HTF. This sperm suspension was washed two times by centrifugation (700 × g, 20 min at 22 °C). Cells were counted and the density of the suspension of swim-up purified sperm was adjusted to 1 × 107 per ml. Sea urchin sperm from Arbacia punctulata were obtained from and disposed by the Marine Resource Center at the Marine Biological Laboratory in Woods Hole. Two hundred microliters of 0.5 M KCl were injected into the body cavity of the animals. Sperm were collected using a Pasteur pipette and stored on ice. Sperm suspensions were prepared in artificial seawater (ASW) at pH 7.8, which contained (in mM): 423 NaCl, 9 KCl, 9.27 CaCl2, 22.94 MgCl2, 25.5 MgSO4, 0.1 EDTA, and 10 HEPES.

This study followed all relevant ethical regulations for animal testing and research. Human semen samples were donated by normozoospermic men (age 20–45) with their prior written consent and the approval of the ethics committee of the University of Bonn (042/17).

### Imaging

All images except those of D. melanogaster were acquired using an inverted microscope (IX71; Olympus, Japan) equipped with a dark-field condenser and a piezo (P-725.xDD PIFOC; Physik Instrumente, Germany) to adjust the axial position of the objective. For MFI, the multi-channel imaging device (QV-2, Photometrics, USA) was installed in front of a high-speed camera (PCO Dimax, Germany). Different objectives were used: ×20 (UPLFLN, NA 0.5; Olympus, Japan), ×10 (UPlanSapo, NA 0.4; Olympus, Japan), ×4 (UPlanFLN, NA 0.13; Olympus, Japan). The total magnification could be increased with an additional ×1.6 magnification changer. Sperm, Amoeba proteus, and Hydra vulgaris were investigated in a custom-made observation chamber of 150 µm depth39. Images of D. melanogaster were recorded with a stereomicroscope (SZX12; Olympus, Japan), equipped with a DF PLAPO 1X PF lens (NA 0.11; Olympus, Japan).

### Characteristics of the multifocal adapter

The multi-channel imaging device QV-2 (Photometrics, USA) was used to split the light coming from the microscope into four light paths (Supplementary Fig. 1a). The light within each optical path was projected to a different location on the camera chip. The focal length of each optical path was varied by an additional lens located at the position designed for holding filters in the QV-2. For measurements, a set of lenses with the following focal lengths was used (in mm): f1 = ∞ (no lens), f2 = 1000, f3 = 750, and f4 = 500 (Supplementary Fig. 1a).

### Intensity normalization across planes

Local differences of image intensity in the four focal planes (Supplementary Fig. 12a) were measured by recording an image without a specimen and generating an intensity heatmap (Supplementary Fig. 12b). For normalization, pixel intensity was divided by the respective fractional intensity value in the heatmap (Supplementary Fig. 12c).

### Alignment of plane images

The four focal planes were aligned at each time step using the ImageJ plugin Multi-Stack Reg.69. An alignment matrix was determined from the first timepoint and applied to all subsequent frames. Because the alignment algorithm uses the first plane as a template and corrects the other planes based on the first plane, this technique also allowed to correct for minute magnification differences appearing in planes 2–4 while not in plane 1 (Supplementary Table 1, Supplementary Table 2).

For the images of freely swimming sperm, an image of a calibration grid was recorded (Supplementary Fig. 2a) and processed in ImageJ by image segmentation (threshold algorithm Li70, Supplementary Fig. 2b), skeletonization71 (Supplementary Fig. 2c), and alignment (Multi-Stack Reg69; Supplementary Fig. 2d). The output alignment matrix was used for plane alignment.

### Determination of the interplane distances

To determine the interplane distances, a series of multifocal images of a calibration grid at different positions set by a piezo was acquired. For each of the four planes, the piezo position at which a reference grid was in focus was determined. The focus position was defined as the position at which the grid image featured the highest standard deviation of pixel intensity72. From the differences in focus position between neighboring planes, the interplane distances were determined.

For low magnification (stereomicroscope, ×4), where the piezo range was insufficient for spanning all planes, the interplane distance was estimated manually by measuring the required focus displacement between two adjacent planes for sharp imaging of the specimen in each plane.

### Simulating images produced by the multifocal adapter

Based on the interplane distances and the focal lengths of the lenses in the QV-2, we performed numerical calculations of images produced by the setup. The different imaging planes produced by the MFI setup were obtained with ray-optics calculations based on the thin-lens equation.

Defocusing was simulated by convolving the point spread function (PSF) with the sharpest image that we acquired of the calibration grid. We assumed a Gaussian PSF with transverse intensity distribution:

$${I}_{G}(r,z)=\frac{{w}_{0}}{w(z)}{e}^{-\frac{{r}^{2}}{{w(z)}^{2}}}.$$
(1)

In Eq. (1), $$w\left(z\right)={w}_{0}\sqrt{1+{\left(\frac{z}{{z}_{R}}\right)}^{2}}$$ is the evolution of the Gaussian beam width with the propagation distance $$z$$, $${z}_{R}=\frac{\pi {w}_{0}^{2}}{\lambda }$$ is the Rayleigh range, $${w}_{0}$$ is the Gaussian beam width at the focal plane, and $$\lambda$$ is the wavelength. $${w}_{0}$$ is given by the optical resolution provided by the experimental setup (Abbe limit, about 2.1 µm). The Abbe limit was calculated as,

$${r}_{z}=\frac{n}{{NA}}\left[\frac{{{{\rm{\lambda }}}}}{{NA}}+\frac{{e}_{{{{\rm{pixel}}}}}}{M}\right],$$
(2)

where $$\lambda$$ is the wavelength, n is the refractive index, NA is the numerical aperture of the objective, epixel is the lateral pixel size, and M is the magnification of the objective. Based on the resolution, the interplane distances, and the Rayleigh range $${z}_{R}$$, we calculated the Gaussian beam width $$w\left(z\right)$$ and PSF at every $$z$$ and for each plane.

### Extended depth-of-field (EDOF)

Multifocal images were processed into extended depth-of-field (EDOF) images using the complex wavelet mode24 of the ImageJ plugin “extended depth-of-field by “Biomedical Imaging Group”, EPFL, Lausanne. We created a customized version of this ImageJ plugin to enable automated processing of time series of image stacks.

### Recording Brownian motion

Carboxylate-modified latex beads of 0.5 µm diameter (C37481, Lot # 1841924, Thermo Fisher Scientific Inc., USA) in an aqueous solution were inserted into the custom-made observation chamber. Images were recorded with the ×20 objective (NA 0.5) and the additional ×1.6 magnification changer at 500 fps. A 530 nm LED was used as a light source. Bead positions were analyzed as described in the next chapter.

### Tracking beads in multifocal images

For each timepoint, a maximum-intensity-projection across the four planes was generated. To remove nonmoving beads and background structures from images of beads around sperm, a time-average projection was subtracted from all timepoints. To determine bead positions in x, y, and time, the maximum-intensity-projection time series was subjected to the FIJI plugin “TrackMate”73 (settings: LoG detector, estimated blob diameter of 10 pixel, threshold of 50, subpixel localization).

Each bead’s z-position was analyzed using a java-based custom-written ImageJ plugin using the “TrackMate”-derived xy and time coordinates as a template.

For each of the focal planes, all pixels within a radius of 10 px (about 3.4 µm) were subjected to an unconstraint 2D radial fit to retrieve the radius. The fit was developed as follows.

For a circle of radius R centered at rc = (xc, yc), the circle equation $${(x-{x}_{c})}^{2}+{(y-{y}_{c})}^{2}={R}^{2}.$$ can be rewritten as $$2x \cdot {x}_{c}+2y \cdot {y}_{c}+{b}={x}^{2}+{y}^{2}$$, where $$b={R}^{2}-{x}_{c}^{2}-{y}_{c}^{2}$$. With the coordinates of the pixel given, we can fit the transformed circle equation linearly to calculate the parameters xc, yc, and b. The linear fit is constructed into the matrix form

$${{{\bf{Ac}}}}={{{\bf{B}}}},$$
(3)

where, $${{{\bf{A}}}}=\left(\begin{array}{ccc}{x}_{1} & {y}_{1} & 1\\ \vdots & \vdots & \vdots \\ {x}_{n} & {y}_{n} & 1\end{array}\right){{{\bf{W}}}}$$, $${{{\bf{B}}}}=\left(\begin{array}{ccc}{x}_{1}^{2}+{y}_{1}^{2}\\ \vdots \\ {x}_{n}^{2}+{y}_{n}^{2}\end{array}\right){{{\bf{W}}}}$$, and $${{{\bf{W}}}}=\left(\begin{array}{ccc}{I}^{2}\left({x}_{1},{y}_{1}\right), & \, & \,\\ \, & \ddots & \,\\ \, & \, & {I}^{2}\left({x}_{n},{y}_{n}\right)\end{array}\right)$$ is the weight matrix defined by the square of the pixel intensity. $${{{\bf{c}}}}=\left(\begin{array}{c}{x}_{{{{\rm{c}}}}}\\ {y}_{{{{\rm{c}}}}}\\ {b}\end{array}\right)$$ is the approximate solution of this overdetermined equation, which can be calculated with the least-squares principle. Finally, the circle radius R can be determined using the equation:

$${{{\rm{R}}}}=\sqrt{{x}_{c}^{2}+{y}_{c}^{2}+b}.$$
(4)

If the quality of the fit r2 > 0.8, the radius was used to infer the z-position. If the fit to the radial profile did not match the criterion in more than two planes, the bead was excluded from analysis, and no z-position was determined. The z-position was determined for each remaining bead as follows. In each focal plane, a difference function of the determined bead radius and the plane’s calibrated relationship between bead radius and z-position was determined. Each difference function shows two minima, which represent the two most probable z-positions (one located above and another below the focal plane). Finally, the position of the bead was unequivocally determined from the multiple measures of the z-positions across different planes: the combination of z-positions from the different planes with the lowest standard deviation was selected. For each z-position, the precision was predicted based on the calibrated relationship of z-position and fit width. The z-position with lowest precision value was selected as the final bead’s z-position.

Images of immobilized beads at different heights set by a piezo-driven objective were used for calibration of z-positions. Radius profiles across z of seven different beads were aligned by minimizing the least-mean-square between all the planes. Finally, aligned profiles were averaged and smoothed. This resulted in a radius function R(z). To estimate the precision error σz resulting from inferring the z-position of beads using this function, we used error propagation of the inverse function: $${\sigma }_{z}=\frac{{dz}(R)}{{dR}}{\sigma }_{R}$$, where σR is the standard deviation of the calibration for different beads.

Individual bead coordinates on each frame were connected to form tracks using custom-made software written in MATLAB. The software searched for the nearest particle across frames within a distance of less than 1.5 µm.

### Determining the x-,y-, and z-precision based on Brownian motion of particles

For characterizing Brownian motion, the normalized histogram of bead displacement along each dimension d (d = x, y, or z) of space was fitted to the characteristic probability density function of diffusive particles74:

$$P\left(d,t\right)=\frac{1}{\sqrt{4\pi {Dt}}}{e}^{-{d}^{2}/4{Dt}}$$
(5)

The measured distribution of bead displacement Δs results from positional variations due to diffusion during each time interval Δt superimposed to positional variations that result from measurement errors due to the precision of the multifocal method. Assuming that measurement errors are independent of diffusion, the distribution of bead displacement measured PMs, t) can be described by the convolution of the two probabilistic distributions characterizing the measurement errors due to precision PP(Δs) and diffusion PDs, Δt):

$${P}_{{{{\rm{M}}}}}\left(\triangle s,\triangle t\right)={P}_{{{{\rm{p}}}}}\left(\triangle s\right)\ast {P}_{{{{\rm{D}}}}}\left(\triangle s,\triangle t\right),$$
(6)

where * denotes the convolution operation. Assuming isotropic diffusion and normally distributed measurement errors, it can be readily found (use the convolution theorem for this) that the variance of the resulting distribution σM2 is characterized by the sum of the variances:

$${\sigma }_{{{{\rm{M}}}}}^{2}={\sigma }_{{{{\rm{p}}}}}^{2}+{\sigma }_{{{{\rm{D}}}}}^{2},$$
(7)

where the standard deviation σp characterizes the precision of the method, and σD2 the variance of the bead displacement due to diffusion. For each dimension of space, the corresponding diffusive variance can be calculated provided that the diffusion coefficient D is known:

$${\sigma }_{{{{\rm{D}}}}}^{2}=2{{{\rm{D}}}}\triangle {{{\rm{t}}}}$$
(8)

For a spherical particle at high Reynolds numbers, the diffusion coefficient is well described by the Stokes-Einstein relation:

$${{{\rm{D}}}}=\frac{{{{{\rm{K}}}}}_{{{{\rm{B}}}}}{{{\rm{T}}}}}{6{{{\rm{\pi }}}}{{{\rm{\eta }}}}{{{\rm{R}}}}},$$
(9)

where KB is the Boltzmann constant, T the temperature (295 K), R the particle radius (250 nm), and η(T) the dynamic viscosity of the embedding fluid (ca 0.95 × 10−3 kgm−1s−1 for water at 295 K). Combining Eqs. (7)–(9), we find the precision of the method to be:

$${\sigma }_{P}=\sqrt{{\sigma }_{M}^{2}-\frac{{K}_{B}T\triangle t}{3\pi \eta R}}$$
(10)

Using Eq. (10) with a frame period Δt of 2 ms, we obtain a z-precision of 150 nm (n = 82 beads recorded for a mean duration of 3.3 s), which is in agreement with the precision found during the calibration procedure (Fig. 2g). Additionally, the precision in the xy-plane can be estimated by this procedure to be 45 and 37 nm for the x- and y-axis, respectively.

### Reconstruction of the flagellar beat

To reconstruct the flagellar beat, we developed a fully automated ImageJ plugin written in Java (SpermQ-MF). This plugin was based on SpermQ, a preexisting plugin for 2D analysis of flagellar beating39. The sharpest (highest standard deviation of pixel intensity) of the four plane images was subjected to the standard SpermQ workflow to reconstruct the flagellum in 2D. Next, the z-position of each point was determined using two methods, which are based on the flagellar width and the pixel intensity.

For the method based on the width, a Gaussian curve was fitted to the image intensity in the direction normal to the flagellum at that particular point. The width of the resulting Gaussian was then converted into two z-positions (one located above and another below the focal plane) using the calibrated relationship between the flagellar width, the position on the flagellum, and the z-distance to the respective plane. Finally, the position of the flagellar point was unequivocally determined from the multiple measures of the z-positions across different planes.

For the z-localization method based on the changes of pixel intensity, we determined the intensity value of a given flagellar point in each plane by linear interpolation from the four surrounding pixels weighted by their distance to the given point. Next, each intensity value, together with the focal position of the corresponding plane, formed a point in a coordinate system of focal position and intensity. A Gaussian curve was fitted to the resulting four points, whose center referred to the z-position. All determined z-positions were averaged and smoothed along the arc length as described39.

z-positions were calibrated using recordings of non-motile sperm at different heights set by a piezo-driven objective. Calibrations from all different analyzed sperm (human: n = 12 sperm from five different donors, sea urchin: n = 5 sperm) were averaged and smoothed by a 6 × 6 median-filter kernel.

For each sperm, the x-, y-, z-, and time coordinates of the flagellar points were further analyzed in MATLAB to plot the flagellar trajectories and to determine the beating plane, flagellar curvatures, and frequency spectra. In 3D visualizations of the flagellum, only arc lengths 5–50 or 5–45 µm were shown for human or sea urchin sperm, respectively.

For determining the flagellar beat plane, Gyration Tensor analysis was performed29. The rolling speed was determined based on the normal vector of the beat plane. The rolling frequency was calculated from the simple average of the rolling speed. The curvature was calculated with a moving window (4 µm arc length) along the flagella. Within the moving window, the flagellar segment was projected to the osculating plane and Taubin’s method75 was used to fit the osculating circle of the projected flagellar segment. This algorithm is available at: https://de.mathworks.com/matlabcentral/fileexchange/47885-frenet_robust-zip.

For each sperm, the frequency spectrum was obtained by subjecting the time course of the 3D curvature at arc lengths 33 µm (human sperm) or 34.4 µm (sea urchin sperm) to the MATLAB function “pwelch”, determining Welch’s power spectral density, with the following settings (moving window of 200 pixels, moving step of 40 pixels, 1 pixel = 11/32 µm). The C0, C1, and C2 values were estimated by fast Fourier transform of flagellar curvature at arc lengths 33 µm (human sperm) or 34.4 µm (sea urchin sperm).

### Flow in aqueous solution around a human sperm cell

Sperm were suspended in an HTF solution containing 100 µg/ml HSA (Ref. 9988, Irvine Scientific, USA) and 40 µg/ml carboxylate-modified latex beads of 0.5 µm diameter (C37481, Lot # 1841924, Thermo Fisher Scientific Inc., USA) and inserted into the custom-made observation chamber. Images were recorded with the ×20 objective (NA 0.5) and the additional ×1.6 magnification changer at 500 fps. The flow field was calculated from the tracked bead positions using a custom-made program written in MATLAB. The volume was divided in a mesh grid xyz of volume 4 × 4 × 2 µm. The spatial and temporal mean velocity of particles within that volume was calculated to produce an estimate of the flow velocity within each voxel. The z glass coordinate was estimated as the mean z-position of 114 nonmoving beads.

### Swimming trajectories and swimming speed

Sperm swimming trajectories were smoothed by Matlab in-built smooth function with the “rlowess” method and a time window of 8 ms. The centreline of sperm swimming trajectories was smoothed with the “rlowess” method and a time window of 80 ms. The swimming speed was estimated by the distance of two points along the centreline divided by the time interval.

### Software

Images were acquired using the software pco.camware version 3.09. Image processing and analysis were performed in ImageJ (v1.52i, National Institutes of Health, Bethesda, MN, USA) and MATLAB 2018b (Mathworks). Calculations were performed in MATLAB 2018b (Mathworks), Rstudio (version 1.2.5019), and R (version 3.6.1). Plots and figures were generated using GraphPad Prism (Version 6.07, GraphPad Software, Inc., La Jolla, CA, USA), OriginPro (Version 9.0.0 G, OriginLab Corporation, Northampton, USA), and MATLAB 2018b (Mathworks). Formatting and arrangement of figures was done using Adobe Illustrator CS5 (Adobe Systems, Inc., v15.0.0, San Jose, CA, USA) and Affinity Designer (Version 1.9.3, Serif (Europe) Ltd.). The Java-based software was developed and compiled using Eclipse Mars.2 (Release 4.5.2, IDE for Java Developers, Eclipse Foundation, Inc., Ottawa, Ontario, Canada) and using parts of the source code of SpermQ (v0.1.7,39).

### Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

## Data availability

All data supporting the findings of this study are contained in the text, the Figures, and the Supplementary Information. Exemplary high-speed video recordings to run the analysis routines, extracted data, and analysis supporting this study have been deposited on GitHub under the accession code https://github.com/hansenjn/MultifocalImaging-AnalysisToolbox. The GitHub repository has been archived on Zenodo under the digital object identifier (DOI) 10.5281/zenodo.505253876. The raw bead tracking data used to generate Fig. 6 were deposited on figshare.com under the DOI 10.6084/m9.figshare.14587509.v177Source data are provided with this paper.

## Code availability

The developed ImageJ plugins along with the source code, manual, and example data are freely accessible on GitHub (https://github.com/hansenjn/MultifocalImaging-AnalysisToolbox). Like ImageJ, all the developed ImageJ plugins are open-source. The version of the code applied in this study has been deposited on Zenodo under the DOI 10.5281/zenodo.505253876.

## References

1. 1.

Eichorn, C. et al. How flies are flirting on the fly. BMC Biol. 15, 2 (2017).

2. 2.

Ellington, C. P. The novel aerodynamics of insect flight: applications to micro-air vehicles. J. Exp. Biol. 202, 3439–3448 (1999).

3. 3.

Kojima, S., Yamamoto, K., Kawagishi, I. & Homma, M. The polar flagellar motor of Vibrio cholerae is driven by an Na+ motive force. J. Bacteriol. 181, 1927–1930 (1999).

4. 4.

Alvarez, L., Friedrich, B. M., Gompper, G. & Kaupp, U. B. The computational sperm cell. Trends Cell Biol. 24, 198–207 (2014).

5. 5.

Weisse, S. et al. A quantitative 3D motility analysis of Trypanosoma brucei by use of digital in-line holographic microscopy. PLoS ONE 7, e37296 (2012).

6. 6.

Saggiorato, G. et al. Human sperm steer with second harmonics of the flagellar beat. Nat. Commun. 8, 1415 (2017).

7. 7.

Gibbons, B. H., Baccetti, B. & Gibbons, I. R. Live and reactivated motility in the 9+0 flagellum of Anguilla sperm. Cell. Motil. 5, 333–350 (1985).

8. 8.

Shimmen, T. & Yokota, E. Cytoplasmic streaming in plants. Curr. Opin. Cell. Biol. 16, 68–72 (2004).

9. 9.

Su, T. W., Xue, L. & Ozcan, A. High-throughput lensfree 3D tracking of human sperms reveals rare statistics of helical trajectories. Proc. Natl Acad. Sci. USA 109, 16018–16022 (2012).

10. 10.

Su, T. W. et al. Sperm trajectories form chiral ribbons. Sci. Rep. 3, 1664 (2013).

11. 11.

Wilson, L. G., Carter, L. M. & Reece, S. E. High-speed holographic microscopy of malaria parasites reveals ambidextrous flagellar waveforms. Proc. Natl Acad. Sci. USA 110, 18769–18774 (2013).

12. 12.

Jikeli, J. F. et al. Sperm navigation along helical paths in 3D chemoattractant landscapes. Nat. Commun. 6, 7985 (2015).

13. 13.

Abrahamsson, S. et al. Fast multicolor 3D imaging using aberration-corrected multifocus microscopy. Nat. Methods 10, 60–63 (2013).

14. 14.

Jikeli, J. F., Pascal, R., Alvarez, L. & Honnef, R. Optical beam splitter, particularly for use in beam path of light microscope, for displaying multiple focal planes of object on optical detector, has monolithic base module comprising beam splitter module. DE102010049751A1 (2010).

15. 15.

Itano, M. S., Bleck, M., Johnson, D. S. & Simon, S. M. Readily accessible multiplane microscopy: 3D tracking the HIV-1 genome in living cells. Traffic 17, 179–186 (2016).

16. 16.

Prabhat, P., Ram, S., Ward, E. S. & Ober, R. J. Simultaneous imaging of different focal planes in fluorescence microscopy for the study of cellular dynamics in three dimensions. IEEE Trans. Nanobiosci. 3, 237–242 (2004).

17. 17.

Walker, B. J. & Wheeler, R. J. High-speed multifocal plane fluorescence microscopy for three-dimensional visualisation of beating flagella. J. Cell. Sci. 132, https://doi.org/10.1242/jcs.231795 (2019).

18. 18.

Dalgarno, P. A. et al. Multiplane imaging and three dimensional nanoscale particle tracking in biological microscopy. Opt. Express 18, 877–884 (2010).

19. 19.

Toprak, E., Balci, H., Blehm, B. H. & Selvin, P. R. Three-dimensional particle tracking via bifocal imaging. Nano Lett. 7, 2043–2045 (2007).

20. 20.

Shain, W. J., Vickers, N. A., Goldberg, B. B., Bifano, T. & Mertz, J. Extended depth-of-field microscopy with a high-speed deformable mirror. Opt. Lett. 42, 995–998 (2017).

21. 21.

Gan, Z., Ram, S., Ober, R. J. & Ward, E. S. Using multifocal plane microscopy to reveal novel trafficking processes in the recycling pathway. J. Cell Sci. 126, 1176–1188 (2013).

22. 22.

Ram, S., Prabhat, P., Ward, E. S. & Ober, R. J. Improved single particle localization accuracy with dual objective multifocal plane microscopy. Opt. Express 17, 6881–6898 (2009).

23. 23.

Ram, S., Prabhat, P., Chao, J., Ward, E. S. & Ober, R. J. High accuracy 3D quantum dot tracking with multifocal plane microscopy for the study of fast intracellular dynamics in live cells. Biophys. J. 95, 6025–6043 (2008).

24. 24.

Forster, B., Van De Ville, D., Berent, J., Sage, D. & Unser, M. Complex wavelets for extended depth-of-field: a new method for the fusion of multichannel microscopy images. Microsc. Res. Tech. 65, 33–42 (2004).

25. 25.

Blayvas, I., Kimmel, R. & Rivlin, E. Role of optics in the accuracy of depth-from-defocus systems. J. Opt. Soc. Am. A 24, 967–972 (2007).

26. 26.

Schechner, Y. Y. & Kiryati, N. Depth from defocus vs. stereo: how different really are they? Int. J. Comput. Vis. 39, 141–162 (2000).

27. 27.

Zammit, P., Harvey, A. R. & Carles, G. Extended depth-of-field imaging and ranging in a snapshot. Optica 1, 209–216 (2014).

28. 28.

Rikmenspoel, R., van, H. & Eijkhout, P. Cinematographic observations of the movements of bull sperm cells. Phys. Med. Biol. 5, 167–181 (1960).

29. 29.

Bukatin, A., Kukhtevich, I., Stoop, N., Dunkel, J. & Kantsler, V. Bimodal rheotactic behavior reflects flagellar beat asymmetry in human sperm cells. Proc. Natl Acad. Sci. USA 112, 15904–15909 (2015).

30. 30.

Wu, Y. et al. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nat. Methods 16, 1323–1331 (2019).

31. 31.

Oku, H., Ishikawa, M., Theodorus & Hashimoto, K. High-speed autofocusing of a cell using diffraction patterns. Opt. Express 14, 3952–3960 (2006).

32. 32.

MacKay, D. J. C. Information Theory, Inference, and Learning Algorithms (Cambridge University Press, 2003).

33. 33.

Farthing, N. E. et al. Simultaneous two-color imaging in digital holographic microscopy. Opt. Express 25, 28489–28500 (2017).

34. 34.

Muschol, M., Wenders, C. & Wennemuth, G. Four-dimensional analysis by high-speed holographic imaging reveals a chiral memory of sperm flagella. PLoS ONE 13, e0199678 (2018).

35. 35.

Gadêlha, H., Hernández-Herrera, P., Montoya, F., Darszon, A. & Corkidi, G. The human sperm beats anisotropically and asymmetrically in 3D. bioRxiv. https://doi.org/10.1101/795245 (2019).

36. 36.

Gallagher, M. T., Cupples, G., Ooi, E. H., Kirkman-Brown, J. C. & Smith, D. J. Rapid sperm capture: high-throughput flagellar waveform analysis. Hum. Reprod. 34, 1173–1185 (2019).

37. 37.

Gong, A. et al. The steering gaits of sperm. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 375, 20190149 (2020).

38. 38.

Dardikman-Yoffe, G., Mirsky, S. K., Barnea, I. & Shaked, N. T. High-resolution 4-D acquisition of freely swimming human sperm cells without staining. Sci. Adv. 6, eaay7619 (2020).

39. 39.

Hansen, J., Rassmann, S., Jikeli, J. & Wachten, D. SpermQ—a simple analysis software to comprehensively study flagellar beating and sperm steering. Cells 8, 10 (2019).

40. 40.

Bohmer, M. et al. Ca2+ spikes in the flagellum control chemotactic behavior of sperm. EMBO J. 24, 2741–2752 (2005).

41. 41.

Gray, J. The movement of sea-urchin spermatozoa. J. Exp. Biol. 32, 775–801 (1955).

42. 42.

Elgeti, J., Kaupp, U. B. & Gompper, G. Hydrodynamics of sperm cells near surfaces. Biophys. J. 99, 1018–1026 (2010).

43. 43.

Wachten, D., Jikeli, J. F. & Kaupp, U. B. Sperm sensory signaling. Cold Spring Harb. Perspect. Biol. 9, https://doi.org/10.1101/cshperspect.a028225 (2017).

44. 44.

Elgeti, J. & Gompper, G. Emergence of metachronal waves in cilia arrays. Proc. Natl Acad. Sci. USA 110, 4470–4475 (2013).

45. 45.

Lenaghan, S. C., Davis, C. A., Henson, W. R., Zhang, Z. & Zhang, M. High-speed microscopic imaging of flagella motility and swimming in Giardia lamblia trophozoites. Proc. Natl Acad. Sci. USA 108, E550–E558 (2011).

46. 46.

Drescher, K., Goldstein, R. E., Michel, N., Polin, M. & Tuval, I. Direct measurement of the flow field around swimming microorganisms. Phys. Rev. Lett. 105, 168101 (2010).

47. 47.

Hamad, A. M. Molecular and physical interactions of human sperm with female tract secretions, University of Birmingham (2017).

48. 48.

Ishimoto, K., Gadelha, H., Gaffney, E. A., Smith, D. J. & Kirkman-Brown, J. Coarse-graining the fluid flow around a human sperm. Phys. Rev. Lett. 118, 124501 (2017).

49. 49.

Chang, J. & Wetzstein, G. Deep Optics for Monocular Depth Estimation and 3D Object Detection. in 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 10192–10201, https://doi.org/10.1109/ICCV.2019.01029 (2019).

50. 50.

Garcia, J., Sanchez, J. M., Orriols, X. & Binefa, X. Chromatic aberration and depth extraction. Proc. 15th Int. Conf. Pattern Recognit. ICPR-2000 761, 762–765 (2000).

51. 51.

Kashiwagi, M., Mishima, N., Kozakaya, T. & Hiura, S. Deep Depth From Aberration Map. in 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 4069–4078, https://doi.org/10.1109/ICCV.2019.00417 (2019).

52. 52.

Huang, B., Wang, W., Bates, M. & Zhuang, X. Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 319, 810–813 (2008).

53. 53.

Izeddin, I. et al. PSF shaping using adaptive optics for three-dimensional single-molecule super-resolution imaging and tracking. Opt. Express 20, 4957–4967 (2012).

54. 54.

Li, Y. et al. Real-time 3D single-molecule localization using experimental point spread functions. Nat. Methods 15, 367–369 (2018).

55. 55.

Siemons, M. et al. Comparing strategies for deep astigmatism-based single-molecule localization microscopy. Biomed. Opt. Express 11, 735–751 (2020).

56. 56.

Zhou, Y. & Carles, G. Precise 3D particle localization over large axial ranges using secondary astigmatism. Opt. Lett. 45, 2466–2469 (2020).

57. 57.

Hernandez-Herrera, P., Montoya, F., Rendon-Mancha, J. M., Darszon, A. & Corkidi, G. 3-D human sperm flagellum tracing in low SNR fluorescence images. IEEE Trans. Med. Imaging 37, 2236–2247 (2018).

58. 58.

Daloglu, M. U. et al. Label-free 3D computational imaging of spermatozoon locomotion, head spin and flagellum beating over a large volume. Light Sci. Appl. 7, 17121 (2018).

59. 59.

Silva-Villalobos, F., Pimentel, J. A., Darszon, A. & Corkidi, G. Imaging of the 3D dynamics of flagellar beating in human sperm. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2014, 190–193 (2014).

60. 60.

Lee, S. H. et al. Characterizing and tracking single colloidal particles with video holographic microscopy. Opt. Express 15, 18275–18282 (2007).

61. 61.

Cheong, F. C., Krishnatreya, B. J. & Grier, D. G. Strategies for three-dimensional particle tracking with holographic video microscopy. Opt. Express 18, 13563–13573 (2010).

62. 62.

Gass, J., Dakoff, A. & Kim, M. K. Phase imaging without 2pi ambiguity by multiwavelength digital holography. Opt. Lett. 28, 1141–1143 (2003).

63. 63.

Kemper, B. & von Bally, G. Digital holographic microscopy for live cell applications and technical inspection. Appl. Opt. 47, A52–A61 (2008).

64. 64.

Kuhn, J. et al. Submicrometer tomography of cells by multiple-wavelength digital holographic microscopy in reflection. Opt. Lett. 34, 653–655 (2009).

65. 65.

Prabhat, P. et al. Elucidation of intracellular recycling pathways leading to exocytosis of the Fc receptor, FcRn, by using multifocal plane microscopy. Proc. Natl Acad. Sci. USA 104, 5889–5894 (2007).

66. 66.

Guasto, J. S., Johnson, K. A. & Gollub, J. P. Oscillatory flows induced by microorganisms swimming in two dimensions. Phys. Rev. Lett. 105, 168102 (2010).

67. 67.

Bush, N. E., Solla, S. A. & Hartmann, M. J. Whisking mechanics and active sensing. Curr. Opin. Neurobiol. 40, 178–188 (2016).

68. 68.

Strunker, T. et al. The CatSper channel mediates progesterone-induced Ca2+ influx in human sperm. Nature 471, 382–386 (2011).

69. 69.

Thevenaz, P., Ruttimann, U. E. & Unser, M. A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process 7, 27–41 (1998).

70. 70.

Li, C. H. & Tam, P. K. S. An iterative algorithm for minimum cross entropy thresholding. Pattern Recognit. Lett. 19, 771–776 (1998).

71. 71.

Arganda-Carreras, I., Fernández‐González, R., Muñoz‐Barrutia, A. & Ortiz-De-Solorzano, C. 3D reconstruction of histological sections: application to mammary gland tissue. Microsc. Res. Tech. 73, 1019–1029 (2010).

72. 72.

Groen, F. C., Young, I. T. & Ligthart, G. A comparison of different focus functions for use in autofocus algorithms. Cytometry 6, 81–91 (1985).

73. 73.

Tinevez, J. Y. et al. TrackMate: an open and extensible platform for single-particle tracking. Methods 115, 80–90 (2017).

74. 74.

Berg, H. C. Random Walks in Biology (Princeton University Press, 1993).

75. 75.

Taubin, G. Estimation of planar curves, surfaces and nonplanar space curves defined by implicit equations, with applications to edge and range image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 13, https://doi.org/10.1109/34.103273 (1991).

76. 76.

Hansen, J. N., Gong, A., Jikeli, J. F. & Alvarez, L. Multifocal imaging for precise, label-free tracking of fast biological processes in 3D. Zenodo, https://doi.org/10.5281/zenodo.5052538 (2021).

77. 77.

Hansen, J. N., Alvarez, L. & Gong, A. 3D particle-imaging-velocimetry around a human sperm cell. figshare, https://doi.org/10.6084/m9.figshare.14587509.v1 (2021).

## Acknowledgements

This work was funded by the German Research Foundation (DFG) under Germany’s Excellence Strategy—EXC2151—project number 390873048 (D.W.), SPP1726 “Microswimmers” (L.A., U.B.K., J.F.J., and D.W.), as well as the Boehringer Ingelheim Fonds (J.N.H.) and the Alexander von Humboldt Foundation (A.T.). We thank Timo Strünker for administering the pool of sperm donors.

## Funding

Open Access funding enabled and organized by Projekt DEAL.

## Author information

Authors

### Contributions

J.F.J., L.A., D.W., and U.B.K. conceived the project. J.N.H., J.F.J., L.A., and U.B.K. designed the experiments. J.N.H., J.F.J., A.G., and R.P. performed the experiments. J.N.H., L.A., J.F.J., and A.G. developed software to analyze the data. J.N.H., A.G., J.F.J., and L.A. analyzed the data. A.T. performed theoretical estimations. J.N.H., L.A., J.F.J., and U.B.K. wrote the manuscript. All authors commented on and edited the text.

### Corresponding authors

Correspondence to Jan N. Hansen or Luis Alvarez.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Peer review information Nature Communications thanks Olivier Thouvenin and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Hansen, J.N., Gong, A., Wachten, D. et al. Multifocal imaging for precise, label-free tracking of fast biological processes in 3D. Nat Commun 12, 4574 (2021). https://doi.org/10.1038/s41467-021-24768-4

• Accepted:

• Published: