Abstract
Tomography has made a radical impact on diverse fields ranging from the study of 3D atomic arrangements in matter to the study of human health in medicine. Despite its very diverse applications, the core of tomography remains the same, that is, a mathematical method must be implemented to reconstruct the 3D structure of an object from a number of 2D projections. Here, we present the mathematical implementation of a tomographic algorithm, termed GENeralized Fourier Iterative REconstruction (GENFIRE), for highresolution 3D reconstruction from a limited number of 2D projections. GENFIRE first assembles a 3D Fourier grid with oversampling and then iterates between real and reciprocal space to search for a global solution that is concurrently consistent with the measured data and general physical constraints. The algorithm requires minimal human intervention and also incorporates angular refinement to reduce the tilt angle error. We demonstrate that GENFIRE can produce superior results relative to several other popular tomographic reconstruction techniques through numerical simulations and by experimentally reconstructing the 3D structure of a porous material and a frozenhydrated marine cyanobacterium. Equipped with a graphical user interface, GENFIRE is freely available from our website and is expected to find broad applications across different disciplines.
Introduction
Tomography has found widespread applications in the physical, biological and medical sciences^{1,2,3,4,5,6,7}. Electron tomography, for example, is experiencing a revolution in highresolution 3D imaging of physical and biological samples. In the physical sciences, atomic electron tomography (AET) has been developed to determine the 3D atomic structure of crystal defects such as grain boundaries, antiphase boundaries, stacking faults, dislocations, chemical order/disorder and point defects, and to precisely localize the 3D coordinates of individual atoms in materials without assuming crystallinity^{1, 8,9,10,11,12}. The atomic coordinates measured by AET have been used as direct input to density functional theory calculations to correlate crystal defects and chemical order/disorder with material properties at the single atomic level^{13}. In the biological sciences, singleparticle cryoelectron microscopy (EM) has been applied to achieve near atomic resolution of purified protein complexes^{2, 7, 14,15,16}, and cryoelectron tomography allows for 3D imaging of pleomorphic samples such as viral infection mechanisms of cells with resolutions on the order of a few nanometers^{17,18,19}. These advances are not limited to electron tomography. Tomographic implementation of synchrotron Xray absorption and phase contrast imaging has also found interdisciplinary applications^{5, 20,21,22,23,24,25}. Using the brilliance of advanced Xray sources, coherent diffractive imaging (CDI) methods^{26} have been combined with tomographic reconstruction for 3D quantitative imaging of thick samples with resolutions in the tens of nanometers^{27,28,29,30,31,32,33}.
Presently, a popular tomographic reconstruction method is filtered back projection (FBP)^{2,3,4}. FBP works well when there are a large number of projections with no missing data. However, when the data is inadequately sampled due to the radiation dose and geometric constraints, it suffers from artifacts, potentially clouding interpretability of the final reconstruction. This difficulty can be partially alleviated by realspace iterative algorithms such as the algebraic reconstruction technique (ART)^{34}, simultaneous ART (SART)^{35} and simultaneous iterative reconstruction technique (SIRT)^{36}. However, these algorithms do not fully exploit the correlated information among all the projections as the iteration process is implemented through local interpolation in real space. In contrast, Fourierbased iterative algorithms use information in both real and Fourier space as part of the iterative process^{13, 37, 38}. A major advantage of these algorithms is that changes made in one space affect the other space globally. Equal slope tomography (EST)^{37}, an example of such an algorithm, has been successfully applied in AET to reconstruct the 3D arrangement of crystal defects in materials, including recovery of Bragg peaks in the missing wedge direction^{1, 8,9,10}. Additionally, EST was shown to produce reconstructions comparable to modern medical CT techniques but using significantly lower radiation dose^{20, 22, 39}. However, the drawback of EST is the requirement that the tilt angles must follow equal slope increments along a single tilt axis, which limits its broader applications.
Very recently, a generalized Fourier iterative reconstruction algorithm (GENFIRE) has been reported for highresolution 3D imaging from a limited number of 2D projections^{13}. GENFIRE first pads zeros to each 2D projection and calculates its oversampled Fourier slice^{40, 41}. The oversampled Fourier slices are used to accurately compute a small fraction of points on a 3D Cartesian grid based on gridding interpolation^{42, 43}. The remaining grid points that cannot be determined with sufficient accuracy are defined as unknown. The algorithm then iterates between real and reciprocal space and enforces constraints in each space. In real space, the negative valued voxels and the voxels in the zeropadding region are set to zero. In reciprocal space, the small fraction of the known grid points are enforced in each iteration, while the unknown grid points are recovered by the iterative process. After several hundred iterations, the algorithm converges to a structure that is concurrently consistent with the measured data and the physical constraints. Furthermore, GENFIRE implements an angular refinement routine to reduce the tilt angle error and can be adapted to any tomographic data acquisition geometry. In this article, we present the mathematical implementation of the GENFIRE algorithm. Using both physical and biological samples, we demonstrate that GENFIRE produces superior 3D reconstructions relative to several other tomographic reconstruction algorithms.
Methods
Assembling a 3D Fourier grid with oversampling
GENFIRE first assembles a rectangular 3D Fourier grid from a set of measured 2D projections. According to the Fourier slice theorem, the Fourier transform of each 2D projection represents a plane slicing through the origin of the Fourier transform of a 3D object^{3, 4}. Care should be taken that the Fourier slice theorem requires each 2D image to be a linear projection of the 3D object at a given angle. This requirement may not be satisfied in some experimental cases due to the presence of nonlinear effects, which will be discussed in later sections. To obtain a Fourier grid point, \({F}_{obs}(\vec{k})\), we compute its perpendicular distance to the Fourier plane, where D _{ j } represents the perpendicular distance and \(({u}_{j},{v}_{j})\) the foot of the perpendicular line to the j ^{th} projection. Since (\({u}_{j},{v}_{j}\)) are not integer coordinates, we use the discrete Fourier transform (DFT) to compute the value of (\({u}_{j},{v}_{j}\)). The use of the DFT to explicitly compute the (\({u}_{j},{v}_{j}\)) value is more accurate than interpolating from the set of 2D FFTs of each projection at the cost of being computationally slower. A faster, but less accurate, FFT gridding method is also provided as an option in the GENFIRE package. After calculating the (\({u}_{j},{v}_{j}\)) values for all the projections, we determine the value of the Fourier grid point by
where D _{ th } is a predefined threshold, \({f}_{obs}^{j}(x,y)\) is the j ^{th} 2D projection with a size of N x N pixels and O is the linear oversampling ratio^{40, 41}. By properly choosing D _{ th } and O, we accurately determine a small fraction of the Fourier grid points, while the remaining grid points are defined as unknown. In the GENFIRE package, the default value for D _{ th } and O is 0.5 and 3, respectively.
The Fourier based iterative algorithm
Due to radiation dose and/or geometric constraints, it is desirable in many tomography applications to achieve highresolution 3D imaging from a limited number of projections. As a result, a significant amount of the assembled Fourier grid points remain unknown after the gridding process. To recover the unknown grid points, GENFIRE iterates between real and reciprocal space with general constraints enforced in each space (Fig. 1). For the 1^{st} iteration, the values of the unknown grid points can be assigned to zero, random numbers or some other predetermined numbers as the algorithm is not very sensitive to the initial input. The i ^{th} GENFIRE iteration consists of following five steps (Fig. 1).

(i)
Apply the inverse FFT to \({F}_{i}(\overrightarrow{k})\) and obtain the i ^{th} image, \({\rho }_{i}(\overrightarrow{r})\).

(ii)
Modify the image by applying the following constraints
$${\rho }_{i}^{^{\prime} }(\overrightarrow{r})=\{\begin{array}{cc}0 & (\overrightarrow{r}\notin S)\cup ({\rho }_{i}(\overrightarrow{r}) < 0)\\ {\rho }_{i}(\overrightarrow{r}) & \,Otherwise\end{array}\,$$(2)where S represents a support, separating the zeropadding region from the sample structure. The zeropadding region is due to oversampling^{40, 41}. This step sets the voxels outside the support or negative valued voxels inside the support to zero, while retaining the values of the other voxels.

(iii)
Apply the FFT to \({\rho }_{i}^{^{\prime} }(\overrightarrow{r})\) to obtain \({F}_{i}^{^{\prime} }(\overrightarrow{k})\).

(iv)
Compute \({F}_{i+1}(\overrightarrow{k})\,\,\)by enforcing the Fourier space constraint,
$${F}_{i+1}(\overrightarrow{k})=\{\begin{array}{cc}{F}_{obs}(\overrightarrow{k}) & \,Known\,voxels\\ {F}_{i}^{^{\prime} }(\overrightarrow{k}) & \,Unknown\,voxels\end{array}\,$$(3)\({F}_{i+1}(\overrightarrow{k})\) is used for the (i + 1)^{th} iteration.

(v)
Calculate two Rfactors, \({R}_{k}\,\,\)and \({R}_{free}\),
where \({\vec{k}}_{known}\) represents the known voxels and \({\vec{k}}_{withheld}\) is a small number of randomly selected known voxels that are not used in the reconstruction. \({R}_{k}\,\,\)is an error metric to monitor the convergence of the iterative process, while \({R}_{free}\) is an unbiased free parameter to evaluate the reconstruction, which is used in crystallography^{44}. \({R}_{free}\) is always larger than \({R}_{k}\), but for a good reconstruction the two Rfactors should be consistent. Significant deviation could indicate overfitting. The algorithm is reliable and usually converges within several hundred iterations.
In the GENFIRE package, there is also an option to use resolution extension/suppression. For experimental data, the signaltonoise ratio decreases with the increase of the spatial frequency. To compensate the high noise level at the high spatial frequency, we implement a resolution extension/suppression technique capable of partially decoupling signal and noise through a simple modification of the way the Fourier constraint is applied. For the first iteration, only the lowest spatial frequency information is enforced. As iterations progress, higher spatial frequency data is gradually applied. This continues, forming the extension step, until half of the total number of iterations has been completed, at which point all measured data is enforced. The process is then reversed for the second half of the reconstruction, and the spatial resolution of the enforced data is gradually reduced to form the suppression step until the final iteration when only the lowest frequency information is constrained once again. While resolution extension has been implemented before^{45, 46}, to our knowledge, resolution extension/suppression has not been previously reported. We have performed extensive numerical simulations and observed that this technique can consistently improve the 3D reconstruction with noisy data (Supplementary Fig. 1). Full exploration of the generality of resolution extension/suppression requires followup studies.
Angular refinement
The experimentally measured tilt angles may not always coincide with the true orientations of the projections. This could be the result of many causes including instrument misalignment, slipping, beaminduced motion, vibration, thermal effects, or software error. To achieve highresolution 3D reconstruction, we implement an angular refinement procedure to reduce the tilt angle error, which consists the following four steps.

(i)
An initial 3D reconstruction is computed using the experimentally measured tilt angles.

(ii)
For the j ^{th} projection, a series of 2D projections are calculated from the 3D reconstruction by varying the three Euler angles: \(\varphi \in [{\varphi }_{j}\delta \varphi ,{\varphi }_{j}+\delta \varphi ]\), \(\theta \in \,[{\theta }_{j}\delta \theta ,{\theta }_{j}+\delta \theta ]\), \(\psi \in [{\psi }_{j}\delta \psi ,{\psi }_{j}+\delta \psi ]\), where \(({\varphi }_{j},{\theta }_{j},{\psi }_{j})\) are the current best fit for the Euler angles of the j ^{th} projection, and (δφ, δθ, δψ) are the userdefined values for the angular search ranges. Each calculated 2D projection is then compared with the corresponding measured projection, \({f}_{obs}^{j}(x,y)\), and a qualityoffit metric is computed. The qualityoffit metric can be implemented by either the normalized crosscorrelation or the real space Rfactor. For the latter, additional translational alignment between two projections have to be performed, whereas using cross correlation the translational search is performed simultaneously. The three Euler angles with either the largest cross correlation or smallest Rfactor are recorded as the refined angles for the j ^{th} projection.

(iii)
Repeat step (ii) for all the projections and a series of the refined angles are obtained.

(iv)
Obtain a new 3D reconstruction with the refined angles for all the projections.

(v)
Repeat steps (ii) – (iv) until no further improvement can be made.
In practice, each projection is refined in parallel, and the calculation of 2D projections from the 3D reconstruction represents the bulk of the computation. This calculation is expedited by applying the FFT to obtain an oversampled Fourier transform from the 3D reconstruction. Central slices are computed from the 3D Fourier transform using the C++ library splinterp for multithreaded linear interpolation. The inverse FFT is used to invert the central slices to the corresponding 2D projections. Care should be taken that while GENFIRE’s reconstruction can find a global minimum, the current angular refinement approach may be trapped into local minima. Further developments are needed to search for a global minimum for angular refinement.
Results
Numerical simulations on the reconstruction of a biological vesicle
Numerical simulations on the 3D reconstruction of a 64 × 64 × 64 voxel vesicle model (Fig. 2a–c) were performed using GENFIRE, EST, FBP and SIRT. Simulated projections were obtained by first calculating 2D Fourier slices of the 3D model for given angles. The corresponding realspace projections were then computed by applying the inverse FFT to the Fourier slices. This code is also included in the GENFIRE package and can be accessed graphically using the Projection Calculator. To evaluate the performance of various reconstruction algorithms with noise, we calculated 71 projections with the tilt angles ranging −70.1° to +70.1°. Noise was added to the projections at levels similar to that observed in cryoEM images of cellular structures. Each set of projections were reconstructed using GENFIRE, EST, FBP and SIRT. The EST and GENFIRE reconstructions were performed using a loose support, the positivity constraint and 250 iterations. The SIRT reconstruction was achieved with the positivity constraint, longobject compensation and 125 iterations.
Figure 2d,g,j and m show a 10voxelthick central slice of the 3D reconstructions in the XY plane using GENFIRE, EST, FBP and SIRT, respectively, where the zaxis is the missing wedge direction. Because there is no missing data in this direction, the reconstructions from all methods exhibit good agreement with the model (Fig. 2a). However, along the missing wedge direction both GENFIRE and EST reconstructions (Fig. 2e,f,h and i) appear to be more isotropic and contain more fine features than FBP and SIRT (Fig. 2k,l,n and o). The Fourier shell correlation^{2} (FSC) between the reconstructions and the model further confirms that the GENFIRE resconstruction is superior at all spatial frequencies compared to other algorithms. This simulation was also performed with no noise and higher noise (Supplementary Figs 2 and 3). In the case of noisefree data with equal slope angles, EST produces slightly better results than GENFIRE as no interpolation is needed in EST. However, in practice this idealized scenario does not occur, and our results show that for even moderate noise levels GENFIRE produces better results. SIRT introduces a form of regularization to the reconstruction, which reduces missing wedge artifacts but also appears to compromise the resolution. By accurately assembling a small fraction of the Fourier grid points and using an iterative algorithm with resolution extension/suppression, GENFIRE is able to simultaneously reduce the effect of noise and retain higher resolution information. This capability will be important as scientists continue to solve important problems by pushing imaging systems to their limits.
Numerical simulations on atomic electron tomography
To quantify the GENFIRE reconstruction of 3D nanostructures at atomic resolution with noise and a missing wedge, we generated a 3D atomic model consisting of a 4.3 nm FePt_{3} nanoparticle with a chemically ordered facecentered cubic (L1_{2}) phase. Using this model, 27 annular darkfield (ADF) projections were computed using multislice simulation^{47} (electron energy: 300 keV, probe size: 0.5 Å, C_{3}: 0 mm, C_{5}: 5 mm, probe convergence semiangle: 30 mrad, and the inner and outer detector angles: 48 mrad and 251 mrad). The multislice calculation accounts for a number of nonlinear effects including dynamical scattering. The angular tilt range is ±70.1° and the pixel size is 0.4 Å. For each tilt angle, a total of 10 frozen phonon configurations were averaged. To simulate the convolution effect resulting from finite probe size and other incoherent effects, each image was convolved with a 2D Gaussian function with σ = 0.51 Å. PoissonGaussian noise was then added to the ADF scanning transmission electron microscopy (STEM) projections.
After denoising was applied to the projections^{48}, this tilt series was reconstructed with GENFIRE, EST, SIRT and FBP, and the results are shown in Fig. 3. Visually, GENFIRE, EST, and SIRT all demonstrate reduction of reconstruction artifacts, though the difference appears more substantial for GENFIRE and EST (Fig. 3a–h). Both SIRT and FBP suffer from aliasing artifacts that produce what appear to be atoms, but are not actually present in the model, outside of the true boundary of the particle (Fig. 3c,d,g and h). These phantom atoms would prove problematic for atom tracing and refinement in AET. The iterative methods have also successfully recovered missing information as indicated by the presence of Bragg peaks in the missing wedge (magenta arrows in Fig. 3i–k). Determination of 3D atomic coordinates is most accurate when the reconstruction is isotropic, thus it is important for the reconstruction algorithm to be robust to noise and the missing wedge problem. Among the four algorithms, GENFIRE produces the best reconstruction of the 3D atomic structure. Individual atoms are clearly visible in the 3D reconstruction (Fig. 3a and e) even though dynamical scattering effects are present in the 2D projections. We attribute the reduction of the dynamical scattering effects to the measurement of many projections at different sample orientations (i.e. a rotational average) in ADFSTEM tomography^{10}. A detailed discussion of dynamical scattering on AET can be found in the Supplementary Information in ref. 10.
Angular refinement simulations
To demonstrate the improvement made by angular refinement, a simulation was performed using the same 27 ADFSTEM projections from Fig. 3. The orientation angle of each projection was randomly shifted up to ±2°, and a random translational shift of ±1 pixel was applied along the x and yaxes. A preliminary GENFIRE reconstruction was performed and used as input to the refinement loop which was run for a total of 5 iterations with an angular search range of ±3° with 0.2° steps, and with normalized crosscorrelation as the error metric. The results of this simulation are shown in Fig. 4. The initial and refined angles were compared with the true ones using a normalized angular distance^{49} (Fig. 4a.), resulting in an improvement from an initial average angular error of 2.1° to a refined value of 1.3°. The reconstruction is improved after angular refinement, shown in Fig. 4b and c. The boundary of the nanoparticle is also better defined, with fewer artifacts around the periphery.
GENFIRE reconstruction on experimental data of a porous material
To perform a quantitative comparison between GENFIRE and other iterative algorithms on experimental data, we acquired a tomographic tilt series of a Mo_{3}Si alloy annealed at 1100 °C. MoSi and MoSiB alloys are resistant to oxidation and creep and are among potential candidates with high melting temperatures to replace currently widely used Nibased superalloys^{50, 51}. The experiment was conducted on an FEI TitanX 60300 in STEM mode equipped with a Gatan highangle annular dark field detector. The microscope was operated at 200 keV with electron beam current ~40 pA, a convergence semiangle of 10 mrad, and a camera length of 91 mm. A total of 129 projections were collected with a tilt range from −58° and +70° in 1° increments. After background subtraction, the projections were aligned along the tilt axis direction by crosscorrelation and along the perpendicular direction using the centerofmass method^{8, 9}. Reconstructions were performed with GENFIRE and SIRT. The SIRT reconstruction was computed using Tomo3D^{52}. Figure 5a and b show the 13.6nmthick central slice of the GENFIRE and SIRT reconstruction of a fragment of the sample, revealing a complex 3D porous structure. Along the 0° direction, both GENFIRE and SIRT produce good reconstructions, although fine features are better resolved by GENFIRE (Fig. 5a and b). However, in the missing wedge direction, GENFIRE exhibits significant improvement over SIRT with sharper boundaries and more distinctive 3D pore structures (Fig. 5c and d). Figure 5e and f show isosurface renderings of the reconstructions, where elongation artifacts due to the missing wedge are clearly visible in the SIRT reconstruction, but are reduced by GENFIRE.
GENFIRE reconstruction of a frozen hydrated cell
GENFIRE was also used to reconstruct the 3D structure of a frozenhydrated marine cyanobacterium in a late stage of infection by cyanophages^{53}. A tilt series of 42 projections ranging from −58° to +65° were acquired on a JEM2200FS electron microscope equipped with a Zernike phase plate and recorded on a 4k × 4k Gatan CCD. The projections were binned by 4 × 4 pixels, resulting in images with approximately 1.8 × 1.8 nm^{2} per pixel. The background was carefully removed from each projection based on the average value in a flat region outside of the cell. A marine cyanobacterium was then cropped out from the surrounding regions by smoothing and thresholding each projection to produce a softedged mask. Finally, each projection was aligned and normalized to have the same total sum as the integrated density should be conserved. The tilt series was separately reconstructed with GENFIRE and FBP (Fig. 6). The GENFIRE reconstruction was performed for 100 iterations with a loose cubic support, while the FBP reconstruction was computed using IMOD^{54}. Several lowcontrast features are visible in the GENFIRE reconstruction that are difficult, if not impossible, to identify with FBP. Of particular interest in this dataset was the interactions between the marine cyanobacterium and cyanophages. Figure 6 shows a slice through the reconstructed volumes capturing the penetration of a cyanophage into the cell membrane during the infection process. This interaction has caused a local depression in the cell membrane, and the shown cross section passes through this depression as well as the viral capsid and appendage (Fig. 6c–j). Based on this geometry the cell membrane should be visible on both sides of the interaction, similar to taking a horizontal crosssection through a Ushape (Fig. 6c and d). Although the top side of the membrane is visible in both reconstructions (magenta arrows), the bottom side is only visible in the GENFIRE reconstruction (yellow arrow). Figure 6i and j show isosurface renderings of the penetration of the cyanophage into the cell membrane, where GENFIRE exhibits higher contrast, less peripheral noise, more easily detectable cell boundaries than FBP.
Discussion
In this article, we present the mathematical implementation of GENFIRE for highresolution 3D reconstruction from a limited number of projections with a missing wedge. Both numerical simulation and experimental results of materials science and biological specimens indicate that GENFIRE produces superior 3D reconstruction to several other tomographic algorithms. As a Fourierbased iterative method, GENFIRE first computes a small fraction of Cartesian grid points with high precision from 2D projections using Fourier gridding and oversampling, where proper choice of the oversampling ratio is critical to the precise calculation of the small fraction of grid points. It then iterates between real and reciprocal space using the FFT and its inversion. Positivity and support are enforced in real space, while the grid points calculated from the measured data are applied in reciprocal space. As the Fourier data, positivity and support are all convex constraint sets, GENFIRE belongs to the method of projections onto convex sets, whose convergence has been mathematically proven^{55, 56}. This allows GENFIRE to search for a global solution that is concurrently consistent with the measured data and physical constraints. One of the unique features of Fourierbased iterative algorithms such as GENFIRE is that any changes in real space globally affect all the points in reciprocal space and vice versa. This global correlation between real and reciprocal space makes GENFIRE robust to the missing data and missing wedge. In contrast, ART, SART and SIRT perform all the iterations in real space through local interpolation. When there is a missing wedge, the local interpolation in that region becomes less accurate. This explains why GENFIRE achieves better 3D reconstructions than several other tomographic algorithms. Furthermore, compared to EST that is only applicable to single tilt axis data, GENFIRE can not only work with any tomographic geometry, but also performs faster due to the use of the FFT and its inversion for iteration. It must be cautioned, however, that GENFIRE users must be aware of the physical implications of the conditions under which images are acquired with respect to the linearity of the projection. Although nonlinear effects such as dynamical scattering can be alleviated due to a rotational average in tomography^{10}, the exact extent to which such nonlinear effects degrade GENFIRE needs to be explored in future work.
Another class of tomographic reconstruction methods based on compressed sensing is presently under rapid development^{57, 58}. Compressed sensing assumes that a physically meaningful structure is usually sparse in some domain. If the sparse domain can be found, the 3D structure can in principle be reconstructed from a small number of 2D projections. Compressed sensing tomography typically incorporates mathematical regularization such as total variation minimization^{59}, which requires manual tuning of parameters^{60}. This is acceptable in certain applications, where the scope of reconstruction targets is limited enough to permit a specialized set of parameters. However, for general tomographic reconstructions, it is not straightforward to optimize these parameters, especially with the presence of missing data and noise. For example, it would be very challenging, if not impossible, for compressed sensing tomography to reconstruct the 3D distribution of point defects in a crystalline specimen. Conversely, GENFIRE uses very general physical constraints and requires minimum manual tuning of parameters. It has recently been used to determine crystal defects such as grain boundaries, chemical order/disorder, antiphase boundaries and point defects with unprecendent 3D detail^{10, 13}. Furthermore, GENFIRE can be easily adapted to incorporate mathematical regularization to reconstruct 3D sparse objects from a small number of projections. Looking forward, we expect GENFIRE can be applied to a plethora of imaging modalities to address a wide range of scientific problems.
Software availability
The GENFIRE software package with a graphical user interface is freely available at www.genfireem.com.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
 1.
Miao, J., Ercius, P. & Billinge, S. J. L. Atomic electron tomography: 3D structures without crystals. Science 353 (2016).
 2.
Frank, J. ThreeDimensional Electron Microscopy of Macromolecular Assemblies: Visualization of Biological Molecules in Their Native State. (Oxford Univ. Press, 2006).
 3.
Kak, A. C. & Slaney, M. Principles of computerized tomographic imaging. (Society for Industrial and Applied Mathematics) doi:10.1137/1.9780898719277 (2001)
 4.
Herman, G. T. and Herman, G. T. Fundamentals of computerized tomography: image reconstruction from projections. (Springer, 2009).
 5.
Momose, A., Takeda, T., Itai, Y. & Hirano, K. Phasecontrast Xray computed tomography for observing biological soft tissues. Nat. Med. 2, 473–475 (1996).
 6.
Pfeiffer, F., Weitkamp, T., Bunk, O. & David, C. Phase retrieval and differential phasecontrast imaging with lowbrilliance Xray sources. Nat. Phys. 2, 258–261 (2006).
 7.
FernandezLeiro, R. & Scheres, S. H. W. Unravelling biological macromolecules with cryoelectron microscopy. Nature 537, 339–346 (2016).
 8.
Scott, M. C. et al. Electron tomography at 2.4angstrom resolution. Nature 483, 444–447 (2012).
 9.
Chen, C.C. et al. Threedimensional imaging of dislocations in a nanoparticle at atomic resolution. Nature 496, 74–77 (2013).
 10.
Xu, R. et al. Threedimensional coordinates of individual atoms in materials revealed by electron tomography. Nat. Mater. 14, 1099–1103 (2015).
 11.
Goris, B. et al. Measuring lattice strain in three dimensions through electron microscopy. Nano Lett. 15, 6996–7001 (2015).
 12.
Haberfehlner, G. et al. Formation of bimetallic clusters in superfluid helium nanodroplets analysed by atomic resolution electron tomography. Nat. Commun. 6, 8779 (2015).
 13.
Yang, Y. et al. Deciphering chemical order/disorder and material properties at the singleatom level. Nature 542, 75–79 (2017).
 14.
Cheng, Y. Singleparticle cryoEM at crystallographic resolution. Cell 161, 450–457 (2015).
 15.
Nogales, E. The development of cryoEM into a mainstream structural biology technique. Nat Methods 13 (2016).
 16.
Bartesaghi, A., Matthies, D., Banerjee, S., Merk, A. & Subramaniam, S. Structure of βgalactosidase at 3.2Å resolution obtained by cryoelectron microscopy. Proc. Natl. Acad. Sci. USA 111, 11709–11714 (2014).
 17.
Lučić, V., Förster, F. & Baumeister, W. Structural studies by electron tomography: from cells to molecules. Annu. Rev. Biochem. 74, 833–865 (2005).
 18.
Oikonomou, C. M. & Jensen, G. J. A new view into prokaryotic cell biology from electron cryotomography. Nat. Rev. Microbiol. 14, 205–220 (2016).
 19.
Lee, E. et al. Radiation dose reduction and image enhancement in biological imaging through equallysloped tomography. J. Struct. Biol. 164, 221–227 (2008).
 20.
Fahimian, B. P., Mao, Y., Cloetens, P. & Miao, J. Lowdose xray phasecontrast and absorption CT using equally sloped tomography. Phys. Med. Biol. 55, 5383 (2010).
 21.
Larabell, C. A. & Nugent, K. A. Imaging cellular architecture with Xrays. Curr. Opin. Struct. Biol. 20, 623–631 (2010).
 22.
Zhao, Y. et al. Highresolution, lowdose phase contrast Xray tomography for 3D diagnosis of human breast cancers. Proc. Natl. Acad. Sci. 109, 18290–18294 (2012).
 23.
Gibbs, J. W. et al. The threedimensional morphology of growing dendrites. Sci. Rep. 5 (2015).
 24.
Meirer, F. et al. Threedimensional imaging of chemical phase transformations at the nanoscale with fullfield transmission Xray microscopy. J. Synchrotron Radiat. 18, 773–781 (2011).
 25.
Krenkel, M. et al. Phasecontrast zoom tomography reveals precise locations of macrophages in mouse lungs. Sci. Rep. 5 (2015).
 26.
Miao, J., Charalambous, P., Kirz, J. & Sayre, D. Extending the methodology of Xray crystallography to allow imaging of micrometresized noncrystalline specimens. Nature 400, 342–344 (1999).
 27.
Miao, J. et al. Threedimensional GaNGa2O3 core shell structure revealed by Xray diffraction microscopy. Phys. Rev. Lett. 97 (2006).
 28.
Nishino, Y., Takahashi, Y., Imamoto, N., Ishikawa, T. & Maeshima, K. Threedimensional visualization of a human chromosome using coherent Xray diffraction. Phys. Rev. Lett. 102, 18101 (2009).
 29.
Jiang, H. et al. Quantitative 3D imaging of whole, unstained cells by using Xray diffraction microscopy. Proc. Natl. Acad. Sci. USA 107, 11234–11239 (2010).
 30.
Dierolf, M. et al. Ptychographic Xray computed tomography at the nanoscale. Nature 467, 436–439 (2010).
 31.
Jiang, H. et al. Threedimensional coherent Xray diffraction imaging of molten iron in mantle olivine at nanoscale resolution. Phys. Rev. Lett. 110, 205501 (2013).
 32.
Miao, J., Ishikawa, T., Robinson, I. K. & Murnane, M. M. Beyond crystallography: Diffractive imaging using coherent xray light sources. Science 348, 530–535 (2015).
 33.
Holler, M. et al. Highresolution nondestructive threedimensional imaging of integrated circuits. Nature 543, 402–406 (2017).
 34.
Gordon, R., Bender, R. & Herman, G. T. Algebraic Reconstruction Techniques (ART) for threedimensional electron microscopy and Xray photography. J. Theor. Biol. 29, 471–481 (1970).
 35.
Andersen, A. Simultaneous Algebraic Reconstruction Technique (SART): A superior implementation of the ART algorithm. Ultrason. Imaging 6, 81–94 (1984).
 36.
Gilbert, P. Iterative methods for the threedimensional reconstruction of an object from projections. J. Theor. Biol. 36, 105–117 (1972).
 37.
Miao, J., Förster, F. & Levi, O. Equally sloped tomography with oversampling reconstruction. Phys. Rev. B 72, 52103 (2005).
 38.
O’Connor, Y. Z. & Fessler, J. A. Fourierbased forward and backprojectors in iterative fanbeam tomographic image reconstruction. IEEE Trans. Med. Imaging 25, 582–589 (2006).
 39.
Fahimian, B. P. et al. Radiation dose reduction in medical xray CT via Fourierbased iterative reconstruction: Dose reduction in CT via Fourierbased iterative reconstruction. Med. Phys. 40, 31914 (2013).
 40.
Miao, J., Sayre, D. & Chapman, H. N. Phase retrieval from the magnitude of the Fourier transforms of nonperiodic objects. J. Opt. Soc. Am. A 15, 1662 (1998).
 41.
Miao, J. & Sayre, D. On possible extensions of Xray crystallography through diffractionpattern oversampling. Acta Crystallogr. A 56(Pt 6), 596–605 (2000).
 42.
Shepard, D. A twodimensional interpolation function for irregularlyspaced data. In 517–524, doi:10.1145/800186.810616 (ACM Press, 1968).
 43.
Franke, R. Scattered data interpolation: tests of some methods. Math. Comput. 38, 181–181 (1982).
 44.
Brünger, A. T. Free R value: crossvalidation in crystallography. Methods Enzymol. 277, 366–396 (1997).
 45.
Rossmann, M. G. Ab initio phase determination and phase extension using noncrystallographic symmetry. Curr. Opin. Struct. Biol. 5, 650–655 (1995).
 46.
Raines, K. S. et al. Threedimensional structure determination from a single view. Nature 463, 214–217 (2010).
 47.
Kirkland, E. J. Advanced computing in electron microscopy (Springer, 2010).
 48.
Dabov, K., Foi, A., Katkovnik, V. & Egiazarian, K. Image Denoising by Sparse 3D TransformDomain Collaborative Filtering. IEEE Trans. Image Process. 16, 2080–2095 (2007).
 49.
Huynh, D. Q. Metrics for 3D rotations: comparison and analysis. J. Math. Imaging Vis. 35, 155–164 (2009).
 50.
Lemberg, J. A. & Ritchie, R. O. MoSiB alloys for ultrahightemperature structural applications. Adv. Mater. 24, 3445–3480 (2012).
 51.
Rioult, F. A., Imhoff, S. D., Sakidja, R. & Perepezko, J. H. Transient oxidation of Mo–Si–B alloys: Effect of the microstructure size scale. Acta Mater. 57, 4600–4613 (2009).
 52.
Agulleiro, J.I. & Fernandez, J.J. Tomo3D 2.0 – exploitation of advanced vector extensions (AVX) for 3D reconstruction. J. Struct. Biol. 189, 147–152 (2015).
 53.
Dai, W. et al. Visualizing virus assembly intermediates inside marine cyanobacteria. Nature 502, 707–710 (2013).
 54.
Mastronarde, D. N. Dualaxis tomography: an approach with alignment methods that preserve resolution. J. Struct. Biol. 120, 343–352 (1997).
 55.
Youla, D. C. & Webb, H. Image restoration by the method of convex projections. IEEE Trans. Med. Imaging 1, 81–94 (1982).
 56.
Sezan, M. I. An overview of convex projections theory and its application to image recovery problems. Ultramicroscopy 40, 55–67 (1992).
 57.
Candes, E. J., Romberg, J. & Tao, T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006).
 58.
Donoho, D. L. Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
 59.
Rudin, L. I., Osher, S. & Fatemi, E. Nonlinear total variation based noise removal algorithms. Phys. Nonlinear Phenom. 60, 259–268 (1992).
 60.
Leary, R., Saghi, Z., Midgley, P. A. & Holland, D. J. Compressed sensing electron tomography. Ultramicroscopy 131, 70–91 (2013).
Acknowledgements
We thank M. Taylor and J. Perepezko for the preparation of the Mo_{3}Si alloy sample. This work was supported by STROBE: A National Science Foundation Science & Technology Center under Grant No. DMR 1548924, the Office of Basic Energy Sciences of the US DOE (DESC0010378), the NSF DMREF program (DMR1437263), ONR MURI (N000141410675), NIH (P41GM103832) and the DARPA PULSE program through a grant from AMRDEC. Work at the Molecular Foundry was supported by the Office of Science, Office of Basic Energy Sciences, of the U.S. Department of Energy under Contract No. DEAC0205CH11231. J.A.R. is supported as a Searle Scholar and a Beckman Young Investigator.
Author information
Author notes
Alan Pryor Jr. and Yongsoo Yang contributed equally to this work.
Affiliations
Department of Physics and Astronomy and California NanoSystems Institute, University of California Los Angeles, California, 90095, USA
 Alan Pryor Jr.
 , Yongsoo Yang
 , Arjun Rana
 , Marcus GallagherJones
 , Jihan Zhou
 , Yuan Hung Lo
 , Georgian Melinte
 & Jianwei Miao
Department of Chemistry & Biochemistry, UCLADOE Institute for Genomics and Proteomics, Los Angeles, California, 900951570, USA
 Marcus GallagherJones
 & Jose A. Rodriguez
Department of Bioengineering, University of California, Los Angeles, CA, 90095, USA
 Yuan Hung Lo
Institut de Physique et Chimie des Matériaux de Strasbourg, CNRSUniversité de Strasbourg, 23, rue du Loess, 67037 Cedex 08, Strasbourg, France
 Georgian Melinte
SLAC National Accelerator Laboratory and Departments of Bioengineering, Microbiology and Immunology, Stanford University, Stanford, California, 94304, USA
 Wah Chiu
Authors
Search for Alan Pryor Jr. in:
Search for Yongsoo Yang in:
Search for Arjun Rana in:
Search for Marcus GallagherJones in:
Search for Jihan Zhou in:
Search for Yuan Hung Lo in:
Search for Georgian Melinte in:
Search for Wah Chiu in:
Search for Jose A. Rodriguez in:
Search for Jianwei Miao in:
Contributions
J.M. directed the research; J.Z., W.C. and J.M. discussed and/or acquired the data; A.P.J., Y.Y. and J.M. discussed and developed the algorithm; A.P.J., Y.Y., A.R., M.G.J., J.Z. Y.H.L., G.M., J.A.R. and J.M. performed the numerical simulation and the reconstruction of the experimental data. A.P.J. and J.M. wrote the manuscript with contributions from all authors.
Competing Interests
The authors declare that they have no competing interests.
Corresponding author
Correspondence to Jianwei Miao.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.