## Abstract

We report a new computed tomography reconstruction method, named quantisation units reconstruction technique (QURT), applicable to electron and other fields of tomography. Conventional electron tomography methods such as filtered back projection, weighted back projection, simultaneous iterative reconstructed technique, etc. suffer from the ‘missing wedge’ problem due to the limited tilt-angle range. QURT demonstrates improvements to solve this problem by recovering a structural image blurred due to the missing wedge and substantially reconstructs the structure even if the number of projection images is small. QURT reconstructs a cross-section image by arranging grey-level quantisation units (QU pieces) in three-dimensional image space via unique discrete processing. Its viability is confirmed by model simulations and experimental results. An important difference from recently developed methods such as discrete algebraic reconstruction technique (DART), total variation regularisation—DART, and compressed sensing is that prior knowledge of the conditions regarding the specimen or the expected cross-section image is not necessary.

### Similar content being viewed by others

## Introduction

Electron tomography (ET) has the potential to analyse three-dimensional (3D) structures of nanomaterials^{1} and the 3D morphology of biological samples^{2,3}. Although ET is common in diverse fields, the ‘missing wedge’ problem due to the limited tilt-angle range remains a serious issue. This problem hinders the interpretation of a structure appearing in a cross-section image because it distorts the structural image and blurs the cross-section image.

Many proposals have been reported to address this issue, including those that are not directly for ET purposes. A unique mathematical operator named the lambda reconstruction operator demonstrates that streak artefacts caused by the Fourier integral calculus, which is due to the limited tilt-angle range, is mitigated in micro local analysis^{4,5}. An interesting method is an image denoising method based on a statistical approach^{6}. Briefly, white Gaussian noise is added to the 3D input tomogram volume, and the original 3D Fourier coefficients before the noise addition are substituted in the 3D spectrum after the noise addition. Next, a 3D denoising algorithm, basically BM4D (Block Matching 4D) algorithm^{7,8}, is executed. Due to its random nature, a smaller amount of added noise appears close to the signal. Consequently, the signal is preserved after denoising. This procedure is iteratively performed by adding an evaluation step based on a dedicated Markov Chain Monte Carlo framework. This article^{6} describes that the method makes the best statistical guess of what the missing data could be based on what has been observed. As a result, spectral recovery in the missing wedge domain is verified.

Recently, sparse modelling and regularisation methods have been applied to various restoration and inverse problems. For ET to solve this issue, methods of the discrete algebraic reconstruction technique (DART)^{9,10}, total variation regularisation (TVR)—DART^{11}, and compressed sensing (CS)^{12,13,14,15} have been proposed. Under the sparse condition given in each method based on prior knowledge regarding the specimen or the expected cross-section image, artefacts due to the issue are reduced. These methods also decrease the required number of projection images. However, even with the aforementioned advances, the missing wedge problem has yet to be fully resolved.

To overcome this deficiency, machine learning methods for image restoration have recently been proposed^{16,17}. These methods require a very large collection of image patch pairs between artificial and artefact-free ones for a learned dictionary. On the other hand, this study aims to improve the reconstruction method itself.

Our research strives to create a unique reconstruction method to fully address this problem^{18,19}. In the early stage, single component objects were the targets to be reconstructed. They were a Pt super crystal^{18} and Pt/C replica specimens of biological samples^{19}. In principle, the projection image intensity distribution of such an object is proportional to the thickness distribution of the object along the tilt direction. A unique reconstruction method was devised by modifying the simultaneous iterative reconstructed technique (SIRT) algorithm to reconstruct a binary cross-section image. This image then becomes the rough object-occupied region when the process converges. A final real cross-section image is obtained with a regular SIRT, where the image reconstruction is limited inside the region. As a result, the missing wedge artefact is considerably reduced because the region to be reconstructed is limited. The way of this thinking has been extended toward the development of the present method.

Our method in this paper does not require prior knowledge or conditions regarding the specimen or the expected cross-section image. It only needs tilt series images and tilt angle data. From the user’s viewpoint, the proposed method can be categorised as a method similar to filtered back projection (FBP)^{20,21}, weighted back projection (WBP)^{20}, SIRT (e.g.,^{22}), etc. Nevertheless, our method provides powerful effects. It can recover a structural image blurred by a missing wedge and reconstruct the structure even if the number of projection images is small.

In the proposed method, the interpretation of an image itself is redefined. A digital image is created by a quantisation process of an analogue image. Then the grey level in a pixel due to quantisation can be defined as the number of quantisation units (QUs). As shown in Fig. 1 the digital image is represented in a 3D space whose axes are two-dimensional (2D) image coordinates and the number of QUs. Therefore, the image can be made by stacking a finite number of QU pieces. This finite number equals a numerical value for the integration of a cross-section image.

As shown in Fig. 2a, the number of QU pieces can be calculated by a numerical integration of the projection data [one-dimensional (1D) as a function of the ‘x’ coordinate] independent of the tilt direction. In addition, the projection data is the distribution of the number of QU pieces in each x belonging to the ray line perpendicular to x within the cross-section image. Under these constraints for the number of QU pieces, we devise a new reconstruction method, called QURT (Quantisation Units Reconstruction Technique), where innumerable QU pieces are sophisticatedly arranged in the 3D image space of the cross-section plane. The above limited number of QU pieces can successfully reconstruct the cross-section image and is unaffected by the missing wedge. This constraint condition and the novel manner using the QU pieces is applicable even if the number of projection images is small.

## Results

### Constraint condition and the grey-level QU

A basic strategy to overcome the missing wedge problem is to determine useful prior knowledge^{23} and devise a calculation algorithm that satisfies the knowledge. In our method (QURT), the integrated value of the projection data (1D distribution) is effective knowledge as it is equal to the 2D integrated value of a cross-section image (Fig. 2a). This relationship is accurate for any projection direction in the ideal case. That is, the object is projected inside the imaging plane without missing parts in the tilt direction.

This relationship can be understood as follows. In linear projection theory, the projection data \(p_{\theta } \left( {x^{{\prime }} } \right)\) of the cross-section image \(f\left( {x,z} \right)\) is obtained by integrating with \(f\left( {x,z} \right)\) along the line perpendicular to the x’ axis, where the line is parallel to the tilt direction θ. Therefore, the 2D integration of \(f\left( {x,z} \right)\) is equal to the 1D integration of \(p_{\theta } \left( {x^{{\prime }} } \right)\) along x′, and the integrated value is a constant, which is independent of the tilt angle as (Fig. 2a)

To adapt the constraint condition in Eq. (1), we devised a novel method to reconstruct a cross-section image. The constant value obtained by Eq. (1) is approximately given as an integer number ‘N’. Here, it is recognised that a digital image is made by quantisation. If a grey-level QU is defined as ‘q’ with a small enough value, a pixel grey value is the number of stacked units (QU pieces). Then the total number of QU pieces needed to generate the cross-section image is approximated to N as

where \(n_{\theta } \left( {x_{j}^{{\prime }} } \right)\) is the distribution of the number of QU pieces in each \(x_{j}^{{\prime }}\). j and k are index numbers for discrete coordinates x′ and z′, respectively. z′ is the coordinate parallel to the tilt direction θ.

### Reconstruction with grey-level QUs

Here we describe our reconstruction method (QURT) using the finite number N of QU pieces. QURT correctly arranges all QU pieces in the cross-section image plane (x, z) by stacking along the grey-level direction while minimising errors of the projection data for all tilt angles.

First, a certain tilt angle is selected as a base angle ‘Θ’ for the reconstruction. The total number N of QU pieces (integer number) is determined as

Thus, the number of QU pieces is determined in each \(x_{j}^{{\prime }}\) column. This is the constraint condition.

The next step is determining how to place the finite number of QU pieces in the cross-section image plane (x′, z′). The basic idea is derived from a study regarding the error between a single QU piece put in a (x′, z′) pixel position and all given projection data (see Fig. 2b). The QU piece must be placed in a specific position to minimise the error. A certain error amount is added when the projection data for all tilt directions changes by placing a QU piece in a (x′, z′) position. If the above-mentioned error is the total value summing up the differences to subtract the given projection data from the current calculated projection data for all tilt directions, a QU piece should be placed in the largest negative error position because the error in the (x′, z′) position indicates the degree of a lack of the QU piece. If the error value is the plus amount, the QU in that position should be erased. This error is calculated as a map of \(E_{map} \left( {x,z} \right)\) in the cross-section image plane with a simple back projection of all such differences in the projection data described above. Finally, when the correct cross-section image (correct arrangement of QU pieces) is reconstructed, the \(E_{map} \left( {x,z} \right)\) becomes a zero distribution. Therefore, determining the method to place the finite number of QU pieces to let \(E_{map} \left( {x,z} \right)\) become zero is important. \(E_{map} \left( {x,z} \right)\) is given as

where \(BP[]\) is a simple back-projection, which is equal to FBP without filtering. \(E_{{\theta_{i} }} \left( {x^{{\prime }} } \right)\) is the error distribution to subtract the given projection data \(p_{{\theta_{i} {\text{ exp}}{.}}} \left( {x^{{\prime }} } \right)\) from the current calculated projection data \(p_{{\theta_{i} \;{\text{cal}}{.}}} \left( {x^{{\prime }} } \right)\). \(FP[]\) is a projection calculation in the θ_{i} direction. \(g\left( {x,z} \right)\) is the current reconstructed image in the iterative procedure. Altering the QU piece arrangement changes \(E_{map} \left( {x,z} \right)\) nonlinearly. Due to the complicated arrangement even if the change is limited to one pixel position only, the simple back projection alters the map of the whole area of the cross-section image plane.

Before providing the entire explanation, we describe the basic procedure to place the QU pieces. In the procedure, they are placed one-by-one in the negative maximum error position in the current error map of \(E_{map} \left( {x,z} \right)\). The error map is updated after each QU placement. This process is repeated until the number of QU pieces reaches the limit of N under the second equation in Eq. (3) (Fig. 3).

Because the basic procedure is only applicable to a very simple binary model (Fig. 3a), we improved the procedure so that it can accommodate general specimens. All procedures have the same purpose, which is to converge into the correct cross-section image. In other words, all are measures that avoid a local solution trap (e.g., the local minimum error) in the nonlinear optimisation. The major improved procedure changes the size and the value of the QU piece in the iterative reconstruction.

The first improvement has the following steps. First assign the QU piece to a large size (e.g., 32 pixels square). Second, reduce the size in a stepwise manner until it reaches a one-pixel size. Once the size is unchanged at the one-pixel level, the cross-section image becomes an artificial granular image composed of many QU dots (Fig. 4b,e). The process of placing a one-pixel size QU piece induces a change in \(E_{map} \left( {x,z} \right)\) with one-pixel fineness. In a nonlinear optimisation, this often falls into a local minimum because the manner is too fine.

Similar to the above procedure for the size of the QU piece, the value (q) of QU is roughly changed stepwise until it reaches the final smallest value. In practice, the final smallest value of q is set to one. According to the adjustment need of the image intensity, the value of the projection data is amplified. Note that the change in the q value helps reduce the calculation time. Even if the value is kept at one during the iterative reconstruction, the cross-section image is correctly reconstructed. However, a long calculation time is necessary.

In the second improvement, the QU pieces are rearranged to further reduce the error after the number of QU reaches the defined N. This rearrangement is realised as QU piece movements. Each movement is performed inside a certain \(x_{j}^{{\prime }}\) column while maintaining the condition of \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\). The movement is iteratively repeated by moving the one with the largest reducible error based on the current \(E_{map} \left( {x,z} \right)\). As shown in Fig. 4g,h of the model simulation, the first and the second improved procedures produce the desired effect, but the error map does not sufficiently converge into the flat zero distribution (see Fig. 4i, arrows). Instead, mosaic-like images are observed in the final reconstructed image (Fig. 4h).

Thus, a third improvement procedure is introduced, where plural base tilt angles Θ are set to generate a cross-section image solution according to each constraint condition of \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\). After the output of each solution, the current solution is updated by averaging their plural ones. Figure 5 shows a typical reconstruction process incorporating all the improved procedures.

### Flow of the procedure

Below the reconstruction procedure is described and the procedure is shown in block flow in Fig. 6. (a) Set the initial size of the QU piece (32-pixel square as a standard) and the initial value (q) of QU (typically with an nth power of 2). Subsequently, null is set in the initial cross-section image. (b) Select the base tilt angle Θ. (c) Determine the constraint conditions for the total number N of QU pieces and the distribution of the number of QU as a function of \(x^{\prime}_{j}\) column \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\). (d) Calculate \(E_{map} \left( {x,z} \right)\). The map indicates the degree (absence or excess) of the QU piece in every pixel calculated from all the projection data and the current projection data of the most recent cross-section image. (e) Place many QU pieces to the number of N one-by-one in the maximum error pixel in the current error map of \(E_{map} \left( {x,z} \right)\), which is updated after each QU placement under the condition of \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\). (f) Selectively repeat QU piece movements inside the certain \(x^{\prime}_{j}\) column while maintaining the condition of \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\) until a predefined convergence condition, which further reduces the error between the current reconstructed image and all given projection data, is reached. (g) Store the result of (f) into a memory, and repeat steps (b) to (f) while changing Θ to the predetermined other plural base tilt angles. When this repeat is finished, average the results to update a reconstruction result for the current QU piece size and for the current QU value (q). Repeat these reconstruction steps for Θ and average the repeated results for every QU piece size until converging on the final pixel size and q value of one. For example, the size and the value decrease step-by-step as follows: (32 and 8), (32 and 1), (16 and 8), (16 and 1), (in sequence), and the last (1 pixel and the value 1).

### Examine with models

As an underlying examination, the basic reconstruction procedure described above was applied to a binary model (Fig. 3a). This model is made with a simple doughnut and bar. The base tilt angle Θ is 0° (the vertical direction in the model). The test condition is that the tilt angle ranges ± 60° with a step angle of 2°. In process (c) described in the above section, the constraint conditions of the total number N of QU pieces and the distribution of the number of QU as a function of \(x^{\prime}_{j}\) column \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\) are determined from the calculated projection data of the model. Figure 3e–h depict the iterative reconstruction process in the procedure (e) with a 1-pixel size QU piece (value: 1), while Fig. 3b–d show the corresponding \(E_{map} \left( {x,z} \right)\), where the white dots denote the QU pieces. Every \(E_{map} \left( {x,z} \right)\) in these processes has a negative distribution (black colour), where the depth of black is proportional to the negative value. In each iteration, the QU pieces are placed in high negative regions of the map, and the map gradually changes toward the flat zero distribution. Although \(E_{map} \left( {x,z} \right)\) is incomplete and distorted due to the limited tilt-angle range, the model is almost perfectly recovered (Fig. 3h). Figure 3i,j show the near ideal \(E_{map} \left( {x,z} \right)\) and an example of a halfway result in the iterative reconstruction for a condition of ± 90° with 2° steps, respectively.

The basic reconstruction procedure fails when applied to general models and experimental samples. Figure 4 shows the result when applying the general model (Fig. 5a) to various structural elements. For the limited tilt-angle range (± 70° with 5° steps), many QU dots create an artificial granular image, which is almost limited to inside the model (Fig. 4e). Even for a case of the full tilt-angle range (± 90° with 5° steps), a similar artificial granular image is reconstructed, but a well-balanced image can be made from the constant dot density (Fig. 4b).

We then applied the improved procedures described in the ‘Flow of the procedure’ section. Figure 5 shows the results of the improved procedure using the same test conditions as those for Fig. 4e. Three base tilt angles Θ are utilised − 70°, 0°, and + 70°, and the initial size of the QU piece is 32 pixels squared. The size is reduced by half stepwise until it reaches 1 pixel. In all processes, the value of QU is kept at 1. The convergence condition is that the change rate of the reconstructed image measured in intervals of 100 iterations is less than 1.0E − 6. Figure 5e–h show the results of the iterative reconstruction progress.

This stepwise reconstruction from rough to fine avoids the local solution trapping problem. In each step, QU pieces are rearranged after the first arrangement of the necessary number QUs specified in the constraint condition. This refinement effectively avoids the problem. Furthermore, by selecting plural base tilt angles of Θ, an average solution can be used to avoid the problem because different plural constraint conditions regarding \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\) can be set. To clearly highlight this effect, Fig. 4g,i show the result by selecting a single base tilt angle of Θ (0°). Although the halfway reconstructed images until 16 pixels size do not differ significantly from the plural base tilt angles case (Fig. 5f), as the QU piece size is decreased below 16, the level of convergence is relatively worse because some mosaic-like images appear (Fig. 4h).

Finally, these improvements realize the optimum solution (Fig. 5h). Compared to Fig. 4f,i, the corresponding error map (Fig. 5l) converges to the flat zero distribution. Figure 5b shows the corresponding result reconstructed with the conventional method (SIRT). The missing wedge artefact is seen in the Fourier transform pattern (Fig. 5c) of the image. On the other hand, QURT result shows no such artefact (Fig. 5d), which is the Fourier transform of Fig. 5h. Hence, the Fourier spectra are generated in the missing part.

Figure 7a shows the convergence process from the viewpoint of the transition of the average reconstruction error as a function of the QU piece size, which is related to the image definition and corresponds to Fig. 5f–h. Around a QU size of four pixels, the error is not significantly decreased, and the reconstructed structure roughly appears at this step. This feature is also observed experimentally. Figure 7b shows the transition of the number of QUs (N) needed for the arrangement and the number of rearranged QUs in refinement procedure (f) in the ‘Flow of the procedure’ section. The rate of the increasing number of N between the image definition steps is four. There is a large difference in the number of rearranged QUs between the single base tilt angle and the plural ones. Because the difference in the errors for the same choice is small, as shown in Fig. 7a, the artificial mosaic-like pattern shown in Fig. 4h is removed by rearranging a large number of QUs.

We examined various test conditions on the tilt-angle range and the step angle (Fig. 8). For comparison, Fig. 9 show the results using conventional methods of SIRT and FBP. The reconstructed results of QURT are deformed little by little from the model (Fig. 5a) due to the missing wedge. This condition deviates from the most ideal one with the ± 90° tilt angle range and a 2° step angle because the number of possible solutions (freedom of solutions) increases. However, compared to the conventional methods, QURT recovers the model relatively well.

This result can be numerically verified by calculating the average error between the model and the reconstructed results (Table 1). As a typical result, when considering the existing installation of a specimen tilting holder, which has an allowable tilt angle range of about ± 70°, our results demonstrate that the missing wedge artefact is negligible, and the structure is reconstructed almost correctly if the step angle is 5°. In addition, artificial images such as streaks and artificial background do not appear. It is noteworthy that the edge parts are clearly resolved compared with those reconstructed with the conventional methods.

When the step angle is 10°, five plural base tilt angles of − 70°, − 50°, 0°, + 50°, and + 70° are adopted because the quality of the reconstructed images is improved compared to that using the standard three plural base tilt angles (Fig. 8). Various tests indicated that a selection of ± 50° is suitable. Considering more than five plural base tilt angles did not result in further improvement. Additionally, when the step angle is 2° or 5°, three base tilt angles is sufficient to obtain the optimum reconstruction image.

## Experiments

QURT was applied to a practical sample of a TiN-Ag nanocomposite. Because nanocomposite particles have clear edges and geometrical forms, the sample is suitable to examine the recovery of blurred structural parts due to conventional reconstruction methods. Figure 10a shows the original STEM image (High-Angle Annular Dark Field (HAADF) image) taken from the 0° direction. The sample contains many dice-like particles. Then we acquired a series of tilt images where the range was ± 70° and the step angle was 2°.

Prior to the experimental application, the projection images of the series were adjusted. That is, the background intensity corresponding to the vacuum region, an unnecessary specimen support film image, the background of the image acquisition system, etc. were roughly subtracted from the image data. Then the image intensity was properly amplified by approximating the projection data as large integers, which were equivalent to the number of QU pieces. Although accurately subtracting the background intensity is impossible in many cases, it is necessary to get as close as possible to the theoretical constraint conditions of Eq. (3). Figure 10c shows a cross-section image of a typical section within the largest particle reconstructed using QURT and all STEM images (71 images).

QURT clearly reconstructs the base TiN dice particle. In contrast, the conventional SIRT method provides blurred sides of the contour (horizontal edges of the square shape) (Fig. 10e). QURT is unaffected by the missing wedge artefact, and artificial background images such as streaks and clouds disappear. Additionally, the fine Ag grains clearly spread on the surface of the TiN dice. The physical and material analyses are described in detail elsewhere^{24}. This sharpness is a superior feature compared to conventional methods. Figure 10g,k show other cross-section images with similar features, which can be compared to the SIRT results in Fig. 10i,l, respectively.

QURT can reconstruct a structure even if the number of the tilt series images is small (Fig. 10d,h). These can be compared to the SIRT results in Fig. 10f,j, respectively. These reconstructions were calculated assuming a tilt range of ± 64° and a step angle of 8°. The SIRT blurs the structures and generates artificial background, whereas QURT well reconstructs the structures even in the worst condition. In these cases (Fig. 10d,h), five plural base tilt angles of − 64°, − 48°, 0°, + 48°, and + 64° are selected due to the improved image quality compared to that using three angles. The selection of ± 48° is determined from various tests.

## Discussion

We devised a unique method (QURT) to reconstruct a cross-section image by arranging grey-level QU pieces in a 3D image space. In the first stage (basic procedure), the predetermined number of QU pieces are placed in the negative maximum error position one-by-one in the current error map of \(E_{map} \left( {x,z} \right)\) (procedure (e)). Then they are rearranged to further reduce the error amount of \(E_{map} \left( {x,z} \right)\) (procedure (f)). Because the QU pieces arrangement depends on \(E_{map} \left( {x,z} \right)\), this map plays an important role. However, the map is affected by the missing wedge because the initial map of \(E_{map} \left( {x,z} \right)\) is calculated with the simple back projection of all data in a limited angular range.

Nevertheless, the influence on the cross-section image is minimal. In the initial stage of the basic procedure, because the error amount in the affected parts of \(E_{map} \left( {x,z} \right)\) by the missing wedge is relatively small compared to those in the structural parts (Fig. 3), QU pieces are not arranged in the affected parts. Hence, the error amount in both the structural parts and the affected parts gradually becomes smaller as the QUs are not arranged in the affected parts (Fig. 3). Eventually all the QU pieces are placed, and the missing wedge artefact in the cross-section image is avoided.

This reason is essentially valid for general specimens. However, the three improvement procedures described in the ‘Reconstruction with grey-level QUs’ section are applied. Consequently, the missing wedge artefact is avoided for the general model and the experimental sample (Figs. 5, 8, 10). A remarkable result is the recovery of the Fourier spectra in the missing part (Fig. 5d). This is attributed to the restoration of the horizontal edges in the model. Moreover, the spectra in every gap between a radial line pair in the Fourier transform are recovered (Fig. 5d). In conventional reconstruction methods, Fourier spectra are limited to the radial lines based on the projection slice theorem (Fig. 5c). On the other hand, QURT is based on the arrangement of QU pieces. Consequently, the Fourier spectra do not have limitations on the radial line and the Fourier transform of the cross-section image are fully reconstructed with QURT. Therefore, QURT produces the typical ring-like diffraction pattern generated from the doughnut model even though the pattern in the missing wedge region is incomplete (Fig. 5d).

The grey-level distribution inside the general model (Fig. 5a) gradually changes from dark to bright. This is intentional to examine the applicability of the present method. The reconstruction simulation confirms that the distribution changes slightly from the model, depending on the deterioration of the tilt-angle condition. However, compared to conventional methods, QURT recovers the grey-level distribution relatively well.

This characteristic is favourable because QURT is also applicable to structures unsuitable for the sparse model adopted in CS, DART, and TVR-DART, which are based on the sparse approximation. In CS, the model is obtained by selecting the proper sparse transform such as by selecting the regularisation parameters^{13}. In TVR-DART, the sparsity is given by prior knowledge about distinct material compositions within the specimen and the boundaries between different compositions in the grey-level. This algorithm starts with an initial solution for the reconstruction using either SIRT or Total Variation minimisation^{11}. On the other hand, QURT does not require a specific model, approximation, or prior knowledge regarding the specimen or the expected cross-section image. It only needs the tilt series images and tilt angle data. Therefore, it is easy to use.

QURT can be categorised as a basic reconstruction method similar to FBP, WBP, and SIRT. We think that TVR-DART and CS methods are in another category of active methods, which are distinct from QURT. To improve the performance of QURT in the near future, for very poor condition projection data, it might be possible to combine QURT with an active method or to process a QURT result using an active method. In QURT, the effect of noise on projection data is mentioned below. In the algorithm, each QU size reconstruction is executed for every base tilt angle Θ which is typically three angles, but can include up to five angles, and all reconstructed results obtained from the plural Θs are averaged before the next reduced QU size reconstruction. This averaging is effective for noise reduction because each constraint condition of \(n_{\Theta } \left( {x_{j}^{{\prime }} } \right)\) for Θ is affected by the noise on projections. Depending on the noise level, if the number of Θ is increased, the efficiency should be improved. Verification of this hypothesis will be reported in the near future. For further denoising, QURT may need to be combined with an active method such as the regularisation method, which is also a future research theme.

As described in the ‘Experiments’ section, the present method has realised useful results. First, it avoids both the missing wedge artefact and artificial background images. Second, it provides clear edges. These features are very useful to segment plural structural parts in post-image processing and in 3D CG demonstration of the reconstructed structure.

QURT reconstructs the structure even if the number of projection images is small, but partial incompleteness may occur. QURT is effective for applications to specimens that are susceptible to electron irradiation damage, and also effective for X-ray energy-dispersive spectroscopy (XEDS) tomography. Currently, the calculation requires a long time. In addition, for large sized projection and cross-section images, QURT requires a very large number of QU pieces. This demands a certain level of computer power. However, as computer processing technologies advance in the near future, the applicability of QURT should increase because our proposed algorithm is simple and straightforward. Although our algorithm is steady, there are alternatives to the one-by-one QU arrangement or movement method proposed here to speed up the calculation. For example, some optimisation algorithms treat the whole QU arranged distribution as one multidimensional vector. Then a cross-section image is reconstructed by obtaining the optimum solution of the vector with an algorithm such as gradient descent type one. We are planning to investigate other optimisation algorithms to improve the performance. Additionally, we will also attempt to realize adaptive change of the QU size based on the the error map \(E_{map} \left( {x,z} \right)\) and an evaluation of the current error reducing rate during the iterative procedure.

## Methods

### Simulation models

A simple binary model (Fig. 3a) and a general model (Fig. 5a) were originally developed, and they are obviously affected by the missing wedge in the reconstruction using conventional methods. The binary model consisted of a doughnut and bar shape, which are assumed to be reconstructed using QU pieces with a value of 1. The general model consisted of a thick doughnut and a rectangular shape. Small holes and cones were added. The model was made with smooth grey-level distributions and steep edges. The maximum value of the model was 255, and reconstruction with QURT gave a QU piece value of 1.

### Reconstruction programs

The programs needed for QURT were custom written in our laboratory using the C++ programming language and Intel (Intel) MPI (Message Passing Interface) library (Parallel Studio XE Composer Edition for C++ Windows 2019). FBP and SIRT reconstructions were also performed using programs written in our laboratory based on the established algorithms (e.g.,^{20–22}). The MPI library was used for the SIRT program to reduce the calculation time.

### Experimental data

TiN–Ag (63.98wt%TiN–36.02wt%Ag) nanocomposites were synthesised by the dc arc-plasma method, which employed a specific arc-melting chamber. The details of the chamber and the method are described elsewhere^{24,25,26}. The nanoparticles collected from the chamber after the appropriate treatment were ultrasonically dispersed in alcohol and mounted on an amorphous carbon coated Cu grid for characterisation by transmission electron microscopy (TEM). For ET, a tilt series of projections by the STEM-HAADF method was acquired with the specimen tilt varying from − 78° to 70°. Data was recorded every 2° using a TEM operated at 300 kV (Titan 80–300, FEI, Netherlands). A STEM-HAADF detector with a specially designed high-tilt holder were attached to the TEM (E.A. Fischione Instruments, U.S.A.).

### Alignment of the tilt series

The tilt series images were accurately aligned with the method proposed in the previous paper^{27}. The alignment was based on the volume cross-correlations between pairs of filtered back-projected ray volume data calculated from neighbouring projection images. The method required neither markers nor image feature points traceable through the tilt series. (See the paper for details.) A remarkable efficiency was the ability to align the series even though the step angle was larger than the normal case of a few degrees. For example, it was practically performed for steps of 8° and 10°. This is expedient when the number of projection images is small with a large step angle. Additionally, the misalignment of the tilt-axis was accurately measured, and the series was realigned with the method proposed in a previous paper^{28}. This was also performed with a similar volume cross-correlation method. In practice, the misalignment of the tilt-axis was − 3.1° (counterclockwise to the vertical axis) for the present experimental tilt series.

## References

Midgley, P. A. & Weyland, M. 3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography.

*Ultramicroscopy***96**, 413–431 (2003).Frank, J.

*Electron Tomography*(Plenum Press, New York, 1992).Baumeister, W., Grimm, R. & Walz, J. Electron tomography of molecules and cells.

*Trends Cell Biol.***9**, 81–85 (1999).Frikel, J. & Quinto, E. T. Characterization and reduction of artifacts in limited angle tomography.

*Inverse Probl.***29**, 125007 (2013).Frikel, J. & Quinto, E. T. Artifacts in incomplete data tomography with applications to photoacoustic tomography and sonar.

*SIAM J. Appl. Math.***75**, 703–725 (2015).Moibel, E. & Kervrann, C. A. Monte-carlo framework for missing wedge restoration and noise removel.

*J. Stuct. Biol. X***4**, 100013 (2020).Dabov, K., Katkovnik, V. & Egiazarian, K. Image denoising by sparse 3-D transform-domain collaborative filtering.

*IEEE Trans. Image Process.***16**, 2080–2095 (2007).Maggioni, M., Katkovnik, V. & Egiazarian, K. Nonlocal transform-domain filter for volumetric data denoising and reconstruction.

*IEEE Trans. Image Process.***22**, 119–133 (2013).Bals, S., Batenburg, K. J., Verbeeck, J., Sijbers, J. & Van Tendeloo, G. Quantitative three-dimensional reconstruction of catalyst particles for bamboo-like carbon nanotubes.

*Nano Lett.***7**, 3669–3674 (2007).Batenburg, K. J.

*et al.*3D imaging of nanomaterials by discrete tomography.*Ultramicroscopy***109**, 730–740 (2009).Zhuge, X.

*et al.*Automated discrete electron tomography—towards routine high-fidelity reconstruction of nanomaterials.*Ultramicroscopy***175**, 87–96 (2017).Saghi, Z.

*et al.*Three-dimensional morphology of iron oxide nanoparticles with reactive concave surfaces. A compressed sensing-electron tomography (CS-ET) approach.*Nano Lett.***11**, 4666–4673 (2011).Leary, R., Saghi, Z., Midgley, P. A. & Holland, D. J. Compressed sensing electron tomography.

*Ultramicroscopy***131**, 70–91 (2013).Saghi, Z.

*et al.*Compressed sensing electron tomography of needle-shaped biological specimens—potential for improved reconstruction fidelity with reduced dose.*Ultramicroscopy***160**, 230–238 (2016).Guay, M. D., Czaja, W., Aronova, M. A. & Leapman, R. D. Compressed sensing electron tomography for determining biological structure.

*Sci. Rep.***6**, 27614–27627 (2016).Zhang, H., Li, L., Quio, K., Wang, L., Yan, B., Li, L. & Hu. G. Image predictin with limited angle tomography via deep learning with convolution network. arXiv: 1607.08707 (2016).

Buchholz, T., Jordan, M., Pigino, G. & Jug, F. Cryo-care: content-aware image restoration for cryo-transmission electron microscopy data. arXiv: 1810.05420 (2018).

Yamasaki, J., Tanaka, N., Baba, N., Kakibayashi, H. & Terasaki, O. Three-dimensional analysis of platinum super-crystals by TEM and HAADF-STEM observations.

*Philos. Mag.***84**, 2819–2828 (2004).Baba, N. & Katayama, E. A novel “ghost”-free tomographic image reconstruction method applicable to rotary-shadowed replica specimens.

*Ultramicroscopy***108**, 239–255 (2008).Radermacher, M. Weighted back-projection methods. In: Electron Tomography (ed. Frank J.) Plenum Press, pp. 91–115 (1992).

Sandberg, K., Mastronarde, D. N. & Beylkin, G. A fast reconstruction algorithm for electron microscope tomography.

*J. Struct. Biol.***144**, 61–72 (2003).Radermacher, M. Three-dimensional reconstruction of single particles in electron microscopy. In: Image Analysis in Biology CRC Press, pp. 219–249 (1992).

Carazo, J. M. The fidelity of 3D reconstructions from incomplete data and the use of restoration methods. In: Electron Tomography (ed. Frank J.) Plenum Press, pp. 117–164 (1992).

Kitawaki, K.

*et al.*Fabrication and characterization of TiN–Ag nano-dice.*Micron***40**, 308–312 (2009).Sakka, Y. & Ohno, S. Hydrogen sorption-desorption characteristics of mixed and composite Ni–TiN nanoparticles.

*Nanostruct. Mater.***7**, 341–353 (1996).Sakka, Y., Okuyama, H., Uchikoshi, T. & Ohno, S. Characterization of degraded surfaces of Al and AlN ultrafine powders.

*Nanostruct. Mater.***5**, 577–588 (1995).Tomonaga, S., Baba, M., Yamazaki, S. & Baba, N. A new field-of-view auto tracking method for online tomography reconstruction based on back-projected ray image cross-correlation.

*Microscopy***63**, 357–369 (2014).Tomonaga, S., Baba, M. & Baba, N. Alternative automatic alignment method for specimen tilt-series images based on back-projected volume data cross-correlations.

*Microscopy***63**, 279–294 (2014).

## Acknowledgements

This work was supported by grants from JSPS Grant-in-Aid for Scientific Research to N.B. (21310075, 24510156, 15H03538 and 18K04891). The authors would like to thank the Japan Science and Technology Agency (JST) for support (JST-CREST Grant No. JPMJCR18R2). This work was partly supported by JSPS KAKENHI Grant No. JP 18K18953.

## Author information

### Authors and Affiliations

### Contributions

N.B. and M.B. designed the research. N.B. wrote the computer code and performed the computations. K.K. performed the experiments and analysed the results. M.B. designed the simulation models. N.B. wrote the paper with guidance from K.K. and M.B.

### Corresponding author

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Additional information

### Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Baba, N., Kaneko, K. & Baba, M. Novel nonlinear reconstruction method with grey-level quantisation units for electron tomography.
*Sci Rep* **10**, 20146 (2020). https://doi.org/10.1038/s41598-020-77156-1

Received:

Accepted:

Published:

DOI: https://doi.org/10.1038/s41598-020-77156-1

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.