Introduction

Over the past decade, advancements in X-ray sources, optics and detector technologies have led to a dramatic increase in the volume and data quality of experimental powder diffraction patterns1,2,3,4,5,6,7. These technical advances are beginning to make high-throughput powder diffraction measurements a reality not just at synchrotron facilities but also at the laboratory8,9. It is now not uncommon to acquire diffraction patterns with sufficient signal-to-noise ratio in matters of minutes with laboratory diffractometers and in milliseconds at X-ray diffraction (XRD) dedicated beamlines at synchrotron facilities. In situ and operando ultra-fast and/or spatially-resolved multi-dimensional XRD experiments of functional materials and devices previously considered technically infeasible have already been demonstrated10,11. For example, we have previously demonstrated the first 5D operando tomographic diffraction imaging experiment (three spatial, one scattering and one dimension to denote time/imposed state) to study a multi-component catalytic reactor for the partial oxidation of methane12. However, these advances come at a cost and this is related to the handling of the big data collected during these experiments.

It is now possible to acquire many Terabytes (TBs) of XRD data per dynamic experiment and this is expected to increase significantly with the advent of the fourth generation synchrotron facilities all around the world, such as the Extremely Brilliant Source (EBS) of the European Synchrotron Radiation Facilities (ESRF) and MAX IV (Sweden) and the scheduled upgrades for Diamond-II (United Kingdom), Petra IV (Germany) and Advanced Photon Source-Upgrade (USA)13. It is currently well-accepted that it is the data analysis that is emerging as the bottleneck for measurement science and not the data acquisition and/or the experiment itself. Conventional data analysis methods, such as least-squares minimisation approaches, are not able to keep up with the data collections rates and there is a need for alternative methods which can provide both fast and accurate results14. As an example, the Rietveld method, often employed in XRD data when performing full profile analysis, can yield invaluable physicochemical information regarding the material system under investigation (e.g., lattice parameters, crystallite sizes/strain, atom/site occupancies and weight percentages for mixed phase systems) but does not scale well with big data. Equally important is the data crunching/visualization capabilities which are not developed in tandem with the data acquisition capabilities and as a result dynamic beamtime experiments are largely driven ‘blind’.

In recent years, deep learning methods involving convolutional neural networks (CNNs) have gained popularity and attracted a lot of attention due to the aforementioned properties and also due to their unique scalability (i.e., ability to handle big data)15,16,17. Although primarily explored for applications in medical imaging and tomography18,19, their potential for materials science and especially for spectroscopic/scattering techniques is gaining momentum20,21,22,23,24,25,26,27,28. Regarding XRD, previous studies have mainly focused on developing CNNs for crystal structure prediction, space group classification and phase identification19,29,30,31,32,33,34. For example, Park et al. demonstrated a classification CNN model that can extract crystal-system and space-group information from an XRD pattern29. Lee et al. proposed a CNN structure used for XRD pattern identification, which also has potential to predict phase fractions19. Wang et al. demonstrated a phase identification CNN which was trained with simulated patterns with noise extracted from experimental data34. Tatlier used an Artificial Neural Network (ANN) to predict the structures of crystalline materials and zeolites from XRD patterns and it was claimed that the ANN performed better than manual regression35. There are also some algorithms that can achieve phase identification without neural networks, but they are generally less efficient and/or less accurate than neural network models36,37,38,39,40,41. It should be pointed out though that most studies focus on developing classification models for phase identification; predicting whether a phase is present or not in an XRD pattern.

In this work, we demonstrate a neural network for parameter quantification (PQ) that we term PQ-Net; a regression CNN model applied in 1D powder diffraction data for extracting physico-chemical information. Specifically, PQ-Net is trained with simulated 1D XRD patterns and it is shown that it can yield accurate predictions for scale factors, crystallite sizes and lattice parameters for both simulated and experimental XRD data. For the latter, a very challenging X-ray diffraction computed tomography (XRD-CT) dataset was chosen in order to test the performance of the network and its feasibility for deployment for real world applications. We focus on the scale factors, lattice parameters and crystallite sizes, because they are most often sufficient to describe and understand materials and process performance under operating conditions12,42,43,44,45,46,47,48. This exemplar dataset was acquired using a multi-component Ni-Pd/CeO2-ZrO2/Al2O3 catalyst and consists of ca. 20,000 diffraction patterns. The results from the analysis of this five-phase system are also compared with results obtained using the Rietveld method with the TOPAS software49.

Results

Single-phase PQ-Net architecture and simulated XRD-CT dataset

The architecture of the PQ-Net for a single-phase of cubic symmetry is presented in Fig. 1. The PQ-Net architecture can be divided into three main parts. The first part (pattern-block), which decreases the size of the XRD pattern and extracts local features, involves three convolutional layers (128 filters, kernel size of 35 and stride equal to one) followed by a max-pooling layer (stride equal to two). The second part, which is replicated for each crystalline phase used in the model, includes a sequence of five convolutional and max-pooling layers ending with a flatten layer; this extracts features from the XRD pattern related to the specific phase (phase-block). The third and final part (parameter-block) contains two fully connected (dense) layers ending with one single output per phase parameter. A glossary explaining the various layers in more depth is available in the Supporting Information (see also Supplementary Fig. 1 and Supplementary Tables 1–2).

Fig. 1: Regression CNN architecture for Ni phantom experimental system featuring a single phase.
figure 1

CONV represents 1-D convolutional layers, Pool represents max-pooling layers, FC represents fully connected layers, and Dropout represents dropout layers with 10% dropout rate. The network consists of eight convolutional layers, six max-pooling layers, one flatten layer, nine fully connected layers and six dropout layers in total. There are three routes connected to the flatten layer which give predictions for scale factor, crystallite size and lattice parameter a, respectively. Each route has three fully connected layers whose scales are shown in the figure. The number of filters, kernel sizes and the stride of convolutional layers are also given in the figure. All max-pooling layers have stride equal to 2.

Full size image

The parameter-block part of the architecture contained the majority of the trained weights so our initial work focused on minimizing both the depth and width of the dense layers. It was found that the width of the second dense layer has to be doubled for the crystallite size and lattice parameters as these are more challenging parameters to quantify compared to scale factors. A dropout layer has been used after each fully connected layer (apart from the final layers) to prevent overfitting. Regarding the training process, it was seen that using mean absolute error (MAE) as the loss function instead of mean squared error (MSE) yielded a more accurate model; this is not surprising as MAE is known to better handle outliers present in the training data. This feature is crucial as there are diffraction patterns in the training data that contain low intensity peaks and the network has to adapt and learn to ignore them. It should be noted that the various parameters were normalised prior to the training of the network since their magnitude varied significantly (i.e., between scale factors, lattice parameters and crystallite sizes). Adam was used as the optimisation algorithm and the learning rate, after initial tuning, was set to 0.0005 for all case studies presented in this work50. In material science aspect

As a proof-of-concept study, the PQ-Net was first trained with noiseless and zero-background XRD patterns of a Ni fcc structure (ICSD: 64989) simulated with the TOPAS software v749. These diffraction patterns were used as the training data with a 10 % validation split. The impact of the diffraction library size (i.e. size of the training data) was investigated and the results are summarised in Supplementary Table 3; there was no apparent change in performance with libraries containing more than 100 K patterns. There is also a sudden drop in MAE above 10 K patterns which implies that accurate results could be obtained with libraries as small as 20 K. It should also be pointed out that the MAE values provide an indication of accuracy regarding the predictions of the network but a PQ-Net model trained with relatively high MAE can potentially still yield accurate predictions.

To evaluate the performance of the PQ-Net we simulated a phantom XRD-CT dataset using a scale factor map derived from the Rietveld analysis of an experimental XRD-CT dataset from a catalyst sample51. One advantage of this dataset is the realistic nature of this intensity map compared to idealised ones, such as the Shepp-Logan phantom. More important though is that one can easily segment the various particles, assign different properties to them and generate multiple XRD-CT datasets with varying chemistry in an efficient manner. More details regarding the phantom simulation can be found in our previous work52. These 120 × 120 (14,400) diffraction patterns were used as test data for the single-phase PQ-Net. The results from a single-phase PQ-Net trained with 100 K patterns applied to this test set are presented in panel in Fig. 2. The ground truth (GT) maps for the various parameters are presented in the first row, the results from the PQ-Net in the second row and the difference between the two maps in the third row. The scale factor is the easiest parameter to train the PQ-Net for, so it is not surprising that it yields very accurate maps with a relative error less than 5 % (Supplementary Fig. 2). Impressively though it can be seen that the error for the crystallite size is below 1 nm for all catalyst particles and similarly the error for the lattice parameter is below 10−3 Å. The Rwp also is below 5% for the majority of particles (Supplementary Fig. 2). To illustrate this further, the mean diffraction pattern from three catalyst particles exhibiting different properties were extracted and analysed with the PQ-Net. The results are presented in panel b of Fig. 2 and it can be clearly seen that the network is able to accurately predict the three patterns (Rwp of 3.144, 1.191 and 1.835% respectively). The corresponding Ni parameters are summarised in Supplementary Table 4.

Fig. 2: PQ-Net results for the single-phase simulated data.
figure 2

a Ni scale factors, crystallite sizes (nm) and lattice parameter (Å) ground truth maps, results obtained with the PQ-Net and their absolute difference. b Comparison between the average diffraction patterns extracted from three particles of interest and the TOPAS generated patterns using the parameters predicted by the PQ-Net.

Full size image

Multi-phase deep ensemble PQ-Net and simulated XRD-CT dataset

The next logical step was to test the PQ-Net against simulated multi-phase XRD patterns which is a closer approximation to most experimental data and closer to real world applications. Here, we chose a five-phase system that corresponds to the chemistry encountered in the experimental XRD-CT dataset presented in the next section. The choice of this material system was based on the complexity and its challenging nature; it is a multi-component system with the various phases exhibiting different symmetries and their physico-chemical properties vary spatially as it will be shown in the next section.

The multi-phase PQ-Net architecture used here is presented in Supplementary Fig. 3 and it can be seen that it is an extension of the one presented in Fig. 1. The five crystalline phases modelled here are NiO (ICSD: 9866), PdO (ICSD: 24692), CeO2 (ICSD: 72155), ZrO2 (ICSD: 66781) and theta-Al2O353. The parameters refined here are the scale factors, lattice parameters and crystallite size for each phase. Poisson noise and a varying linear background (2nd degree Chebyshev polynomial) were also introduced in the diffraction libraries used to train the multi-phase PQ-Net (Supplementary Table 5). To improve the performance and robustness of our model, necessary for complex tasks such as this, we implemented and evaluated deep ensembles of the PQ-Net54,55,56. Deep ensembles are based on the assumption that different initial weights of a CNN will lead to different local minima which may not necessarily be a minimum on the validation data, but these local minima are evenly distributed in parameter space and therefore sample different possible solutions efficiently57. Therefore, simply retraining a CNN model several times and taking the average of the outputs can usually have better results than using only one model.

Supplementary Fig. 4 shows the influence of deep ensemble acting on the results’ mean absolute error (MAE). It can be seen that the more individual models in a deep ensemble model, the more accurate results it can generate (Supplementary Table 6). Another benefit of deep ensembles is that the method can give a referable uncertainty of the results. By taking the standard deviation among all individual models’ predictions on a parameter, one can easily estimate the precision of a deep ensemble result. However, because the time taken by training and merging individual models can rise proportionally as the number of models in deep ensemble increases and the curve tends to flatten out, using a deep ensemble of size between five to ten is the most realistic option for real world applications. It is worth mentioning that the MAEs for the validation dataset have a lower limit which can not be surpassed by increasing the size of the training dataset or the deep ensemble, because the crystallite sizes and lattice parameters are very hard to predict when scale factors are approaching zero.

The performance of the trained multi-phase PQ-Net was evaluated with a simulated XRD-CT dataset using the same intensity image as for the single-phase dataset. Here, the same strategy was followed creating five single-phase XRD-CT datasets, one for each component which were then added together to form the multi-phase simulated XRD-CT dataset consisting of 14,400 diffraction patterns (test data). This dataset was then passed to the PQ-Net for analysis and quantification of the various parameters. The scale factor results from the deep ensemble PQ-Net consisting of 10 models and trained with 100 K patterns (training data) are presented in Fig. 3. It can be clearly seen that the PQ-Net is able to create accurate phase distribution maps for all components and retain the local features and relative intensities. Importantly, it is able to also create accurate crystallite size and lattice parameter maps for all phases (Supplementary Figs. 5–9) while the Rwp remains <10 % for all particles (Supplementary Fig. 10).

Fig. 3: PQ-Net scale factor results for the multi-phase simulated data.
figure 3

Scale factor ground truth maps, results obtained with the PQ-Net and their absolute difference for the simulated multi-phase NiO-PdO-CeO2-ZrO2-Al2O3 system.

Full size image

To further illustrate the ability of the PQ-Net to handle multi-phase systems, we also extracted three diffraction patterns from the XRD-CT which correspond to the mean diffraction patterns from three particles. The results from the analysis of these three patterns are presented in Fig. 4 where it is clearly shown that the PQ-Net is able to model them accurately (Rwp of 5.024, 5.419 and 4.798 % respectively; see also Supplementary Tables 7–11).

Fig. 4: Examples of fitted simulated diffraction patterns from regions of interest.
figure 4

Comparison between the average diffraction patterns extracted from three particles of interest and the TOPAS generated patterns using the parameters predicted by the PQ-Net. The main reflections for NiO (blue ticks), PdO (green ticks), CeO2 (red ticks) and ZrO2 (cyan ticks) are also presented.

Full size image

Experimental XRD-CT dataset

The final and most challenging dataset is an experimental XRD-CT dataset acquired using the Ni-Pd/CeO2-ZrO2/Al2O3 catalyst. Details regarding its preparation and phase identification are provided in our previous work12. Here we use the previous deep ensemble PQ-Net consisting of 10 models and trained with 100 K patterns (training data) to analyse the 22,801 XRD patterns (test data) present in the XRD-CT dataset (XRD-CT images of 151 × 151 pixels). We also perform conventional Rietveld analysis which serves as an approximation to the ground truth and the benchmark to assess the performance of the PQ-Net.

The results for the scale factors for all phases using the Rietveld method and the PQ-Net are presented in Fig. 5. It can be seen that the PQ-Net is able to accurately predict the scale factors of both the main phases, such as Al2O3, NiO and ZrO2, and also minor components such as the PdO. It should be noted here that the PdO should not have been detectable at this resolution due to its low content (<1 wt.%) and it is only possible due to areas of high concentration near the surface of some particles. Importantly, the PQ-Net results are consistent with the ones obtained with the Rietveld method showing for example that the ZrO2 (or Zr-rich CexZr1-xO2 phase, where x ≤ 1) is located only at the periphery of the catalyst particles while the CeO2 (or Ce-rich CexZr1-xO2 phase) is present also at the interior of the catalyst particles. The Al2O3 distribution is correctly shown to be homogeneous in all catalyst particles while NiO, the primary catalyst active component, is also present in all particles. The uncertainty maps shown in the last row of Fig. 5 correspond to the standard deviation of the calculated values from the 10 models of the deep ensemble PQ-Net (Supplementary Fig. 11).

Fig. 5: PQ-Net scale factor results for the multi-phase experimental data.
figure 5

Scale factor maps obtained with the Rietveld method, results obtained with the PQ-Net, their absolute difference for the experimental multi-phase NiO-PdO-CeO2-ZrO2-Al2O3 system and the uncertainty maps of the deep ensemble PQ-Net (last row).

Full size image

Importantly, apart from the scale factors, the PQ-Net is able to capture the chemical gradients present in this challenging XRD-CT dataset. As shown in Fig. 6, it is able to resolve the heterogeneities in the crystallite size and lattice parameter of the CeO2-ZrO2 phases. As we have previously reported, the CeO2 crystallite size and lattice parameter follow an egg-shell distribution in this catalyst with lower values for lattice parameter and crystallite size at the shell compared to the core of the particles. As expected, this result is reproducible with the Rietveld method but it can also be seen that the PQ-Net yields the same results. Moreover, the difference between the CeO2-ZrO2 lattice parameter maps obtained with the Rietveld method and the PQ-Net is below 2 × 10−2 Å for the majority of the particles while for the crystallite size maps the difference is in the order of 1–2 nm. The uncertainty maps presented in the last row of Fig. 6 indicate that the error for the crystallite size for both phases is in the order of 1 nm.

Fig. 6: PQ-Net CeO2-ZrO2 crystallite size and lattice parameter a results for the multi-phase experimental data.
figure 6

Crystallite size (colorbar axis corresponding to nm) and lattice parameter a (colorbar axis corresponding to Å) maps for CeO2 and ZrO2 obtained with the Rietveld method, results obtained with the PQ-Net, their absolute difference for the experimental multi-phase NiO-PdO-CeO2-ZrO2-Al2O3 system and the uncertainty maps of the deep ensemble PQ-Net.

Full size image

Diffraction patterns from two representative regions-of-interest were extracted from the XRD-CT for further analysis. As shown in Fig. 7, one pattern is derived from the periphery of three particles (light blue region) while the second is derived from the whole area of two other particles (magenta region). The first region corresponds to an area where the ZrO2 phase is present while the second where it is absent. Both patterns are the mean XRD patterns from the respective regions in the sample. These two patterns were then analysed with the PQ-Net and the results are presented in Fig. 7. It can be clearly seen that the PQ-Net is able to model accurately the data (Rwp of 8.353 and 7.749 % respectively).

Fig. 7: Examples of fitted experimental diffraction patterns from regions of interest.
figure 7

Comparison between the average diffraction patterns extracted from two regions of interest and the TOPAS generated patterns using the parameters predicted by the PQ-Net. The main reflections for NiO (blue ticks), PdO (green ticks), CeO2 (red ticks) and ZrO2 (cyan ticks) are also presented.

Full size image

Discussion

The results presented in this work demonstrate that the PQ-Net model is able to extract accurate physico-chemical information from XRD patterns. Its performance was evaluated with different datasets of increasing complexity, varying from simulated noiseless single-phase to experimental five-phase systems. In all cases, PQ-Net was able to quantify the scale factors, lattice parameters and crystallite sizes of the various phases providing predictions with errors within acceptable ranges (i.e. compared to ground truth and Rietveld results for the simulated and experimental data respectively). The work presented here therefore serves also as an exemplar study demonstrating PQ-Net’s flexibility and scalability as the method applied to datasets of different complexity and it can be scaled up or down depending on the data requirements. The robustness of the PQ-Net was further improved through the implementation of deep ensembles that allow for uncertainty quantification.

The deep ensemble PQ-Net model was able to provide results with less than 2% difference in Rwp compared to the one obtained from the analysis of the same data using the Rietveld method. These results were obtained (i.e. analysis of ca. 20 K XRD patterns) in ca. 10 s while the state-of-the-art Rietveld for 9 K XRD patterns required ca. 4.4 h. A comparison table and corresponding figures demonstrating how the performance of PQ-Net and Rietveld scale with increasing data size are provided in the Supporting Information (Supplementary Tables 12–14 and Supplementary Figs. 12–14). It should be noted though that the PQ-Net was not designed to replace the Rietveld method but to assist and provide very fast and accurate predictions for the various physico-chemical parameters; conventional least-square minimisation techniques, such as the Rietveld method, can be used for fine parameter tuning using the PQ-Net results as the starting point. The PQ-Net in its current form, similar to the Rietveld method, requires some a priori knowledge of the sample and its chemistry. However, in case there is evolving/unpredictable chemistry or unknown phases form/are present in the system, it can be easily spotted through the Rwp, the difference between the experimental and predicted patterns and the uncertainty maps provided by the deep ensemble.

The key advantage of the PQ-Net is that the diffraction libraries can be generated prior to an experiment and used to pretrain the model. This approach can allow for real-time assessment of diffraction data acquired during a dynamic experiment and enable the user to better guide the experiment (e.g., through the applied operating conditions) and intervene when necessary. In future, we plan to increase the complexity of the PQ-Net and explore its ability to model other parameters such as complex backgrounds, to quantify amorphous/crystalline components and to perform simultaneous calculation of weight percentages and strain analysis. It should be pointed out that the PQ-Net could in principle also be used to benefit X-ray Free Electron Laser (XFELs) facilities, like the European XFEL (Germany), for fast analysis of powder diffraction data. We will also try to apply the PQ-Net on other analytical scattering/spectroscopic techniques such as X-ray absorption fine structure spectroscopy (XAFS) data which is a technique suitable for large volume data analysis (e.g., XAFS-CT). Last but not least, we expect that the PQ-Net has the potential to become an essential tool for diffraction applications beyond synchrotron experiments such as real-time quality inspection in manufacturing/synthesis.

Methods

XRD-CT measurements

A Ni-Pd/CeO2-ZrO2/Al2O3 catalyst was measured with XRD-CT at beamline station ID31 of the ESRF using a 70 keV monochromatic X-ray beam focused to have a spot size of 20 × 20 μm. Information regarding the sample preparation can be found in our previous work12. The Pilatus3 X CdTe 2 M hybrid photon counting area detector was used for the acquisition of the 2D powder diffraction patterns. The total acquisition time per point was 20 ms. 225 translation steps (translation step size of 20 μm) covering 0–180° angular range, in steps of 1.125° (i.e., 160 line scans) were used for the tomographic measurements. A CeO2 NIST standard was used for the detector calibration. Every 2D powder diffraction pattern was transformed to a 1D powder diffraction pattern after applying a 1% trimmed mean filter to remove outliers; this was done using in-house developed MATLAB scripts58. The tomographic images were reconstructed using the filtered back projection algorithm and were cropped to a 151 × 151 × 2048 reconstructed data volume (22801 diffraction patterns).

Rietveld analysis

Full profile analysis of the spatially-resolved diffraction patterns present in the XRD-CT data was performed using the Rietveld method. Inhouse developed MATLAB scripts were used in combination with the TOPAS software v7 to perform the analysis in a sequential manner, processing one line at a time (i.e., corresponding to 225 diffraction patterns). A mask was applied so that only the pixels in the images corresponding to sample regions would be processed (decreasing significantly the number of patterns to be analysed in each line) decreasing the number of patterns to 9027. Full profile analysis was initially performed using the mean diffraction pattern of the XRD-CT dataset before performing the batch fitting (i.e., in order to have a good starting model for the Rietveld analysis). The background was fairly linear and a 2nd degree Chebyshev polynomial was used to model it. For each phase, the scale factor, lattice parameters and crystallite size were refined in the model.

Diffraction libraries

All diffraction libraries were created using inhouse developed MATLAB scripts and the TOPAS software v7 using CPU. The neural networks were trained using GPU and a CPU/GPU comparison Table is provided in the Supporting Information (Supplementary Table 1). A 10% validation split was applied to the training data for each diffraction library used in this work to train the PQ-Net. The scale factors, lattice parameters and crystallite size values for each pattern were created using random sampling from a fixed range. Details for each parameter are presented in the Supporting Information. Each parameter corresponds to a 1D array with length equal to the library size. For each parameter, the minimum value was subtracted from the corresponding array and then it was normalised. This step proved very important for the stability of the network as it allowed all parameters to have values at the same range (0–100). The minimum and maximum values are recorded in a text file, then the models’ predictions on the experimental dataset can be easily converted to the real values before normalisation. A 3XS Data Science Workstation C264X2 with 2x Intel Xeon Silver 4216, 350 GB RAM and 2x Quadro RTX 8000 was used for the development and training of the neural networks used in this work and the Rietveld analysis of the diffraction patterns.