Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Automated crystal structure analysis based on blackbox optimisation

## Abstract

In the present study, we show that time-consuming manual tuning of parameters in the Rietveld method, one of the most frequently used crystal structure analysis methods in materials science, can be automated by considering the entire trial-and-error process as a blackbox optimisation problem. The automation is successfully achieved using Bayesian optimisation, which outperforms both a human expert and an expert-system type automation despite the absence of expertise. This approach stabilises the analysis quality by eliminating human-origin variance and bias, and can be applied to various analysis methods in other areas which also suffer from similar tiresome and unsystematic manual tuning of extrinsic parameters and human-origin variance and bias.

## Introduction

The physical properties of materials are governed by their crystal structure; thus, crystal structure analysis is an indispensable element in materials science research1,2. Compared to the drastic improvement in material fabrication and measurement throughput, the throughput of crystal structure analysis has not been improved because the analysis heavily relies on manual time-consuming trial-and-error methods3,4,5,6,7,8. Rietveld refinement or the Rietveld method9,10, one of the most widely used crystal structure analysis methods for powder diffraction data, such as X-ray diffraction (XRD) and neutron diffraction, faces this problem as well.

Rietveld refinement involves complex curve fitting using an experimental result and a simulation based on a physical model. The physical model comprises intrinsic parameters for governing crystal structure such as lattice constants and atomic positions and extrinsic parameters such as background signals and peak shape defined by instrumental resolution. The method is designed to minimise the difference between the observed and simulated results by updating the parameters in a given physical model until the difference reaches a threshold. All parameters need initial values, including categorical ones such as background and peak shape functions. For each numerical parameter, a binary parameter (fix or refine) to control the scope of the refinement needs to be set. The number of intrinsic parameters depends on the complexity of the crystal structure and ranges from 10 to 100, while that of extrinsic parameters is typically less than 20.

It is commonly known that refining all parameters at once often leads to physically unreasonable results. The basic strategy to avoid such an issue is to gradually increase the number of parameters to be refined; that is, beginning with a few to finally incorporate all in order to keep them physically acceptable. Consequently, one needs to update the scope of the refinement at each step of the refinement process and, eventually, to determine the sequence of scopes. To guide researchers among a massive number of possible combinations and sequences of different refinement scopes with several tens of refinable parameters, empirical guidelines which significantly limit the size of "scope space” are often employed. However, even with the limited scope space, one has to search the right path to a reasonable solution in the search space by trial and error; furthermore, it is not guaranteed whether the guidelines will lead researchers to the optimal crystal structure in the whole scope space. Considering the wide use of Rietveld refinement in various materials science communities, this situation, that only proficient experts can exploit Rietveld refinement properly, should be improved.

Such complexity in parameter tuning would have consequences on analysis quality in terms of both variance and bias. For example, the consistency in experts’ results shown by round robin tests in refs. 11,12,13 would have declined due to a wider distribution of individual skills to deal with the complexity if more researchers had participated in the tests. On the contrary, the consistency itself indicates the success of the refinement guidelines shared by experts. While such guidelines contribute to the consistency as expected, they can also cause a systematic bias in refinement results. In this study, based on blackbox optimisation (BBO)14, we offer a general automation framework for Rietveld refinement to alleviate these problems and allow practitioners to spend more time examining candidate structures proposed by the framework, and not searching for them in a vast parameter space.

BBO refers to a problem setup, in which the objective function and/or constraint function are given by blackboxes, i.e., the analytic forms of the function are unknown. BBO has recently emerged as a popular concept in the machine learning community because it performs an important task of tuning hyperparameters in machine learning models automatically, which is termed as hyperparameter optimisation (HPO)15. HPO plays a vital role in machine learning; (1) it reduces the human effort necessary for applying machine learning, (2) it improves the performance of machine learning algorithms, and (3) it facilitates fair comparisons among the results of machine learning studies by providing the same level of tuning15. HPO is formalised as the maximisation of machine learning model performance in the hyperparameter configuration space. For this case, the relationship between a hyperparameter configuration and the machine learning model performance is considered as a blackbox. At present, Bayesian optimisation16, a global search method for BBO, is the most successful in this HPO problem17,18,19,20,21. This topic is closely related to our focus, i.e., to stabilise the analysis quality and bypassing unsystematic (and often unexciting) tasks in scientific studies. Herein, we realised that the concept of HPO can be applied to Rietveld refinement.

Whereas numerous studies have been conducted on Rietveld refinement9,10,22,23,24,25,26,27,28,29,30,31,32,33 and software34,35,36,37,38,39, up to now, the number of studies that aim to address aforementioned problems is very limited. Powder 3D Parametric40 is a semi-automated software toolkit for Rietveld refinement based on user-provided configuration. This software is particularly designed for sequential analysis of a large number of data such as time-resolved measurement. SrRietveld41,42, an automated software toolkit for Rietveld refinement, provides several pre-written scripts for automating the common tasks in the method. AutoFP43 uses an expert system algorithm to simulate the manual process performed by human experts. The expert system consists of simple if–then rules, based on a collection of empirical strategies and simple ideas. Recently, PowderBot44, which applies reinforcement learning45 to automate Rietveld refinement, has been proposed. Reinforcement learning is suitable for solving problems with stochastic state transitions which are formulated as a Marcov decision process. This method is famous for its great success in playing games like Go. While the reinforcement learning-based approach is conceptually interesting, it seems that another simpler approach such as BBO is sufficient for our purpose because Rietveld refinement is deterministic. In single-crystal diffraction, SCAR46, machine-learning-based automated crystal structure analysis software has been proposed very recently. It supports precise analysis and is applicable to various single-crystal diffraction data, and is of interest in the field of crystal structure analysis. However, in powder X-ray diffraction, there is still room to apply ideas from machine learning. To the best of our knowledge, no attempt has been made to apply the BBO approach to Rietveld refinement for automating the time-consuming trial-and-error process and search beyond the boundaries defined by the common sense of human experts.

## Results and discussion

### Formalisation

Here, we introduce our "BBO-Rietveld” approach that automates the tuning of the refinement scope in the Rietveld method in the same manner as previously done in HPO. Fig. 1 illustrates the overview of the BBO-Rietveld approach described below, and Table 1 shows the list of optimised parameters. Most of them are for defining the scope and an extrinsic categorical parameter for the background function. Note that we did not perform numerical initial parameter search contrary to the possible expectation that some of the readers would have.

In Rietveld refinement, the weighted profile residual factor (Rwp), which is the difference between calculated and observed diffraction intensity, is widely used as a metric to evaluate the Rietveld refinement result29. Let x be a fixed parameter configuration of Rietveld refinement including the scope of the refinement, $${y}_{i}^{{\rm{Obs}}}$$ be the i-the observed diffraction intensity, $${y}_{{\boldsymbol{x}},i}^{{\rm{Calc}}}$$ be the diffraction intensity calculated using the physical model with the configuration x, and wi be the weight corresponding to the inverse of the uncertainty of i-th intensity. Then, Rwp is defined as follows:

$${R}_{{\rm{wp}}}({\boldsymbol{x}})={\left(\frac{{\sum }_{i}{w}_{i}{\left({y}_{i}^{{\rm{Obs}}}-{y}_{{\boldsymbol{x}},i}^{{\rm{Calc}}}\right)}^{2}}{{\sum }_{i}{w}_{i}{\left({y}_{i}^{{\rm{Obs}}}\right)}^{2}}\right)}^{1/2}.$$
(1)

The crystal structure with the minimum Rwp is the most probable structure based on the given experimental data.

Our task can be formalised as the following BBO problem:

$${\mathrm{minimise}}\,\,\,{R_{\rm{wp}}}({\boldsymbol{x}}),$$
$${\text{subject}} \, {\text{to}}\,\,{\boldsymbol{c}}({\boldsymbol{x}})\,\ge\, 0,$$
$${\boldsymbol{x}}\in {\bf{X}},$$
(2)

where X is the configuration space and c(x) is the set of constraint functions. In the problem, a configuration x is considered as a variable and it affects Rwp through $${y}_{{\boldsymbol{x}},i}^{{\rm{Calc}}}$$. The constraints are set to ensure that the refined parameters are physically valid, or sometimes to limit the search space and reduce manual trial-and-error attempts with a plausible explanation. In this study, we limit ourselves to introduce the latter type of constraints and we only introduce an essential constraint to keep human-origin biases to a minimum (see Methods section for more details).

By solving the problem, we obtain the best configuration $${{\boldsymbol{x}}}^{\star }={{\rm{argmin}}}_{{\boldsymbol{x}}}{R}_{{\rm{wp}}}({\boldsymbol{x}})$$, which achieves the best fit without manual process.

Additionally, goodness-of-fit (GOF) is used as a metric to evaluate the optimised Rietveld refinement result. The definition of GOF is given as follows:

$${\rm{GOF}}=\frac{{\sum }_{i}{w}_{i}{\left({y}_{i}^{{\rm{Obs}}}-{y}_{{\boldsymbol{x}},i}^{{\rm{Calc}}}\right)}^{2}}{{N}_{obs}-p},$$
(3)

where Nobs is the number of data points of observed XRD pattern, and p is the number of variables in the fitting model. This metric is called as GOF in GSAS-II39 and χ2 in FullProf35.

It should be noted that small Rwp or GOF does not guarantee that the corresponding refined crystal structure would be appropriate for the obtained experimental data. The examination of the analysis result is still an important task for human experts29.

### Experiments

To evaluate the proposed BBO-Rietveld approach, we optimised parameter configurations of Rietveld refinement for XRD patterns of Y2O3, Dy0.5Sr0.5MnO3 (DSMO) and LiCoO2 which are chosen as benchmark materials. The tree-structured Parzen estimator (TPE)17,20, one of the variants of Bayesian optimisation, is used as the optimiser. The details of our optimisation system are provided in the Methods section. Here we discuss the results of Y2O3 and DSMO. Readers who are interested in the result of LiCoO2 should refer to the supplementary resources provided in the BBO-Rietveld repository (see Code availability). For each optimisation run, the optimiser iteratively evaluated 200 configurations including the initial 20 random configurations and returned the best one with the minimum Rwp. The search spaces for each parameter are shown in Table 1. The initial crystal structure models were taken from crystallographic information files (CIFs) on the GitHub repository of AutoFP47, presumably those used in ref. 43. To evaluate statistical property and reproducibility, optimisations have been performed 100 times with different random seeds each for Y2O3 and DSMO. Histograms of 100 Rwp values for both materials are shown in Fig. 2. Reference values for a human expert and AutoFP are also shown for comparison. The latter ones are taken from the previous study43.

Surprisingly, 90% and 99% of our optimisation runs for both test datasets, Y2O3 and DSMO, achieve better Rwp than the human expert and AutoFP, although no additional expertise in Rietveld refinement is used other than the conventional refinement procedure (see Methods section for details). Fig. 3 shows the refinement results of the best configuration derived by BBO for each benchmark material. We cannot find apparent flaws in the results refined with the best configuration. The changes of Rwp and goodness-of-fit (GOF) are shown in Fig. 4 (Y2O3) and 5 (DSMO). Both Rwp and GOF were improved as optimisation progresses, and in both metrics, our proposed method exceeded human expertise at around 100 evaluations. Rwp and GOF will be further improved if we continue the optimisation.

To visualise the refined structure features (x, y, z positions and Uiso of each atom) in two-dimensional space, we utilise the multidimensional scaling (MDS)48,49, one of the dimensionality reduction algorithms. MDS can represent high-dimensional data in a low-dimensional space by approximating the distance in the original space. The number of original structure feature dimension is 16 for Y2O3 and 20 for DSMO. The MDS visualisation for DSMO is shown in Fig. 6. Each point in this figure represents a refined crystal structure, and the distance between points indicates their similarity. The majority of the refined structures form a loose cluster at the lower-right side of the figure, which means they are similar to each other. In the cluster, there are three special points; the best one from BBO, the one from the human expert, and the one from BBO and very close to the human expert result as shown in Fig. 6. While the best structure obtained by BBO (Table 2) scores smaller Rwp than that of the human expert result (Table 2), Uiso of O1 atom in the BBO result is noticeably smaller than that of O2 atom, which violates a conventional criterion that Uiso of atoms with similar mass should be comparable. On the other hand, the third special point, obtained by BBO and close to the human expert result, passes the criterion (Table 2). This demonstrates that BBO efficiently obtains candidates with small Rwp values but additional examinations are still required to choose the most preferable one.

Next, we discuss the interesting outlier at the upper left of Fig. 6. The outlier (Table 2) has good converged Rwp, but the x position of O1 atom in the refined structure is considerably different from others. The difference exceeds the uncertainty calculated by the software, and, from the structural point of view, the positional shift corresponding to 5% of the lattice parameter would be enough to affect physical properties. This implies that the outlier corresponds to a local minimum not belonging to the cluster discussed above. In such a situation, multiple crystal structure candidates can explain the experimental data sufficiently with a similar level, which may affect the conclusion of a study or evoke a new discussion. Despite that it meets the constraint in the optimisation (positive Uiso), the outlier can be rejected due to the violation of the conventional criterion for Uiso as well as the best result by BBO. We believe that imposing constraints other than universal physical requirements in BBO can result in biased optimisation of results toward human expectations. Experts can examine the list of candidates using conventional criteria and other knowledge, and can eliminate inappropriate solutions at any time. This ability to propose multiple candidates without human effort and bias is a great advantage of automation and may lead to the discovery of hitherto unnoticed new knowledge beyond the standard practices of conventional manual tuning and the expert system simulating the practices. The use of empirical constraints in a manual analysis by experts also has the purpose of bounding the search space to reach a valid solution as soon as possible. However, Rietveld refinement using BBO can evaluate a large number of parameter combinations and suggest candidate structures much more efficiently than manual approach, that is, the constraints imposed to save time are no longer essential.

The results demonstrate that the proposed approach is reliable, allowing more practitioners without much expertise to utilise Rietveld refinement. In addition, the automation of the trial-and-error processes not only gives better results, but also stabilises the quality of results by eliminating human-origin variance and bias accumulated in manual tuning processes as HPO does in machine learning.

The efficiency against execution (wall-clock) time is also important in practice. In this study, each optimisation run (200 trials) takes less than an hour using a general workstation (2.3 GHz 18 core CPU, 256 GB RAM), and it can be performed within a practical time limit using a general laptop. The execution time of the optimiser is improved by parallelisation. For the same number of trials, the performance of parallel optimisation is typically slightly worse than in sequential optimisation, but the execution time will be about 1/n when the number of workers is n. Increasing the number of experts is difficult, but increasing computational resources is relatively easy. It is also an advantage of automation over manual work.

The problems of human-origin variance and bias and low stability of the quality of model fitting results are very common in other research areas. We consider that the framework of automation as a blackbox optimisation in this study is powerful and our approach can be easily applied to similar problems in other fields.

## Methods

### Details of implementation

To solve the problem (Eq. (2)), we connect GSAS-II39, a crystallography data analysis software, which provides Rietveld refinement to Optuna50, an automatic HPO software framework. Extrinsic parameters listed in Table 1 are optimised using the TPE. Other extrinsic parameters related to Rietveld refinement are initialised with GSAS-II default values, and the intrinsic parameters for a crystal structure are obtained from ref. 43. The TPE is easy to use and supports real, integral, categorical, and conditional types of parameters, making it suitable for application in our case. It iteratively samples a new promising configuration based on the past observations (i.e. the pairs of the configurations and the corresponding Rwp values) and then runs GSAS-II to perform Rietveld refinement with the sampled configuration to find the best one. The refinement procedure described herein consists of the following conventional steps: (1) set initial 2θ range, (2) refine the background, (3) refine the 2θ zero correction and the unit cell parameters, (4) refine sample displacement parameters and the histogram scale factor, (5) refine atom parameters, and (6) refine the above parameters without 2θ bounds in step 1. During the optimisation, the constraint that all refined Uiso must be non-negative is imposed to avoid unreasonable structure models by setting Rwp to a large constant (=109) for penalty when the constraint is violated. The TPE does not mimic human experts but simply optimises the objective function. This is completely different from a conventional expert system that aims to reproduce the empirical strategies of human experts.

### Visualisation of results

To visualise the crystal structure features with MDS in Fig. 6, feature standardisation, which makes the values of each feature in the data have zero-mean and unit-variance, was performed before applying MDS. We used the MDS implementation in scikit-learn51, a machine learning toolkit for Python.

## Data availability

The dataset of diffraction patterns and initial crystal structures are available in the GitHub repository of AutoFP47 (Y2O3, DSMO) and PowBase52 (LiCoO2).

## Code availability

The codes for automated crystal structure analysis, documents, and supplementary resources that support the findings of this study are available at https://doi.org/10.5281/zenodo.3747147. These are also available at the BBO-Rietveld repository on GitHub (https://github.com/quantumbeam/BBO-Rietveld). As the supplementary resources, we provide not only the experimental results of Y2O3, DSMO that we discussed in this paper but also the result of LiCoO2 that are also comparable to the result of human expert.

## References

1. 1.

Callister, W. D. & Rethwisch, D. G. Materials Science and Engeneering (John Wiley and Sons, 2010).

2. 2.

De Graef, M. & McHenry, M. E. Structure of Materials. An Introduction to Crystallography, Diffraction and Symmetry (Cambridge University Press, 2012).

3. 3.

Koinuma, H. & Takeuchi, I. Combinatorial solid-state chemistry of inorganic materials. Nat. Mater. 3, 429–438 (2004).

4. 4.

Hofmann, D. W. M. & Kuleshova, L. N. Data Mining in Crystallography, vol. 134 of Structure and Bonding (Springer Science & Business Media, Berlin, Heidelberg, 2009).

5. 5.

Potyrailo, R. et al. Combinatorial and high-throughput screening of materials libraries: Review of state of the art. ACS Comb. Sci. 13, 579–633 (2011).

6. 6.

Gregoire, J. M. et al. High-throughput synchrotron X-ray diffraction for combinatorial phase mapping. J. Synchrotron Rad. 21, 1262–1268 (2014).

7. 7.

Lookman, T., Alexander, F. J. & Rajan, K. Information Science for Materials Discovery and Design (Springer-Verlag, 2015).

8. 8.

Alberi, K. et al. The 2019 materials by design roadmap. J. Phys. D 52, 013001 (2018).

9. 9.

Rietveld, H. M. Line profiles of neutron powder-diffraction peaks for structure refinement. Acta Crystallogr. 22, 151–152 (1967).

10. 10.

Rietveld, H. M. A profile refinement method for nuclear and magnetic structures. J. Appl. Crystallogr. 2, 65–71 (1969).

11. 11.

Hill, R. Rietveld refinement round robin. i. analysis of standard x-ray and neutron data for pbso4. J. Appl. Crystallogr. 25, 589–610 (1992).

12. 12.

Hill, R. & Cranswick, L. International union of crystallography. commission on powder diffraction. rietveld refinement round robin. ii. analysis of monoclinic zro2. J. Appl. Crystallogr. 27, 802–844 (1994).

13. 13.

León-Reina, L. et al. Round robin on rietveld quantitative phase analysis of portland cements. J. Appl. Crystallogr. 42, 906–916 (2009).

14. 14.

Audet, C. & Hare, W. Introduction: tools and challenges in derivative-free and blackbox optimization. In Derivative-Free and Blackbox Optimization, 3–14 (Springer, 2017).

15. 15.

Feurer, M. & Hutter, F. Hyperparameter optimization. In Automated Machine Learning, 3–33 (Springer, 2019).

16. 16.

Mockus, J., Tiesis, V. & Zilinskas, A. The application of Bayesian methods for seeking the extremum. Towards Glob. Optim. 2, 2 (1978).

17. 17.

Bergstra, J., Bardenet, R., Bengio, Y. & Bengio, Y. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems, 2546–2554 (2011).

18. 18.

Hutter, F., Hoos, H. H. & Leyton-Brown, K. Sequential model-based optimization for general algorithm configuration. In International Conference on Learning and Intelligent Optimization, 507–523 (Springer, 2011).

19. 19.

Snoek, J., Larochelle, H. & Adams, R. P. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, 2951–2959 (2012).

20. 20.

Bergstra, J., Yamins, D. & Cox, D. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In International Conference on Machine Learning, 115–123 (2013).

21. 21.

Eggensperger, K. et al. Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. In NIPS workshop on Bayesian Optimization in Theory and Practice (2013).

22. 22.

Young, R. A., Prince, E. & Sparks, R. A. Suggested guidelines for the publication of Rietveld analyses and pattern decomposition studies. J. Appl. Crystallogr. 15, 357–359 (1982).

23. 23.

LeBail, A., Duroy, H. & Fourquet, J. L. Ab-initio structure determination of LiSbWO6 by X-ray powder diffraction. Mater. Res. Bull. 23, 452 (1988).

24. 24.

Hill, R. J. Rietveld refinement round robin. I. Analysis of standard X-ray and neutron data for PbSO4. J. Appl. Crystallogr. 25, 589–610 (1992).

25. 25.

Harris, K. D. M., Tremayne, M., Lightfoot, P. & Bruce, P. G. Crystal structure determination from powder diffraction data by Monte Carlo methods. J. Am. Chem. Soc. 116, 3543–3547 (1994).

26. 26.

Young, R. A.The Rietveld Method. IUCr monographs on crystallography (Oxford University Press, 1995).

27. 27.

Shankland, K., David, W. I. F. & Csoka, T. Crystal structure determination from powder diffraction data by the application of a genetic algorithm. Z. Kristallog. 212, 550–552 (1997).

28. 28.

Harris, K. D. M., Johnston, R. L. & Kariuki, B. M. The genetic algorithm: foundations and apllications in structure solution from powder diffraction data. Acta Crystallogr. Sect. A 54, 632–645 (1998).

29. 29.

Toby, B. H. R factors in Rietveld analysis: how good is good enough? Powder Diffr. 21, 67–70 (2006).

30. 30.

Tsubota, M. & Jiro, K. A necessary criterion for obtaining accurate lattice parameters by Rietveld method. Sci. Rep. 7, 15381 (2017).

31. 31.

Fancher, C. M. et al. Use of Bayesian inference in crystallographic structure refinement via full diffraction profile analysis. Sci. Rep. 6, 31625 (2016).

32. 32.

Oishi-Tomiyasu, R. Application of convex optimization to identification of atomic species from diffraction patterns. Powder Diffr. 32, S179–S185 (2017).

33. 33.

Tomiyasu, K., Oishi-Tomiyasu, R., Matsuda, M. & Matsuhira, K. A new mathematical approach to finding global solutions of the magnetic structure determination problem. Sci. Rep. 8, 16228 (2018).

34. 34.

Rodríguez-Carvajal, J. Recent advances in magnetic structure determination by neutron powder diffraction. Phys. B: Condens. Matter 192, 55–69 (1993).

35. 35.

Rodríguez-Carvajal, J. Recent developments of the program FULLPROF. IUCR 26, 12–19 (2001).

36. 36.

Toby, B. H. EXPGUI, a graphical user interface for GSAS. J. Appl. Crystallogr. 34, 210–213 (2001).

37. 37.

Larson, A. C. & VonDreele, R. B. General structure analysis system (gsas). Tech. Rep. LAUR-86–748, Los Alamos National Laboratory, New Mexico (2004).

38. 38.

Izumi, F. & Momma, K. Three-dimensional visualization in powder diffraction. Solid State Phenom. 130, 15–20 (2007).

39. 39.

Toby, B. H. & VonDreele, R. B. GSAS-II: The genesis of a modern open-source all-purpose crystallography software package. J. Appl. Crystallogr. 46, 544–549 (2013).

40. 40.

Rajiv, P., Dinnebier, R. E. & Jansen, M. “Powder 3D Parametric” a program for automated sequential and parametric Rietveld pefinement using topas. Mater. Sci. Forum 651, 97–104 (2010).

41. 41.

Tian, P. & Billinge, S. Testing different methods for estimating uncertainties on Rietveld refined parameters using srrietveld. Z. Kristallogr. 226, 898–904 (2011).

42. 42.

Tian, P. et al. SrRietveld: a program for automating Rietveld refinements for high-throughput powder diffraction studies. J. Appl. Crystallogr. 46, 255–258 (2013).

43. 43.

Cui, X. et al. AutoFP: a GUI for highly automated Rietveld refinement using an expert system algorithm based on FullProf. J. Appl. Crystallogr. 48, 1581–1586 (2015).

44. 44.

Feng, Z. et al. Method of artificial intelligence algorithm to improve the automation level of Rietveld refinement. Computational Mater. Sci. 156, 310–314 (2019).

45. 45.

Sutton, R. S. & Barto, A. G. Reinforcement learning: an introduction. In Adaptive Computation and Machine Learning Series (MIT Press, 2018).

46. 46.

Viswanathan, G. et al. Single-crystal automated refinement (SCAR): a data-driven method for determining inorganic structures. Inorg. Chem. 58, 9004–9015 (2019).

47. 47.

AutoFP GitHub repository. https://github.com/xpclove/autofp/.

48. 48.

Torgerson, W.S. Theory and Methods of Scaling (Wiley, 1958).

49. 49.

Borg, I. & Groenen, P. Modern multidimensional scaling: theory and applications. J. Educ. Meas. 40, 277–280 (2003).

50. 50.

Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2623–2631 (ACM, 2019).

51. 51.

Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

52. 52.

Powbase. http://www.cristal.org/powbase/.

## Acknowledgements

This work is partly supported by JST-Mirai Program, Grant Number JPMJMI19G1. K.O. and T.H. are partly supported by the Elements Strategy Initiative Center for Magnetic Materials (ESICMM), Grant Number 12016013, through the Ministry of Education, Culture, Sports, Science and Technology (MEXT). Y.O. and M.O. are supported by the New Energy and Industrial Technology Development Organization (NEDO). Y.S. is supported by the Japan Science and Technology Agency (JST), ACT-I, grant number JPMJPR18UE. K.S. has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie grant agreement No. 701647. K.O. and K.S. gratefully acknowledge the financial support by Toyota Motor Corporation.

## Author information

Authors

### Contributions

Y.O., Y.S. and K.O. conceived the idea for the present work. Y.O. and Y.S. carried out the numerical experiments. T.H. and K.S. examined the Rietveld refinement results as experts in materials science. All authors discussed the results and wrote the manuscript.

### Corresponding author

Correspondence to Kanta Ono.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Ozaki, Y., Suzuki, Y., Hawai, T. et al. Automated crystal structure analysis based on blackbox optimisation. npj Comput Mater 6, 75 (2020). https://doi.org/10.1038/s41524-020-0330-9

• Accepted:

• Published:

• DOI: https://doi.org/10.1038/s41524-020-0330-9

• ### Chemical crystallography by serial femtosecond X-ray diffraction

• Elyse A. Schriber
• Daniel W. Paley
• J. Nathan Hohman

Nature (2022)

• ### Automation of diffusion database development in multicomponent alloys from large number of experimental composition profiles

• Jing Zhong
• Li Chen
• Lijun Zhang

npj Computational Materials (2021)

• ### Automated stopping criterion for spectral measurements with active learning

• Tetsuro Ueno
• Hideaki Ishibashi
• Kanta Ono

npj Computational Materials (2021)

• ### Visualization of judgment regions in convolutional neural networks for X-ray diffraction and scattering images of aliphatic polyesters

• Yoshifumi Amamoto
• Hiroteru Kikutake
• Kei Terayama

Polymer Journal (2021)

• ### "What is This?" A Structure Analysis Tool for Rapid and Automated Solution of Small Molecule Structures

• Takashi Matsumoto
• Akihito Yamano
• Mathias Meyer

Journal of Chemical Crystallography (2021)