Abstract
Parameter optimization is a long-standing challenge in various production processes. Particularly, powder film forming processes entail multiscale and multiphysical phenomena, each of which is usually controlled by a combination of several parameters. Therefore, it is difficult to optimize the parameters either by numerical-model-based analysis or by “brute force” experiment-based exploration. In this study, we focus on a Bayesian optimization method that has led to breakthroughs in materials informatics. Specifically, we apply this method to exploration of production-process-parameter for the powder film forming process. To this end, a slurry containing a powder, polymer, and solvent was dropped, the drying temperature and time were controlled as parameters to be explored, and the uniformity of the fabricated film was evaluated. Using this experiment-based Bayesian optimization system, we searched for the optimal parameters among 32,768 (85) parameter sets to minimize defects. This optimization converged at 40 experiments, which is a substantially smaller number than that observed in brute-force exploration and traditional design-of-experiments methods. Furthermore, we inferred the mechanism corresponding to the unknown drying conditions discovered in the parameter exploration that resulted in uniform film formation. This demonstrates that a data-driven approach leads to high-throughput exploration and the discovery of novel parameters, which inspire further research.
Similar content being viewed by others
Introduction
Background
The examination of production processes is necessary for attaining the inherent characteristics of any novel material and for realizing the desired performance of products. Among such processes, powder film forming is a fundamental process that must be used in either prototyping or mass production to fabricate functional devices such as rechargeable batteries1, fuel cells2,3,4, solar cells5, and water electrolysis systems6.
The powder film forming process is a production process that is used to manufacture thin-film products from powdery materials, such as carbon or ceramic powders. It consists of several major subprocesses (typically dispersion, mixing, coating, and drying), all of which have a significant impact on the performance of the final product. However, similar to most production processes, the powder film forming process is controlled by a number of parameters, and the phenomena entailed in the process are complex. Efforts to elucidate the details are still ongoing7,8,9. Given the complexity of the phenomena, it is almost impossible to build an adequate model that describes the entire process10, which indicates that numerical approaches are impractical. In addition, the large number of parameters hinder the evaluation of all combinations of parameters using a brute-force approach. Determining the parameters of the process depends on empirical rules and the extensive effort and skill of engineers and researchers; therefore, it cannot be guaranteed that any adopted parameter set would be optimal. Therefore, a non-parametric, sample-efficient parameter exploration method should be developed for the powder film forming process.
Related work
A similar problem has persisted in the field of materials science. However, in recent years, the application of machine learning to materials exploration has achieved some success11,12,13,14, enabling high-throughput exploration of, particularly, alloy compositions15, which was previously based on empirical or brute-force methods. In particular, in previous studies using active learning, including Bayesian optimization (BO), efficient searches were performed with a small number of samples (a few percent of the entire candidates)16,17,18. BO is a powerful optimization method that often uses Gaussian process regression (GPR) to predict the relationship between parameters and performance19,20. GPR can estimate a model considering uncertainty, and BO using GPR is effective for the efficient exploration of a large parameter space17. The application of BO has produced remarkable results in a wide range of fields. For example, in the fields of robotics, BO has been applied to learn a controller for a bipedal robot21 and robotics grasping22,23. Prior work also demonstrated that BO is effective for transfer learning in natural language processing24. More recently, BO was applied to optimize the hyperparameters of deep learning for AI, which beat a professional human Go player25. Therefore, BO is also expected to be a suitable approach for production processes, in which it is challenging to search for the optimal process parameters manually and heuristically. In fact, BO has been applied to search for parameters in processes such as polymer fiber synthesis26, TiO2 film depositions27, and gas atomization of alloy powders28. However, there were some challenges that needed to be addressed, e.g., the explored parameter space was not large enough for practical use and the product evaluation was subjective and not sufficiently reliable. When applying BO to a production process that does not have any suitable numerical model, it is necessary to repeat the experiments (fabrication and evaluation of the test specimens) during the optimization. Because the exploration in BO progresses based only on the results of the experiments conducted without assuming any prior knowledge, the experiments used to train the optimization system must be reproducible and quantitative. Therefore, most attempts to apply BO methods to materials development have used numerical simulations instead of experiments29,30,31,32. Nevertheless, pioneering research using industrial robots to overcome the problem of reproducibility and to conduct searches autonomously are being conducted for catalytic33, organic34, and inorganic materials27.
Aim of this study
Film manufacturing processes that utilize raw materials in powder form are often used in the fabrication of polymer electrolyte fuel cell (PEFC) electrodes35,36,37. A PEFC is a type of fuel cell that has attracted considerable attention as a power source for automobiles and for portable use because of its low operating temperature. The catalyst layer in PEFC electrodes consists of carbon particles with Pt loaded on the surface and a polymer. The polymer acts as an ionomer that provides a conduction pathway for protons during power generation; Nafion™ is generally used as the polymer. In the dispersion step, these materials are dispersed in water and alcohol. Carbon particles that are several tens of nanometers in diameter are widely used, and the particles agglomerate in the solvent to form secondary particles that are several hundreds of nanometers in diameter38. The ionomers are generally well dispersed in alcohol; however, similar to carbon particles, the ionomers agglomerate and become stable in a colloidal state. The ionomers also agglomerate around the secondary particles of carbon to form shells39,40. The degree of dispersion and agglomeration of the materials depend on the fabrication steps used. This ultimately affects the microstructure and distribution of components after drying. Notably, these features affect the power generation performance of PEFCs depending on electrochemical reaction activity, gas diffusivity and proton conductivity; therefore, improvements to the fabrication process of electrodes are essential for increasing the efficiency of PEFCs36,41,42. The dispersion and agglomeration of material particles also cause surface defects like cracks on the fabricated film43. It is possible to evaluate the power generation performance indirectly by evaluating the easily observable defects like cracks44. Therefore, in this study, cracks in the film are selected as targets for optimization. In the industry, PEFC catalyst layers are fabricated by brush coating, doctor blade, inkjet printing, or other methods45,46; however, we conducted experiments using drop casting, which can easily produce multiple samples with a small amount of slurry. Prototyping with a small amount of material is also meet the demands of the product development stage. Drop casting also suffers from the aforementioned problems of film forming and is appropriate for use in a case study as a simple and fundamental example of film forming47.
In this study, we demonstrate the optimization of the drying process in PEFC electrode fabrication. Figure 1 shows a conceptual diagram of the proposed experiment-based BO. The material slurry was dried while controlling the heating temperature, and the homogeneity of the resulting electrode film was quantitatively evaluated. Based on the obtained datasets of temperature profiles and electrode film states, the machine learning system estimated a surrogate model and proposed the parameters under which more homogeneous films were expected to be formed. Then, experiments were conducted accordingly. By repeating the experiment cycles, we could explore the parameters with a smaller number of experiments than that required in conventional methods. Moreover, the detailed mechanism of the drying phenomenon could be determined. In our previous study48, we confirmed that the parameters determined by BO were partially applicable for this process. However, the effectiveness of the parameter exploration system was not evaluated. In this study, we further discuss the process phenomena using the exploration results and evaluate the performance of the parameter exploration system to demonstrate its effectiveness using BO. This approach is useful for improving the efficiency of parameter exploration of production processes, and it is the first step towards elucidating the complicated phenomena entailed in such processes.
Experimental procedure
To estimate the surrogate model for the first iteration of the search, we conducted 30 experiments using random temperature profiles (see “Methods” section for details). Although a difference in the initial states has some effect on the number of experiments required for convergence26, the random profiles were adopted to evaluate the exploration from an initial state without bias. Thirty microliters of the slurry were dropped onto a glass slide placed on a heater and dried while controlling the temperature of the heater. The temperature profile had five time steps, and each time step was assigned one of the eight temperature levels. There were 85 = 32,768 candidate combinations in the exploration space. In the initial dataset preparation phase, experiments were conducted with 30 temperature profiles that were randomly selected from all the possible combinations. For each fabricated electrode, cracks and exceedingly thin regions were extracted based on the brightness values of the surface images, and the proportion of the detected areas to the total area where the dropped ink spread was evaluated as the “defect ratio”.
Using the obtained initial data as a starting point, we conducted parameter exploration via BO. To predict the defect ratio, we built a surrogate model using GPR49, which is commonly used in BO17,29. After each fabrication, an acquisition function was calculated for each temperature profile using the predicted values of the surrogate model, and the temperature profile with the highest score was adopted as the subsequent experimental parameter set. The acquisition function was set to the upper confidence bound (UCB)50,51,52. The UCB strategy balances exploration and exploitation during optimization. In BO, the superiority of the UCB strategy over other strategies, such as expected improvement and probability of improvement has been reported21,53. Although the advantages of other acquisition functions have been suggested54, investigating the best acquisition function is out of the scope of this study. We performed the fabrication process ten times to evaluate the defect ratio, and the experimental parameter set for this procedure was selected each time.
Results
Observation of the fabricated electrodes
For most of the temperature profiles, defect ratios of 30% to 50% were observed; however, in rare samples (3 out of 30 samples), the defect ratios were below 15% (Fig. 2a). In the samples with high defect ratios, clear radial cracks at the center of the electrodes and an exceedingly thin region at the periphery were typically observed, and carbon was unevenly deposited in concentric circles (Fig. 2b). By contrast, the samples with a low defect ratio had no thin regions at the periphery, and sparse cracks were observed; however, they were not clearly radial (Fig. 2c). The diameter of the fabricated films was approximately 13 mm and the thickness was approximately 30 μm at the thickest area.
Parameter exploration
Figure 3 shows the relationship between the temperature profiles and the defect ratios predicted by GPR during the exploration, using two of the five parameters for visualization. From the figures, it can be confirmed that as the search progresses, the regions wherein low defect ratios are predicted become more limited, while the variance of the predictions become smaller. The temperature profiles and observed defect ratios in each experiment are shown in Fig. 4, and the comparison of the defect ratio distribution with the initial data is shown in Fig. 5. In the exploration phase, the defect ratios were less than 10% for all the tested parameter sets, and the lowest value was 2.8%, which was obtained in experiments four and nine. Moreover, a lower defect ratio was observed as the exploration progressed. These results suggest that the application of BO makes parameter exploration more efficient than iterations of random experiments. During the initial data preparation phase, wherein the process parameters are selected randomly, even if a temperature parameter that achieves a relatively low defect ratio is found, it is challenging to guarantee the optimality of the temperature profile because of the large search space. However, exploration using the combination of GPR and UCB strategy found the temperature profile that outperforms other temperature profiles, and the convergence of the defect ratio can be a criterion for stopping the exploration in practice. Performance prediction maps obtained during the exploration, such as the one shown in Fig. 3, provide a basis for the relative usefulness of the proposed process parameters. Moreover, the maps are clues that lead to the elucidation of the phenomena occurring in the process of interest, as discussed in the subsequent section.
Examination of the drying mechanism based on knowledge extracted from the regression results
Upon examining the relationship between the temperature profiles and defect ratios, presented in Fig. 4, low defect ratios tended to be observed in the temperature profiles wherein the temperature of the first step was low (30 °C), that of the second step was relatively high (above 60 °C), and that of the third step was also relatively low (below 60 °C). This trend is also confirmed by the performance prediction map obtained after ten experiments (Fig. 3c). Comparing experiments five and six, it seems that raising the temperature after lowering it once has the effect of reducing the defect ratio. In industrial powder drying processes, it is common to employ a single drying temperature, rate of temperature increase, and holding time, without finely controlling the temperature profile. Therefore, it is likely that the low–high–low (–high) temperature profile proposed and validated based on the machine learning system cannot be discovered via trials following conventional methods. The phenomena occurring in the drying process are discussed as follows (Fig. 6).
Immediately after the drop, the slurry had a low solid phase ratio, and maintaining a low temperature caused the solvent to evaporate gently, leading to an increase in the solid phase ratio. If the slurry was heated to a high temperature immediately after the drop, the carbon particles would have flowed outward, forming an uneven distribution, and the final film would have had an uneven thickness. In addition, concentric and uneven precipitation of carbon particles is observed in the samples heated rapidly immediately after dropping (Fig. 6a), and this non-uniformity is commonly observed upon the drying of dilute solutions55,56,57. Once the solid phase ratio is sufficiently increased by drying at a low temperature, accelerated evaporation by increasing the temperature would not cause uneven precipitation. Nonetheless, it was possible for the evaporation of the solvent to progress on the surface of the liquid film, forming a skin of carbon particles bonded to each other and suppressing intensive evaporation. During the drying of colloidal solutions, the packing of particles on the surface generally lowers the evaporation rate58. A low evaporation rate may be realized if a low temperature is maintained throughout the drying process; nevertheless, the evaporation rate can be lower if the aforenoted skin is formed by heating once. Moreover, the defect ratio was not substantially low under the continuous-low-temperature profiles (Fig. 6c). After the skin is formed, the temperature is decreased, and the remaining solvent is gently evaporated from the inside of the film, followed by complete drying of the film. This results in a crack-free film with a uniform thickness (Fig. 6d). The effect of heating on the formation of the skin and the subsequent defect reduction was confirmed in our previous study48. In the samples that continued to be heated at high temperatures even after the skin was formed, radial cracks were observed. This can be attributed to the rapid evaporation of the solvent, inevitably leading to the fixing of the position of the carbon particles and causing residual stress in the film (Fig. 6b). Heating at the last stage of drying reduces the defect ratio, likely because precipitated Nafion™ can deform near the glass-transition temperature and relax the stress (Fig. 6e)59. Considering the drying mechanism discussed above, the temperature profile, which enabled low defect ratios, discovered by the machine learning system was in an exceedingly limited process window and was not expected to be found as an extension of the traditional drying methods.
Discussion
We demonstrated that appropriate process windows can be discovered with a high throughput and a small number of samples by repeating experiments using the parameters provided by the machine learning system. The total number of experiments conducted in this study was 40, corresponding to approximately 0.1% of all the possible parameter combinations. Therefore, the exploration was 1,000 times more efficient than evaluating all the parameters individually. Even if a researcher or an engineer, rather than a machine learning system, was to consider previous results and propose subsequent experimental parameters for each experiment, it would be almost impossible for a human to understand the distribution of the predicted performance in a five-dimensional (in this study) parameter space. Thus, such an efficient exploration would not be possible. Furthermore, while conventional parameter adjustments in trial-and-error methods require experts to analyze the results for each experiment, in our method, human intervention is required only for setting the search space and for the analysis of the entire exploration result. Thus, in addition to the time cost, the human cost can be significantly reduced. The advantage of machine learning systems becomes even more significant when the number and/or range of parameters is expanded. However, this does not undermine the value of specialists who have a deeper understanding and keen insight into the processes and phenomena involved, and their focus remains downstream. The map of the relationship between process parameters and performances, obtained via exploration using BO, contains a large amount of useful information. By examining this map closely, specific inferences regarding the phenomena that govern a process can be made, as we have done in this study. Such analyses are expected to lead to subsequent higher-throughput explorations. This study is a pioneering example of a “human-in-the-loop” system, wherein artificial intelligence sensitizes human intelligence to repeat high-throughput explorations.
Another highlight of this study is the application of existing powerful machine learning methods to process exploration by setting up appropriate optimization parameters and experimental/evaluation techniques for the target process. Although simultaneous exploration of numerous parameters is acceptable, parameters that do not affect the process or are difficult to control may increase the cost of the experiments. This contradicts the original purpose of a highly efficient search and makes the interpretation of the proposed parameter sets and the resulting maps difficult. In addition, if the evaluation criterion is not set appropriately, the experiments become more difficult and the measurement error may crucially affect the prediction results. We address these limitations by devising experimental equipment and using image processing, which are also areas that require the knowledge of specialists.
The experiment-based process parameter exploration demonstrated in this study based on PEFC electrode film-forming is a pioneering example of the “process informatics” methodology that follows materials informatics. Process informatics methods can be developed in various directions, such as by expanding the number and/or range of parameters, applying such methods to slightly different conditions, and applying such methods to processes in other fields. In this study, we used five-dimensional parameters and a single evaluation criterion. However, considering the demonstrated exploration efficiency, it is acceptable to explore larger parameter spaces and adopt more complex evaluation criteria. Moreover, even if the process preconditions that are not used for exploration change slightly, the trends of the performance maps are presumed to have a commonality. Therefore, more efficient exploration can be performed by referring to the results of previous explorations in similar processes60. The application of process informatics is not limited to powder film forming, and this method can be applied to other industrial processes to obtain outstanding results. This study is only a case study of a specific process for a certain material, and is insufficient to examine the robustness of the method for other materials and processes or the influence of the properties of the input data (such as number, error, and density of the data). Nevertheless, process informatics is useful as a method for the rapid optimization of processes associated with novel materials proposed in materials informatics. Once a sufficiently high-throughput process informatics method is established, the bottleneck in materials development may become the prototyping of the materials.
Methods
Preparation of slurry for dropping and fabrication of samples
The specimen slurry comprised carbon particles (Vulkan® XC-72R, Cabot) and 5% Nafion™ dispersion solution (DE520 CS type, FUJIFILM Wako Chemical Corporation). These materials were added to water and 2-propanol at an ionomer/carbon weight ratio of 0.8, solid phase ratio of 0.1, and water ratio of 0.3. In this study, we did not conduct a power generation evaluation; therefore, carbon particles without Pt catalyst were used. Just before the experiments, this slurry was diluted three times with IPA and was subsequently subjected to 10 min of stirring using a planetary centrifugal mixer (mixing at 2,000 rpm for 5 min and defoaming at 2,200 rpm for 5 min) and 10 min sonication at 45 kHz.
The slurry was dropped onto a glass slide on a heater using a motorized pipette (dPette 30–300, DLAB) handled by a robot arm (DOBOT Magician, Shenzhen Yuejiang Technology). We used an automated dropping system for reproducible dropping. In each experiment, 30 μL of the slurry was dropped and dried. The heater temperature was maintained at 30 °C even before the drop, and one of eight temperature levels, with equal intervals from 30 to 100 °C, was applied every 120 s for 600 s in the range of 30–630 s after the drop. After 630 s of drying following the drop, the fabricated electrode was removed from the heater and observed using transmitted light. Regarding the quantitative evaluation, sample images were processed using OpenGL. The films were photographed at 504 × 504 pixels using transmitted light. The captured area corresponded to approximately 16 mm2. The defect ratio was calculated by separating the area where the dropped ink spread (droplet area) from the background and the detection of the defect areas in the droplet area. First, the raw images were binarized, and the contour of the droplet was detected according to the brightness values. To prevent cracks reaching the outer edge of the droplet from being treated as background, the contour was smoothed by expanding the contour line width once and then shrinking the width. The interior area of the resulting contour was evaluated as the droplet area. The black regions in Fig. 2b and c were excluded as background by the above process. The droplet areas were binarized again, and the areas with high brightness values, that is, areas where the formed electrode layer was thin or almost absent and most of the reference light was transmitted, were detected as defects, as demonstrated by the gray regions in Fig. 2b and c. All samples were captured and analyzed under the same conditions, and the binarization thresholds were set at 150 out of 256 shades for background separation and 50 out of 256 shades for defect detection. The proportion of the defect area to the droplet area was defined as the defect ratio.
Gaussian process
We used GPR to estimate the relationship between the temperature profiles and defect ratios. The GPR algorithm used in this study is available in scikit-learn61.To estimate the relationship, we have to find the model \(p(y|{\varvec{x}},{ \mathcal{D}})\), given a dataset \({\mathcal{D}} = \left\{ {{\varvec{x}}_{i} , y_{i} } \right\}_{i = 1}^{n}\), where \({\varvec{x}}\) is the parameter of the temperature profile and \(y\) is the resulting defect ratio. Because the fabricated samples are observed with noise, the regression problem to solve is given by
where we assume a zero mean Gaussian noise. By using GPR:
where \(\mu ({\varvec{x}})\) is the mean function and \(k({\varvec{x}}, {\varvec{x}}\boldsymbol{^{\prime}})\) is the covariance function of the Gaussian process. In this study, the covariance is modeled by the radial basis function (RBF):
where \(C\) is a constant and \(l\) is the length-scale parameter. Considering the initial conditions, we set \(\mu \left( {\varvec{x}} \right) = 0\) as the mean function. The set of hyper parameters \({\varvec{\theta}} = \left\{ {C, l, \beta } \right\}\) can be optimized by maximizing the marginal likelihood given by
In scikit-learn, the optimization of \({\varvec{\theta}}\) is implemented using the L-BFGS-B algorithm62 and executed for each regression. Given the dataset \({\mathcal{D}}\), the covariance matrix between the previously observed \({\varvec{x}}\) and \({\varvec{y}}\) is given as
The joint Gaussian probability of the dataset \({\mathcal{D}}\) and the prediction of \(y^{ + }\), which is expected to be observed with a new parameter \({\varvec{x}}^{ + }\), is, assuming a zero mean prior, given by
with
Then, conditioning the Gaussian process on \({\varvec{x}}^{ + }\), the predictive posterior \(p\left( {y^{ + } {|}{\varvec{x}},{\mathcal{D}}} \right)\) of \({\varvec{x}}^{ + }\) is given by the Gaussian:
with the mean and variance calculated by
The obtained means \(\mu \left( {{\varvec{x}}^{ + } } \right)\) and variance \(\sigma \left( {{\varvec{x}}^{ + } } \right)\) were used to calculate the acquisition function.
Acquisition function
In this study, since the objective is to minimize the defect ratio, the objective function \(R\left( {\varvec{x}} \right)\) is given as follows:
where \(D\left( {\varvec{x}} \right)\), which is an unknown function, is the value of the defect ratio in the experiment with parameter \({\varvec{x}}\). The UCB strategy was used to determine the temperature profiles of the experiments. The acquisition function \(U\left( {\varvec{x}} \right)\) for the UCB strategy is given by
where \(\mu \left( {R\left( {\varvec{x}} \right)} \right)\) is the mean value and \(\sigma \left( {R\left( {\varvec{x}} \right)} \right)\) is the standard deviation of \(R\left( {\varvec{x}} \right)\). The coefficient \(c\) of \(\sigma \left( {R\left( {\varvec{x}} \right)} \right)\) is a constant that determines the balance between exploration and exploitation. The UCB strategy shows good performance for various tasks when \(c\) is in the range of 0.1 to 154. In this study, we set \(c = 0.5\).
Data availability
The data that support the findings of this study are available from the corresponding author on request.
References
Kwade, A. et al. Current status and challenges for automotive battery production technologies. Nat. Energy 3, 290–300. https://doi.org/10.1038/s41560-018-0130-3 (2018).
Mehta, V. & Cooper, J. S. Review and analysis of PEM fuel cell design and manufacturing. J. Power Sources 114, 32–53. https://doi.org/10.1016/S0378-7753(02)00542-6 (2003).
Will, J., Mitterdorfer, A., Kleinlogel, C., Perednis, D. & Gauckler, L. J. Fabrication of thin electrolytes for second-generation solid oxide fuel cells. Solid State Ionics 131, 79–96. https://doi.org/10.1016/S0167-2738(00)00624-X (2000).
Duan, N. Q., Yan, D., Chi, B., Pu, J. & Jian, L. High performance anode-supported tubular solid oxide fuel cells fabricated by a novel slurry-casting method. Sci. Rep. 5, 1–4. https://doi.org/10.1038/srep08174 (2015).
Hashmi, G. et al. Review of materials and manufacturing options for large area flexible dye solar cells. Renew. Sustain. Energy Rev. 15, 3717–3732. https://doi.org/10.1016/j.rser.2011.06.004 (2011).
Carmo, M., Fritz, D. L., Mergel, J. & Stolten, D. A comprehensive review on PEM water electrolysis. Int. J. Hydr. Energy 38, 4901–4934. https://doi.org/10.1016/j.ijhydene.2013.01.151 (2013).
Font, F., Protas, B., Richardson, G. & Foster, J. M. Binder migration during drying of lithium-ion battery electrodes: Modelling and comparison to experiment. J. Power Sources 393, 177–185. https://doi.org/10.1016/j.jpowsour.2018.04.097 (2018).
Kobayashi, N. et al. Crack formation in polymer nanocomposite thin films containing surface-modified nanoparticles during solution casting. J. Chem. Eng. Japan 51, 460–468. https://doi.org/10.1252/jcej.17we323 (2018).
Fernandes, I. J. et al. Silver nanoparticle conductive inks: Synthesis, characterization, and fabrication of inkjet-printed flexible electrodes. Sci. Rep. 10, 1–11. https://doi.org/10.1038/s41598-020-65698-3,Pubmed:32483302 (2020).
Maki, K. L. & Kumar, S. Fast evaporation of spreading droplets of colloidal suspensions. Langmuir 27, 11347–11363. https://doi.org/10.1021/la202088s (2011).
Ward, L. & Wolverton, C. Atomistic calculations and materials informatics: a review. Curr. Opin. Solid State Mater. Sci. 21, 167–176. https://doi.org/10.1016/j.cossms.2016.07.002 (2017).
Ramprasad, R., Batra, R., Pilania, G., Mannodi-Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: Recent applications and prospects. npj Comput. Mater. 3, 1–13. https://doi.org/10.1038/s41524-017-0056-5 (2017).
Jablonka, K. M., Ongari, D., Moosavi, S. M. & Smit, B. Big-data science in porous materials: Materials genomics and machine learning. Chem. Rev. 120, 8066–8129. https://doi.org/10.1021/acs.chemrev.0c00004,Pubmed:32520531 (2020).
Chen, C. et al. A critical review of machine learning of energy materials. Adv. Energy Mater. 10, 1–36. https://doi.org/10.1002/aenm.201903242 (2020).
Wen, C. et al. Machine learning assisted design of high entropy alloys with desired property. Acta Mater. 170, 109–117. https://doi.org/10.1016/j.actamat.2019.03.010 (2019).
Kusne, A. G. et al. On-the-fly closed-loop materials discovery via Bayesian active learning. Nat. Commun. 11, 5966. https://doi.org/10.1038/s41467-020-19597-w,Pubmed:33235197 (2020).
Lookman, T., Balachandran, P. V., Xue, D. & Yuan, R. Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design. npj Comput. Mater. 5, 1–17. https://doi.org/10.1038/s41524-019-0153-8 (2019).
Ju, S. et al. Designing nanostructures for phonon transport via Bayesian optimization. Phys. Rev. X 7, 1–10. https://doi.org/10.1103/PhysRevX.7.021024 (2017).
Jones, D. R., Schonlau, M. & Welch, W. J. Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492. https://doi.org/10.1023/A:1008306431147 (1998).
Snoek, J., Larochelle, H. & Adams, R. P. Practical Bayesian optimization of machine learning algorithms. Preprint at https://arxiv.org/abs/1206.2944 (2012).
Calandra, R., Seyfarth, A., Peters, J. & Deisenroth, M. P. Bayesian optimization for learning gaits under uncertainty. Ann. Math. Artif. Intell. 76, 5–23. https://doi.org/10.1007/s10472-015-9463-9 (2016).
Osa, T., Peters, J. & Neumann, G. Hierarchical reinforcement learning of multiple grasping strategies with human instructions. Adv. Robot. 32, 955–968. https://doi.org/10.1080/01691864.2018.1509018 (2018).
Daniel, C., Viering, M., Metz, J., Kroemer, O. & Peters, J. Active reward learning. In Proceedings of Robotics: Science and Systems (2014).
Ruder, S. & Plank, B. Learning to select data for transfer learning with Bayesian Optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 372–382 (2017).
Chen, Y. et al. Bayesian optimization in alphago. Preprint at https://arxiv.org/abs/1812.06855 (2018).
Li, C. et al. Rapid Bayesian optimisation for synthesis of short polymer fiber materials. Sci. Rep. 7, 1–10. https://doi.org/10.1038/s41598-017-05723-0,Pubmed:28720869 (2017).
Shimizu, R., Kobayashi, S., Watanabe, Y., Ando, Y. & Hitosugi, T. Autonomous materials synthesis by machine learning and robotics. APL Mater. 8, 111110. https://doi.org/10.1063/5.0020370 (2020).
Tamura, R. et al. Machine learning-driven optimization in powder manufacturing of Ni-Co based superalloy. Mater. Des. 198, 109290. https://doi.org/10.1016/j.matdes.2020.109290 (2021).
Balachandran, P. V., Xue, D., Theiler, J., Hogden, J. & Lookman, T. Adaptive strategies for materials design using uncertainties. Sci. Rep. 6, 19660. https://doi.org/10.1038/srep19660,Pubmed:26792532 (2016).
Kiyohara, S., Oda, H., Tsuda, K. & Mizoguchi, T. Acceleration of stable interface structure searching using a kriging approach. Jpn. J. Appl. Phys. 55, 045502. https://doi.org/10.7567/JJAP.55.045502 (2016).
Jalem, R. et al. Bayesian-driven first-principles calculations for accelerating exploration of fast ion conductors for rechargeable battery application. Sci. Rep. 8, 1–10. https://doi.org/10.1038/s41598-018-23852-y,Pubmed:29643423 (2018).
Gopakumar, A. M., Balachandran, P. V., Xue, D., Gubernatis, J. E. & Lookman, T. Multi-objective optimization for materials discovery via adaptive design. Sci. Rep. 8, 1–12. https://doi.org/10.1038/s41598-018-21936-3,Pubmed:29487307 (2018).
Burger, B. et al. A mobile robotic chemist. Nature 583, 237–241. https://doi.org/10.1038/s41586-020-2442-2,Pubmed:32641813 (2020).
MacLeod, B. P. et al. Self-driving laboratory for accelerated discovery of thin-film materials. Sci. Adv. 6, eaaz8867 (2020). https://doi.org/10.1126/sciadv.aaz8867, Pubmed: 32426501.
Litster, S. & McLean, G. PEM fuel cell electrodes. J. Power Sources 130, 61–76. https://doi.org/10.1016/j.jpowsour.2003.12.055 (2004).
Holdcroft, S. Fuel cell catalyst layers: A polymer science perspective. Chem. Mater. 26, 381–393. https://doi.org/10.1021/cm401445h (2014).
Huang, J., Li, Z. & Zhang, J. Review of characterization and modeling of polymer electrolyte fuel cell catalyst layer: The blessing and curse of ionomer. Front. Energy 11, 334–364. https://doi.org/10.1007/s11708-017-0490-6 (2017).
Xu, F. et al. Investigation of a catalyst ink dispersion using both ultra-small-angle X-ray scattering and cryogenic TEM. Langmuir 26, 19199–19208. https://doi.org/10.1021/la1028228,Pubmed:21090580 (2010).
Malek, K., Mashio, T. & Eikerling, M. Microstructure of catalyst layers in PEM fuel cells redefined: a computational approach. Electrocatalysis 2, 141–157. https://doi.org/10.1007/s12678-011-0047-0 (2011).
Kusano, T. et al. Structural evolution of a catalyst ink for fuel cells during the drying process investigated by CV-SANS. Polym. J. 47, 546–555. https://doi.org/10.1038/pj.2015.36 (2015).
Huang, D. C. et al. Effect of dispersion solvent in catalyst ink on proton exchange membrane fuel cell performance. Int. J. Electrochem. Sci. 6, 2551–2565 (2011).
Uchida, M. et al. Effect of the state of distribution of supported Pt nanoparticles on effective Pt utilization in polymer electrolyte fuel cells. Phys. Chem. Chem. Phys. 15, 11236–11247. https://doi.org/10.1039/c3cp51801a,Pubmed:23715296 (2013).
Kumano, N. et al. Controlling cracking formation in fuel cell catalyst layers. J. Power Sources 419, 219–228. https://doi.org/10.1016/j.jpowsour.2019.02.058 (2019).
Zhang, J. et al. Effect of catalyst layer microstructures on performance and stability for high temperature polymer electrolyte membrane fuel cells. J. Power Sources 505, 230059. https://doi.org/10.1016/j.jpowsour.2021.230059 (2021).
Wee, J. H., Lee, K. Y. & Kim, S. H. Fabrication methods for low-Pt-loading electrocatalysts in proton exchange membrane fuel cell systems. J. Power Sources 165, 667–677. https://doi.org/10.1016/j.jpowsour.2006.12.051 (2007).
Strong, A., Thornberry, C., Beattie, S., Chen, R. & Coles, S. R. Depositing catalyst layers in polymer electrolyte membrane fuel cells: A review. J. Fuel Cell Sci. Technol. 12, 064001. https://doi.org/10.1115/1.4031961 (2015).
Zhao, C. et al. Formation of uniform reduced graphene oxide films on modified PET substrates using drop-casting method. Particuology 17, 66–73. https://doi.org/10.1016/j.partic.2014.02.005 (2014).
Nagai, K. et al. Parameter optimization in the drying process of catalyst ink for PEFC electrode films with few cracks. ECS Trans. 104, 17–23. https://doi.org/10.1149/10409.0017ecst (2021).
Rasmussen, C. E. & Williams, C. K. I. Gaussian processes for machine learning (MIT, 2006).
Auer, P., Cesa-Bianchi, N. & Fischer, P. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256. https://doi.org/10.1023/A:1013689704352 (2002).
Auer, P. Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3, 397–422 (2002).
Srinivas, N., Krause, A., Kakade, S. & Seeger, M. Gaussian process optimization in the bandit setting: No regret and experimental design. Preprint at https://arxiv.org/abs/0912.3995 (2009).
Kandasamy, K., Schneider, J. & Póczos, B. High dimensional Bayesian optimisation and bandits via additive models. In Proceedings of the 32nd International Conference on Machine Learning (ICML’15), 295–304 (2015).
Merrill, E., Fern, A., Fern, X. & Dolatnia, N. An empirical study of Bayesian optimization: acquisition versus partition. J. Mach. Learn. Res. 22, 1–25 (2021).
Routh, A. F. Drying of thin colloidal films. Rep. Prog. Phys. 76, 046603. https://doi.org/10.1088/0034-4885/76/4/046603 (2013).
Deegan, R. D. et al. Capillary flow as the cause of ring stains from dried liquid drops. Nature 389, 827–829. https://doi.org/10.1038/39827 (1997).
Kumar, A. K. S., Zhang, Y., Li, D. & Compton, R. G. A mini-review: How reliable is the drop casting technique?. Electrochem. Commun. 121, 106867. https://doi.org/10.1016/j.elecom.2020.106867 (2020).
Vanderhoff, J. W., Bradford, E. B. & Carrington, W. K. The transport of water through latex films. J. Polym. Sci. C Polym. Symp. 41, 155–174. https://doi.org/10.1002/polc.5070410116 (1973).
Lee, W. P. & Routh, A. F. Temperature dependence of crack spacing in drying latex films. Ind. Eng. Chem. Res. 45, 6996–7001. https://doi.org/10.1021/ie051256m (2006).
Li, X. et al. A transfer learning approach for microstructure reconstruction and structure-property predictions. Sci. Rep. 8, 13461. https://doi.org/10.1038/s41598-018-31571-7,Pubmed:30194426 (2018).
Fabian, P. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Zhu, C., Byrd, R. H., Lu, P. & Nocedal, J. Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans. Math. Softw. 23, 550–560. https://doi.org/10.1145/279232.279236 (1997).
Acknowledgements
The authors thank Mr. Masato Kurosu (Yokohama National University) for his support in constructing the drying system; Mr. Masaya Honda (Kanazawa University) for his assistance in sample slurry preparation; and Dr. Park Kayoung (Kyushu University) and Mr. Shota Ishikawa (Kyushu University) for the analysis of the samples. This study was supported by JST-Mirai Programs (Grant Numbers JPMJMI19G3 and JPMJMI21G2), Japan. We would like to thank Editage (www.editage.com) for English language editing.
Author information
Authors and Affiliations
Contributions
The main idea of the study was conceived by K. Nagato. and K. Nagai. T.O. designed and examined the machine learning system. T.T., T.A., and K. Nagai examined the preparation of the slurry and the methods of drying. K. Nagai, K. Nagato, T.A., and M.T. constructed the experimental system, and K. Nagai performed the exploration experiments. All authors discussed the results and contributed to writing the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Nagai, K., Osa, T., Inoue, G. et al. Sample-efficient parameter exploration of the powder film drying process using experiment-based Bayesian optimization. Sci Rep 12, 1615 (2022). https://doi.org/10.1038/s41598-022-05784-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-05784-w
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.