Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Visualization of judgment regions in convolutional neural networks for X-ray diffraction and scattering images of aliphatic polyesters

Polymer Journal volume 53, pages 1269–1279 (2021)Cite this article

Subjects

Abstract

The construction of a deep learning model and visualization of judgment regions were conducted for X-ray diffraction and scattering images of aliphatic polyesters. Due to recent progress in measurement methods, a large amount of image data can be obtained in a short time; therefore, machine learning methods are useful to determine the important regions for a given objective. Although techniques to visualize the judgment regions using deep learning have recently been developed, there have been few reports discussing whether such models can determine the important regions of X-ray diffraction and scattering images of polymeric materials. Herein, we demonstrate classification models based on convolutional neural networks (CNNs) for wide-angle X-ray diffraction and small-angle X-ray scattering images of aliphatic polyesters to predict the types of polymers and several crystallization temperatures. Furthermore, the judgment regions of the X-ray images used by the CNNs were visualized using the Grad-CAM, LIME, and SHAP methods. The main regions were diffraction and scattering peaks recognized by experts. Other areas, such as the beam centers were recognized when the intensity of the images was randomly changed. This result may contribute to developing important features in deep learning models, such as the recognition of structure–property relationships.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Nguyen HK, Inutsuka M, Kawaguchi D, Tanaka K. Direct observation of conformational relaxation of polymer chains at surfaces. ACS Macro Lett. 2018;7:1198–202.

    Article  CAS  Google Scholar 

  2. Ueda E, Liang XB, Ito M, Nakajima K. Dynamic moduli mapping of silica-filled styrene-butadiene rubber vulcanizate by nanorheological atomic force microscopy. Macromolecules. 2019;52:311–9.

    Article  CAS  Google Scholar 

  3. Kakubo T, Shimizu K, Kumagai A, Matsumoto H, Tsuchiya M, Amino N, et al. Degradation of a metal-polymer interface observed by element-specific focused ion beam-scanning electron microscopy. Langmuir. 2020;36:2816–22.

    Article  CAS  Google Scholar 

  4. Kobayashi S, Kaneko S, Kiguchi M, Tsukagoshi K, Nishino T. Tolerance to stretching in Thiol-terminated single-molecule junctions characterized by surface-enhanced raman scattering. J Phys Chem Lett. 2020;11:6712–7.

    Article  CAS  Google Scholar 

  5. Yamamoto Y, Hoshina H, Sato H. Differences in intermolecular interactions and flexibility between Poly(ethylene terephthalate) and poly(butylene terephthalate) studied by far-infrared/Terahertz and low-frequency raman spectroscopy. Macromolecules. 2021;54:1052–62.

    Article  CAS  Google Scholar 

  6. Amamoto Y, Kikuchi M, Otsuka H, Takahara A. Arm-replaceable star-like nanogels: arm detachment and arm exchange reactions by dynamic covalent exchanges of alkoxyamine units. Polym J. 2010;42:860–7.

    Article  CAS  Google Scholar 

  7. Mitamura K, Yamada NL, Sagehashi H, Torikai N, Arita H, Terada M, et al. Novel neutron reflectometer SOFIA at J-PARC/MLF for in-situ soft-interface characterization. Polym J. 2013;45:100–8.

    Article  CAS  Google Scholar 

  8. Dechnarong N, Kamitani K, Cheng CH, Masuda S, Nozaki S, Nagano C, et al. In situ synchrotron radiation x-ray scattering investigation of a microphase-separated structure of thermoplastic elastomers under uniaxial and equi-biaxial deformation modes. Macromolecules. 2020;53:8901–9.

    Article  CAS  Google Scholar 

  9. Hiroi T, Hirosawa K, Okazumi Y, Pingali SV, Shibayama M. Mechanism of heat-induced gelation for ovalbumin under acidic conditions and the effect of peptides. Polym J. 2020;52:1263–72.

    Article  CAS  Google Scholar 

  10. Mayumi K. Molecular dynamics and structure of polyrotaxane in solution. Polym J. 2021;53:581–6.

    Article  CAS  Google Scholar 

  11. Tashiro K, Yamamoto H, Yoshioka T, Ninh TH, Tasaki M, Shimada S, et al. Hierarchical structural change in the stress-induced phase transition of poly(tetramethylene terephthalate) as studied by the simultaneous measurement of FTIR spectra and 2D synchrotron undulator WAXD/SAXS data. Macromolecules. 2014;47:2052–61.

    Article  CAS  Google Scholar 

  12. Diep PTN, Mochizuki M, Doi M, Takagi H, Shimizu N, Igarashi N, et al. Effects of a special diluent as an agent of improving the crystallizability of poly(L-lactic acid). Polym J. 2019;51:283–94.

    Article  Google Scholar 

  13. Kishimoto M, Mita K, Ogawa H, Takenaka M. Effect of submicron structures on the mechanical behavior of polyethylene. Macromolecules. 2020;53:9097–107.

    Article  CAS  Google Scholar 

  14. Nishitsuji S, Watanabe Y, Takebe T, Fujii N, Okano M, Takenaka M. X-ray scattering study on the changes in the morphology of low-modulus polypropylene under cyclic uniaxial elongation. Polym J. 2020;52:279–87.

    Article  CAS  Google Scholar 

  15. Toda A, Taguchi K, Nozaki K, Guan XC, Hu WB, Furushima Y, et al. Crystallization and melting of poly(butylene terephthalate) and poly (ethylene terephthalate) investigated by fast-scan chip calorimetry and small angle X-ray scattering. Polymer. 2020;192:122303.

    Article  CAS  Google Scholar 

  16. Fancher CM, Han Z, Levin I, Page K, Reich BJ, Smith RC, et al. Use of Bayesian inference in crystallographic structure refinement via full diffraction profile analysis. Sci Rep. 2016;6:31625.

    Article  CAS  Google Scholar 

  17. Mototake Y, Mizumaki M, Akai I, Okada M. Bayesian hamiltonian selection in X-ray photoelectron spectroscopy. J Phys Soc Jpn. 2019;88:034004.

    Article  Google Scholar 

  18. Saito K, Yano M, Hino H, Shoji T, Asahara A, Morita H, et al. Accelerating small-angle scattering experiments on anisotropic samples using kernel density estimation. Sci Rep. 2019;9:1526.

    Article  Google Scholar 

  19. Suzuki Y, Hino H, Kotsugi M, Ono K. Automated estimation of materials parameter from X-ray absorption and electron energy-loss spectra with similarity measures. npj Comput Mater. 2019;5:39.

    Article  Google Scholar 

  20. Miyazaki Y, Nakayama R, Yasuo N, Watanabe Y, Shimizu R, Packwood DM, et al. Bayesian statistics-based analysis of AC impedance spectra. Aip Adv. 2020;10:045231.

    Article  CAS  Google Scholar 

  21. Ozaki Y, Suzuki Y, Hawai T, Saito K, Onishi M, Ono K. Automated crystal structure analysis based on blackbox optimisation. npj Comput Mater. 2020;6:75.

    Article  Google Scholar 

  22. Amamoto Y, Kojio K, Takahara A, Masubuchi Y, Ohnishi T. Complex network representation of the structure-mechanical property relationships in elastomers with heterogeneous connectivity. Patterns. 2020;1:100135.

    Article  Google Scholar 

  23. Hagita K, Higuchi T, Jinnai H. Super-resolution for asymmetric resolution of FIB-SEM 3D imaging using AI with deep learning. Sci Rep. 2018;8:5877.

    Article  Google Scholar 

  24. Yamada H, Liu C, Wu S, Koyama Y, Ju SH, Shiomi J, et al. Predicting materials properties with little data using shotgun transfer learning. ACS Cent Sci. 2019;5:1717–30.

    Article  CAS  Google Scholar 

  25. Ma BY, Wei XY, Liu CN, Ban XJ, Huang HY, Wang H, et al. Data augmentation in microscopic images for material data mining. npj Comput Mater. 2020;6:125.

    Article  Google Scholar 

  26. Aoyagi T. Deep learning model for predicting phase diagrams of block copolymers. Comp Mater Sci. 2021;188:110224.

    Article  CAS  Google Scholar 

  27. Ienaga N, Higuchi K, Takashi T, Gen K, Tsuda K, Terayama K. Vision-based egg quality prediction in Pacific bluefin tuna (Thunnus orientalis) by deep neural network. Sci Rep. 2021;11:6.

    Article  CAS  Google Scholar 

  28. Matsumoto S, Ishida S, Araki M, Kato T, Terayama K, Okuno Y. Extraction of protein dynamics information from cryo-EM maps using deep learning. Nat Mach Intell. 2021;3:153–60.

    Article  Google Scholar 

  29. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis. 2020;128:336–59.

    Article  Google Scholar 

  30. Lundberg S, Lee S-I. A unified approach to interpreting model predictions. https://arxiv.org/abs/1705.07874v2. 2017.

  31. Ribeiro MT, Singh S, Guestrin C. “Why Should I Trust You?”: explaining the predictions of any classifier. https://arxiv.org/abs/1602.04938. 2016.

  32. Pokuri BSS, Ghosal S, Kokate A, Sarkar S, Ganapathysubramanian B. Interpretable deep learning for guided microstructure-property explorations in photovoltaics. npj Comput Mater. 2019;95:95.

    Article  Google Scholar 

  33. Horwath JP, Zakharov DN, Megret R, Stach EA. Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images. npj Comput Mater. 2020;108:108.

  34. Chollet F. Keras. https://keras.io. 2015.

  35. Ihn KJ, Yoo ES, Im SS. Structure and morphology of poly(Tetramethylene Succinate) crystals. Macromolecules. 1995;28:2460–4.

    Article  CAS  Google Scholar 

  36. Gan ZH, Abe H, Kurokawa H, Doi Y. Solid-state microstructures, thermal properties, and crystallization of biodegradable poly(butylene succinate) (PBS) and its copolyesters. Biomacromolecules. 2001;2:605–13.

    Article  CAS  Google Scholar 

  37. Puchalski M, Szparaga G, Biela T, Gutowska A, Sztajnowski S, Krucinska I. Molecular and supramolecular changes in polybutylene succinate (PBS) and polybutylene succinate adipate (PBSA) copolymer during degradation in various environmental conditions. Polymers. 2018;10:251.

  38. Hoogsteen W, Postema AR, Pennings AJ, Tenbrinke G, Zugenmaier P. Crystal-structure, conformation, and morphology of solution-spun poly(L-Lactide) fibers. Macromolecules. 1990;23:634–42.

    Article  CAS  Google Scholar 

  39. Huang SY, Li HF, Jiang SC, Chen XS, An LJ. Crystal structure and morphology influenced by shear effect of poly(L-lactide) and its melting behavior revealed by WAXD, DSC and in-situ POM. Polymer. 2011;52:3478–87.

    Article  CAS  Google Scholar 

  40. Wasanasuk K, Tashiro K, Hanesaka M, Ohhara T, Kurihara K, Kuroki R, et al. Crystal structure analysis of poly(L-lactic Acid) alpha form on the basis of the 2-dimensional wide-angle synchrotron X-ray and neutron diffraction measurements. Macromolecules. 2011;44:6441–52.

    Article  CAS  Google Scholar 

  41. Molnar C. Interpretable machine learning. Lulu.com; 2020.

Download references

Acknowledgements

The synchrotron radiation experiments were performed at the BL40B2 and BL40XU beamlines of SPring-8 with the approval of the Japan Synchrotron Radiation Research Institute (JASRI) (Proposal no. 2020A1525 and 2019B1667). This work was supported by the Cabinet Office, Government of Japan, Cross-ministerial Strategic Innovation Promotion Program (SIP), “Technologies for Smart Bio-industry and Agriculture” (funding agency: Bio-oriented Technology Research Advancement Institution, NARO). The computer resources were supported by the “Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures” and the “High Performance Computing Infrastructure” in Japan (Project ID: jh200016-NAH). This work was also supported by the JSPS Grant-in-Aid for Scientific Research on Innovative Areas, Discrete Geometric Analysis for Materials Design: 20H04644, by the Grant-in-Aid for Scientific Research (B): 20H02800, and by Early-Career Scientists: 18K14273 from JSPS. YA and KT acknowledge the financial support of the Grant-in-Aid for RIKEN-Kyushu University Science and Technology Hub Collaborative Research Program.

Author information

Authors and Affiliations

  1. Institute for Materials Chemistry and Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, 819-0395, Japan

    Yoshifumi Amamoto & Ken Kojio

  2. Graduate School of Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, 819-0395, Japan

    Yoshifumi Amamoto, Hiroteru Kikutake & Ken Kojio

  3. Center for Polymer Interface and Molecular Adhesion Science, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, 819-0395, Japan

    Yoshifumi Amamoto, Ken Kojio & Atsushi Takahara

  4. Research Center for Negative Emission Technology, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, 819-0395, Japan

    Ken Kojio & Atsushi Takahara

  5. Graduate School of Medical Life Science Department of Medical Life Science, Yokohama City University, 1-7-29 Suehiro-cho, Tsurumi-ku, Yokohama, Kanagawa, 230-0045, Japan

    Kei Terayama

  6. RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan

    Kei Terayama

Authors
  1. Yoshifumi Amamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiroteru Kikutake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ken Kojio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Atsushi Takahara
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kei Terayama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoshifumi Amamoto.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amamoto, Y., Kikutake, H., Kojio, K. et al. Visualization of judgment regions in convolutional neural networks for X-ray diffraction and scattering images of aliphatic polyesters. Polym J 53, 1269–1279 (2021). https://doi.org/10.1038/s41428-021-00531-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41428-021-00531-w

This article is cited by

Search

Advanced search

Quick links