Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Progress Article
  • Published:

Big–deep–smart data in imaging for guiding materials design

Nature Materials volume 14, pages 973–980 (2015)Cite this article

Subjects

Abstract

Harnessing big data, deep data, and smart data from state-of-the-art imaging might accelerate the design and realization of advanced functional materials. Here we discuss new opportunities in materials design enabled by the availability of big data in imaging and data analytics approaches, including their limitations, in material systems of practical interest. We specifically focus on how these tools might help realize new discoveries in a timely manner. Such methodologies are particularly appropriate to explore in light of continued improvements in atomistic imaging, modelling and data analytics methods.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Bridging theory and imaging for understanding materials structure and functionalities.
Figure 2: Big data in image analysis.
Figure 3: Deep-data approaches allow scientists to establish or improve the link between theory, simulation and experiment.
Figure 4: Illustration of an envisioned smart-data approach to materials discovery.

Similar content being viewed by others

References

  1. Hoddeson, M. R. a. L. Crystal Fire: The Invention of the Transistor and the Birth of the Information Age (W. W. Norton & Company, 1998).

    Google Scholar 

  2. Sze, S. M. Physics of Semiconductor Devices 2nd edn (Wiley-Interscience, 1981).

    Google Scholar 

  3. Shockley, W. Electrons and Holes in Semiconductors: With Applications to Transistor Electronics (D. Van Nostrand, 1950).

    Google Scholar 

  4. Fuechsle, M. et al. A single-atom transistor. Nature Nanotech. 7, 242–246 (2012).

    Article  CAS  Google Scholar 

  5. Woodward, D. I., Knudsen, J. & Reaney, I. M. Review of crystal and domain structures in the PbZrxTi1–xO3 solid solution. Phys. Rev. B 72, 104110 (2005).

    Article  CAS  Google Scholar 

  6. Vugmeister, B. E. Polarization dynamics and formation of polar nanoregions in relaxor ferroelectrics. Phys. Rev. B 73, 174117 (2006).

    Article  CAS  Google Scholar 

  7. Fiebig, M. Revival of the magnetoelectric effect. J. Phys. D 38, R123–R152 (2005).

    Article  CAS  Google Scholar 

  8. Binder, K. & Young, A. P. Spin-glasses—experimental facts, theoretical concepts, and open questions. Rev. Mod. Phys. 58, 801–976 (1986).

    Article  CAS  Google Scholar 

  9. Dagotto, E. Complexity in strongly correlated electronic systems. Science 309, 257–262 (2005).

    Article  CAS  Google Scholar 

  10. Dagotto, E., Hotta, T. & Moreo, A. Colossal magnetoresistant materials: The key role of phase separation. Phys. Rep. 344, 1–153 (2001).

    Article  CAS  Google Scholar 

  11. Spaldin, N. A. & Fiebig, M. The renaissance of magnetoelectric multiferroics. Science 309, 391–392 (2005).

    Article  CAS  Google Scholar 

  12. Adler, S. B. Factors governing oxygen reduction in solid oxide fuel cell cathodes. Chem. Rev. 104, 4791–4843 (2004).

    Article  CAS  Google Scholar 

  13. Fischer, C. C., Tibbetts, K. J., Morgan, D. & Ceder, G. Predicting crystal structure by merging data mining with quantum mechanics. Nature Mater. 5, 641–646 (2006).

    Article  CAS  Google Scholar 

  14. Curtarolo, S. et al. The high-throughput highway to computational materials design. Nature Mater. 12, 191–201 (2013).

    Article  CAS  Google Scholar 

  15. Crewe, A. V. Scanning electron microscopes—is high resolution possible. Science 154, 729–738 (1966).

    Article  CAS  Google Scholar 

  16. Pennycook, S. J. & Nellist, P. D. (eds) Scanning Transmission Electron Microscopy: Imaging and Analysis (Springer, 2011).

    Book  Google Scholar 

  17. Ardenne, M. v. Das elektronen-rastermikroskop. Praktische Ausführung. Z. Tech. Phys. 19, 407–416 (1938).

    Google Scholar 

  18. Binnig, G., Rohrer, H., Gerber, C. & Weibel, E. 7 × 7 reconstruction on Si(111) resolved in real space. Phys. Rev. Lett. 50, 120–123 (1983).

    Article  CAS  Google Scholar 

  19. Binnig, G. & Rohrer, H. Scanning tunneling microscopy. Helv. Phys. Acta 55, 726–735 (1982).

    CAS  Google Scholar 

  20. Gerber, C. & Lang, H. P. How the doors to the nanoworld were opened. Nature Nanotech. 1, 3–5 (2006).

    Article  CAS  Google Scholar 

  21. Pennycook, S. J. & Kalinin, S. V. Microscopy: Hasten high resolution. Nature 515, 487–488 (2014).

    Article  CAS  Google Scholar 

  22. Van Tendeloo, G., Bals, S., Van Aert, S., Verbeeck, J. & Van Dyck, D. Advanced electron microscopy for advanced materials. Adv. Mater. 24, 5655–5675 (2012).

    Article  CAS  Google Scholar 

  23. Yankovich, A. B. et al. Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts. Nature Commun. 5, 4155 (2014).

    Article  CAS  Google Scholar 

  24. Jia, C. L. et al. Atomic-scale study of electric dipoles near charged and uncharged domain walls in ferroelectric films. Nature Mater. 7, 57–61 (2008).

    Article  CAS  Google Scholar 

  25. Chang, H. J. et al. Atomically resolved mapping of polarization and electric fields across ferroelectric/oxide interfaces by Z-contrast imaging. Adv. Mater. 23, 2474–2479 (2011).

    Article  CAS  Google Scholar 

  26. Nelson, C. T. et al. Spontaneous vortex nanodomain arrays at ferroelectric heterointerfaces. Nano Lett. 11, 828–834 (2011).

    Article  CAS  Google Scholar 

  27. Chisholm, M. F., Luo, W. D., Oxley, M. P., Pantelides, S. T. & Lee, H. N. Atomic-scale compensation phenomena at polar interfaces. Phys. Rev. Lett. 105, 197602 (2010).

    Article  CAS  Google Scholar 

  28. Borisevich, A. et al. Mapping octahedral tilts and polarization across a domain wall in BiFeO3 from Z-contrast scanning transmission electron microscopy image atomic column shape analysis. ACS Nano 4, 6071–6079 (2010).

    Article  CAS  Google Scholar 

  29. Jia, C. L. et al. Oxygen octahedron reconstruction in the SrTiO3/LaAlO3 heterointerfaces investigated using aberration-corrected ultrahigh-resolution transmission electron microscopy. Phys. Rev. B 79, 081405 (2009).

    Article  Google Scholar 

  30. Kim, Y. M. et al. Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level. Nature Mater. 11, 888–894 (2012).

    Article  CAS  Google Scholar 

  31. Van Aert, S., Van Dyck, D. & den Dekker, A. J. Resolution of coherent and incoherent imaging systems reconsidered—Classical criteria and a statistical alternative. Opt. Express 14, 3830–3839 (2006).

    Article  Google Scholar 

  32. Van Aert, S., den Dekker, A. J., Van Dyck, D. & van den Bos, A. High-resolution electron microscopy and electron tomography: Resolution versus precision. J. Struct. Biol. 138, 21–33 (2002).

    Article  CAS  Google Scholar 

  33. Pan, S. H. et al. Imaging the effects of individual zinc impurity atoms on superconductivity in Bi2Sr2CaCu2O8+δ . Nature 403, 746–750 (2000).

    Article  CAS  Google Scholar 

  34. Roushan, P. et al. Topological surface states protected from backscattering by chiral spin texture. Nature 460, 1106–1109 (2009).

    Article  CAS  Google Scholar 

  35. Lin, J. H. et al. Flexible metallic nanowires with self-adaptive contacts to semiconducting transition-metal dichalcogenide monolayers. Nature Nanotech. 9, 436–442 (2014).

    Article  CAS  Google Scholar 

  36. Ishikawa, R. et al. Direct observation of dopant atom diffusion in a bulk semiconductor crystal enhanced by a large size mismatch. Phys. Rev. Lett. 113, 155501 (2014).

    Article  CAS  Google Scholar 

  37. Huang, P. Y. et al. Imaging atomic rearrangements in two-dimensional silica glass: Watching silica's dance. Science 342, 224–227 (2013).

    Article  CAS  Google Scholar 

  38. Zheng, H. M. et al. Observation of transient structural-transformation dynamics in a Cu2S nanorod. Science 333, 206–209 (2011).

    Article  CAS  Google Scholar 

  39. Eigler, D. M. & Schweizer, E. K. Positioning single atoms with a scanning tunnelling microscope. Nature 344, 524–526 (1990).

    Article  CAS  Google Scholar 

  40. Garcia, R., Knoll, A. W. & Riedo, E. Advanced scanning probe lithography. Nature Nanotech. 9, 577–587 (2014).

    Article  CAS  Google Scholar 

  41. Balke, N., Bdikin, I., Kalinin, S. V. & Kholkin, A. L. Electromechanical imaging and spectroscopy of ferroelectric and piezoelectric materials: State of the art and prospects for the future. J. Am. Ceram. Soc. 92, 1629–1647 (2009).

    Article  CAS  Google Scholar 

  42. Runkler, T. A. Data Analytics: Models and Algorithms for Intelligent Data Analysis (Vieweg, 2012).

    Book  Google Scholar 

  43. Bonnet, N. in Advances in Imaging and Electron Physics Vol. 114 (ed. P. W. Hawkes) 1–77 (Elsevier Academic Press, 2000).

    Google Scholar 

  44. Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference and Prediction 2nd edn (Springer, 2009).

    Book  Google Scholar 

  45. Belianinov, A. et al. Big data and deep data in scanning and electron microscopies: Deriving functionality from multidimensional data sets. Adv. Struct. Chem. Imaging 1, 1–25 (2015).

    Article  Google Scholar 

  46. Parr, R. G. & Weitao, Y. Density-Functional Theory of Atoms and Molecules (Oxford Univ. Press, 1994).

    Google Scholar 

  47. Sumpter, B. G. & Noid, D. W. On the design, analysis, and characterization of materials using computational neural networks. Annu. Rev. Mater. Sci. 26, 223–277 (1996).

    Article  CAS  Google Scholar 

  48. Sumpter, B. G., Getino, C. & Noid, D. W. Theory and applications of neural computing in chemical science. Annu. Rev. Phys. Chem. 45, 439–481 (1994).

    Article  CAS  Google Scholar 

  49. Phillips, J. C. & Rabe, K. M. Transport anomalies and internal structural models of stable quasi-crystals. Phys. Rev. Lett. 66, 923–925 (1991).

    Article  CAS  Google Scholar 

  50. Villars, P., Phillips, J. C. & Chen, H. S. Icosahedral quasi-crystals and quantum structural diagrams. Phys. Rev. Lett. 57, 3085–3088 (1986).

    Article  CAS  Google Scholar 

  51. Dongarra, J. et al. The International Exascale Software Project roadmap. Int. J. High Perform. Comput. Appl. 25, 3–60 (2011).

  52. Materials Genome Initiative; http://go.nature.com/Rkw2mj

  53. The Materials Project; https://www.materialsproject.org

  54. Jain, A. et al. A high-throughput infrastructure for density functional theory calculations. Comput. Mater. Sci. 50, 2295–2310 (2011).

    Article  CAS  Google Scholar 

  55. Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Mater. 1, 011002 (2013).

    Article  CAS  Google Scholar 

  56. AFLOW; http://materials.duke.edu/aflow.html

  57. Setyawan, W. & Curtarolo, S. High-throughput electronic band structure calculations: Challenges and tools. Comput. Mater. Sci. 49, 299–312 (2010).

    Article  Google Scholar 

  58. Ramakrishnan, R., Dral, P. O., Rupp, M. & von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data 1, 140022 (2014).

    Article  CAS  Google Scholar 

  59. Sohlberg, K., Rashkeev, S., Borisevich, A. Y., Pennycook, S. J. & Pantelides, S. T. Origin of anomalous Pt-Pt distances in the Pt/alumina catalytic system. ChemPhysChem 5, 1893–1897 (2004).

    Article  CAS  Google Scholar 

  60. Bachelet, G. B. & Schluter, M. Relativistic norm-conserving pseudopotentials. Phys. Rev. B 25, 2103–2108 (1982).

    Article  CAS  Google Scholar 

  61. von Lilienfeld, O. A., Tavernelli, I., Rothlisberger, U. & Sebastiani, D. Optimization of effective atom centered potentials for London dispersion forces in density functional theory. Phys. Rev. Lett. 93, 153004 (2004).

    Article  CAS  Google Scholar 

  62. Baumeier, B., Kruger, P. & Pollmann, J. Self-interaction-corrected pseudopotentials for silicon carbide. Phys. Rev. B 73, 195205 (2006).

    Article  CAS  Google Scholar 

  63. von Lilienfeld, O. A. & Schultz, P. A. Structure and band gaps of Ga-(V) semiconductors: The challenge of Ga pseudopotentials. Phys. Rev. B 77, 115202 (2008).

    Article  CAS  Google Scholar 

  64. von Lilienfeld, O. A. Force correcting atom centred potentials for generalised gradient approximated density functional theory: Approaching hybrid functional accuracy for geometries and harmonic frequencies in small chlorofluorocarbons. Mol. Phys. 111, 2147–2153 (2013).

    Article  CAS  Google Scholar 

  65. Zhou, F., Cococcioni, M., Marianetti, C. A., Morgan, D. & Ceder, G. First-principles prediction of redox potentials in transition-metal compounds with LDA + U. Phys. Rev. B 70, 235121 (2004).

    Article  CAS  Google Scholar 

  66. Marzouk, Y. M., Najm, H. N. & Rahn, L. A. Stochastic spectral methods for efficient Bayesian solution of inverse problems. J. Comp. Phys. 224, 560–586 (2007).

    Article  Google Scholar 

  67. Marzouk, Y. & Xiu, D. A stochastic collocation approach to Bayesian inference in inverse problems. Commun. Computational Phys. 6, 826–847 (2009).

    Article  Google Scholar 

  68. Howson, C. & Urbach, P. Scientific Reasoning: The Bayesian Approach (Open Court, 2006).

    Google Scholar 

  69. Robert, C. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation (Springer Texts in Statistics) (Springer, 2001).

    Google Scholar 

  70. Lingerfelt, E. J., Messer, O. E. B., Desai, S. S., Holt, C. A. & Lentz, E. J. Near real-time data analysis of core-collapse supernova simulations with Bellerophon. Procedia Comput. Sci. 29, 1504–1514 (2014).

    Article  Google Scholar 

  71. Rupp, M., Tkatchenko, A., Muller, K. R. & von Lilienfeld, O. A. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett. 108, 058301 (2012).

    Article  CAS  Google Scholar 

  72. Montavon, G. et al. Machine learning of molecular electronic properties in chemical compound space. New J. Phys. 15, 095003 (2013).

    Article  CAS  Google Scholar 

  73. Lopez-Bezanilla, A. & von Lilienfeld, O. A. Modeling electronic quantum transport with machine learning. Phys. Rev. B 89, 235411 (2014).

    Article  CAS  Google Scholar 

  74. Ramakrishnan, R., Dral, P. O., Rupp, M. & Anatole von Lilienfeld, O. Big data meets quantum chemistry approximations: The Δ-machine learning approach. J. Chem Theory Comput. 11, 2087–2096 (2015).

    Article  CAS  Google Scholar 

  75. Pyzer-Knapp, E. O., Suh, C., Gómez-Bombarelli, R., Aguilera-Iparraguirre, J. & Aspuru-Guzik, A. What is high throughput virtual screening? A perspective from organic materials discovery. Annu. Rev. Mater. Sci. 45, 195–216 (2015).

    Article  CAS  Google Scholar 

  76. Hachmann, J. et al. Lead candidates for high-performance organic photovoltaics from high-throughput quantum chemistry—the Harvard Clean Energy Project. Energy Environ. Sci. 7, 698–704 (2014).

    Article  CAS  Google Scholar 

  77. Bartok, A. P., Gillan, M. J., Manby, F. R. & Csanyi, G. Machine-learning approach for one- and two-body corrections to density functional theory: Applications to molecular and condensed water. Phys. Rev. B 88, 054104 (2013).

    Article  CAS  Google Scholar 

  78. Machta, B. B., Chachra, R., Transtrum, M. K. & Sethna, J. P. Parameter space compression underlies emergent theories and predictive models. Science 342, 604–607 (2013).

    Article  CAS  Google Scholar 

  79. Katsoulakis, M. A. & Plechac, P. Information-theoretic tools for parametrized coarse-graining of non-equilibrium extended systems. J. Chem. Phys. 139, 074115 (2013).

    Article  CAS  Google Scholar 

  80. Materials Genome Initiative Strategic Plan; http://www.nist.gov/mgi/upload/MGI-StrategicPlan-2014.pdf

  81. Spiegelhalter, D. The future lies in uncertainty. Science 345, 264–265 (2014).

    Article  CAS  Google Scholar 

  82. Ovsjanikov, M., Bronstein, A. M., Bronstein, M. M. & Guibas, L. J. Shape Google: a computer vision approach to invariant shape retrieval. Proc. NORDIA 1, 1 (2009).

    Google Scholar 

  83. Zhu, J., Ferguson, D. I. & Dolgov, D. A. System and method for predicting behaviors of detected objects. US patent 8660734 B2 (2014).

  84. Tourassi, G. D., Vargas-Voracek, R., Catarious, D. M. & Floyd, C. E. Computer-assisted detection of mammographic masses: A template matching scheme based on mutual information. Med. Phys. 30, 2123–2130 (2003).

    Article  Google Scholar 

  85. Scharcanski, J. & Celebi, M. E. (eds) Computer Vision Techniques for the Diagnosis of Skin Cancer (Springer, 2013).

    Google Scholar 

  86. Mody, C. C. M. Instrumental Community (The MIT Press, 2011).

    Book  Google Scholar 

  87. Reed, J. W. et al. TF-ICF: A new term weighting scheme for clustering dynamic data streams. 5th Int. Conf. Machine Learning Appl. 258–263 (IEEE, 2006).

    Google Scholar 

  88. http://cda.ornl.gov/piranha.shtml

  89. Bollen, J. et al. Clickstream data yields high-resolution maps of science. PLoS ONE 4, e4803 (2009).

    Article  CAS  Google Scholar 

  90. Aiello, L. M., Schifanella, R. & State, B. Reading the source code of social ties. Preprint at http://arXiv.org/abs/1407.5547v1 (2014).

  91. He, Q., Woo, J., Belianinov, A., Guliants, V. V. & Borisevich, A. Y. Better catalysts through microscopy: Mesoscale M1/M2 intergrowth in molybdenum-vanadium based complex oxide catalysts for propane ammoxidation. ACS Nano 9, 3470–3478 (2015).

    Article  CAS  Google Scholar 

  92. Lin, W. Z. et al. Direct probe of interplay between local structure and superconductivity in FeTe0.55Se0.45 . ACS Nano 7, 2634–2641 (2013).

    Article  CAS  Google Scholar 

  93. Tselev, A. et al. Oxygen control of atomic structure and physical properties of SrRuO3 surfaces. ACS Nano 7, 4403–4413 (2013).

    Article  CAS  Google Scholar 

  94. Cruz-Silva, E. et al. Edge-edge interactions in stacked graphene nanoplatelets. ACS Nano 7, 2834–2841 (2013).

    Article  CAS  Google Scholar 

  95. Romo-Herrera, J. M., Terrones, M., Terrones, H., Dag, S. & Meunier, V. Covalent 2D and 3D networks from 1D nanostructures: Designing new materials. Nano Lett. 7, 570–576 (2007).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors thank A. Borisevich, H. Christen, J. Morris, and D. Levy, as well as multiple colleagues at ORNL and elsewhere for valuable discussions. R.K.A. acknowledges The Compute and Data Environment (CADES) for continuous support. E. Strelcov and R. Vasudevan are gratefully acknowledged for help with figure preparation. Research was sponsored by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the US Department of Energy. A portion of this research was conducted at the Center for Nanophase Materials Sciences, which is a DOE Office of Science User Facility. The algorithmic aspects were sponsored by the applied mathematics program at the DOE and the computational aspects made use of the Oak Ridge Leadership Computing Facility, a DOE Office of Science User Facility at ORNL supported under contract no. DE-AC05-00OR22725.

Author information

Authors and Affiliations

  1. Institute for Functional Imaging of Materials, Oak Ridge National Laboratory, Oak Ridge, 37831, Tennessee, USA

    Sergei V. Kalinin, Bobby G. Sumpter & Richard K. Archibald

  2. Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, 37831, Tennessee, USA

    Sergei V. Kalinin & Bobby G. Sumpter

  3. Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, 37831, Tennessee, USA

    Bobby G. Sumpter & Richard K. Archibald

Authors
  1. Sergei V. Kalinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bobby G. Sumpter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard K. Archibald
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergei V. Kalinin.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalinin, S., Sumpter, B. & Archibald, R. Big–deep–smart data in imaging for guiding materials design. Nature Mater 14, 973–980 (2015). https://doi.org/10.1038/nmat4395

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmat4395

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing