Article | Open | Published:

Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2

npj Computational Materialsvolume 5, Article number: 12 (2019) | Download Citation


Recent advances in scanning transmission electron microscopy (STEM) allow the real-time visualization of solid-state transformations in materials, including those induced by an electron beam and temperature, with atomic resolution. However, despite the ever-expanding capabilities for high-resolution data acquisition, the inferred information about kinetics and thermodynamics of the process, and single defect dynamics and interactions is minimal. This is due to the inherent limitations of manual ex situ analysis of the collected volumes of data. To circumvent this problem, we developed a deep-learning framework for dynamic STEM imaging that is trained to find the lattice defects and apply it for mapping solid state reactions and transformations in layered WS2. The trained deep-learning model allows extracting thousands of lattice defects from raw STEM data in a matter of seconds, which are then classified into different categories using unsupervised clustering methods. We further expanded our framework to extract parameters of diffusion for sulfur vacancies and analyzed transition probabilities associated with switching between different configurations of defect complexes consisting of Mo dopant and sulfur vacancy, providing insight into point-defect dynamics and reactions. This approach is universal and its application to beam-induced reactions allows mapping chemical transformation pathways in solids at the atomic level.


Chemical reactions and phase transformations underpin phenomena ranging from cosmological processes, to the emergence of life on Earth, to modern technologies and are therefore of tremendous interest for both basic and applied sciences. Simple gas phase reactions of small molecules can be readily studied using well-established spectroscopy methods (infrared,1,2 mass,3 nuclear magnetic resonance4), utilizing the spatial homogeneity of reaction volumes when the same process occurs multiple times. In conjunction with first-principles-based modeling,5,6 a reliable picture of molecular reactivity can be built. For studies of more complex organic and biochemical reactions, time-resolved cryogenic microscopy7 and femtosecond X-ray pump–probes8 provide a reliable investigative framework, again relying on the statistical similarity between multiple orientations of the same molecule.

The situation is far more complicated for solid state reactions involving continuous solids. Traditionally, solid-state phase transformations and reactions were explored by bulk measurements and X-ray techniques. However, such techniques may not be able to provide sufficient spatial resolution for understanding elementary mechanisms behind the observed transformations. This problem can be partially solved by direct ex situ visualization of reaction zones,9,10 providing information on the geometry and, in certain cases, atomic configurations at the reaction fronts. Similarly, utilization of colloid models11 allows for the development of model systems, even though the nature of local interactions is significantly different from those found in atomic systems.

In recent years, advances in scanning transmission electron microscopy ((S)TEM) have enabled the direct visualization of dynamic phenomena at the atomic level.12,13,14,15,16,17,18,19,20 The physical/chemical phenomena studied by in situ STEM are wide ranging and now include e-beam induced defect evolution,21,22,23,24,25,26,27,28,29,30 dislocation migration,31,32,33 observation of e-beam induced production of single layer Fe and ZnO membranes in graphene nanopores,34,35 e-beam induced chemical etching and growth from nanoparticle and single atom catalysts,36,37,38,39,40,41 sub-10 nm scale lithography,42 graphene healing,43 conductive nanowire formation,44 crystallization and amorphization at 2D interfaces,45,46 formation of fullerenes,47 and graphene edge dynamics.48,49 This list can hardly be considered comprehensive, but it serves to illustrate the vast array of dynamic changes that are being observed and rapidly explored via in situ (S)TEM techniques. A tantalizing development, which was published just last year (2017), is the introduction of a single dopant atom into a graphene lattice, the controlled movement of the atom through the lattice, and the assembly of a few primitive structures atom-by-atom.50,51,52,53 Such efforts harken back to the work of Don Eigler who first demonstrated controlled atomic motion via scanning probe techniques.54 However, given the colorful array of other atomic, chemical processes observed in (S)TEM, and the continuously growing portfolio of commercially available in situ equipment (heating, electrical biasing, gas and liquid cells, etc.), it seems likely that many more processes can be brought under direct control, turning the (S)TEM into an atomic-scale fabrication platform.55

Successes in e-beam atom-by-atom fabrication and atom-by-atom mapping of solid-state reactions will require more than explorational research and instrumental improvements. A key piece of the puzzle will involve successfully grappling with the enormous amount of data which can be generated by these instruments to infer material-specific information describing kinetics and thermodynamics of point and extended defects, reaction paths, and mechanisms for extended defect and second phase nucleation and growth. The “by hand” analysis of years past is no longer a tractable solution considering the dimensionality and number of datasets which are now routinely obtained. This necessitates the creation of methods which allow for automated analysis of dynamic transformations to extract relevant materials descriptors and reconstruct reaction pathways from various sources of detector readouts, such as the variety of imaging and spectroscopic modes. In this article, we attempt to forge an inroad in one aspect of this challenge, namely automated image analysis for the detection and tracking of defects in STEM of 2D materials and further proceed to extract microscopic point-defect reaction mechanisms from these observations.

Here, we analyze the phase evolution of Mo-doped WS2 during electron beam irradiation. In this process, the electron irradiation results in removal of sulfur atoms, rendering the system oversaturated with respect to low-valence tungsten–sulfur moieties. We develop a deep-learning network for rapid analysis of this dynamic data, analyze transformation pathways, create a library of defects, and explore minute distortions in local atomic environment around the defects of interest, ultimately building a complete framework for exploring point-defect dynamics and reactions.

Results and discussion

Figure 1 shows several selected frames from a STEM “movie” of lattice transformations in the Mo-doped WS2 monolayer under 100 kV electron beam irradiation. The full movie is available in the Supplementary Material. This movie was previously analyzed by some of the authors in the context of mesoscale phase transformations.56 It can be seen that the system evolves with time, evolving numerous point defects. On accumulation of nonstoichiometry, the latter start to segregate, forming extended defects, nucleating secondary phases, and resulting in the segmentation and rearrangement of the 2D layer. The key task is to obtain information of interest about the atomic-scale defects. Unfortunately, most of the methods for localizing and identifying/classifying defects available to date are slow, inefficient and require frequent manual inputs.

Fig. 1
Fig. 1

Defects evolution under e-beam irradiation in Mo-doped WS2. a Ball-and-stick representations of WS2 structure. b–e Four selected frames from the STEM movie of Mo-doped WS2 obtained at 100 kV illustrating formation of defects and lattice transformations as a function of time. The full movie can be found in the Supplementary material

To overcome the limitations of the available approaches, we developed a physics-based machine learning method for localizing and identifying defects. We exploit the fact that each defect is associated with violation of ideal periodicity of the lattice. Therefore, we train a convolutional neural network (cNN) using a single image at the early stage of the beam-induced transformation, when macroscopic periodicity is still maintained, and each defect can be readily discovered providing the “ground truth” for network training. Thus, a trained network relies only on the local characteristics of the image, and hence can identify defects on the later stages of system evolution when the long range periodicity of the lattice is broken due to a second phase evolution and displacement and rotation of unreacted WS2 fragments. Furthermore, we find that the network can discover via “extrapolation” other defects which may not necessarily be a part of the initial training set. Such “extrapolation” is possible due to generalization abilities of deep-learning models. Indeed, we have recently demonstrated57 that a deep cNN trained on the simulated images of an idealistic lattice vacancy structure can in principle generalize well enough to detect larger and more complex lattice vacancy structures in the system (e.g., double and triple vacancies, as well as reconstructed vacancies). The extracted defect structures can be identified/classified using unsupervised clustering and unmixing techniques. Finally, the selected defects can be studied further using local crystallography techniques,58 such as a combination of atom finding and principal component analysis for analyzing minute atomic distortions in their vicinity in the “movies”, as well as with a Markov analysis for identifying transition probabilities between different defect configurations.

As a first step of analysis, we defined the topology of a neural network to target specific physics of beam-induced transformations. The network must be able (i) to separate atomic-scale lattice disorder from the rest of the lattice, (ii) to return the precise location of the detected defects, and (iii) to be able to generalize to previously unseen defect structures. One possible candidate is the class activation maps-based deep-learning analysis, in which a model trained on image-level labels is capable in principle of discriminating the image regions used to identify the specific class59 (defect). The disadvantage of such an approach is that one must start with manually selecting the isolated single defect structures to create a training set. In addition, we found that while this approach allows certain atomic defect structures to be located with sufficient accuracy, it has shown relatively poor generalization ability. The alternative approach is to use a fully cNN model,60 which can be trained to output a pixel-wise classification map, with the same size/resolution as the original input image, that shows a probability of each pixel belonging to certain type of object (defect). This type of model has been recently successfully applied to finding lattice atoms in raw STEM data57 and we therefore chose it for the current problem.

The next task is to create a training set that will be used to “teach” a model to find lattice defects in STEM movies, allowing for sufficient flexibility to discover all the defects but at the same time avoiding over-classification for classes that physically cannot be present in the data. We found that it is possible to train a network using only the first frame of such a movie or a single image obtained before recording a movie, and then let the trained network analyze the remaining part of the movie. This approach utilizes the fact that macroscopically (i.e., on the length scale of the image) the defects can be trivially discovered via the Fourier method,61 providing the ground truth for network training. However, when trained, the network relies solely on local edge properties for identification and is thus stable toward formation of extended defects, rotations, and fragmentations of the lattice.

To identify the defects, we select a single image (frame) at the beginning of transformations (Fig. 2a). Once the image is selected, we performed a global Fast Fourier Transform (FFT) on the selected experimental image and applied a high-pass filter in reciprocal space in order to remove nonperiodic components of the lattice (Fig. 2b). We then performed an inverse FFT to obtain the periodic image and subtracted the original image from it (and vice versa) such that only the deviations from the ideal periodic lattice remained.61 In this image difference, vacancies show up as bright spots. Next, the image difference is thresholded to find locations of the single defects (Fig. 2c). Note that the thresholded image represents the “ground truth” which will be used to train the cNN. The training set is created by performing data augmentation of the selected experimental image and the corresponding ground truth image. This augmented dataset can be used to train a neural network to return positions of atomic lattice disorder from raw experimental data (Fig. 2d). Once trained, not only is this cNN-based method for finding defects faster and more efficient than the method based on FFT subtraction, but it also allows, unlike the FFT method, to find the position of defects in the images of fragmented atomic lattices where multiple (joint and/or disjoint) lattice domains can be rotated by different angles with respect to each other. Because our model allows finding defects that break lattice periodicity irrespective of the exact type of the defect, we consider it to be a “universal” defect finder for a given material.

Fig. 2
Fig. 2

Training a deep convolutional neural network to recognize defects that break lattice periodicity. a The first frame from STEM movie on Mo-doped WS2. b Global FFT and global FFT with high-pass filter applied. c Binary masks for image differences between the original data in a and inverse of filtered FFT in b. The image in a is a training image and the data in c serves as ‘ground truth’ (pixel-wise labeling). d Schematics of convolutional neural network with an encoder–decoder type of structure

We now use the cNN model trained according to the method described above (accuracy on the test set 99%) to locate atomic defects in dynamic STEM data on Mo-doped WS2. Interestingly, although our model was trained using only the first frame (out of 100) of the movie, it was able to accurately identify the positions of atomic defects in the remaining 99 frames (see Supplementary Figure 1; full movie of the defects found can be downloaded from Supplementary Material). Once a sufficient number of defects (~104 in this case) is extracted via the cNN model it becomes possible to categorize them into different classes. To perform such a defect classification in an unsupervised fashion, we adapted a Gaussian mixture model (GMM).62 The GMM was applied to a stack of defect “windows” (images of the identical size, usually 32px*32px cropped around the center of each defect) extracted using the pixel-wise classification maps in the cNN output. Here, we chose the number of GMM components to be five as this appears to be the optimal number of components for understanding the type of defect structures present in the data. Indeed, an increase in the number of components resulted in fine (sub-) structures of the detected defects, while decreasing the number of components produced some physically meaningless structures (see Supplementary Figure 2). We also note that the number of components past the purely exploratory stage can be adjusted based on known defect chemistry of the material (either from common physics principles, density functional theory calculations, or combinatorial analysis)

The defect structures associated with the unmixed components of GMM are shown in Fig. 3a–e. The class 1 and class 3 (Fig. 3a, c) were found to correspond to a substitutional atom in W sublattice with a lower Z number, which we interpret as Mo dopant (Mow). Note that Mo atom does not occupy a symmetric central spot in these structures as one would expect for a lone Mo dopant. This suggests that there are additional distortions present in the defects that form classes 1 and 3, likely associated with a disorder in the S sublattice. Interestingly, presence of a coupling between distinct defect species has been recently observed in static STEM images from Mo-doped WS2 system and attributed to merging of defects during growth and postgrowth procedures.63 This comparison illustrates that as in other cases, systematic application of statistical and machine learning methods allows us both to recover earlier observations and, as we show next, derive new information about underlying physical and chemical processes. The classes 4 and 5 (Fig. 3d, e) are associated with a vacancy in the W sublattice (Vw) and in the S sublattice (Vs), respectively. The presence of adatoms/“contaminations” created during the e-beam surface transformations (e.g., chemical species from initial WS2 material deposited back on to the surface in combination with carbon atoms) can explain the defect structure in class 2 (Fig. 3b).

Fig. 3
Fig. 3

Unsupervised classification of defects located by the deep convolutional neural network. ae Results of GMM unmixing into five components (classes). f Spatiotemporal trajectories of the detected defects. Color scheme is the same as in (ae)

Figure 3f shows spatiotemporal trajectories (“brush diagram”64) for the identified defects. Based on the analysis of the diagram, we identify three characteristic statistical behaviors: weakly moving trajectories, stronger diffusion, and “uncorrelated events”/“flickering”. Presence of more than one characteristic behavior of the atomic defects may be potentially connected to the complex spatial character of strain fields during the material transformation, which may impact diffusion properties as well as create certain “localization regions”, in which the motion of defects is restrained.65,66 In the following, we will focus on the analysis of the continuous and quasi-continuous trajectories only. The most well-defined trajectories are associated with Mo dopants (class 1 and class 3). These Mo defects show different diffusion behaviors depending on their location in the lattice and are characterized by reversible switching between two configurations (class 1 and class 3) along their trajectories. The defects associated with S and W vacancies typically form shorter (compared to Mo defects) trajectories. One possible explanation is that these vacancies are becoming filled by the W and S species from the extended clusters of the deposited WS2 material (although we did not find any associated correlations with point defects of class 2).

We now demonstrate that, based on the results produced by a combination of cNN and GMM, it becomes possible to estimate diffusion characteristics of the selected defect species. Particularly, we studied diffusion properties of S vacancies. We first collapse the 3-d spatiotemporal diagram for a chosen class of defect into a 2-d representation. For this purpose, we project the “windows” of specific classes of defect, which allows separating defects that are continuous in time from the randomly occurring ones (see Supplementary Figure 3). This analysis is complemented by a density-based clustering algorithm,67 which yields similar results. After extracting defect coordinates for each selected defect “flow” (Fig. 4), we can obtain variance of each distribution and estimate a diffusion coefficient within a framework of a random walk model in two dimensions. This yields values of diffusion coefficient between 3 × 10−4 nm2/s and 6 × 10−4 nm2/s.

Fig. 4
Fig. 4

Analysis of diffusion behavior for selected defect structure. a 2-d projections (X–Y coordinates) of the 3-d defect “flow” of the S vacancies (inset) with 95% prediction ellipses overlaid. b 1-d r(t) representation of the same data

We further proceed to the analysis of another type of defect, namely, the defect associated with a Mo dopant (classes 1 and 3). Here, it is worth noting that while the GMM-based decomposition into five components provides us with a good understanding of the major types of defects present in the system, it does not allow studying the fine details (variations) of the detected structures. Performing such an analysis is especially important for classes 1 and 3 that show peculiar switching behavior in Fig. 3f. We therefore investigated the “internal” structures of classes 1 and 3 using the so-called local crystallography analysis.58 Specifically, we studied statistically significant deformation of the nearest neighborhood for each defect structure using principal component analysis (PCA). We first employ a deep-learning-based “atom finder”68 that allows extracting positions of atoms from thousands of noisy images of defects in a matter of seconds (note that S atoms cannot be reliably identified at the current experimental resolution and hence we omit them). The first two PCA components associated with displacements from the averaged structure of the central Mo atom and six W neighbor atoms for each defect class are plotted in Fig. 5a, b. Since the Mo dopant does not considerably distort the WS2 lattice,63 the structural variations in PCA analysis must be associated with a disorder in the S sublattice. In general, one must exercise caution in assigning a specific physical meaning to the PCA components. However, the results shown in Fig. 5 strongly suggest a presence of strong variations in a relative position of central Mo atom with respect to neighbor W atoms, thus it is possible that these variations originate from the presence of S vacancies next to Mo dopant.

Fig. 5
Fig. 5

Local crystallography analysis for the selected defect classes. a, b The principal component analysis-derived first two eigenmodes of atomic displacements for defects associated with a Mo dopant (class 1 and class 3 in Fig. 3a, c, respectively) presented as vectors of deformation from atom positions in the averaged defect structure. The length of the arrows is magnified by a factor of 5

Based on the PCA analysis of the atomic displacements in Fig. 5a, b and general lattice symmetry considerations we use GMM to split the defect structures from classes 1 and 3 into four subclasses (Fig. 6b) associated with undistorted Mow defect (no coupling to S vacancy) and three (Mow + Vs) complexes (it is worth noting that the similar result can be achieved by splitting the entire stack of all the defect images into >12 classes, see Supplementary Figure 2). Our next goal is to analyze the switching behavior between different states. Using the same approach as described for the analysis of diffusion parameters we first identified continuous-in-time defect trajectories for all the four subclasses from Fig. 6b, isolated them, and then converted them into the r(t) 1-d representation (Fig. 6a). In this case, each “flow” represents a sequence (in time) of defect structures undergoing switching between four different states. This observation suggests that the switching between different states can be analyzed as a Markov process, defining corresponding reaction constants on a single defect level.

Fig. 6
Fig. 6

Analysis of transition probabilities between different defect states. a 1-d representation of defect “flows” for the four subclasses of defect associated with Mo dopant in b. Color scheme is the same as in a and b. c Illustration of Markov transition processes between four states. d Markov transition matrix for the four subclasses (lone Mo dopant and three complexes of Mo dopant with S vacancy) based on the analysis of trajectories in a

The corresponding Markov transition matrix is depicted in Fig. 6d (see also the schematics of transitions in Fig. 6c). This analysis suggests the Mow defect may couple to a S vacancy in the dynamic STEM experiment. To explain transitions between Mow and (Mow + Vs) we argue that, due to a lower diffusion barrier of a S vacancy, as well as higher probability of S sublattice atoms being knocked-out during the e-beam irradiation, it is likely that the S vacancy created in the vicinity of Mo dopant can get captured by it. Interestingly, we also found transitions between different (Mow + Vs) structures. While the detailed explanation of such a behavior would require rigorous first-principles calculations and additional experiments, one can argue that the (Mow + Vs + Vs) structures are not stable and/or have a very short lifetime compared to the experimental time resolution (to the best of our knowledge such structures have not been observed even when (Mow + Vs) defects are abundant) and therefore attachment of the second S vacancy leads to pushing one of the two S vacancies out of the structure. The noticeably different values of transition probabilities for the (Mow + Vs) − I structure can be explained by a different rate of supply of S vacancies from different lattice directions, for example due to nontrivial distribution of strain fields during the e-beam induced transformation and their effect on diffusion characteristics in different lattice directions.

In summary, we have presented a deep-learning-based approach for analysis of dynamic transformation of the lattice structure in STEM “movies” from Mo-doped WS2. We started by teaching a deep neural network how the defects that break lattice periodicity appear in STEM data using a single experimental image (frame 0) and then used the generalization abilities of the network to find various types of atomic defects in the rest of the experimental data. We then performed unsupervised classification of the detected defect structures using a Gaussian mixture model and showed that the classification results can be linked to specific physical structures. We were then able to (i) identify dominant point defects and their characteristic statistical behaviors in spatiotemporal diagrams, (ii) analyze diffusion for the selected defect species (S vacancy), and (iii) study transformation pathways for Mo–S complexes, including detailed transition probabilities. In this manner, point-defect dynamics and solid state reactions in the material are studied on the atomic level, and corresponding reaction constants are determined for just one point defect.

As far as the future studies are concerned, we believe that one particularly promising direction is incorporating specific physics-based constraints into machine learning based analysis of STEM videos. Indeed, the current approaches treat observed lattice defects as collections of pixels, without “understanding” the physics behind the observations. One possible way of overcoming such physics-agnostic classification is by integrating a Markov model into the initial search and identification/classification scheme. The Markov model can be guided by the theoretical calculations of interaction potentials on the atomic level, enforcing physical constraints to transition probabilities of atoms and molecules, effects of electron beam irradiation, and operating both in space (hidden Markov random field) and time (hidden Markov model) domains. For example, one may incorporate transition probabilities between certain types of defects (e.g., reconstructed vs. nonreconstructed defect), as well as a maximum diffusion length of a defect for a given time scale calculated from first principles, and with a Markov model use it to refine the results of the initial classification. This would be an important step towards creating a fully autonomous, AI microscope that could make decisions based on the knowledge of physics that it was “taught”.


Sample preparation

The Mo-doped WS2 monolayers were grown on SiO2/Si substrate at 800oC by a low-pressure chemical vapor deposition.43 To prepare STEM samples, poly(methyl methacylate), PMMA (A4), was first spun onto the SiO2/Si substrate with monolayer crystals at 3500 rpm for 60 s. After being cured at 100 °C for 15 min, the PMMA/W1−xMoxS2 sample was detached from the substrate with a 30% KOH solution (100 °C and 0.5−1.0 h). The sample was then transferred to deionised water to remove the KOH residue. The washed film was scooped onto a QUANTIFOIL TEM grid. The PMMA was then removed with acetone, and the samples were soaked in methanol for 12 h to achieve a clean surface with flakes. To remove the polymer, the TEM grids were then annealed in an Ar flow (90 sccm, 10 torr) at 350 °C for 3 h.

STEM experiment

STEM imaging was performed using a Nion UltraSTEM U100 STEM operated at 100 kV. The images were acquired in high angle annular dark field imaging mode and were introduced to the deep cNN without any post processing.

Data analysis

The deep cNN was implemented using Keras 2.0 ( Python deep-learning library, with the TensorFlow backend. The cNN had an encoder–decoder type of structure. The encoder part had alternating convolutional layers for feature extraction with filters of the size 3 × 3 and stride 1 activated by a rectified linear unit function and max-pooling layers of the size 2 × 2 and stride 2 to account for translational invariance as well as for reducing the size of processed data. The decoder part of the network, whose role was to map the encoded low-resolution feature maps to full input-resolution feature maps, consisted of the same filters (in reverse order) and upsampling layers. The feature maps from the final convolutional layer of the network were fed into a softmax classifier for pixel-wise classification, providing us with information on the probability of each pixel being a defect. The Adam optimizer69 was used for training. The Gaussian mixture model was implemented with scikit-learn machine learning library (

Data availability

The complete workflow for studying defects in dynamic STEM data, which includes creation and training/testing of DL model, unsupervised defect classification, analysis of diffusion characteristics, local crystallography analysis and Markov transition matrix analysis, is available in a form of Jupyter notebooks in the Supplemental Material and at

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


  1. 1.

    Attygalle, A. B., Svatos, A., Wilcox, C. & Voerman, S. Gas-phase infrared spectroscopy for determination of double bond configuration of monounsaturated compounds. Anal. Chem. 66, 1696–1703 (1994).

  2. 2.

    Iskra, A. et al. Infrared spectroscopy of gas-phase M+(CO2)n (M=Co, Rh, Ir) ion–molecule complexes. J. Phys. Chem. A 121, 133–140 (2017).

  3. 3.

    Holman, R. W., Rozeboom, M. D., Gross, M. L. & Warner, C. D. Mass spectrometry for investigations of gas-phase radical cation chemistry: the two step cycloaddition of the benzene radical cation and 1,3-butadiene. Tetrahedron 42, 6235–6244 (1986).

  4. 4.

    Jameson, C. J. Gas-phase NMR spectroscopy. Chem. Rev. 91, 1375–1395 (1991).

  5. 5.

    Frank, I., Parrinello, M. & Klamt, A. Insight into chemical reactions from first-principles simulations: the mechanism of the gas-phase reaction of OH radicals with ketones. J. Phys. Chem. A 102, 3614–3617 (1998).

  6. 6.

    Snyder, J. A., Hanway, D., Mendez, J., Jamka, A. J. & Tao, F.-M. A density functional theory study of the gas-phase hydrolysis of dinitrogen pentoxide. J. Phys. Chem. A 103, 9355–9358 (1999).

  7. 7.

    Frank, J. Time-resolved cryo-electron microscopy: recent progress. J. Struct. Biol. 200, 303–306 (2017).

  8. 8.

    Pande, K. et al. Femtosecond structural dynamics drives the trans/cis isomerization in photoactive yellow protein. Science 352, 725 (2016).

  9. 9.

    Barmak, K. et al. Ex situ characterization of phase transformations and associated microstructures in polycrystalline thin films. J. Vac. Sci. Technol. A 17, 1950–1957 (1999).

  10. 10.

    Lucadamo, G., Barmak, K., Carpenter, D. T. & Rickman, J. M. Microstructure evolution during solid state reactions of Nb/Al multilayers. Acta Mater. 49, 2813–2826 (2001).

  11. 11.

    Suresh, S. Colloid modspriel for atoms. Nat. Mater. 5, 253 (2006).

  12. 12.

    Taheri, M. L. et al. Current status and future directions for in situ transmission electron microscopy. Ultramicroscopy 170, 86–95 (2016).

  13. 13.

    Jiang, N. et al. Atom-by-atom fabrication by electron beam via induced phase transformations. Mrs. Bull. 42, 653–659 (2017).

  14. 14.

    Kalinin, S. V. & Pennycook, S. J. Single-atom fabrication with electron and ion beams: from surfaces and two-dimensional materials toward three-dimensional atom-by-atom assembly. Mrs. Bull. 42, 637–643 (2017).

  15. 15.

    Mishra, R., Ishikawa, R., Lupini, A. R. & Pennycook, S. J. Single-atom dynamics in scanning transmission electron microscopy. Mrs. Bull. 42, 644–652 (2017).

  16. 16.

    Zhao, X. et al. Engineering and modifying two-dimensional materials by electron beams. Mrs. Bull. 42, 667–676 (2017).

  17. 17.

    Susi, T. et al. Towards atomically precise manipulation of 2D nanostructures in the electron microscope. 2D Mater. 4, 042004 (2017).

  18. 18.

    Zheng, H. & Zhu, Y. Perspectives on in situ electron microscopy. Ultramicroscopy 180, 188–196 (2017).

  19. 19.

    Jesse, S. et al. Directing matter: toward atomic-scale 3D nanofabrication. ACS Nano 10, 5600–5618 (2016).

  20. 20.

    Gonzalez-Martinez, I. et al. Electron-beam induced synthesis of nanostructures: a review. Nanoscale 8, 11340–11362 (2016).

  21. 21.

    He, Z. et al. Atomic structure and dynamics of metal dopant pairs in graphene. Nano Lett. 14, 3766–3772 (2014).

  22. 22.

    Robertson, A. W. et al. Stability and dynamics of the tetravacancy in graphene. Nano Lett. 14, 1634–1642 (2014).

  23. 23.

    Robertson, A. W. et al. Dynamics of single Fe atoms in graphene vacancies. Nano Lett. 13, 1468–1475 (2013).

  24. 24.

    Yang, Z. et al. Direct observation of atomic dynamics and silicon doping at a topological defect in graphene. Angew. Chem. 126, 9054–9058 (2014).

  25. 25.

    Lee, J., Zhou, W., Pennycook, S. J., Idrobo, J.-C. & Pantelides, S. T. Direct visualization of reversible dynamics in a Si6 cluster embedded in a graphene pore. Nat. Commun. 4, 1650 (2013).

  26. 26.

    Kotakoski, J. et al. Stone–Wales-type transformations in carbon nanostructures driven by electron irradiation. Phys. Rev. B 83, 245420 (2011).

  27. 27.

    Kotakoski, J., Mangler, C. & Meyer, J. C. Imaging atomic-level random walk of a point defect in graphene. Nat. Commun. 5, 3991 (2014).

  28. 28.

    Kotakoski, J., Krasheninnikov, A. V., Kaiser, U. & Meyer, J. C. From point defects in graphene to two-dimensional amorphous carbon. Phys. Rev. Lett. 106, 105505 (2011).

  29. 29.

    Ishikawa, R. et al. Direct observation of dopant atom diffusion in a bulk semiconductor crystal enhanced by a large size mismatch. Phys. Rev. Lett. 113, 155501 (2014).

  30. 30.

    Susi, T. et al. Atomistic description of electron beam damage in nitrogen-doped graphene and single-walled carbon nanotubes. ACS Nano 6, 8837–8846 (2012).

  31. 31.

    Lehtinen, O., Kurasch, S., Krasheninnikov, A. V. & Kaiser, U. Atomic scale study of the life cycle of a dislocation in graphene from birth to annihilation. Nat. Commun. 4, 2098 (2013).

  32. 32.

    Robertson, A. W. et al. Partial dislocations in graphene and their atomic level migration dynamics. Nano Lett. 15, 5950–5955 (2015).

  33. 33.

    Warner, J. H. et al. Dislocation-driven deformations in graphene. Science 337, 209–212 (2012).

  34. 34.

    Zhao, J. et al. Free-standing single-atom-thick iron membranes suspended in graphene pores. Science 343, 1228–1232 (2014).

  35. 35.

    Quang, H. T. et al. In situ observations of free-standing graphene-like mono- and bilayer ZnO membranes. ACS Nano 9, 11408–11413 (2015).

  36. 36.

    Wang, H. et al. Unraveling the atomic structure of ultrafine iron clusters. Sci. Rep. 2, 995 (2012).

  37. 37.

    Ta, H. Q. et al. Single Cr atom catalytic growth of graphene. Nano Res. 11, 2405–2411 (2017).

  38. 38.

    Zhao, J. et al. Direct in situ observations of single Fe atom catalytic processes and anomalous diffusion at graphene edges. Proc. Natl Acad. Sci. 111, 15641–15646 (2014).

  39. 39.

    Wang, W. L. et al. Direct observation of a long-lived single-atom catalyst chiseling atomic structures in graphene. Nano Lett. 14, 450–455 (2014).

  40. 40.

    Ramasse, Q. M. et al. Direct experimental evidence of metal-mediated etching of suspended graphene. ACS Nano 6, 4063–4071 (2012).

  41. 41.

    Zan, R., Bangert, U., Ramasse, Q. & Novoselov, K. S. Metal−graphene interaction studied via atomic resolution scanning transmission electron microscopy. Nano Lett. 11, 1087–1092 (2011).

  42. 42.

    Dorp, W. Fv. et al. Nanometer-scale lithography on microscopically clean graphene. Nanotechnology 22, 505303 (2011).

  43. 43.

    Zan, R., Ramasse, Q. M., Bangert, U. & Novoselov, K. S. Graphene reknits its holes. Nano Lett. 12, 3936–3940 (2012).

  44. 44.

    Lin, J. et al. Flexible metallic nanowires with self-adaptive contacts to semiconducting transition-metal dichalcogenide monolayers. Nat. Nanotechnol. 9, 436 (2014).

  45. 45.

    Jesse, S. et al. Atomic-level sculpting of crystalline oxides: toward bulk nanofabrication with single atomic plane precision. Small 11, 5895–5900 (2015).

  46. 46.

    Jesse, S. et al. Direct atomic fabrication and dopant positioning in Si using electron beams with active real-time image-based feedback. Nanotechnology 29, 255303 (2018).

  47. 47.

    Chuvilin, A., Kaiser, U., Bichoutskaia, E., Besley, N. A. & Khlobystov, A. N. Direct transformation of graphene to fullerene. Nat. Chem. 2, 450 (2010).

  48. 48.

    Girit, Ç. Ö. Graphene at the edge: stability and dynamics. Science 323, 1705 (2009).

  49. 49.

    Kotakoski, J., Santos-Cottin, D. & Krasheninnikov, A. V. Stability of graphene edges under electron beam: equilibrium energetics versus dynamic effects. ACS Nano 6, 671–676 (2012).

  50. 50.

    Susi, T. et al. Silicon-carbon bond inversions driven by 60-keV electrons in graphene. Phys. Rev. Lett. 113, 115501 (2014).

  51. 51.

    Susi, T., Meyer, J. C. & Kotakoski, J. Manipulating low-dimensional materials down to the level of single atoms with electron irradiation. Ultramicroscopy 180, 163–172 (2017).

  52. 52.

    Dyck, O., Kim, S., Kalinin, S. V. & Jesse, S. Placing single atoms in graphene with a scanning transmission electron microscope. Appl. Phys. Lett. 111, 113104 (2017).

  53. 53.

    Dyck, O. et al. Building structures atom by atom via electron beam manipulation. Small 14, 1801771 (2018).

  54. 54.

    Eigler, D. M. & Schweizer, E. K. Positioning single atoms with a scanning tunnelling microscope. Nature 344, 524–526 (1990).

  55. 55.

    Kalinin, S. V., Borisevich, A. & Jesse, S. Fire up the atom forge. Nature 539, 485–487 (2016).

  56. 56.

    Vasudevan, R. K. et al. Mapping mesoscopic phase evolution during E-beam induced transformations via deep learning of atomically resolved images. npj Comput. Mater. 4, 30 (2018).

  57. 57.

    Ziatdinov, M. et al. Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations. ACS Nano 11, 12742–12752 (2017).

  58. 58.

    Belianinov, A. et al. Identification of phases, symmetries and defects through local crystallography. Nat. Commun. 6, 7801 (2015).

  59. 59.

    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A. & Torralba, A. Learning deep features for discriminative localization. "Learning Deep Features for Discriminative Localization," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 2921–2929 (2016).

  60. 60.

    Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. Computer Vision and Pattern Recognition (CVPR). (IEEE: Boston, MA, USA, 2015).

  61. 61.

    Wenzhi, L. et al. Local crystallography analysis for atomically resolved scanning tunneling microscopy images. Nanotechnology 24, 415707 (2013).

  62. 62.

    McLachlan, G., Peel, D. Finite Mixture Models (John Wiley & Sons, Inc., New York, 2000).

  63. 63.

    Azizi, A. et al. Defect coupling and sub-angstrom structural distortions in W1–xMoxS2 monolayers. Nano Lett. 17, 2802–2808 (2017).

  64. 64.

    Ievlev, A. V. et al. Quantitative description of crystal nucleation and growth from in situ liquid scanning transmission electron microscopy. ACS Nano 9, 11784–11791 (2015).

  65. 65.

    Rushton, M. J. D. & Chroneos, A. Impact of uniaxial strain and doping on oxygen diffusion in CeO2. Sci. Rep. 4, 6068 (2014).

  66. 66.

    Li, W. & Li, J. Ferroelasticity and domain physics in two-dimensional transition metal dichalcogenide monolayers. Nat. Commun. 7, 10843 (2016).

  67. 67.

    Arlia, D. & Coppola, M. Experiments in parallel clustering with dbscan. In: European Conference on Parallel Processing. pp. 326–331. (Springer Berlin Heidelberg, 2001).

  68. 68.

    Ziatdinov, M. et al. Deep analytics of atomically-resolved images: manifest and latent features. arXiv:1801.05133.

  69. 69.

    Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. arXiv:1412.6980.

Download references


Research was conducted at the Center for Nanophase Materials Sciences, which is a DOE Office of Science User Facility. R.V.K. and S.V.K. were supported by the U.S. Department of Energy, Office of Science, Basic Energy Sciences, Materials Sciences and Engineering Division. The synthesis of Mo-doped WS2 2D materials (K.W., K.X., D.G.) was supported by the U.S. Department of Energy, Office of Science, Basic Energy Sciences, Materials Sciences and Engineering Division. M.Z., O.D., S.J. acknowledge support by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U.S. Department of Energy. A.M. acknowledges fellowship support from the UT/ORNL Bredesen Center for Interdisciplinary Research and Graduate Education.

Author information


  1. Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA

    • Artem Maksov
    • , Ondrej Dyck
    • , Kai Wang
    • , Kai Xiao
    • , David B. Geohegan
    • , Bobby G. Sumpter
    • , Rama K. Vasudevan
    • , Stephen Jesse
    • , Sergei V. Kalinin
    •  & Maxim Ziatdinov
  2. Institute for Functional Imaging of Materials, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA

    • Artem Maksov
    • , Ondrej Dyck
    • , Rama K. Vasudevan
    • , Stephen Jesse
    • , Sergei V. Kalinin
    •  & Maxim Ziatdinov
  3. Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, TN, 37996, USA

    • Artem Maksov
  4. Computational Sciences & Engineering Division, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA

    • Bobby G. Sumpter
    •  & Maxim Ziatdinov


  1. Search for Artem Maksov in:

  2. Search for Ondrej Dyck in:

  3. Search for Kai Wang in:

  4. Search for Kai Xiao in:

  5. Search for David B. Geohegan in:

  6. Search for Bobby G. Sumpter in:

  7. Search for Rama K. Vasudevan in:

  8. Search for Stephen Jesse in:

  9. Search for Sergei V. Kalinin in:

  10. Search for Maxim Ziatdinov in:


M.Z. and S.V.K. conceived the idea. O.D. acquired the STEM data. A.M. and M.Z. wrote the code and analyzed the data. M.Z. and S.V.K. wrote the paper. K.W., D.G., and K.X. were involved in fabrication of the WS2 film. R.V.K., S.J., and B.G.S. aided in the interpretation of results. All authors commented on the manuscript.

Competing interests

The authors declare no competing interests.

Corresponding authors

Correspondence to Sergei V. Kalinin or Maxim Ziatdinov.

Supplementary information

About this article

Publication history