Introduction

Superhard materials are important in a wide variety of applications including cutting and polishing, or as abrasives and coatings. They typically contain light elements such as B, C, N, and O, which can form short and strong covalent bonds, and they may have complex potential energy surfaces (PES) with numerous low lying minima. Because the main atomic constituents of superhard materials all have similar masses, it is sometimes impossible to determine their crystal structures from X-ray diffraction patterns alone. As a result, first-principles calculations have been instrumental in uncovering the structures of a number of superhard phases including R3m-BC2N,1 the M-carbon phase formed upon cold compression of graphite,2,3,4 Pnnm-CN,5 and cubic BC3.6

Many superhard materials are metastable at ambient pressure/temperature conditions, and the synthesis procedure can have an impact on the product that is made. Therefore, a synergistic feedback loop between theory and experiment is required for the rational design of superhard phases with specific properties. For example, recently an entropy-forming ability descriptor was developed and used to predict novel carbides containing five metals with a high hardness that were subsequently synthesized.7 Another example is the development of machine learning (ML) models for the elastic moduli, which were subsequently employed to screen hundreds of thousands of compounds found in a crystal structure database. The most promising superhard ternaries, Re0.5W0.5C and ReWC0.8, were subsequently synthesized and experiments confirmed they were superhard at low load.8

The a priori prediction of superhard materials is a global optimization problem that requires maximizing the computed hardness in the space of the low-lying local minima on the PES. Microscopic models employing quantities that can be readily obtained such as the geometry of the crystal, atomic connectivity, valence electron density, and electronegativity or ionicity of the constituent atoms9,10,11,12,13 can be used to estimate a structure’s hardness. Recently, the well-known crystal structure prediction (CSP) algorithms CALYPSO12 and USPEX13 have been extended towards the prediction of superhard phases. The structures generated in the course of a CSP search with these programs are typically optimized using first-principles calculations, and their hardnesses are calculated using modified versions of either the Šimunek-Vackář (SV)10 or the Li et al.11 methods, respectively. The main deficiency of the original Li and SV equations is that they predict unreasonably high hardness values for crystals with non-bonding interactions that are necessary for maintaining the three dimensional structure of the system, such as the van der Waals forces between different layers of graphene in graphite. The hardness models developed in CALYPSO and USPEX overcome these limitations by using chemical graph theory to determine which atoms are bonded to each other.

The hardness of a material can also be estimated given its elastic properties, such as the bulk and shear moduli.14,15,16 One of the advantages of these macroscopic hardness models is that they do not depend explicitly upon quantities that are ambiguous or hard to define, such as the atomic radii, bond electronegativity, or the bond strength. However, because elastic properties are expensive to calculate from first-principles, they have so far not been coupled with CSP searches where the hardness of hundreds, if not thousands, of structures would need to be obtained within a single run. Models that require the computed elastic properties are typically only employed to determine the hardness of a handful of promising structures found within a search.

Large materials databases that contain many measured and calculated observables17,18,19,20,21,22 have allowed for the advent of ML models that have demonstrated predictive power for numerous properties, for example superconducting critical temperatures,23 electronic band gaps,24,25 elastic properties,24 and the melting temperatures of unary and binary component solids.26 AFLOW (Automatic FLOW) is an automatic framework for high-throughput materials discovery17 that includes a materials database with over 2 million entries.27 Many properties have been calculated for the materials in the AFLOW database including vibrational properties with the Automatic Phonon Library (APL),17 thermal properties with the Automatic GIBBS Library (AGL),28 and thermomechanical properties such as the bulk and shear moduli with the Automatic Elasticity Library (AEL).22,29 It is now possible to interact with the AFLOW database via a RESTful API30 and the AFLUX materials search API.31

Herein, we illustrate that the Vickers hardnesses, Hv, of a wide variety of crystalline materials predicted by using a macroscopic hardness model in conjunction with ML-derived bulk and shear moduli obtained via the RESTful API32 available on AFLOW are in excellent agreement with results obtained from first-principles calculations. Both are in good agreement with experiment. These developments make it possible to quickly calculate reasonable hardness values for a given crystal structure using ML-based elastic properties, and these hardness estimates can subsequently be employed to calculate an individual’s fitness in a CSP algorithm designed for the prediction of superhard phases. This technique is implemented within the XtalOpt evolutionary algorithm (EA),33,34 and is subsequently applied towards the carbon system to search for stable and superhard phases. Seventy-nine dynamically stable, low energy, distinct topologies with Hv > 40 GPa are found in our searches. Forty-three of the predicted structures have not been reported previously.

Results and discussion

Macroscopic hardness models coupled with machine learning

Because the chemical bonding within a crystalline lattice affects the bulk modulus, B, of the material, it has been proposed that B can be a good indicator of hardness.35,36 While this is true for specific classes of materials, such as diamond-like semiconductors, it turns out that, although some exceptions exist, the shear modulus, G, is a much better predictor of hardness.14,37,38 The linear correlation between hardness and G was originally noted by Teter in 1998,14 and in 2011, Chen et al. used the geometric shape of the Vickers indenter to derive the correlation coefficient.15 We refer to the resulting equation for estimating hardness,

$$H_{{\mathrm{v,Teter}}} = 0.151G,$$
(1)

as the Teter model to distinguish it from another model developed by Chen and co-workers in the same manuscript. Chen et al. suggested that discrepancies between Hv,Teter and the experimentally measured Vickers hardnesses resulted from neglecting the plastic deformations in Eq. (1). In order to better account for these effects, they proposed a new formula that employs the famous Pugh’s modulus ratio, k = G/B, which correlates well with a material’s brittleness. The parameters in the modified equation, which we refer to as the Chen model, were obtained by an empirical fit yielding

$$H_{{\mathrm{v,Chen}}} = 2(k^2G)^{0.585} - 3.$$
(2)

Equation (1) was found to work well for brittle materials with a large k. The Chen model, Eq. (2), yielded better estimates of the hardness in most cases, with one exception being crystals with a low k where the bonding was primarily metallic, such as fcc Al.

In 2012 Tian and co-workers noted that the intercept term in Eq. (2) did not have a physical basis, and would yield negative values for some materials such as KI and KCl.16 Therefore, they obtained a revised formula, which we refer to as the Tian model, via refitting the original function proposed by Chen et al. without the intercept term as

$$H_{{\mathrm{v,Tian}}} = 0.92k^{1.137}G^{0.708}.$$
(3)

The Vickers hardnesses estimated with the formulae of Chen and Tian are in good agreement with experimental measurements for many systems, but they tend to overestimate the hardness of ionic compounds and materials for which Hv is less than ~5 GPa.

The two parameters in the above equations, G and B, have been computed for over 5000 unique materials in the AFLOW database via AEL.27 We employed their Voigt-Reuss-Hill average values, GVRH, and BVRH, to estimate the Vickers hardnesses of 64 systems, including many of those studied by Chen15 (data for the full set can be found in Section S1 in the Supplementary Information, SI; Table 1 lists results only for those systems considered in ref. 15). This set of materials spanned a wide range of hardness values (\(H_{\mathrm{v}}^{{\mathrm{Exp}}} = 0.2 - 96.0\) GPa), and included ionic and covalently bonded crystals, as well as intermetallics. It has been demonstrated that AFLOW-AEL calculates reasonable bulk and shear moduli,29 so it should not be a surprise that the Vickers hardnesses obtained using them are in good agreement with experiment, and the predicted values reflect the variation in the results of the models themselves.

Table 1 A comparison of the Vickers hardness, Hv, for various materials computed via the Teter (Eq. (1)), Chen (Eq. (2)), and Tian (Eq. (3)) models using the Voigt-Reuss-Hill (VRH) averages of the bulk, B, and shear, G, moduli obtained from the Automatic Elasticity Library (AEL) and via a Machine Learning (ML) model trained on the AFLOW database, with experiment. Hv, B, and G are given in units of GPa. The dataset used was taken from ref. 15, which also provides the experimental values. A full table with results for all of the 64 structures used in the correlation analyses and the plots in Fig. 1 is provided in Section S1 in the SI
Full size table

To analyze the agreement between the Vickers hardnesses computed via AFLOW-AEL and the three different models in more detail, a correlation analysis was performed, and the results are provided in Fig. 1, which also plots the theoretical vs. the experimental data. The Pearson and Spearman coefficients measure the linear correlation between two variables, and the monotonicity of the relation between them, respectively. The normalized root-mean-squared relative deviation, RMSrD, between the experimental and computed values was also found, as it can be useful for distinguishing between different methods that may have similar correlations with, but different deviations from, experiment. Whereas the Pearson and Spearman coefficients should ideally be equal to 1, a lower RMSrD is indicative of a better agreement between theory and experiment. All three models showed a high correlation, though the Pearson coefficient was somewhat closer to unity for the Chen and Tian models, as compared to the Teter model. The RMSrD is clearly lower for the Teter model than the other two for this data set when AEL moduli are used. However, the Teter model tends to underestimate the Vickers hardnesses of materials with Hv > 40 GPa. Many of the systems in the full data set are soft, with Hv < 5 GPa, and, as previously mentioned, the macroscopic models are not so good at predicting their hardness values. A correlation analysis was also carried out for the subset of materials studied by Chen,15 see Table 1, and the results are provided in Section S2 in the SI. The correlation coefficients for this subset of systems are higher than for the full data set, and the RMSrD values are significantly lower, <1.

Fig. 1
figure 1

A comparison of the Vickers hardness, Hv in GPa, for 64 materials (see the SI) computed via the Teter (Eq. (1)), Chen (Eq. (2)), and Tian (Eq. (3)) models using the Voigt-Reuss-Hill averages of the bulk and shear moduli obtained from the Automatic Elasticity Library (AEL) and via a Machine Learning (ML) model trained on the AFLOW database, with experiment. The line represents a perfect correlation. The Pearson and Spearman coefficients between the experimental and theoretical data are provided, as is the root-mean-squared relative deviation, RMSrD

Full size image

The overwhelming majority of structures generated in a CSP search will not be found within a repository such as AFLOW. Moreover, it would not be feasible to carry out the first-principles geometry optimizations (24 per structure, unless reducible by symmetry) that AFLOW-AEL requires to obtain the bulk and shear moduli for the hundreds, if not thousands, of structures generated within a single CSP search. Therefore, we wondered if it would be possible to take advantage of the power of data science by using bulk and shear moduli obtained from the ML model that was trained on the materials within the AFLOW repository instead?24 This Property-Labeled Materials-Fragments model24 uses atomic distance and a Voronoi tessellation to characterize the connectivity of a crystal structure: Voronoi cells that are centered on each atom are constructed and two atoms are considered connected if they are within a bonding distance threshold and their Voronoi cells share a face. The connected atoms form a graph, and sections of the graph define the “materials fragments”. The nodes of the graph are decorated with elemental properties of the atomic species at that site, such as electronegativity, ionization energy, and atomic number. Combinations of these elemental properties, along with other properties of the crystal such as lattice parameters and space group, form the feature vector representing the material. The ML model is based on gradient boosted decision trees,39 and is trained on elastic properties calculated using AEL. This model was able to predict the GVRH and BVRH of over 85% of the systems tested to within 20 GPa of the values calculated via AEL, with the ML moduli typically being smaller. The deviation was most pronounced for systems with a large GVRH, and significant differences for BVRH were found for graphite and two theoretical high-pressure B-N phases.

While it would be preferable to train the ML model directly to predict hardness values, this would require training directly on experimental data, since hardness cannot be calculated directly from first-principles.15,16,37 However, because experimental hardness data is not readily available in the quantity and quality required to train ML models (reported experimental values can vary by more than 10%,37 as discussed in Section S4 in the SI), models based on calculated elastic moduli must be used. For example, a recent combined theoretical/experimental ML directed search for superhard materials trained an ML model to predict G and B, which were assumed to be good proxies for hardness, and no attempts to estimate hardness were made.8

Unsurprisingly, Table 1 shows that the AFLOW-ML model also typically underestimated the bulk and shear moduli of the structures considered herein, and it tended to do a better job in predicting BVRH as opposed to GVRH. This suggests that the ML-derived Vickers hardnesses will be smaller than those obtained using AEL for all three models. Comparison of the hardness values obtained with ML vs. AEL in Table 1 and Fig. 1 shows that in general these expectations hold, with the exception of c-BN and AlN for all three models, and SiO2, ZrO2, and Y2O3 for the Chen and Tian models. The correlation coefficients for the ML and AEL results shown in Fig. 1 are similar, but the RMSrD values are somewhat smaller for the results obtained using the ML moduli.

Our findings suggest that the ML moduli can be used to obtain reasonable estimates of the hardness of a given material quickly, making it possible to employ them to determine the fitness of an individual in an EA based structure search. But which one of the macroscopic models should be used? During our EA searches on the elemental carbon system we found that in some instances the ML-derived BVRH were extremely small, and when they were used in the denominator in Eqs. (2) and (3), unrealistically hard values were obtained, with several systems being predicted to be much harder than diamond itself. Only the Teter model was able to predict that these crystals had a very low Vickers hardness. For the 64 systems in our test set, the Pearson and Spearman correlation coefficients between the Vickers hardness values calculated using the Teter model and AEL vs. ML were 0.993 and 0.980, respectively, and the RMSrD was 0.156. Moreover, the Chen and Tian models are theoretically less satisfying than the Teter model, since they are empirically fitted and result in an incorrect dimension. Thus, the Teter model, Eq. (1), was chosen for the hardness evaluations since it can differentiate between the hard crystals that are kept in the gene pool during the evolutionary search, as described in the following section, and those that are soft.

The XtalOpt evolutionary algorithm

EAs employ concepts from biological evolution to find an optimal solution for problems that have many degrees of freedom. When applied towards a priori CSP, EAs search for the lattice parameters and atomic coordinates that minimize or maximize a computed quantity. Because EAs are typically concerned with finding the global minimum (along with important local minima) on the PES, they attempt to minimize the computed energy/enthalpy.40 An EA starts by generating a set of random structures (structures chosen by the user, also known as seeds, may also be employed) that are relaxed to the nearest local minima by an external program. The fitness of each optimized individual is calculated and used to determine the probability that it will be chosen for procreation. Child structures are created either via a two-parent breeding operation, or by a mutation of a single parent. Further details about the XtalOpt EA, and its subprograms can be found in refs. 33,34,41,42,43 (https://xtalopt.github.io/).

The original implementation of the open source EA for CSP, XtalOpt,33 uses roulette wheel selection where the probability pi that a structure with energy/enthalpy Ei is chosen for procreation is equal to its fitness. The probability is calculated as:

$$p_i = N\left( {1 - \frac{{E_i - E_{{\mathrm{min}}}}}{{E_{{\mathrm{max}}} - E_{{\mathrm{min}}}}}} \right),$$
(4)

where Emin and Emax are the lowest and highest energies/enthalpies in the breeding pool, respectively, and N is a normalization constant chosen to ensure that \({\sum} {p_i} = 1\). The lower the energy/enthalpy of an individual, the more likely it is to be selected as a parent for the subsequent generation.

In the algorithms designed to predict superhard materials that are implemented within CALYPSO12 and USPEX,13 the hardness, as calculated via a microscopic model, is employed to determine an individual’s fitness. Using roulette wheel selection, and maximizing the Vickers hardness instead of minimizing the energy/enthalpy, Eq. (4) becomes

$$p_i = N\left( {1 - \frac{{H_{{\mathrm{v,max}}} - H_{{\mathrm{v}},i}}}{{H_{{\mathrm{v,max}}} - H_{{\mathrm{v,min}}}}}} \right).$$
(5)

The higher the hardness of an individual, Hv,i, the more likely it is to be selected for breeding.

Because we are interested in predicting the structures of superhard materials that could potentially be synthesized, it is desirable that they correspond to low lying local minima. Therefore, the new fitness function implemented within XtalOpt combines Eqs. (4) and (5) to favor the selection of structures that are both low in enthalpy and high in hardness via:

$$p_i = N\left[ {1 - w\left( {\frac{{H_{{\mathrm{v,max}}} - H_{{\mathrm{v}},i}}}{{H_{{\mathrm{v,max}}} - H_{{\mathrm{v,min}}}}}} \right) - (1 - w)\left( {\frac{{E_i - E_{{\mathrm{min}}}}}{{E_{{\mathrm{max}}} - E_{{\mathrm{min}}}}}} \right)} \right],$$
(6)

where the weight, w, is a fractional number between 0 and 1. Herein, we employed a w = 0.5, and the Vickers hardnesses used to determine pi were computed via the Teter model, Eq. (1), using the ML values of the shear modulus.

Most EAs do not keep every optimized structure within the breeding pool, but rather only a user-specified number of the fittest individuals. In the original version of XtalOpt, the lowest energy/enthalpy structures were placed in the breeding pool. In the current implementation, the fitness of each individual is computed prior to normalization using Eq. (6). The crystals with the highest pi are placed in the breeding pool, and their normalized probabilities are computed and used.

Applications to carbon

Above we have shown that the Vickers hardness of a compound can be computed with reasonable accuracy via Teter’s equation using the shear modulus obtained with a ML algorithm, and proposed a fitness function that can be used in conjunction within an evolutionary structure search to predict superhard materials. To determine how well this approach works, we performed an EA search on elemental carbon. This system was chosen because of the diversity of the crystalline carbon family that arises from the possibility of sp, sp2, and sp3 hybridized bonds, leading to an infinite number of structures within its chemical space. Moreover, the hardness of its family members ranges from that of soft layered graphite to diamond, the hardest known material on this planet, as well as many well-known superhard species including lonsdaleite, M-carbon, Z-carbon, F-carbon, among many others. The wide range of possible Vickers hardnesses and crystalline topologies within the carbon system creates a tremendous space for XtalOpt to explore, and we are able to evaluate its performance by observing its efficiency on finding pearls, the superhard species, from the ocean of carbon crystals. We expect our EA search will find many reported structures, illustrating that it works, and show that it is a predictive tool that can discover new topologies.

Figure 2 plots \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) for all of the 5624 optimized individuals vs. their energy. The plot is partitioned into four quadrants following the standard suggested by Zhang et al.12. We define superhard structures to have a Vickers hardness larger than 40 GPa, and thermodynamically stable structures to be those whose 0 K energies are less than −8.80 eV/atom, which is within 0.30 eV/atom of diamond. For comparison, the energy of M-carbon, which has been synthesized under pressure,2,3,4 was computed to be −8.93 eV/atom. The superhard species that could potentially be synthesized lie in the top left quadrant of the hardness-energy map, and soft stable ones are found in the bottom left. Superhard phases with comparably high energies are located in the top right hand side of the plot, and unstable soft systems are at the bottom right. The energy of 1324 structures was less than −8.80 eV/atom, and from these 827 had a Vickers hardness greater than 40 GPa. Duplicates were removed from the yellow quadrant in Fig. 2, resulting in 89 distinct topologies, which were found by XtalOpt, that are both hard and thermodynamically favored. Phonon calculations showed that ten of these phases, all containing sp2 carbons, were dynamically unstable. Phonon dispersion curves and phonon density of states plots for all structures, as well as thermodynamic properties for stable structures can be found in Section S3 in the SI. Therefore, in our searches, which employed cells containing 8, 12, 16, and 20 carbon atoms, 79 hard and stable individuals were found.

Fig. 2
figure 2

The hardness vs. energy map of all carbon topologies predicted by the XtalOpt searches. The horizontal line corresponds to a \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}} = 40\) GPa, and the vertical line to an energy of −8.80 eV/atom. The structures in the yellow quadrant are both stable and superhard

Full size image

To determine if the predicted structures are already known, and avoid making false claims of novelty, the Samara Carbon Allotrope Database (SACADA) was employed (http://sacada.sctms.ru).44 SACADA has collected the crystal structures and physical properties from both experiment and theory of more than 500 3D carbon allotropes. Among the hard species found via XtalOpt, 36 are already known, including diamond (#1, dia), lonsdaleite (#37, lon), a chiral-framework (#29, unj), M-carbon (#224, cbn), Z-carbon (#141, sie), F-carbon (#225, 44T35), bct4-carbon (#60, crb), and BC8-carbon (#20, gsi). Some of these are also present in the AFLOW crystal prototype library.45 The SACADA ID numbers and densities of the previously known phases are reported in Table 2, which also provides a comparison between our \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) results with the hardness values computed by others. The reason why the \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) for diamond given in Table 1 differs slightly from that in Table 2 is because in the first case the structure was taken as-is from the AFLOW database, and in the second case it was optimized again by us.

Table 2 Stable superhard structures discovered using the XtalOpt evolutionary algorithm that have been predicted before. Either the well-known names, or those used in the first paper reporting the structure, are provided, along with the SACADA ID and topology. The Vickers hardness (GPa) as calculated via the Teter model using the ML shear modulus, \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\), or computed in other studies, \(H_{\mathrm{v}}^{{\mathrm{others}}}\), as well as the density, ρ, in g/cm3 of the optimized structures is also given
Full size table

Figure 1 clearly illustrates that the Teter model tends to underestimate the Vickers hardnesses of superhard materials. Therefore, it is not surprising that almost all of our \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) values are lower than the Vickers hardness estimates of others. However, once a structure has been predicted by an EA, it is always possible to compute its hardness using a more accurate and computationally expensive hardness model. What is important is that for the unit cell sizes employed herein, our EA is able to find most of the hard stable carbon structures predicted earlier in refs. 12,13. Plus, we find many new stable superhard phases not reported in SACADA, or known as of yet in the literature.

Table 2 shows that the computed \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) for the previously known individuals is typically 10–20 GPa lower than the values obtained using the microhardness models implemented within CALYPSO12 and USPEX.13 One exception is for the chiral-framework structure (#29, unj) originally predicted in ref. 46, which we find to be 28.5 GPa less hard than diamond. The aforementioned microhardness models computed diamond to be only 8–8.4 GPa harder than this P6522 symmetry structure. It should be noted, however, that the chiral framework is significantly less dense than diamond (ρ = 3.21 g/cm3 vs. 3.50 g/cm3), and it would be quite remarkable if it were nearly as hard. The value of \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) obtained herein for this phase, 47.1 GPa, is close to Vickers hardnesses we calculate for phases with a similar density: bikitaite carbon, 6B, G158 and 3,4,4T154.

In addition to the previously predicted phases, our algorithm identified 53 new low-energy topologies that were superhard. Calculations of the phonons and elastic constants confirmed that 43 of these were dynamically and mechanically stable. A summary table, which lists the hardness, relative stability and topological properties of these phases is provided in Section S5 in the SI, and their fascinating structures, properties and electronic structures will be discussed in detail in a follow-up paper. Table 3 lists the ML and AEL values of their bulk and shear moduli, as well as the corresponding Vickers hardnesses computed with the Teter model. Their energy relative to diamond, density, and pressure above which their enthalpies become lower than that of graphite are also given. The caption of Table 3 provides the correlation coefficients and RMSrD between the ML and AEL Vickers hardnesses computed for these new phases using the Teter, Chen and Tian equations. The best agreement between the ML and AEL values is obtained for the Teter model.

Table 3 New stable superhard structures predicted by the XtalOpt evolutionary algorithm. Eref is the energy relative to diamond (eV/atom). The Voigt-Reuss Hill (VRH) average of the bulk, B, and shear, G, moduli obtained from the Automatic Elastic Library (AEL) and via a Machine Learning (ML) model trained on the AFLOW database, and the Vickers hardness, Hv, as computed via the Teter model (Eq. (1)) are provided. The density, ρ, in g/cm3, percent of diamond and lonsdaleite found in the structure, and pressure, P, above which the structures are computed to become more stable than graphite are listed. B, G, Hv, and P are given in units of GPa. The Pearson and Spearman coefficients, along with the RMSrD between the AEL and ML results are 0.858, 0.895, 0.115 for the Teter model, 0.811, 0.849, 0.201 for the Chen model and 0.818, 0.854, and 0.200 for the Tian model
Full size table

M-carbon (#224, cbn), whose \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) is computed to be 68.0 GPa, can be made by compressing graphite to ~17–19 GPa.2,3 Thus, it might be possible to synthesize some of the newly predicted superhard phases under pressure by an appropriate choice of the carbon allotrope starting material and the synthesis route. Metastable carbon phases have been previously synthesized by compressing glassy carbon47,48 or onion carbon,49,50 for instance. To study this further, we computed the pressures above which the newly predicted superhard phases become more stable than graphite. The transition pressures for diamond-like structures (more on this below) were typically between 7 and 8 GPa, whereas other structures have transition pressures within 23 and 47 GPa. Generally speaking, the higher density phases were predicted to become enthalpically favored over graphite at lower pressures, and they had larger Vickers hardnesses than the low density phases, as expected.

Since both the ML and AEL shear moduli of lonsdaleite are larger than that of diamond, but the bulk moduli are about the same, lonsdaleite is predicted to have a slightly higher Vickers hardness by all three macroscopic hardness models. Lonsdaleite has, in fact, been previously predicted to have superior mechanical properties as compared to diamond,51,52 but its experimental existence has been questioned,53 although there are some recent claims of its synthesis both through static47 and shock compression.54

\(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) for a few of the predicted structures was computed to be at least as high as that of diamond. This includes the following from Table 2: 4H and 12R diamond, G153, and 8-layered SiC. These are all composed entirely of mixtures of lonsdaleite and diamond: they are members of the infinite family of dia-lon polytypes. The tiling approach, wherein a 3D periodic net is represented as a tiling of generalized polyhedra, has previously been employed to analyze the topology of sp3 bonded and amorphous carbon allotropes.55,56 Using this method to extract the content of diamond and lonsdaleite topological building blocks in each superhard allotrope found in our EA searches revealed two new dia-lon polytypes that were also, unsurprisingly, found to be harder than diamond (\(R\overline 3 m\)-16 and R3m-16, see Table 3).

The same analysis technique revealed that G178 (#470 48T44), and 16D (#336 416T3), illustrated in Fig. 3a, b, as well as mP16 (#294 48T16), shown in Fig. 2 in ref. 55, contained more than 47% of diamond/lonsdaleite tiles (see Section S5 in the SI). The newly discovered \(P\overline 1\)-16d and \(P\overline 1\)-12 superhard species that are shown in Fig. 3c, d contained 61.6 and 33.7% diamond/lonsdaleite, respectively, along with tiles found in mC12 (#147, 42T112) and oC16-I (#175, 43T84), respectively. The mC12 and oC16-I phases have previously been predicted to be superhard,57 and herein their \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) values were computed to be somewhat lower than that of diamond, 72.6 and 73.5 GPa, respectively. Thus, the newly predicted species that are estimated to be at least as hard as diamond contain a mixture of tiles found in diamond, lonsdaleite, or other superhard phases.

Fig. 3
figure 3

a, b Previously known (see Table 2), and (c–h) newly predicted (see Table 3) superhard phases. The computed \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) are provided. The cages colored in blue are structurally related to diamond (dia), and the cages colored in yellow and green are structurally related to lonsdaleite (lon). The arrows in (f–h) point towards the sp2 carbons

Full size image

The first superhard phase that did not contain diamond or lonsdaleite, P1-16d, is shown in Fig. 3e. It is very asymmetric and appears close to amorphous, or disordered, with 16 different atom types. Indeed, it has been shown that disorder in a material can enhance its hardness.7 The topological analysis further revealed that systems with sp2 hybridized carbon atoms that had an \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) > 70 GPa contained a high percentage of diamond and/or lonsdaleite, at least 40%. They are illustrated in Fig. 3f–h.

Finally, we mention some classic cases where the microhardness models have failed, and illustrate that even for these difficult systems our simple approach yields results that are sufficiently accurate to guide a CSP search. T-carbon (#33, dia-a) is a hypothetical carbon allotrope58 whose prediction prompted scrutiny of the various computational descriptors of hardness. The Gao9 and SV10 microhardness models yielded Vickers hardnesses of 61.1 GPa and 40.5 GPa, respectfully, but it was unclear why such a porous structure with a highly anisotropic distribution of the sp3-like C–C bonds should be superhard. The macroscopic Chen model yielded results that were more in-line with physical reasoning, Hv,Chen = 5.6 GPa.59 The failure of the Gao and SV models to accurately predict the hardness was attributed to the fact that they both assume uniformly distributed bonds. Herein, we find \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\)/\(H_{{\mathrm{v,Teter}}}^{{\mathrm{AEL}}} = 8.7/7.0\) GPa for T-carbon, which is a reasonable estimate. Layered van der Waals bonded systems such as graphite have also proven to be challenging for the microhardness models. For example, the Li model11 predicts a hardness of 57.4 GPa for graphite,13 whereas experimental measurements yield <1 GPa.60 The value we calculate for a graphite structure that was optimized with Grimme’s D3 dispersion correction, 27.8 GPa, is too high for a quantitative estimate. However, the fitness of structures with such a small Vickers hardness would be quite low, and it is unlikely that any of them would be present in the pool in the final stages of an EA search. Herein, 62.5% of the structures whose energy was below −8.80 eV/atom had an \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}}\) greater than 40 GPa, confirming the preferential exploration of the PES of hard structures, which can further be tuned by adjusting the weight in the relation used to determine fitness, Eq. (6).

In the past, a priori crystal structure prediction (CSP) algorithms designed to pinpoint novel superhard phases have estimated a structure’s hardness using microscopic models. Macroscopic hardness models overcome these limitations, but they depend upon elastic properties that are expensive to calculate from first principles.

We have shown that the Vickers hardnesses, Hv, of a wide range of materials computed via the Teter, Chen and Tian macroscopic models along with the bulk and/or shear moduli calculated using AFLOW-AEL agree well with experiment. Moreover, the Hv calculated using the moduli obtained via a ML model that was trained on the AFLOW repository are in excellent agreement with the AFLOW-AEL results. Because the ML based values can be computed quickly given only a crystal’s structure, it is possible to employ them to determine fitness in a CSP search. This methodology has been implemented in the XtalOpt evolutionary algorithm, which can now calculate an individual’s probability for procreation using both the Vickers hardness and the energy/enthalpy, to accelerate the search for hard and stable species.

The method was applied towards the carbon system. In addition to finding 36 known phases, 43 dynamically stable superhard individuals that have not been reported previously were identified. Their structures were analyzed using a topological analysis, which showed that all but one of the newly predicted phases with \(H_{{\mathrm{v,Teter}}}^{{\mathrm{ML}}} \;>\; 70\) GPa were comprised of a substantial fraction of diamond and/or lonsdaleite. The other superhard phase was close to amorphous or disordered. We expect that in the future CSP algorithms will become increasingly coupled with ML derived values to predict materials with desired properties.

Methods

All of the calculations were carried out using density functional theory as implemented within the Vienna ab initio Simulation Package (VASP)61 coupled with the Perdew, Burke and Ernzerhof (PBE) functional,62 and the projector augmented wave (PAW) method.63

Detailed computational settings for the AFLOW-AEL29 calculations used to determine the elastic properties are described in ref. 64. A set of 4 normal and shear deformations (2 compressive and 2 expansive) are applied in each independent direction (1 to 3 depending on the structure’s symmetry) to the fully relaxed structure of a given compound. The ionic positions are optimized while keeping the cell size and shape fixed, and the stress tensors for each deformed structure are calculated. The resulting stress-strain data is fitted to obtain the elastic constants in the form of the symmetric 6 × 6 elastic stiffness tensor (Voigt notation). The bulk and shear moduli are calculated from elastic constants as described in ref. 22.

The EA searches were carried out using XtalOpt version r1234 on unit cells containing 8, 12, 16, and 20 formula units per primitive cell, and a total of 5624 structures were generated. These cell sizes were chosen because they are able to predict phases with 1–6, 8, 10, 12, 16, and 20 atoms in the primitive cell, and because refs. 12,13 considered primitive cells with up to 24 carbon atoms. Tests showed that the likelihood of predicting superhard individuals increased when the initial volumes of the generated crystals were constrained to be about the same as that of diamond. The first generation was constructed using the RandSpg method,43 and duplicates were removed from the breeding pool via the XtalComp algorithm.42 The spacegroups of the predicted superhard phases were determined using AFLOW-SYM,65 and their structures were analyzed via the tiling approach as implemented in the ToposPro package.66

The carbon 2s22p2 electrons were treated as valence, and the core states were described using the PAW method with an energy cutoff of 500 eV in the structure searches, and 600 eV otherwise. Γ-centered Monkhorst-Pack k-meshes were employed where the number of divisions along each reciprocal lattice vector was chosen such that the product of this number with the real lattice constant was 25 Å for the final step in the EA searches, 40 Å for the phonon calculations, and 50 Å for the final optimizations and equation of state calculations. Crystals that satisfied the hardness and stability criteria described in the section “Applications to carbon” were optimized so that all forces were smaller than 10−5 eV/Å. Phonon calculations were carried out using AFLOW-APL (Automatic Phonon Library) in conjunction with VASP.17 APL uses the finite displacement method to calculate the phonon dispersion. Supercells are generated with displaced atoms, and the interatomic forces are calculated using VASP. APL uses these forces to calculate the interatomic force constants and generate the dynamical matrix, which is diagonalized to obtain the phonon eigenmodes and dispersions.