Introduction

Since its initial formulation, density functional theory (DFT)1,2 has become one of the main methods to solve the many-body Schrodinger equation. The source of the major ambiguity in DFT theory is the lack of a universal form of the exchange-correlation (XC) functional. Therefore, the success of DFT is mostly determined by the improvement of XC representations. The seminal Monte Carlo (MC) simulations of Ceperley and Adler3 have resulted in a number of practical local density approximations (LDAs)4,5,6. The next major progress was produced by the introduction of the generalized gradient approximation (GGA), which takes into account local gradients of electron density7,8,9. This greatly enhanced the ability of DFT to describe systems with inhomogeneous electron densities, such as molecules and surfaces. A number of other functionals have been suggested with various degrees of complexity, extending DFT to a broad class of material properties10. The seeking for better functionals is still a very active area of research11,12,13,14.

Despite the obvious success of the LDA and GGA, their construction is a highly complicated process that includes many heuristics steps. Even the XC functional form itself is based on some assumptions about the local nature of XC interactions. A large number of analytical conditions to fulfill9 and a commitment to the local functional form make it difficult to incorporate new numerical results from high-level quantum mechanics theories. For example, only numerical data from MC simulations of homogeneous electron gas are used to produce LDA6 and GGA9 functionals. Therefore, much of the progress in quantum MC15,16 and post-Hartree-Fock17 methods is not adopted by XC approximations. One possible solution to overcome this problem is to use neural networks (NNs) for the interpolation of XC functionals. The NN provides a general way to represent any functional relationship18 and a highly flexible framework to embody both analytical and numerical training data.

There have been a number of studies in which NNs have been used to describe the potential energy surface to speed up electronic structure calculations19,20,21,22,23,24,25,26, while fewer studies have been devoted to developing NN XC functionals27,28,29,30. NN was firstly utilized to approximate XC potential by Tozer et al.31. This study showed that NN approach is flexible and principally applicable to generate XC potentials to be used in the real physical systems DFT calculations.

In this work, we focus on constructing NN XC functionals for 3D systems without any feature engineering. We show that even rather simple NN architectures reproduce LDA and GGA XC potentials with satisfactory accuracy. To construct the LDA, potential point-to-point mapping is used due to the local idea behind the LDA itself. GGA potential region-to-point mapping is achieved by increasing the number of input neurons to take into account the local environment.

This paper is outlined as follows: In methods section, we describe the data generation process, NN topologies and hyperparameters. In results section, we show that the constructed NN XC potentials have the capability to reproduce LDA and GGA results for real physical systems. In conclusion section, we summarize our results and consider future research that could be done with the proposed framework, with a particular emphasis on incorporating data from high-level methods such as quantum Monte Carlo and post-Hartree-Fock.

Methods

To obtain the training data needed to build the NN XC potential, we carry out calculations of three-dimensional (3D) electron gases in a framework of the Kohn-Sham formalism1,2 for potentials of a simple harmonic oscillator (SHO). This potential takes the form:

$${V}_{ext}({\bf{r}})=\frac{1}{2}\mathop{\sum }\limits_{i}^{3}\,{k}_{i}{x}_{i}^{2}$$
(1)

where ki is the spring constant and r = (x1, x2, x3) is the radius vector. Parameter ki pertains to a simple harmonic oscillator randomly sampled from intervals 0.01 ≤ ki ≤ 0.9 Ha/\({{\rm{a}}}_{0}^{2}\), where a0 is 1 Bohr radius.

The XC potential is evaluated on a 40 × 40 × 40 a0 parallelepiped with 32 × 32 × 32 mesh, which corresponds to a spacing of 1.25 a0. Three electrons are placed in this 3D space. For each chosen parameter, DFT calculations in real space were performed using the Octopus code32,33,34. The parametrizations of the LDA exchange-correlation functional used include the Slater exchange35,36 and Perdew-Zunger (modified) correlation part5. The GGA parameters used include the exchange and correlation parts from9,37. The total number of DFT calculations performed is 150 for each XC parametrization. The ranges of electron density represented by the Wigner-Seitz radius rs are [1; 5 × 106] and [1; 108] for LDA and GGA, respectively. One calculation then provides a 32 × 32 × 32 dataset size for LDA and a 28 × 28 × 28 size for GGA. These yield overall dataset sizes of approximately 5.0 million and 3.3 million for LDA and GGA, respectively, but only 3.3 million samples are used for both functionals. 15% of all datasets are used for the test set.

The composition of Tensorflow38 and TFLearn39 is utilized for training the NN. The Adam algorithm with a learning rate of 0.001 is used for training, with MSE loss. The neural architecture used for the LDA is a fully connected network with one input neuron, one linear output neuron and two ReLU40 hidden layers with 1000 and 500 neurons (see Fig. 1). The only difference in the GGA case is that it takes into account the contribution of the local environment, which corresponds to an increase in the number of neurons in the first layer to 125 (5 × 5 × 5 cube). It should be noted that in principle the local environment could be taken into account explicitly by passing gradient and higher order derivatives of electron density. We delegate a computation of electron density derivatives to NN itself, making preprocessing step easy and convenient. Considered 5 × 5 × 5 input cube gives information to calculate gradient and Jacobian matrix with O(Δr2) and O(Δr) precision correspondingly, where Δr is mesh spacing.

Figure 1
figure 1

Topology of the developed neural network. Two hidden layers are used, with 1000 and 500 neurons, respectively. A base 10 logarithm is applied to the input data (electron density). This helps to scale the input data to the range that the neural network is most sensitive to.

Full size image

Results and Discussion

Multiple neural architectures, including different activation functions, numbers of layers and numbers of neurons, are tested. We choose robust and rather simple architectures, namely, the point-to-point and region-to-point architectures described above. We also test NNs with only one hidden layer with 1000 neurons, which have poor generalizability and result in a larger loss in comparison with the topology used. It should be noted that the topology chosen is heuristics, and it is possible that another number of hidden layers and neurons could give better results than those provided in this study. The search for the optimal NN topology is a large task that needs to be carried out in future studies.

Because the density range contains mostly small values, base-10 logarithmic transformation is chosen for preprocessing the dataset. This scaling greatly improves the convergence speed of the backpropagation algorithm during the NN training cycle. No other preprocessing techniques and feature engineering are applied to the input data.

Training of the LDA NN is stopped after 1300 epochs. The NN LDA mean absolute error (MAE) is 0.03 mHa × Bohr3. The mean minimum and maximum values of Vxc averaged over all configurations used for the LDA NN construction are 0.73 and 510 mHa × Bohr3, respectively. Training of the GGA NN is terminated after 320 epochs. The NN GGA mean absolute error (MAE) is 0.156 mHa × Bohr3. The mean minimum and maximum values of Vxc averaged over all configurations used for the GGA NN construction are 0 and 553 mHa × Bohr3, respectively. The results of the training for both the LDA and GGA NNs are summarized in Table 1.

Table 1 General information about NN XC functionals training process.
Full size table

The results of applying an NN to the electron density from the test set are presented in Fig. 2. Both XC potentials provided by the LDA and GGA NNs are in good agreement with the numerical calculations made by Octopus. The special features of the Vxc form, for example, the “two humps”, are also well reproduced by them. For LDA it is also possible to provide comparison between \({V}_{xc}^{NN}({r}_{s})\) and analytical reference function Vxc(rs). Plotted curves are indistinguishable in the range of rs from 1 to 15, the absolute maximum difference is 1.7 mHa × Bohr3 (see Fig. S1 of supplementary materials).

Figure 2
figure 2

The results of applying NN XC functionals to the self-consistent electron density obtained from solving the Schrodinger equation in the framework of DFT for 3 electrons in a simple harmonic potential. LDA and GGA “true” Vxc results are produced by the Octopus package. All presented images are slices of 3D data in the z = 0 plane; (a) - the slice of the electron density obtained in the DFT calculation using LDA XC funtional, (b) - the slice of the XC potential obtained in the same as (a) DFT calculation using LDA XC functional, (c) - the slice of the NN XC potential obtained from the (a) electron density, (d) - the slice of the electron density obtained in the DFT calculation using GGA XC funtional, (e) - the slice of the XC potential obtained in the same as (d) DFT calculation using GGA XC functional, (f) - the slice of the NN XC potential obtained from the (d) electron density.

Full size image

To test the NN XCs for real physical systems, we consider Si in a diamond structure with 8 basis atoms in a cubic cell (lattice constant: 10.2 a0). The numerical calculation is performed on a 10.2 × 10.2 × 10.2 a0 parallelepiped with a 14 × 14 × 14 mesh, which corresponds to a spacing of 0.729 a0. The NN LDA and NN GGA MAEs are 0.6 and 18.1 mHa × Bohr3, respectively. The NN LDA results are in excellent agreement with the reference data obtained from Octopus. The NN GGA describes Vxc reasonably well, except for regions with high density variations (see Fig. 3). We believe that this is because of the different spacings of the training/test data, which lead to the change in electron density gradients perceived by the NN XC.

Figure 3
figure 3

The results of applying NN XC functionals to the self-consistent electron density obtained from solving the Schrodinger equation in the framework of DFT for a Si diamond structure with 8 basis atoms in a cubic cell. The LDA and GGA “true” Vxc results are produced by the Octopus package. All presented images are slices of 3D data in the (001) plane defined by the Miller indices; (a) - the slice of the electron density obtained in the DFT calculation using LDA XC funtional, (b) - the slice of the XC potential obtained in the same as (a) DFT calculation using LDA XC functional, (c) - the slice of the NN XC potential obtained from the (a) electron density, (d) - the slice of the electron density obtained in the DFT calculation using GGA XC funtional, (e) - the slice of the XC potential obtained in the same as (d) DFT calculation using GGA XC functional, (f) - the slice of the NN XC potential obtained from the (d) electron density.

Full size image

The NN XC potentials are also tested on slices of 3D data in the plane of the benzene molecule. The numerical calculation is performed on a 14 × 14 × 14 a0 parallelepiped with a 44 × 44 × 44 mesh, which corresponds to a spacing of 0.318 a0. The NN LDA and NN GGA MAEs are 1.36 and 18.3 mHa × Bohr3, respectively. The overall form of the XC potential obtained by Octopus is similar to the form produced by both NNs. In the case of benzene, the MAE is substantially larger than the MAE obtained during the training/test procedure. This is due to the high electron density values obtained for benzene, which are not presented in the training/test dataset. Moreover, the mesh spacing in the benzene calculation is smaller than that in the dataset, which is important for NN GGA potential evaluation. Despite the lack of appropriate input data, both NN XC potentials show an impressive ability to generalize the XC functional (see Fig. 4).

Figure 4
figure 4

The results of applying NN XC functionals to the self-consistent electron density obtained from solving the Schrodinger equation in the framework of DFT for a benzene molecule. The LDA and GGA “true” Vxc results are procuced by the Octopus package. All presented images are slices of 3D data in the plane of the benzene molecule; (a) - the slice of the electron density obtained in the DFT calculation using LDA XC funtional, (b) - the slice of the XC potential obtained in the same as (a) DFT calculation using LDA XC functional, (c) - the slice of the NN XC potential obtained from the (a) electron density, (d) - the slice of the electron density obtained in the DFT calculation using GGA XC funtional, (e) - the slice of the XC potential obtained in the same as (d) DFT calculation using GGA XC functional, (f) - the slice of the NN XC potential obtained from the (d) electron density.

Full size image

The results obtained from the developed NN XC potentials show that NN LDA works well, provided that the electron density values are presented in the training/test dataset. Based on Table 2, the NN LDA MAE is rather small for silicon and becomes larger for benzene, in which the electron density contains values that are outside the training/test density range. It is not surprising that the NN GGA (see Table 3) demonstrates a large MAE for both systems considered, as the mesh spacing is not presented in the training procedure. Electron density spatial information not included explicitly in the training set leads to the fact that neural networks can take gradients into account in a way that cannot be properly generalized. Changing the mesh corrupts the learned representations and makes it not quite correct to apply the NN GGA to systems on meshes other than the one learned.

Table 2 Summary of results for NN LDA.
Full size table
Table 3 Summary of training results for NN GGA.
Full size table

To construct mesh-independent nonlocal functionals such as the GGA, it is necessary to use the coordinates of mesh points at which the electron density is defined. This additional information will help the NN evaluate mesh-independent gradients or other higher-order variations of electron density and include them in the process of the interpolation of the exchange-correlation potential. The type of local coordinate system to use remains the subject of further research.

We also implemented both NN LDA and NN GGA potentials in the OCTOPUS package. The xc-energy was taken from original LDA5,35,36 and GGA9,37 functionals. A behaviour during self-consistent cycle and an ability to perform geometry optimization were tested. Lattice constant of Si was scanned in range from 9.6 to 10.95 Bohr with a step 0.15 Bohr. Bond length of a benzene molecule was changed in range from 2.37 to 2.90 Bohr with a step 0.053 Bohr. All performed self-consistent calculations converged with density tolerance 0.5 × 10−5 Bohr−3. In case of LDA dependencies of a total energy on a geometry parameter for both Si and the benzene molecule are in good agreement with results obtained for original xc-potentials (see Fig. 5). There is a notable shift to higher values for NN GGA dependencies of a total energy on a geometry parameter in comparison with one obtained from original GGA xc-potential. We attribute this differences to the fact that total xc-energy was not fully consistent with the NN xc-potential. In general, obtained results for self-consistent calculations show that NN xc-potentials are converged and can be used in geometry optimization problems.

Figure 5
figure 5

The results of applying NN XC potentials in self-consistent cycle. The xc-energy is calculated from original functional that was used for training NN; (a) - Si dependence of total energy on lattice parameter for NN LDA, (b) - The benzene molecule dependence of total energy on bond length for NN LDA, (c) - Si dependence of total energy on lattice parameter for NN GGA, (d) - The benzene molecule dependence of total energy on bond length for NN GGA.

Full size image

Conclusions

In this work, we developed, verified and tested a neural-network approach to interpolating XC functionals. We show that the NN approach gives adequate results for both the LDA and GGA considered. When considering a model system based on three electrons placed in a parallelepiped box with an external potential of a 3D simple harmonic oscillator, the NN is in perfect agreement with numerical calculations from the test dataset. Moreover, the neural network shows a satisfactory error in real 3D physical systems not presented in the training/test data, such as silicon and benzene. Despite the lack of generalizability of the NN GGA for both systems, the analysis of those results led us to the idea of improving the NN approach by searching for a better type of coordinate system for proper representation of the input data. It is important to note that the neural network topology used in this work is very simple and has good potential for further improvements.

The main advantage of the NN approach in comparison with other interpolation techniques for XC functionals is its flexibility to incorporate exchange-correlation data from different sources, such as post-Hartree-Fock and quantum Monte Carlo. It is possible that application of the NN to interpolate high-level XC quantum data could eliminate many heuristics used in the traditional construction of XC functionals.