Introduction

Density functional theory (DFT)1,2 is an essential method for describing electronic states in all manner of nanoscale phenomena, including chemical bonds in molecules, band structures of materials, electron transfer, and reactive metal clusters of proteins. In principle, an exact theory, DFT in practice3,4,5,6,7,8, has remained far from exact due to the unavailability of exact exchange-correlation (xc) potentials (\({v}_{{\rm{xc}}}\)), which are responsible for describing the quantum mechanical behavior of electrons. Fortunately, \({v}_{{\rm{xc}}}\) is a unique functional of the electron density (\(\rho ({\bf{r}})\)), so there exists a one-to-one relationship from \({v}_{{\rm{xc}}}({\bf{r}})\) to \(\rho ({\bf{r}})\) and vice versa. This observation presents a possible route forward to construct accurate xc functionals via the transformation of the electron density into \({v}_{{\rm{xc}}}({\bf{r}})\) through the so-called inverse DFT problem9,10,11,12,13 (refer to the schematic in Fig. 1). The inverse problem not only provides a route for finding the sole unknown quantity in DFT, it is also central for describing quantum mechanics without resorting to complicated multi-electron wave functions.

Fig. 1
figure 1

Schematic of the inverse DFT problem. The exact ground-state many-body wavefunction (\(\Psi ({{\bf{r}}}_{1},{{\bf{r}}}_{2},\ldots ,{{\bf{r}}}_{{N}_{e}})\)) and, hence, the ground-state electron density (\(\rho ({\bf{r}})\)) is obtained from configuration interaction calculation. The inverse DFT calculation evaluates the exact exchange-correlation potential (\({v}_{{\rm{xc}}}({\bf{r}})\)) that yields the given \(\rho ({\bf{r}})\). The ability to accurately solve the inverse DFT problem, presented in this work, presents a powerful tool to construct accurate density functionals (\({v}_{{\rm{xc}}}[\rho ({\bf{r}})]\)), either through conventional approaches or via machine learning. The schematic shows the ground-state density and the exact exchange-correlation potential for \({{\rm{H}}}_{2}{\rm{O}}\) obtained in this work

Full size image

Given the large importance of this problem, there have been several attempts to solve the inverse DFT problem, employing either iterative updates10,11,14,15,16 or constrained optimization approaches9,12,17,18. However, these approaches have suffered from ill-conditioning, thereby resulting in non-unique solutions or causing spurious oscillations in the resultant \({v}_{{\rm{xc}}}({\bf{r}})\). This ill-conditioning has been largely attributed to the incompleteness of the Gaussian basis sets that were employed to solve the inverse DFT problem18,19,20. Recent efforts21,22,23 have presented a different approach, which utilizes the two-electron reduced density matrix to remedy the non-uniqueness and the spurious oscillations in the obtained \({v}_{{\rm{xc}}}({\bf{r}})\). However, this does not represent the solution of the inverse DFT problem, i.e., the \({v}_{{\rm{xc}}}\) obtained from this approach is not guaranteed to yield the input electron density23. Thus, the inverse DFT problem has, heretofore, remained an open challenge.

In this work, we present an advance that provides an accurate solution to the inverse DFT problem, enabling the evaluation of the exact \({v}_{{\rm{xc}}}\) from an ab-initio density. Specifically, the approach uses a finite-element (FE) basis that is systematically convergent and complete, thereby eliminating ill-conditioning in the discrete solution of the inverse DFT problem. Our approach is tested on a range of molecular systems, both weakly and strongly correlated, showing robustness and efficacy in treating realistic polyatomic molecules. The proposed approach therefore unlocks the door to constructing accurate xc functionals that provide precise energies and electronic properties of a huge range of chemical, materials, and biological systems. To elaborate, we envisage the inverse DFT problem to be instrumental in generating \(\{{\rho }^{(i)},{v}_{{\rm{xc}}}^{(i)}\}\) pairs, using \({\rho }^{(i)}\)’s from correlated ab-initio calculations. Subsequently, these can be used as training data to model \({v}_{{\rm{xc}}}[\rho ]\) through machine-learning algorithms24,25, which are designed to preserve the functional derivative requirement on \({v}_{{\rm{xc}}}[\rho ]\)26. Furthermore, the xc energy (\({E}_{{\rm{xc}}}[\rho ]\)) can be directly evaluated through line integration on \({v}_{{\rm{xc}}}[\rho ]\).

Results

Constrained optimization for inverse DFT

We cast the inverse DFT problem of finding the \({v}_{{\rm{xc}}}({\bf{r}})\) that yields a given density \({\rho }_{{\rm{data}}}({\bf{r}})\) as the following partial differential equation (PDE)-constrained optimization:

$$\arg \ \mathop{min}\limits_{{v}_{{\rm{xc}}}({\bf{r}})}\int w({\bf{r}}){\left({\rho }_{{\rm{data}}}({\bf{r}})-\rho ({\bf{r}})\right)}^{2}\ d{\bf{r}},$$
(1)

subject to

$$\left(-\frac{1}{2}{\nabla }^{2}+{v}_{{\rm{ext}}}({\bf{r}})+{v}_{{\rm{H}}}({\bf{r}})+{v}_{{\rm{xc}}}({\bf{r}})\right){\psi }_{i}={\epsilon }_{i}{\psi }_{i},$$
(2)
$$\int | {\psi }_{i}({\bf{r}}){| }^{2}\ d{\bf{r}}=1\ .$$
(3)

In the above equation, \(w({\bf{r}})\) is an appropriately chosen weight to expedite convergence, \({v}_{{\rm{ext}}}({\bf{r}})\) represents the nuclear potential, \({v}_{{\rm{H}}}({\bf{r}})\) is the Hartree potential corresponding to \({\rho }_{{\rm{data}}}({\bf{r}})\), and \({\psi }_{i}\) and \({\epsilon }_{i}\) denote the Kohn–Sham orbitals and eigenvalues, respectively. For simplicity, we restrict ourselves to only closed-shell systems and, hence, the Kohn–Sham density \(\rho ({\bf{r}})=2{\sum }_{i=1}^{{N}_{e}/2}| {\psi }_{i}({\bf{r}}){| }^{2}\). Equivalently, the above PDE-constrained optimization can be solved by minimizing the following Lagrangian,

$$ {\mathcal{L}}\left({v}_{{\rm{xc}}},\{{\psi }_{i}\},\{{p}_{i}\},\{{\epsilon }_{i}\},\{{\mu }_{i}\}\right)=\int w({\bf{r}}){\left({\rho }_{{\rm{data}}}({\bf{r}})-\rho ({\bf{r}})\right)}^{2}\ d{\bf{r}}\\ \!+{\sum }_{i=1}^{{N}_{e}/2}\int {p}_{i}({\bf{r}})\left(\hat{H}-{\epsilon }_{i}\right){\psi }_{i}\ d{\bf{r}} +{\sum }_{i=1}^{{N}_{e}/2}{\mu }_{i}\left(\int | {\psi }_{i}({\bf{r}}){| }^{2}\ d{\bf{r}}-1\!\right),$$
(4)

with respect to all its constituent variables—\({p}_{i}\), \({\mu }_{i}\), \({\psi }_{i}\), \({\epsilon }_{i}\) and \({v}_{{\rm{xc}}}\). In the above equation, \(\hat{H}=-\frac{1}{2}{\nabla }^{2}+{v}_{{\rm{ext}}}({\bf{r}})+{v}_{{\rm{H}}}({\bf{r}})+{v}_{{\rm{xc}}}({\bf{r}})\) is the Kohn–Sham Hamiltonian, \({p}_{i}\) is the adjoint function that enforces the Kohn–Sham eigenvalue equation corresponding to \({\psi }_{i}\), and \({\mu }_{i}\) is the Lagrange multiplier corresponding to the normality condition of \({\psi }_{i}\). The optimality of \({\mathcal{L}}\) with respect to \({p}_{i}\), \({\mu }_{i}\), \({\psi }_{i}\), and \({\epsilon }_{i}\) are given by:

$$\hat{H}{\psi }_{i}={\epsilon }_{i}{\psi }_{i},$$
(5)
$$\int | {\psi }_{i}({\bf{r}}){| }^{2}\ d{\bf{r}}=1,$$
(6)
$$(\hat{H}-{\epsilon }_{i}){p}_{i}({\bf{r}})={g}_{i}({\bf{r}}),$$
(7)
$$\int {p}_{i}({\bf{r}}){\psi }_{i}({\bf{r}})\ d{\bf{r}}=0,$$
(8)

where \({g}_{i}({\bf{r}})=8w({\bf{r}})({\rho }_{{\rm{data}}}({\bf{r}})-\rho ({\bf{r}})){\psi }_{i}-2{\mu }_{i}{\psi }_{i}\). We remark that the operator \(\hat{H}-{\epsilon }_{i}\) in Eq. 7 is singular with \({\psi }_{i}\) as its null vector. However, the orthogonality of \({g}_{i}\) and \({\psi }_{i}\) (consequence of Eq. 7) along with the orthogonality of \({p}_{i}\) and \({\psi }_{i}\) (Eq. 8) guarantee a unique solution for \({p}_{i}\). Having solved the above optimality conditions in Eqs. 5–8, the variation (gradient) of \({\mathcal{L}}\) with respect to \({v}_{{\rm{xc}}}\) is given by

$$\frac{\delta {\mathcal{L}}}{\delta {v}_{{\rm{xc}}}}={\sum }_{i=1}^{{N}_{e}/2}{p}_{i}{\psi }_{i}.$$
(9)

This constitutes the central equation for updating \({v}_{{\rm{xc}}}({\bf{r}})\) via any gradient-based optimization technique.

Summing up, the proposed approach involves: (i) obtaining \({\rho }_{{\rm{data}}}({\bf{r}})\) from correlated ab-initio calculations (i.e., configuration interaction (CI) calculations); (ii) using an initial guess for \({v}_{{\rm{xc}}}({\bf{r}})\); (iii) solving Eqs. 5–8 using the current iterate of \({v}_{{\rm{xc}}}\); (iv) updating \({v}_{{\rm{xc}}}\) using Eq. 9 as the gradient; (v) repeating (iii)–(iv) until \(\rho ({\bf{r}})\) converges to \({\rho }_{{\rm{data}}}({\bf{r}})\). We note that the general idea of PDE-constrained optimization has been explored recently in ref. 13. However, its utility had only been demonstrated on non-interacting model systems in one dimension.

Verification with LDA-based densities

To assess the accuracy and robustness of the proposed approach, we use \({\rho }_{{\rm{data}}}\) obtained from local density approximation (LDA)27,28-based DFT calculations, discretized using the FE basis—a systematically improvable and complete basis constructed from piecewise polynomials. This verification test allows us to compare the \({v}_{{\rm{xc}}}\) obtained from the inverse DFT calculation against \({v}_{{\rm{xc}}}^{{\rm{LDA}}}[{\rho }_{{\rm{data}}}]\). As remarked earlier, most of the previous attempts at this verification test have suffered from either non-unique solutions or had resulted in unphysical oscillations in \({v}_{{\rm{xc}}}\), owing to the incompleteness of the Gaussian basis employed in these works. Figure 2 presents the comparison of \({v}_{{\rm{xc}}}^{{\rm{LDA}}}[{\rho }_{{\rm{data}}}]\) against the \({v}_{{\rm{xc}}}\) obtained from the inverse calculation, for various atomic systems (also see Supplementary Fig. 2). We also provide, in Fig. 3, the \({v}_{{\rm{xc}}}\) for 1,3-dimethylbenzene (\({{\rm{C}}}_{8}{{\rm{H}}}_{10}\)) obtained from the inverse calculation with LDA-based \({\rho }_{{\rm{data}}}\) (cf. Supplementary Fig. 3 for the error in \({v}_{{\rm{xc}}}\)), highlighting the efficacy of our approach in accurately treating large systems. We note that all the inverse DFT calculations have been performed in three dimensions and the \({L}^{2}\) norm error in the density, \(| | {\rho }_{{\rm{data}}}-\rho | {| }_{{L}^{2}}\), is driven below \(1{0}^{-5}\). As evident from these figures, the \({v}_{{\rm{xc}}}\) determined from the inverse DFT calculation is devoid of any spurious oscillations and is in excellent agreement with \({v}_{{\rm{xc}}}^{{\rm{LDA}}}[{\rho }_{{\rm{data}}}]\). In addition, the Kohn–Sham eigenvalues computed using the inverted \({v}_{{\rm{xc}}}\) are in excellent agreement (i.e., \(| {\epsilon }_{i}^{{\rm{LDA}}}-{\epsilon }_{i}| < 1\) mHa), further validating the accuracy of the method. Although we have reported the verification of our method for LDA-based densities, similar accuracy was obtained using generalized gradient approximation (GGA)-based densities. We refer to the Supplementary Discussion for a comparison of these verification results against similar studies conducted using existing methods.

Fig. 2
figure 2

Verification study on atomic systems using LDA-based density. The density (\({\rho }_{{\rm{data}}}\)) is obtained from a ground-state DFT calculation using an LDA functional. The solid line corresponds to the direct evaluation of the LDA exchange-correlation potential corresponding to \({\rho }_{{\rm{data}}}\), i.e., \({v}_{{\rm{xc}}}^{{\rm{LDA}}}[{\rho }_{{\rm{data}}}]\). The dashed line corresponds to the exchange-correlation potential obtained from the inverse DFT calculation using \({\rho }_{{\rm{data}}}\) as the input. The atomic systems considered are as follows: (a) He; (b) Be; (c) Ne

Full size image
Fig. 3
figure 3

Inverse DFT calculation on \({{\rm{C}}}_{8}{{\rm{H}}}_{10}\). The exchange-correlation potential (in a.u.) determined from the inverse DFT algorithm, using an LDA-based density, is displayed on the plane of the benzene ring. Refer to Supplementary Table 3 for the coordinates

Full size image

Removing Gaussian basis-set artifacts

We next turn to employing the proposed method with input densities generated from CI calculations. All the CI calculations reported in this work are performed using the incremental full-CI approach presented in ref. 29 and discretized using the universal Gaussian basis set (UGBS)30 or polarized triple zeta (cc-PVTZ) Gaussian basis set31. It is known that Gaussian basis-set densities, owing to their lack of cusp at the nuclei as well as incorrect far-field decay, induce highly unphysical features in the \({v}_{{\rm{xc}}}\)s obtained from inverse calculations. To this end, we provide two numerical strategies, which, for all practical purposes, remedy the Gaussian basis-set artifacts and thereby allow for accurate evaluation of the exact \({v}_{{\rm{xc}}}\)s from CI densities. It is to be noted that the following numerical strategies are only necessitated due to the unphysical asymptotics in the Gaussian basis-set densities and not due to any inadequacy of the proposed inverse DFT algorithm.

To begin with, the CI density obtained from a Gaussian basis has wrong decay characteristics away from the nuclei (i.e., Gaussian decay instead of exponential decay). This, in turn, results in incorrect asymptotics in the \({v}_{{\rm{xc}}}\) obtained from an inverse DFT calculation. Thus, to ensure the correct asymptotics in \({v}_{{\rm{xc}}}\), we employ the following approach. First, we use an initial guess for \({v}_{{\rm{xc}}}\) that satisfies the correct \(-1/r\) decay. In particular, we use the Fermi–Amaldi potential (\({v}_{{\rm{FA}}}\))32. Next, we enforce homogeneous Dirichlet boundary condition on the adjoint function (\({p}_{i}\)) in the low-density region (i.e., \({\rho }_{{\rm{data}}}< 1{0}^{-6}\)), while solving Eq. 7. In effect, this fixes the \({v}_{{\rm{xc}}}\) to its initial value in the low-density region, thereby ensuring correct far-field asymptotics in the \({v}_{{\rm{xc}}}\). This approach is also crucial to obtaining an agreement between the highest occupied Kohn–Sham eigenvalue (\({\epsilon }_{{\rm{H}}}\)) and the negative of the ionization potential (\({I}_{{\rm{p}}}\)), as mandated by the Koopmans’ theorem33,34.

Furthermore, the Gaussian basis-set-based CI densities lack the cusp at the nuclei, which, in turn, leads to undesirable oscillations in the \({v}_{{\rm{xc}}}\) near the nuclei in any inverse DFT calculation35,36,37. We demonstrate this in the case of equilibrium \({{\rm{H}}}_{2}\) molecule (bond-length \({R}_{\text{H-H}}=1.4\) a.u.), henceforth denoted as \({{\rm{H}}}_{2}(eq)\). Figure 4 shows the \({v}_{{\rm{xc}}}\) profile for \({{\rm{H}}}_{2}(eq)\) corresponding to the \({\rho }_{{\rm{data}}}({\bf{r}})\) obtained from a CI calculation, discretized using UGBS. As evident, we observe large unphysical oscillations in the \({v}_{{\rm{xc}}}\) near the nuclei. We remedy these oscillations by adding a small correction, \(\Delta \rho ({\bf{r}})\) to \({\rho }_{{\rm{data}}}({\bf{r}})\), so as to correct for the missing cusp at the nuclei. The \(\Delta \rho ({\bf{r}})\) is given by

$$\Delta \rho ({\bf{r}})={\rho }_{{\rm{FE}}}^{{\rm{DFT}}}({\bf{r}})-{\rho }_{{\rm{G}}}^{{\rm{DFT}}}({\bf{r}}),$$
(10)

where \({\rho }_{{\rm{FE}}}^{{\rm{DFT}}}({\bf{r}})\) is the ground-state density obtained from a forward DFT calculation using a known xc functional (e.g., LDA and GGA) and discretized using the FE basis, and \({\rho }_{{\rm{G}}}^{{\rm{DFT}}}({\bf{r}})\) denotes the same, albeit obtained using the Gaussian basis employed in the CI calculation. The key idea here is that \({\rho }_{{\rm{FE}}}^{{\rm{DFT}}}({\bf{r}})\), obtained from the FE basis, contains the cusp. Thus, one can expect \(\Delta \rho\) to reasonably capture the Gaussian basis-set error near the nuclei. In addition, \(\int \Delta \rho ({\bf{r}})\ d{\bf{r}}=0\), preserving the number of electrons. A conceptually similar approach has been explored in ref. 37, wherein one post-processes the \({v}_{{\rm{xc}}}\) instead of pre-processing the \({\rho }_{{\rm{data}}}\), to remove the oscillations arising from the lack of cusp in \({\rho }_{{\rm{data}}}\). We illustrate the efficacy of the \(\Delta \rho\) correction with the \({{\rm{H}}}_{2}(eq)\) molecule as an example. Figure 5 presents the \({v}_{{\rm{xc}}}\) corresponding to the cusp-corrected density (i.e., \({\rho }_{{\rm{data}}}+\Delta \rho\)) for \({{\rm{H}}}_{2}(eq)\), with two different \(\Delta \rho\): \(\Delta {\rho }_{{\rm{LDA}}}\) evaluated using an LDA functional27,28 and \(\Delta {\rho }_{{\rm{GGA}}}\) evaluated using a GGA functional38. As evident, both \(\Delta {\rho }_{{\rm{LDA}}}\)- and \(\Delta {\rho }_{GGA}\)-based cusp correction generate smooth \({v}_{{\rm{xc}}}\) profiles. More importantly, both the profiles are nearly identical, except for small differences in the bonding region between the H atoms. Further, a comparison of both these \({v}_{{\rm{xc}}}\)s against the LDA-based xc potential (\({v}_{{\rm{xc}}}^{{\rm{LDA}}}\)) elucidates the significant difference between the exact \({v}_{{\rm{xc}}}\) and \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\) even for a simple system that is not strongly correlated. Lastly, for both the \({v}_{{\rm{xc}}}\)s, we obtain the same \({\epsilon }_{{\rm{H}}}\) of \(-0.601\) Ha, which, in turn, is in excellent agreement with the \(-{I}_{{\rm{p}}}\) (listed in Table 1). We remark that the agreement of \({\epsilon }_{{\rm{H}}}\) with \(-{I}_{{\rm{p}}}\) is a stringent test of the accuracy of the inversion and is particularly sensitive to the \({v}_{{\rm{xc}}}\) in the far field.

Fig. 4
figure 4

Artifact of Gaussian basis-set-based density. The exchange-correlation potential (\({v}_{{\rm{xc}}}\)) is evaluated from inverse DFT, using \({\rho }_{{\rm{data}}}\) obtained from a Gaussian basis-set-based configuration interaction (CI) calculation for the equilibrium hydrogen molecule (\({{\rm{H}}}_{2}(eq)\)). The lack of cusp in \({\rho }_{{\rm{data}}}\) at the nuclei induces wild oscillations in the \({v}_{{\rm{xc}}}\) obtained through inversion. The two atoms are located at \(r=\pm 0.7\) a.u

Full size image
Fig. 5
figure 5

Exchange-correlation potentials (\({v}_{{\rm{xc}}}\)) for equilibrium \({{\rm{H}}}_{2}\). A comparison is provided between the exact and the LDA-based \({v}_{{\rm{xc}}}\) potential. The exact exchange-correlation potential is evaluated using the cusp-corrected configuration interaction (CI) density. The effect of the choice of the functional used in evaluating the cusp correction is demonstrated using two different functionals—LDA (\({\rm{exact}}-\Delta {\rho }_{{\rm{LDA}}}\)) and GGA (\({\rm{exact}}-\Delta {\rho }_{{\rm{GGA}}}\))

Full size image
Table 1 Comparison of the highest occupied Kohn–Sham eigenvalue (\({\epsilon }_{{\rm{H}}}\)) and the negative of the ionization potential (\({I}_{{\rm{p}}}\)) (all in Ha)
Full size table

Exact \({v}_{{\rm{xc}}}\) from CI densities for molecules

We now combine the above numerical strategies to evaluate the exact \({v}_{{\rm{xc}}}\) for four other benchmark systems—two stretched \({{\rm{H}}}_{2}\) molecules and two polyatomic systems (water and ortho-benzyne molecules). The CI calculations for all the molecules, excepting ortho-benzyne, are performed using the UGBS. For ortho-benzyne, we used the cc-PVTZ basis. Given the weak sensitivity of the inverted \({v}_{{\rm{xc}}}\) to the choice of xc functional used in \(\Delta \rho\), we employ \(\Delta {\rho }_{{\rm{LDA}}}\) for performing the cusp correction in all our calculations. Further, for all the systems, the inverse problem is deemed to have converged when \(| | {\rho }_{{\rm{data}}}-\rho | {| }_{{L}^{2}}< 1{0}^{-4}\). We remark that the \({L}^{2}\) error norm is a natural convergence criterion, given the form of the objective function in Eq. 1. However, given that previous works on this inverse problem have reported the \({L}^{1}\) error, we provide the same in the Supplementary Table 2, for all the benchmark systems considered. Figure 6 compares the \({v}_{{\rm{xc}}}\) for two stretched \({{\rm{H}}}_{2}\) molecules—\({{\rm{H}}}_{2}(2eq)\) (\({R}_{\text{H-H}}=2.83\) a.u., roughly twice the equilibrium bond length) and \({{\rm{H}}}_{2}(d)\) (\({R}_{\text{H-H}}=7.56\) a.u., at dissociation). We emphasize that these are prototypical systems where all existing xc approximations perform poorly, owing to their failure in handling strong correlations. We could successfully solve the inverse DFT problem for these systems (\(| | {\rho }_{{\rm{data}}}-\rho | {| }_{{L}^{2}} \sim 8\times 1{0}^{-5}\)), thereby suggesting that our approach works equally well for strongly correlated systems. As indicated in Table 1, we get remarkable agreement between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\) for \({{\rm{H}}}_{2}(2eq)\). However, for \({{\rm{H}}}_{2}(d)\), we obtain \({\epsilon }_{{\rm{H}}}\) within 19 mHa of \(-{I}_{{\rm{p}}}\). We attribute this larger difference between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\) (as compared with \({{\rm{H}}}_{2}(eq)\) and \({{\rm{H}}}_{2}(2eq)\)) to the use of \({v}_{{\rm{FA}}}\) as the boundary condition for \({v}_{{\rm{xc}}}\) in the low-density region. To elaborate, for a single-orbital system, \({v}_{{\rm{FA}}}\) is the exact \({v}_{{\rm{x}}}\) (exchange-only potential) and, hence, represents the exact \({v}_{{\rm{xc}}}\) in regions where the correlations are negligible. Although for the \({{\rm{H}}}_{2}(eq)\) and \({{\rm{H}}}_{2}(2eq)\) molecules the correlations are short-ranged, they are relatively longer-ranged for \({{\rm{H}}}_{2}(d)\). We highlight this in Fig. 7 by comparing the \({v}_{{\rm{xc}}}\) against \({v}_{{\rm{x}}}\) for \({{\rm{H}}}_{2}(eq)\), \({{\rm{H}}}_{2}(2eq)\), and \({{\rm{H}}}_{2}(d)\). As evident, \({{\rm{H}}}_{2}(d)\) has strong correlations extending to a significantly larger domain (in the far-field) in comparison with \({{\rm{H}}}_{2}(eq)\) and \({{\rm{H}}}_{2}(2eq)\). Thus, for \({{\rm{H}}}_{2}(d)\), the use of \({v}_{{\rm{FA}}}\) is warranted only in regions of much lower density (i.e., \({\rho }_{{\rm{data}}}\ll 1{0}^{-6}\)) than considered here. However, at such low densities, the wrong far-field asymptotics of a Gaussian basis-set density produces spurious oscillations in the far-field \({v}_{{\rm{xc}}}\). Thus, for the want of more accurate densities, we are restricted to using \({v}_{{\rm{FA}}}\) in regions where \({\rho }_{{\rm{data}}}< 1{0}^{-6}\), at the cost of incurring some error in \({\epsilon }_{{\rm{H}}}\).

Fig. 6
figure 6

Exact \({v}_{{\rm{xc}}}\) for stretched \({{\rm{H}}}_{2}\) molecules. The exact \({v}_{{\rm{xc}}}\) is provided for two stretched hydrogen molecules: one at twice the equilibrium bond length (\({{\rm{H}}}_{2}(2eq)\)) and the other at dissociation (\({{\rm{H}}}_{2}(d)\)). The H atoms for \({{\rm{H}}}_{2}(2eq)\) and \({{\rm{H}}}_{2}(d)\) are located at \(r=\pm 1.415\) a.u. and \(r=\pm 3.78\) a.u., respectively

Full size image
Fig. 7
figure 7

Nature and extent of electronic correlations in \({{\rm{H}}}_{2}\) molecules. A comparison of the exact exchange-correlation (\({v}_{{\rm{xc}}}\)) and the exchange-only (\({v}_{\text{x}}\)) potentials is provided for \({{\rm{H}}}_{2}\) molecules at three different bond lengths: (a) equilibrium bond length (\({{\rm{H}}}_{2}(eq)\)); (b) twice the equilibrium bond length (\({{\rm{H}}}_{2}(2eq)\)); (c) at dissociation (\({{\rm{H}}}_{2}(d)\)). The relative difference between \({v}_{{\rm{xc}}}\) and \({v}_{{\rm{x}}}\) indicates the nature and extent of electronic correlations. The correlations become stronger with bond stretching

Full size image

We now turn to a polyatomic system—the \({{\rm{H}}}_{2}{\rm{O}}\) molecule. Figure 8 compares the exact \({v}_{{\rm{xc}}}\) against \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\), on the plane of the \({{\rm{H}}}_{2}{\rm{O}}\) molecule. In particular, Fig. 8c provides the comparison along the O–H bond. For the exact \({v}_{{\rm{xc}}}\), we observe an atomic inter-shell structure—marked by a yellow ring around the O atom in Fig. 8b (as well as the local maxima and minima at around \(r=\pm 0.4\) a.u. in Fig. 8c). This atomic inter-shell structure is a distinctive feature of the exact \({v}_{{\rm{xc}}}\)39,40 and is absent in the standard xc approximations, as evident from \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\). Further, we observe a deeper potential around the O atom, as compared with \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\), thereby suggesting a higher electronegativity on the O atom than that predicted by LDA. Moreover, we observe a distinct local maximum at the H atom, as opposed to a local minimum in \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\). Lastly, as indicated in Table 1, we obtain striking agreement between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\) for this polyatomic system.

Fig. 8
figure 8

Comparison of exchange-correlation potentials (\({v}_{{\rm{xc}}}\)) for \({{\rm{H}}}_{2}{\rm{O}}\). a LDA-based exchange-correlation potential. b Exact exchange-correlation potential. c Comparison of the LDA-based and the exact exchange-correlation potential along the O–H bond. In a and b, the \({v}_{{\rm{xc}}}\) (in a.u.) is presented on the plane of the molecule. Refer to Supplementary Table 3 for the coordinates

Full size image

Finally, we evaluate the exact \({v}_{{\rm{xc}}}\) for the singlet state of the ortho-benzyne radical (\({{\rm{C}}}_{6}{{\rm{H}}}_{4}\))—a strongly correlated species that has previously served as a test for accurate wavefunction theories41. Figure 9 compares the exact \({v}_{{\rm{xc}}}\) against \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\), on the plane of the benzyne molecule. This example underscores the efficacy of our approach in handling both large and strongly correlated systems. As expected for the exact \({v}_{{\rm{xc}}}\), we observe an atomic inter-shell structure—marked by a yellow ring around the C atoms, which, on the other hand, are absent in the case of \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\). As is the case with \({{\rm{H}}}_{2}{\rm{O}}\), we observe a deeper potential around the C atom, as compared with \({v}_{{\rm{xc}}}^{{\rm{LDA}}}\), suggesting a higher electronegativity on the C atom than that predicted by LDA. Furthermore, as indicated in Table 1, we obtain remarkable agreement between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\).

Fig. 9
figure 9

Comparison of exchange-correlation potentials (\({v}_{{\rm{xc}}}\)) for \({{\rm{C}}}_{6}{{\rm{H}}}_{4}\). a LDA-based exchange-correlation potential. b Exact exchange-correlation potential. In both the cases, the \({v}_{{\rm{xc}}}\) (in a.u.) is presented on the plane of the molecule. Refer to Supplementary Table 3 for the coordinates

Full size image

Discussion

We have presented an accurate and robust method to evaluate the exact \({v}_{{\rm{xc}}}\), solely from the ground-state electron density. The key ingredients in our approach are—(a) the effective use of FE basis, which is a systematically convergent and complete basis, and, in turn, results in a well-posed inverse DFT problem; (b) the use of \(\Delta \rho\) correction and appropriate far-field boundary conditions to alleviate the unphysical artifacts associated with Gaussian basis-set densities. We emphasize that the proposed approach can easily drive the error in the target densities, i.e., \(| | {\rho }_{{\rm{data}}}-\rho | {| }_{{L}^{2}}\), to tight tolerances of \({\mathcal{O}}(1{0}^{-5}-1{0}^{-4})\)—which represents a stringent accuracy (see the Supplementary Discussion for a comparison with existing methods). Notably, as demonstrated through the 1,3-dimethylbenzene and the ortho-benzyne molecules, our approach can competently handle system sizes, which have, heretofore, remained challenging for other inverse DFT methods. Furthermore, for all the exact \({v}_{{\rm{xc}}}\)s obtained from CI densities, we obtain excellent agreement between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\) (excepting in the case of \({{\rm{H}}}_{2}(d)\)), further validating the accuracy and robustness of the approach. We remark that the larger discrepancy between \({\epsilon }_{{\rm{H}}}\) and \(-{I}_{{\rm{p}}}\) in the case of \({{\rm{H}}}_{2}(d)\) is a consequence of long-range (static) correlations in this system coupled with incorrect far-field asymptotics of Gaussian basis-set densities and can be remedied with the availability of more accurate densities. The ability to evaluate the exact xc potentials from ground-state electron densities, enabled by this method, will provide a powerful tool in the future testing and development of approximate xc functionals. Further, it paves the way for using machine learning to construct the functional dependence of \({v}_{{\rm{xc}}}\) on \(\rho\), i.e., \({v}_{{\rm{xc}}}[\rho ]\), providing another avenue to develop density functionals24,42,43 that can systematically improve both ground-state densities and energies44 as well as satisfy the known conditions on the exact functional45,46,47.

Methods

Discretization

We employ spectral FE basis to discretize all the spatial fields—\({v}_{{\rm{xc}}}\), \(\{{\psi }_{i}\}\), \(\{{p}_{i}\}\). The FE basis is constructed from piecewise polynomials on non-overlapping subdomains called elements. The basis, thus constructed, can be systematically improved to completeness by reducing the element size and/or increasing the polynomial order48. We remark that the spectral FE basis are not orthogonal and, hence, result in a generalized eigenvalue problem as opposed to the more desirable case of standard eigenvalue problem. To this end, we use special reduced-order quadrature (Gauss–Legendre–Lobatto quadrature rule) to render the overlap matrix diagonal and, thereby, trivially transform the generalized eigenvalue problem into a standard one. For all the \({{\rm{H}}}_{2}\) molecules, we used adaptively refined quadratic FEs to discretize the \(\{{\psi }_{i}\}\) and \(\{{p}_{i}\}\), whereas for all other systems we used adaptively refined fourth-order FEs. The \({v}_{{\rm{xc}}}\), in all the calculations, is discretized using linear FEs. Most importantly, the form of the FE basis is chosen carefully, so as to guarantee the cusp in \({\psi }_{i}\)s (and hence in \(\rho\)) at the nuclei, which in turn is critical to obtaining accurate \({v}_{{\rm{xc}}}\)s near the nuclei (refer to the Supplementary Note 1 for more information).

Solvers

In order to efficiently solve the Kohn–Sham eigenvalue problem in Eq. 5, we employ the Chebyshev polynomial-based filtering technique48,49,50. We remark that, compared with a forward ground-state DFT calculation, the inverse DFT calculation warrants much tighter accuracy in solving the Kohn–Sham eigenvalue equation(s). However, the use of a very high polynomial degree Chebyshev filter can generate an ill-conditioned subspace, akin to any power iteration-based eigen solver. To circumvent the ill conditioning and attain higher accuracy, we employ multiple passes of a low polynomial degree Chebyshev filter (polynomial order \(\sim 1000\)) and orthonormalize the Chebyshev-filtered vectors between two successive passes. The number of passes is determined adaptively so as to guarantee an accuracy of \(1{0}^{-9}\) in \(| | \hat{H}{\psi }_{i}-{\epsilon }_{i}{\psi }_{i}| {| }_{{L}^{2}}\).

The discrete adjoint function (\({p}_{i}\)) is solved by, first, projecting Eq. 7 onto a space orthogonal to the corresponding \({\psi }_{i}\) (or degenerate \({\psi }_{i}\)s) and then employing the conjugate-gradient method to compute the solution. The discrete adjoint problem is solved to an accuracy of \(1{0}^{-12}\) in \(| | (\hat{H}-{\epsilon }_{i}){p}_{i}-{g}_{i}| {| }_{{L}^{2}}\).

The update for \({v}_{{\rm{xc}}}\) is computed using limited-memory Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, a memory-efficient quasi-Newton solver, which constructs approximate Hessian matrices using the history of the gradients51. In all the calculations, we used a history of size \(100\) to construct the approximate Hessian. We refer to Supplementary Discussion for details on the rate of convergence and the factors affecting it.

Weights

To expedite the convergence of the nonlinear solver, we make use of two different weights, \(w({\bf{r}})=1\) and \(w({\bf{r}})=1/{\rho }_{{\rm{data}}}^{\alpha }\) (\(1\le \alpha \le 2\)), in sequence. The latter penalizes the objective function in the low-density region.

Ab initio densities

Accurate electron densities were generated using the incremental full CI (iFCI) method29 in the Q-Chem software package52. This method solves the electronic Schrödinger equation via a many-body expansion and asymptotically produces the exact electronic energy and density as the number of bodies in the expansion approaches the all-electron limit. For this study, electron densities were provided in the all-valence-electron limit of iFCI, i.e., where the full valence set is fully correlated and the core orbitals of \({{\rm{H}}}_{2}{\rm{O}}\) and \({{\rm{C}}}_{6}{{\rm{H}}}_{4}\) are treated as uncorrelated electron pairs. Reference ionization energies were obtained at the same level of theory, for each system with one less electron.