Quadrupole topological insulators in Ta2M3Te5 (M = Ni, Pd) monolayers

Higher-order topological insulators have been introduced in the precursory Benalcazar-Bernevig-Hughes quadrupole model, but no electronic compound has been proposed to be a quadrupole topological insulator (QTI) yet. In this work, we predict that Ta2M3Te5 (M = Pd, Ni) monolayers can be 2D QTIs with second-order topology due to the double-band inversion. A time-reversal-invariant system with two mirror reflections (Mx and My) can be classified by Stiefel-Whitney numbers (w1, w2) due to the combined symmetry TC2z. Using the Wilson loop method, we compute w1 = 0 and w2 = 1 for Ta2Ni3Te5, indicating a QTI with qxy = e/2. Thus, gapped edge states and localized corner states are obtained. By analyzing atomic band representations, we demonstrate that its unconventional nature with an essential band representation at an empty site, i.e., Ag@4e, is due to the remarkable double-band inversion on Y–Γ. Then, we construct an eight-band quadrupole model with Mx and My successfully for electronic materials. These transition-metal compounds of A2M1,3X5 (A = Ta, Nb; M = Pd, Ni; X = Se, Te) family provide a good platform for realizing the QTI and exploring the interplay between topology and interactions.


INTRODUCTION
In higher-order topological insulators, the ingap states can be found in (d−n)-dimensional edges (n > 1), such as the corner states of two-dimensional (2D) systems or the hinge states of three-dimensional systems [1][2][3][4][5][6][7][8][9][10][11][12] . Different from topological insulators with (d−1)-dimensional edge states, the Chern numbers or Z 2 numbers in higher-order topological insulators are zero. The higher-order topology can be captured by topological quantum chemistry [13][14][15][16][17][18] , nested Wilson-loop method 1,8 and second Stiefel-Whitney (SW) class [19][20][21][22][23][24][25] . Using topological quantum chemistry 13 , the higher-order topological insulator can be diagnosed by the decomposition of atomic band representations (aBRs) as an unconventional insulator (or obstructed atomic insulator) with mismatching of electronic charge centers and atomic positions [16][17][18]26 . In contrast to dipoles (Berry phase) for topological insulators, the higher-order topological insulators can be understood by multipole moments 1 . In a 2D system, the second-order topology corresponds to the quadrupole moment, which can be diagnosed by the nested Wilson-loop method 1,8,23 . When the system contains space-time inversion symmetries, such as PT and C 2z T, where the P and T represent inversion and timereversal symmetries, the second-order topology can be described by the second SW number (w 2 ) 23,27 . The second SW number w 2 is a well-defined 2D topological invariant of an insulator only when the first SW number w 1 = 0. Usually, a 2D quadrupole topological insulator (QTI) with w 1 = 0, w 2 = 1 has gapped edge states and degenerate localized corner states, which are pinned at zero energy (being topological) in the presence of chiral symmetry. When the degenerate corner states are in the energy gap of bulk and edge states, the fractional corner charge can be maintained due to filling anomaly 28 .
So far, various 2D systems are proposed to be SW insulators with second-order topology, such as monolayer graphdiyne 29,30 , liganded Xenes 25,31 , β-Sb monolayer 18 and Bi/EuO 32 . However, no compound has been proposed to be a QTI with M x and M y symmetries. After considering many-body interactions in transition-metal compounds, superconductivity, exciton condensation and Luttinger liquid could emerge in a transition-metal QTI. In recent years, van der Waals layered materials of A 2 M 1,3 X 5 (A = Ta, Nb; M = Pd, Ni; X = Se, Te) family have attracted attentions because of their special properties, such as quantum spin Hall effect in Ta 2 Pd 3 Te 5 monolayer 33,34 , excitons in Ta 2 NiSe 5 35-37 , and superconductivity in Nb 2 Pd 3 Te 5 and doped Ta 2 Pd 3 Te 5 38 . In particular, the monolayers of A 2 M 1,3 X 5 family can be exfoliated easily, serving as a good platform for studying topology and interactions in lower dimensions.
In this work, we predict that based on first-principles calculations, Ta 2 Ni 3 Te 5 monolayer is a 2D QTI. Using the Wilson-loop method, we show that its SW numbers are w 1 = 0 and w 2 = 1, corresponding to the second-order topology. We also solve the aBR decomposition for Ta 2 Ni 3 Te 5 monolayer, and find that it is unconventional with an essential band representation (BR) at an empty Wykoff position (WKP), A g @4c, which origins from the remarkable double-band inversion on Y-Γ line. To verify the QTI phase, we compute the energy spectrum of Ta 2 Ni 3 Te 5 monolayer with open boundary conditions in both x and y directions and obtain four degenerate corner states. Then, we construct an eightband quadrupole model with M x and M y successfully. The doubleband-inversion picture widely happens in the band structures of A 2 M 1,3 X 5 family. The Ta 2 M 3 Te 5 monolayers are 2D QTI candidates for experimental realization in electronic systems.

Band structures
The band structure of Ta 2 Ni 3 Te 5 monolayer suggests that it is an insulator with a band gap of 65 meV. We have checked that spin-orbit coupling (SOC) has little effect on the band structure ( Supplementary Fig. 2b). We also checked the band structures using GW method and SCAN method, and we find that the band gap remains using these methods (the corresponding band structures are shown in Supplementary Note 1). As shown in the orbital-resolved band structures of Fig. 3a-c, although low-energy bands near the Fermi level (E F ) are mainly contributed by Ta-d z 2 orbitals (two conduction bands) and Ni A -d xz orbitals (two valence bands), the inverted bands of {Y2; GM1−, GM3−} come from Te-p x orbitals. The irreducible representations (irreps) 39 at Y and Γ are denoted for the inverted bands in Fig. 1b. We notice that the double-band inversion between {Y2; GM1−, GM3−} bands and {Y1; GM1+, GM3+} bands is remarkable, about 1 eV.

Atomic band representations
To analyze the band topology, the decomposition of aBR is performed. In a unit cell of Ta 2 Ni 3 Te 5 monolayer in Fig. 1a, four Ta atoms, four Ni B atoms and eight Te atoms are located at different 4e WKPs. The rest two Te atoms and two Ni A atoms are located at 2a and 2b WKPs, respectively. The aBRs are obtained from the crystal structure by pos2aBR [16][17][18] , and irreps of occupied states are calculated by IRVSP 39 at high-symmetry kpoints. Then, the aBR decomposition is solved online-http:// tm.iphy.ac.cn/UnconvMat.html. The results are listed in Supplementary Table 2 of Supplementary Note 2. Instead of being a sum of aBRs, we find that the aBR decomposition of the occupied bands has to include an essential BR at an empty WKP, i.e., A g @4c. As illustrated in Fig. 1a, the charge centers of the essential BR are located at the middle of Ni B -Ni B bonds (i.e., the 4c WKP), indicating that the Ta 2 Ni 3 Te 5 monolayer is a 2D unconventional insulator with second-order topology.

Double-band inversion
In an ideal atomic limit, Te-p orbitals and Ni-d orbitals are occupied, while Ta-d orbitals are fully unoccupied. Thus, all the occupied bands are supposed to be the aBRs of Te-p and Ni-d orbitals, as shown in the left panel of Fig. 2a. However, in the monolayers of A 2 M 1,3 X 5 family (see their band structures in Supplementary Note 1), a double-band inversion happens between the occupied aBR A″@4e (Te-p x and Ni-d) and unoccupied aBR A 0 @4e (Ta-d z 2 ), as shown in the right two panels of  Fig. 2c, the system becomes a 2D QTI for Ta 2 Ni 3 Te 5 monolayer (235-type), resulting in the essential BR of A g @4c.
Second Stiefel-Whitney class w 2 = 1 To identify the second-order topology of the monolayers, we compute the second SW number by the Wilson-loop method. The first SW class (w 1 ) is  Fig. 1c as a function of k x . The results show that the first SW class is w 1 = 0. In addition, there is one crossing of Wilson bands at θ = π [ Fig. 1d], indicating the second SW class w 2 = 1. The quadruple moment q xy = e/2 calculated by the nested Wilson-loop method in Supplementary Note 3. Therefore, the Ta 2 Ni 3 Te 5 monolayer is a QTI with a nontrivial second SW number.

Edge spectrum and corner states
From the orbital-resolved band structures (Fig. 3), the maximally localized Wannier functions of Ta-d z 2 , Ni A -d xz and Te-p x orbitals are extracted, to construct a 2D tight-binding (TB) model of Ta 2 Ni 3 Te 5 monolayer. As shown in Fig. 3d, the obtained TB model fits the density functional theory (DFT) band structure well. First, we compute the (01)-edge spectrum with open boundary condition along y. Instead of gapless edge states for a 2D Z 2 -nontrivial insulator, gapped edge states are obtained for the 2D QTI [ Fig. 3e]. Then, we explore corner states as the hallmark of the 2D QTI. We compute the energy spectrum for a nanodisk. For concreteness, we take a rectangular-shaped nanodisk with 50 × 10 unit cells, preserving both M x and M y symmetries in the 0D geometry. The obtained discrete spectrum for this nanodisk is plotted in the inset of Fig. 3f. Remarkably, one observes four degenerate states near E F . The spatial distribution of these four-fold modes can be visualized from their charge distribution, as shown in Fig. 3f. Clearly, they are well localized at the four corners, corresponding to isolated corner states.
Minimum model for the 2D QTI As shown in Fig. 2, the minimum model for the 2D QTI should be consisted of two BRs of A 0 @4e and A″@4e. Based on the situation of Ta 2 Ni 3 Te 5 monolayer in Fig. 2c, the minimum effective model is derived as below: The terms of H Ta (k), H Ni (k) and H int (k) are 4 × 4 matrices, which read The γ 1,2,3 (k) matrices are given explicitly in Supplementary Note 4. We find that t s1 < 0 and t p1 , t p2 > 0 for the A 2 M 1,3 X 5 family (t s3 and t p3 are small). When ε s + 2t s1 − 2|t s2 | < ε p + 2t p1 + 2t p2 , the double-band inversion happens in the monolayers of this family. By fitting the DFT bands, we obtain t s2 > 0 for the 215-type, while t s2 < 0 for the 235-type. When ε p = − ε s and t pi = − t si (i = 1, 2, 3), the model is chiral symmetric (i.e., δ = 0 in Table 1). Since the second SW insulator or QTI is topological in the presence of chiral symmetry, we would focus on the model (almost respecting chiral symmetry) in the following discussion.
Analytic solution of (01)-edge states As the remnants of the QTI phase, the localized edge states can be solved analytically for the minimum model. For the (01)-edge, one can treat the model H TB (k) as two parts, H 0 (k) and H 0 ðkÞ, Note that there is a pair of Dirac points ð ± k D x ; 0Þ, with Since k x is still a good quantum number on the (01)-edge, expanding k y to the second order, the zeromode equation H 0 (k x , − i∂ y )Ψ(k x , y) = 0 can be solved for y ∈ [0, +∞). Taking the trial solution of Ψ(k x , y) = ψ(k x )e λy , we obtain the secular equation and the solution of λ = ±λ ± , where With the boundary conditions Ψ(k x , 0) = Ψ(k x , +∞) = 0, only − λ ± are permitted.
In the k x regime of Àk D x ; k D x Â Ã , the edge zero-mode states are Ψ(k x , y) = [C 1 (k x )ϕ 1 (k x ) + C 2 (k x )ϕ 2 (k x )]ðe Àλþy À e ÀλÀy Þ with The edge zero states are Fermi arcs that linking the pair of projected Dirac points ð± k D x ; 0Þ. Once H 0 ðkÞ included, the effective (01)-edge Hamiltonian is, where Φ j i ϕ 1 ðk x Þ; ϕ 2 ðk x Þ j i . Two edge spectra are obtained in Fig. 4a.
Effective Su-Schrieffer-Heeger model on (10)-edge and corner states Similarly, we derive the (10)-edge modes as ½F 1 ðk y Þφ 1 ðk y Þ þ F 2 ðk y Þφ 2 ðk y Þ e ÀΛþx À e ÀΛÀx ð Þ with Here, Then we obtain the effective Hamiltonian on (10) edge below,  When δ = 0, the minimum QTI model is chiral symmetric and it is gapless on the (10) edge (preserving M y symmetry). When the chiral symmetry is slightly broken (δ ≠ 0), the H eff 10 becomes an Su-Schrieffer-Heeger model (δ < 0 nontrivial; δ > 0 trivial), as presented in Fig. 4b-d. As a result, we obtain a solution state on the end of the edge mode, i.e., the corner. As long as the energy of the corner state is located in the gap of bulk and edge states, the corner state is well localized at the corners, as shown in Fig. 4c, d.

DISCUSSION
In Ta 2 NiSe 5 monolayer, the double-band inversion has also happened between Ta-d z 2 and Te-p x states, about 0.4 eV, resulting in a semimetal with a pair of nodal lines in the 215-type. The highest valence bands on Y-Γ are from the inverted Ta-d z 2 states. However, in Ta 2 Ni 3 Te 5 monolayer, the double-band inversion strength becomes remarkable,~1 eV, which is ascribed to the filled B-type voids and more extended Te-p states. On the other hand, the highest valence bands become Ni A -d xz states (slightly hybridized with Te-p x states). It is insulating with a small gap of 65 meV. When it comes to A 2 Pd 3 Te 5 monolayer, the remarkable inversion strength is similar to that of Ta 2 Ni 3 Te 5 . But the Pd A -d xz states go upwards further due to more expansion of the d orbitals and the energy gap becomes almost zero in Ta 2 Pd 3 Te 5 . In short, the double-band inversion happens in all these monolayers, while the band gap of the 235-type changes from positive (Ta 2 Ni 3 Te 5 ), to nearly zero (Ta 2 Pd 3 Te 5 with a tiny band overlap), to negative (Nb 2 Pd 3 Te 5 ), as shown in Supplementary Fig. 1. Although the band structure of Ta 2 Pd 3 Te 5 bulk is metallic without SOC 33,34,38 , the monolayer could become a QSH insulator upon including SOC in ref. 33 . Since their bulk materials are van der Waals layered compounds, the bulk topology and properties strongly rely on the band structures of the monolayers in the A 2 M 1,3 X 5 family.
As we find in ref. 33 , the band topology of Ta 2 Pd 3 Te 5 monolayer is lattice sensitive. By applying >1% uniaxial compressive strain along b, it becomes a Z 2 -trivial insulator, being a QTI. On the other hand, due to the quasi-1D crystal structure, the screening effect of carriers is relatively weak and the electron-hole Coulomb interaction may be substantial for exciton condensation. The 1D in-gap edge states as remnants of the QTI are responsible for the observed Luttinger-liquid behavior.
In conclusion, we predict that Ta 2 M 3 Te 5 monolayers can be QTIs by solving aBR decomposition and computing SW numbers. Through aBR analysis, we conclude that the second-order topology comes from an essential BR at the empty site (A g @4c), and it origins from the remarkable double-band inversion. The double-band inversion also happens in the band structure of Ta 2 NiSe 5 monolayer. The second SW number of Ta 2 Ni 3 Te 5 monolayer is w 2 = 1, corresponding to a QTI. Therefore, we obtain edge states and corner states of the monolayer. The eight-band quadrupole model with M x and M y has been constructed successfully for electronic materials. With the large double-band inversion and small band energy gap/overlap, these transition-metal materials of A 2 M 1,3 X 5 family provide a good platform to study the interplay between the topology and interactions (Fig. 4e).

Calculation method
Our first-principles calculations were performed within the framework of the DFT using the projector augmented wave method 40,41 , as implemented in Vienna ab-initio simulation package (VASP) 42,43 . The Perdew-Burke-Ernzerhof (PBE) generalized gradient approximation exchange-correlations functional 44 was used. SOC was neglected in the calculations except in Supplementary Fig. 2b. We also used SCAN 45 and GW 46 method when checking the band gap. In the self-consistent process, 16 × 4 × 1 k-point sampling grids were used, and the cut-off energy for plane wave expansion was 500 eV. The irreps were obtained by the program IRVSP 39 . The maximally localized Wannier functions were constructed by using the Wannier90 package 47 . The edge spectra are calculated using surface Green's function of semi-infinite system 48,49 .

DATA AVAILABILITY
The data sets that support the findings in this study are available from the corresponding author upon request.

CODE AVAILABILITY
The codes that support the findings in this study are available from the corresponding author upon request.