Quantum spin-valley Hall effect in AB-stacked bilayer silicene

Our density functional theory calculations show that while AB-stacked bilayer silicene has a non-quantized spin-valley Chern number, there exist backscattering-free gapless edge states within the bulk gap, leading to a quantum spin-valley Hall effect. Using a tight-binding model for a honeycomb bilayer, we found that the interlayer potential difference and the staggered AB-sublattice potential lead to abrupt and gradual change of the valley Chern number from a quantized value to zero, respectively, while maintaining backscattering-free gapless edge states if the valley Chern number is not too close to zero. Under an inversion symmetry-breaking potential in the form of the staggered AB-sublattice potential, such as an antiferromagnetic order and a hexagonal diatomic sheet, a finite but non-quantized (spin-)valley Chern number can correspond to a quantum (spin-)valley Hall insulator.

In this work, our density functional theory (DFT) calculations show that while the intralayer ferrimagnetic and interlayer antiferromagnetic order of AB-stacked bilayer silicene gives a non-quantized spin-valley Chern number, there exist backscattering-free gapless edge states corresponding to a quantum spin-valley Hall effect. Using a tight-binding (TB) model for a honeycomb bilayer, we found that the valley mixing occurs abruptly as the interlayer potential difference increases above a threshold, but occurs gradually as the staggered AB-sublattice potential increases. The interlayer potential difference induces an abrupt change of the valley Chern number from a quantized value to zero. The staggered AB-sublattice potential induces a gradual decrease of the valley Chern number from a quantized value, while maintaining backscattering-free gapless edge states corresponding to a quantum valley Hall effect. As a result, under an inversion symmetry-breaking potential in the form of the staggered AB-sublattice potential, such as an antiferromagnetic order and a hexagonal diatomic sheet, a finite but non-quantized (spin-)valley Chern number can correspond to a quantum (spin-)valley Hall insulator. Figure 1(a) shows the spin density of AB-stacked bilayer silicene sheet. The spin density is mostly localized on the nondimer sites with intralayer ferrimagnetic and interlayer antiferromagnetic order as previously reported 14 . Figure 1(b) shows the band structure of AB-stacked bilayer silicene, where the energy bands with opposite spins were completely overlapped. The band structure indicates that the AB-stacked bilayer silicene is an antiferromagnetic insulator with an indirect band gap ~0.36 eV, consistently with a previous work 14 . Figure 1(c,d) show the Berry curvature maps for opposite spins. The Berry curvature shows opposite signs at opposite valleys and for opposite spins, indicating a possible spin-valley Hall effect. From the spin-valley-resolved Chern number ξ C s ( , ) ~sign(sξ)0.67, four types of the Chern number can be defined 4,18 . The Chern numbers were zero except for the spin-valley Chern number C sv ~ 2.68, which is not quantized but still indicates a spin-valley Hall effect.
The Berry curvature shows broad peaks of small height. The Berry curvature peak pertaining to each valley can be partially mixed, giving the non-quantized spin-valley Chern number. A small inversion-symmetry breaking potential ensures the sharply peaked Berry curvature 20,23 , and the staggered magnetization in the AB-stacked bilayer silicene decreases with increasing interlayer distance 24 . Figure 1(e,f) show the band gap and the spin-valley Chern number as a function of The sublattice magnetization on the nondimer sites, obtained from the Mulliken population analysis, is 0.3 μ B at equilibrium. As the interlayer distance increases, the sublattice magnetization gradually decreases to zero, becoming nonmagnetic when Δd~35%. As the interlayer distance increases, the antiferromagnetic band gap decreases due to the reduction of staggered magnetization, and the spin-valley Chern number gradually approaches C sv~4 , confirming that the non-quantized spin-valley Chern number is due to a large staggered magnetization. Next, we have investigated AB-stacked bilayer silicene nanoribbons with zigzag edges in order to confirm the corresponding spin-valley Hall effect. The ribbon width is represented by the number N of the C-C pairs on a layer in the unit cell. Figure 2 shows the band structures of the zigzag-edge nanoribbons with N = 16 near the Fermi level E F , where the energy bands with opposite spins were completely overlapped. As shown in Fig. 2(a), nanoribbons with bare edges have gapless edge states within the bulk gap, indicating that bilayer silicene can be a topologically nontrivial insulator while the spin-valley Chern number is not quantized. As can be seen in Fig. 2(b-d), there are gapless edge states when the dimer edge sites are terminated by hydrogen, but the gapless edge states disappear when the nondimer edge sites are terminated with hydrogen.
The gapless edge states in Fig. 2(a,b) are not connected with valence and conduction bands, which can be attributed to edge potentials. Nanoribbons inevitably have edge potentials, which can be induced by dangling σ-bonds, functional groups passivating the dangling bonds and edge-localized magnetic moments [25][26][27] . In a quantum valley Hall insulator, the dispersion of the edge state band is sensitive to edge potentials, varying from gapped flat-band to gapless chiral modes, because the edge potentials can introduce bearded-edge-like properties on the zigzag-edges 20 .
To check whether the gapless edge states correspond to any quantum Hall effect, we have investigated the square of the wavefunction |Ψ| 2 near E F . Figure 3(a) shows |Ψ| 2 at E F in the nanoribbons with bare edges, where the gapless edge states are confined on an edge, possibly indicating a quantum Hall effect. Figure 3(b) shows the schematics for the propagation of the gapless edge states constructed from |Ψ| 2 at E F and the band velocity. Since each state contributes e h 2 to the conductivity, the Hall conductivity can be estimated in units of e h 2 by counting the number of states propagating in a given direction. In Fig. 3(b), we can obtain a spin-valley Hall conductivity σ sv = 4 e h 2 and can confirm that backscattering of the gapless edge states requires inversion of the spin-valley index sξ. Time-reversal symmetry for a fixed valley and inter-valley separation for a fixed spin protects the gapless edge states from backscattering. As a result, the AB-stacked bilayer silicene is believed to be a quantum spin-valley Hall insulator, while the spin-valley Chern number is not quantized. In zigzag-edge nanoribbons with N = 32, we obtained the same results.
The gapless edge states with σ sv = 4 e h 2 correspond to C sv = 4, which is consistent with the spin-valley Chern number obtained at the limits of small staggered magnetization shown in Fig. 1

(f). For a quantum spin-valley
Hall insulator, the bulk-edge correspondence is known to be exactly established only at a topological domain wall 4,18 . At a vacuum interface, the number of gapless edge states depends on the edge condition and may not correspond to the spin-valley Chern number. Nevertheless, the gapless edge states in Figs. 2 and 3 show that the AB-stacked bilayer silicene is a quantum spin-valley Hall insulator while the spin-valley Chern number is not quantized.
The (spin-)valley Chern number and the bulk-edge correspondence are based on the assumption that opposite valleys are well separated [19][20][21][22][23] . At an armchair edge, where opposite valleys are completely mixed, the bulk-edge correspondence has no meaning. For a large inversion symmetry-breaking potential, the Berry curvature still centered on each valley may be broadened and partially mixed with each other, and thus the (spin-)valley Chern number obtained by integrating the Berry curvature around each valley may deviate from the quantized value.
To clarify the effect of partial valley mixing, the TB model of Eq. (1) for the AB-stacked honeycomb bilayer was investigated (see the Methods section). Figure 4(a) shows the absolute value of spin-valley-resolved Chern number ξ | | C s ( , ) as a function of the inversion symmetry-breaking potentials. We can consider two types of potential, interlayer potential difference 2V and staggered AB-sublattice potential Δ. For a small enough V and Δ, ( , ) abruptly changes from 1 to 0 as V increases to t 0 , ξ | | C s ( , ) gradually decreases as Δ increases. In the intralayer ferrimagnetic and the interlayer antiferromagnetic order, the staggered magnetization M i corresponds to a combination of a interlayer potential difference and a staggered AB-sublattice potential, with opposite signs for opposite spins. As M increases, ξ | | C s ( , ) gradually decreases, too. The abrupt and gradual change of the spin-valley-resolved Chern number can be attributed to the Berry curvature. , where d (d 0 ) is the strained (equilibrium) interlayer distance. Figure 4(b,c) show the Berry curvature maps for V = 0.9 and Δ = 0.9, respectively. For a small V and Δ, the Berry curvature is sharply peaked at each valley. The increase in V shifts the Berry curvature peak away from the valley while maintaining its sharp shape as shown in Fig. 4(b), and the mixing of the Berry curvature peak pertaining to each valley abruptly occurs when the Berry curvature peak reaches the Brillouin zone border. As shown in Fig. 4(c), the increase in Δ broadens and lowers the Berry curvature peak while maintaining its center at each valley, and the mixing of the tails of the Berry curvature peak pertaining to each valley gradually increases with Δ. www.nature.com/scientificreports www.nature.com/scientificreports/ Under a staggered AB-sublattice potential Δ, the Berry curvature in a honeycomb monolayer was reported to be ξ Ω = Δ Δ + q q ( ) 3 /2( 3 ) 2 23/2 for small q, where q is the wavevector measured from the K point in the Brillouin zone 20 . As Δ increases, the Berry curvature peak broadens while maintaining its center on each valley, consistently with our results. Under an interlayer potential difference 2V, the Berry curvature in a honeycomb bilayer was reported to be ξ Ω = − + q tVq q t V ( ) 2 /( ) 1 2 4 1 2 2 3/2 for small q 23 . As V increases, the Berry curvature peak moves away from the valley, consistently with our results.
Regardless of the abrupt and gradual change of the spin-valley-resolved Chern number, the band gap increases as the inversion symmetry-breaking potential increases. It is clear that ξ | | = C s ( , ) 1 corresponds to a quantum (spin-)valley Hall insulator with gapless edge states at a topological domain wall, and ξ | | = C s ( , ) 0 corresponds to a topologically trivial insulator. To clarify whether a finite non-quantized ξ | | C s ( , ) due to the partial valley mixing corresponds to a quantum (spin-)valley Hall effect, we investigated N = 80 zigzag-edge honeycomb bilayer nanoribbons with a kink-type topological domain wall. Figure 5(a,c,e) show the band structure, |Ψ| at E = 0 and a schematic for the propagating states at E = 0, respectively, for Δ = ±0.5 in each half of the nanoribbon. Figure 5(b,d,f) show the band structure, |Ψ| at E = 0 and a schematic for the propagating states at E = 0, respectively, for M = ±1 in each half of the nanoribbon. In both cases, ξ ∼ ± .  In the quantum (spin-)valley Hall effect with large staggered AB-sublattice potentials, the (spin-)valley Chern number is not well defined due to the valley mixing, but the gapless edge states still correspond to the (spin-)valley Chern number obtained at small potential limits. Even in the presence of spin mixing, the spin Chern number could be well defined by partitioning the occupied valence band into two spin sectors using the projected spin operator 28,29 . In a similar way, it is expected that the (spin-)valley Chern number can be well defined even in the presence of valley mixing, which requires further study.
To summarize, we have investigated AB-stacked bilayer silicene, known to be an antiferromagnetic insulator, by using the density functional theory calculations. While the spin-valley Chern number is not quantized, there exist backscattering-free gapless edge states within the bulk gap, indicating that the bilayer silicene is a quantum spin-valley Hall insulator. Using a tight-binding model for a honeycomb bilayer, we found that a non-quantized (spin-)valley Chern number may correspond to a quantum (spin-)valley Hall insulator under an inversion symmetry-breaking potential in the form of staggered AB-sublattice potential such as an antiferromagnetic order.

Methods
A SIESTA package 30 using a localized linear combination of numerical atomic-orbital basis sets was employed for the DFT calculations. A generalized gradient approximation of Perdew-Burke-Ernzerhof (GGA-PBE) was used for the exchange and correlation potential 31 . A double-ζ polarized basis set and norm-conserving Troullier-Martins pseudopotentials generated with a Perdew-Burke-Ernzerhof functional were used. The plane-wave cutoff energy of 350 Ry was used for the real-space grid, and k-points of 100 × 100 × 1 and 40 × 1 × 1 meshes in a Monkhorst-Pack scheme were used for bilayer silicene sheet and nanoribbons, respectively. The atomic coordinates were optimized by using the conjugated gradients method with a maximum force tolerance of 0.1 eV/nm. The equilibrium lattice constant a 0 = 0.386 nm and the interlayer distance d 0 = 0.251 nm between the covalently-bonded dimer sites were obtained from the total energy minimum, in agreement with a previous work 14 . Zigzag-edge nanoribbons were considered as a one-dimensional system periodic in a zigzag direction. The ribbon width is represented by the number N of the C-C pairs on a layer in the unit cell. The spin-orbit coupling, much smaller than the antiferromagnetic gap, was neglected.
Using the wavefunctions obtained from the DFT calculations on the bilayer silicene sheet, maximally localized Wannier functions were constructed within the Wannier90 code 32 . Berry curvatures were calculated based on the