Gapless edge states localized to odd/even layers of AA′-stacked honeycomb multilayers with staggered AB-sublattice potentials

In honeycomb multilayers with staggered AB-sublattice potentials, we predict gapless edge states localized to either of the odd and the even layers for the AA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\prime }$$\end{document}′ stacking order in which the sublattice-pseudospin polarizations of adjacent layers are antiparallel. Gaps in the projected layer-pseudospin spectrum suppress interlayer hopping between odd and even layers. The layer-valley Chern number corresponding to the edge states was obtained by decomposing the occupied state into two layer-pseudospin sectors by using a projected layer-pseudospin operator. For the AB\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\prime }$$\end{document}′ stacking, the sublattice-pseudospin polarizations of adjacent layers are antiparallel, but the layer-pseudospin spectrum gap closes at the interface of the topologically different states, leading to gapped edge states. For the AA and AB stackings where the sublattice-pseudospin polarizations of the adjacent layers are parallel, the gapless edge states corresponding to quantum valley Hall states are evenly distributed across the layers.

the AA and AB stackings, sublattice-pseudospin polarizations of the two layers are parallel and the inversion symmetry is broken.In the AA ′ and AB ′ stackings, the sublattice-pseudospin polarizations of the two layers are antiparallel and the inversion symmetry is preserved.
Coupling between different degrees of freedom leads to the expectation of new topological states related to the combined degrees of freedom 16 .Most topological states are characterized by Chern numbers.According to Prodan's approach, a (pseudo)spin Chern number can be defined if the eigenvalue spectrum of the projected (pseudo)spin operator consists of two isolated sectors with nonzero Chern numbers 19 .Spin Chern insulators can be characterized by the spin Chern number obtained by decomposing the occupied state into two spin sectors using the projected spin operator 20 .Since the occupied state of a honeycomb bilayer can accommodate two pseudospin states 21 , pseudospin Chern numbers related to sublattice-or layer-pseudospins are expected to be obtained following Prodan's approach.Bilayer configurations generated by folding monolayers with domain walls were used to predict valley Chern states 22 , and bilayers with opposite Rashba-type spin-orbit coupling in adjacent layers have been proposed as building blocks for constructing three-dimensional topological insulators 23 .
In this work, we show that interlayer hopping of gapless edge states can be suppressed by the gap in the projected layer-pseudospin spectrum, leading to gapless edge states localized to either of the odd and the even layers.The layer-valley Chern number corresponding to the edge states was obtained by decomposing the occupied state into two layer-pseudospin sectors.The layer-valley Chern number is a topological invariant unless the band gap or the layer-pseudopsin spectrum gap is closed.At the interface of topologically different states, either  the band gap or the layer-pseudospin spectrum gap needs to be closed.We found that in the AA ′ stacking, the layer-pseudospin spectrum has a gap at the interface, suppressing the interlayer hopping of the edge states, and there exist gapless edge states localized to the odd and the even layers, respectively.On the other hand, in the AB ′ stacking, the layer-pseudospin spectrum has no gap at the interface, allowing the interlayer hopping of the edge states, and there exist gapped edge states.In the AA and AB stackings, gapless edge states corresponding to quantum valley Hall states are evenly distributed across the layers.

Results and discussion
First, we consider noninteracting honeycomb bilayers with staggered AB-sublattice potentials.A bilayer with no interlayer interactions can be considered a bilayer in which the separation between the layers is large enough so that the wavefunctions do not overlap.The valley Chern number of the bilayer with no interlayer interactions is given by the sum of the valley Chern numbers of the individual layers, C v = N=2 i=1 τ i , where τ i corresponds to the orientation of the sublattice-pseudospin polarization of the i-th layer.For the parallel sublattice-pseudospin polarizations where τ i = τ , C v =± 2. For the antiparallel sublattice-pseudospin polarizations where τ i = (−1) i τ , C v =0.Since the quantum valley Hall state can only occur when the inversion symmetry is broken, C v must be zero in the inversion-symmetric bilayers.However, in the absence of interlayer interactions, the individual layers of the inversion-symmetric bilayer is still quantum valley Hall insulators with gapless edge states.Although the valley Chern number is zero, the gapless edge states indicate that the inversion-symmetric bilayer is still topologically nontrivial.
A non-zero Chern number corresponding to the gapless edge states of the inversion-symmetric bilayer can be defined by subtracting the valley Chern numbers of the two layers, C pv = N=2 i=1 (−1) i τ i , which can be referred to as the layer-valley Chern number, just like the spin-valley Chern number 16 .Thus, C pv =0 for the inversion symmetry-broken bilayer and C pv =± 2 for the inversion-symmetric bilayer.The gapless edge states corresponding to the layer-valley Chern number can be described by the coupled layer-valley degree of freedom ξµ defined as the product of the valley index ξ = ±1 and the layer-pseudospin index µ = ±1 , similarly to the coupled spin- valley degree of freedom 24 .If the layer-valley Chern number is not zero, the coupled layer-valley index is locked to the propagation direction of the edge state.In the inversion-symmetric bilayer, backscattering of the edge states requires an inversion of the coupled layer-valley index ξµ .For example, if the valley index ξ is fixed as shown in Fig. 1b, backscattering of the edge states requires an inversion of µ , which corresponds to the interlayer hopping and is not allowed in the noninteracting bilayer.
Moreover, the definition of C pv is independent of an inversion symmetry and can be extended to an arbitrary number N of layers.For honeycomb multilayers where τ i = τ , C v =±N and C pv =±N mod 2. For honeycomb multilayers where τ i = (−1) i τ , C v =±N mod 2, and C pv =±N .If the number of layers is even, only one of the two Chern numbers has a nonzero value.If the number of layers is odd, interestingly, both Chern numbers have nonzero values.For example, in all types of trilayers, we can expect a gapless edge state localized to the layer.Of course, this prediction is trivial for noninteracting multilayers where all the gapless edge states are already localized to individual layers.For interacting mulatilayers, this prediction should be verified.
In the presence of interlayer interactions, the superposition of edge states propagating in opposite directions can lead to gapped edge states in the inversion-symmetric bilayer.Below, however, we show that interlayer hopping of the edge states can be suppressed by the gap in the projected layer-pseudospin spectrum by using a tight binding (TB) model.And we show that the layer-valley Chern numbers obtained by decomposing the occupied state into two layer-pseudospin sectors by using the projected layer-pseudospin operator and subtracting the valley Chern numbers of the two sectors agree with those predicted in the noninteracting case.
Figure 3 shows the TB model calculations for honeycomb bilayers for = 0.2t o (see "Methods" section for calculation details).The first and second columns correspond to the AB and AA ′ stackings, respectively.As shown in Fig. 3a,e, both the AB and AA ′ stackings have band gaps at the K point of the Brillouin zone.Inversion symmetry is broken in the AB stacking and preserved in the AA ′ stacking.As shown in Fig. 3b,f, the Berry curvature has peaks of opposite signs at the opposite valleys in the AB stacking, while the Berry curvature is zero over the entire Brillouin zone in the AA ′ stacking.Therefore, the valley Chern number obtained by integrating the Berry curvature is nonzero for the AB stacking and zero for the AA ′ stacking.Figure 3c,g shows the (projected) layer-pseudospin spectrum obtained by diagonalizing the projected layer-pseudospin operator.Since the layer-pseudospin spectrum has gaps, the Berry curvature can be obtained separately in the two layer-pseudospin sectors.Figure 3d,h show the Berry curvature in the two layer-pseudospin sectors.The Berry curvatures of the two sectors have the same signs in the AB stacking and the opposite signs in the AA ′ stacking.Therefore, the layer-valley Chern number obtained by subtracting the valley Chern numbers of the two sectors is zero for the AB stacking and nonzero for the AA ′ stacking.
Figure 4 shows the band gap, the minimal gap in the layer-pseudospin spectrum, the valley Chern number, and the layer-valley Chern number obtained from the TB model calculations for the AA, AA ′ , AB, and AB ′ stackings as a function of .As shown in Fig. 4a, the band gap E g = 2| | opens in both the AA ′ and AB stack- ings.In the AA stacking, the band gap E g = 2(|�| − t 1 ) opens when |�| − t 1 > 0 .In the AB ′ stacking, the band gap E g = 2| | − t 1 opens when 2|�| − t 1 > 0 .The band gap is accompanied by the layer-pseudospin spectrum gap S g , as shown in Fig. 4b.In the AA ′ stacking, the layer-pseudospin spectrum gap has a simple form, S g = 2|�|/ � 2 + t 2 1 , indicating that the layer-pseudospin spectrum gap opens concurrently with the band gap.As shown in Fig. 4c, the valley Chern number is C v = ±2 for the AA and AB stackings, and zero for the AA ′ and AB ′ stackings.As shown in Fig. 4d, the layer-valley Chern number is C pv = ±2 for the AA ′ and AB ′ stack- ings, and zero for the AA and AB stackings.The Chern numbers C v and C pv are consistent with those predicted in the noninteracting bilayers.The gradual deviation of C v and C pv from integer values as increases can be attributed to the broadening and overlapping of the Berry curvature peaks centered on the opposite valleys 25,26 .
Figure 5 shows the TB model calculations for zigzag-edge bilayer ribbons with a topological domain wall in the middle of the ribbons for = 0.2t o (see "Methods" section for the definition of topological domain walls).Figure 5a,b show the band structure and probability density | | 2 at E = 0 for the AA stacking.It can be seen that the gapless edge states fully crossing the bulk band gap are well confined to the domain wall and evenly distributed across the two layers.The propagation direction of the gapless edge state can be determined from the sign of the band velocity (band slope) at E = 0 .As shown in Fig. 5a, the band slopes have the same sign at the same valley and the opposite sign at the opposite valley (negative at the K valley and positive at the K ′ valley), indicating that the gapless edge states belonging to the same valley propagate in the same direction.The gapless edge states belonging to the opposite valley propagate in the opposite direction, so an inversion of the valley is required to reverse the direction of propagation.This indicates that the gapless edge states are prevented from backscattering by the large separation of the valleys, which is typical of the quantum valley Hall states.As shown in Fig. 5c,d, the same is true for the AB stacking.
Figure 5e,f show the band structure and probability density | | 2 at E = 0 for the AA ′ stacking, where the gap- less edge states that fully cross the bulk band gap are well confined to the domain wall.Interestingly, the gapless edge states are predominantly localized to one of the two layers rather than evenly distributed across the two layers.In Fig. 5e, the gapless edge states belonging to the same valley propagate in the opposite directions, where the valley separation cannot prevent backscattering.As shown in Fig. 5f, the gapless edge states belonging to the same valley are localized to different layers, and the gapless edge states localized to the same layer belong to the opposite valleys.Thus, reversing the propagation direction of an edge state requires an inversion of the valley index or the layer-pseudospin index, indicating the coupled layer-valley index is locked to the propagation www.nature.com/scientificreports/direction.Gapless edge states belonging to the same valley is prevented from backscattering by the gap in the layer-pseudospin spectrum shown in Fig. 5g, because the gap in the layer-pseudospin spectrum suppresses the inversion of the layer-pseudospin index and therefore interlayer hopping of the gapless edge states.Figure 5h shows the band structure for the AB ′ stacking, where the edge states are gapped.In spin Chern insulators characterized by spin Chern numbers, it has been reported that either the band gap or the gap in the projected spin spectrum needs to close on the sample edge 20 .In Fig. 5i, it can be seen that the gap the layer-pseudospin spectrum is closed.The absence of gaps in the layer-pseudospin spectrum allows interlayer hopping of the edge states and therefore backscattering of edge states, resulting in the gapped edge states.Both the spin Chern number and the layer-valley Chern number are defined according to Prodan's approach, which decomposes the occupied state into two (pseudo)spin sectors by using the projected (pseudo)spin operator 19 .The two Chern numbers are topological invariants unless the band gap or the projected (pseudo)spin spectrum gap in the bulk is closed 20 .Except for the fact that we use layer-pseudospins instead of spins to decompose the occupied state, the two Chern numbers are defined in exactly the same way and should have many similarities.The fundamental difference is that each sector in the occupied state has a nonzero valley Chern number in our case and a nonzero (charge) Chern number in the case of a spin Chern insulator 20 .Details of the Chern number calculation can be found in the "Methods" section.
In honeycomb trilayers, C v and C pv obtained from the TB model calculations are consistent with those pre- dicted in the noninteracting cases.For the AAA and ABA stackings, C v = ±3 and C pv = ±1 .For the AA ′ A and AB ′ A stackings, C v = ±1 and C pv = ±3 .Figure 6 shows the TB model calculations for zigzag-edge trilayer ribbons with a topological domain wall in the middle of the ribbons for = 0.2t o .As shown in Fig. 6a-d, in the AAA and ABA stackings, since the gapless edge states propagating in opposite directions belong to the opposite valleys, the gapless edge states are prevented from backscattering by the large separation of the valley.However, as shown in Fig. 6b,d, while the other edge states are distributed across the layers, a pair of the edge states propagating in opposite directions (highlighted by red lines) are localized to the odd layers.Since the edge states have the same layer-pseudospin index, inversion of the valley index is the same as inversion of the coupled layer-valley index.
In the AA ′ A stacking, as shown in Fig. 6e-g, the gapless edge states are localized to the odd/even layers.Interlayer hopping between the odd and the even layers is suppressed by the gap in the layer-pseudospin spectrum.Backscattering requires an inversion of the valley index if the edge states have the same layer-pseudospin index, and requires an inversion of the layer-pseudospin index if the edge states have the same valley index.In other words, backscattering requires an inversion of the coupled layer-valley index.In the AB ′ A stacking, the gap in the layer-pseudospin spectrum is closed, as shown in Fig. 6j, and therefore the edge states should be gapped because interlayer hopping between odd and even layers is not suppressed.However, as shown in Fig. 6h,i, there exist a pair of gapless edge states with opposite valley indices where backscattering is prevented by the valley separation.As shown in Fig. 6i, the gapless edge states are localized to the odd layers with the same layer-pseudospin index, Figure 5. TB model calculations for zigzag-edge bilayer ribbons.80-ZNR with a topological domain wall in the middle of the ribbon.The first, second, and third columns correspond to the band structure, probability density | | 2 at E = 0 , and layer-pseudospin spectrum, respectively.The first through fourth rows correspond to the AA, AB, AA ′ , and AB ′ stackings, respectively.In (e, f), K and K ′ correspond to the two valleys.In (f), red and blue arrows indicate the opposite direction of propagation.The broken curve in (g) represents the gap in the layerpseudospin spectrum.so the opposite valley index corresponds to the opposite layer-valley index.The layer-localized gapless edge states identified in all trilayers are consistent with predictions by the definition of the layer-valley Chern number.
Finally, we briefly discuss experimental validation.In AB-stacked graphene bilayers with stacking domain walls, valley-polarized conducting channels have been experimentally observed 27,28 .In AA ′ -stacked MoS 2 bilayers under vertical electric field, valley Hall effect has been experimentally confirmed through a spin-valleydependent optical selection rule 7 .AA ′ -stacked honeycomb bilayers have an inversion symmetry in the absence of a vertical electric field and therefore should not have valley Hall effects.Conducting channels along the stacking domain walls of AA ′ -stacked bilayers, if found, can be attributed to gapless edge states localized to odd/even www.nature.com/scientificreports/layers.Previous studies have shown that electron spin resonance spectroscopy can be an effective tool to detect surface conduction electrons in topological insulators 29,30 .
To summarize, we have shown that honeycomb multilayers with staggered AB-sublattice potentials can have topologically nontrivial states characterized by layer-valley Chern numbers.The layer-valley Chern number is obtained by decomposing the occupied state into two layer-pseduspin sectors and subtracting the valley Chern numbers of the two sectors.The gapless edge states corresponding to the layer-valley Chern number are localized to the odd/even layers, and the gap in the layer-pseudospin spectrum can suppress interlayer hopping of edge states between the odd and the even layers.The coupled layer-valley degree of freedom is defined as the product of the layer-pseudospin index and the valley index.The gapless edge states free from backscattering can be well described by the coupled layer-valley degree of freedom locked to the direction of propagation.

Methods
A single orbital TB model for honeycomb multilayers can be described as 16,25,31 c † p,i ( c p,i ) is a creation (annihilation) operator of an electron at the site p (A or B) in the i-th layer.t 0 =1 is the nearest-neighbor hopping energy within a layer.t 1 is the nearest-neighbor hopping energy between layers and was set to t 1 = 0.1t 0 .p,i is the on-site potential.On the basis of ( ψ A1 , ψ B1 , ψ A2 , ψ B2 , ...), the on-site potential was set to −�(I N ⊗ τ z ) for the AA and AB stackings, and was set to −�(M ⊗ τ z ) for the AA ′ and AB ′ stack- ings, each of which corresponds to parallel and aniparallel alignments of subalttice-pseudospin polarizations in the adjacent layers.I N is the N×N identity matrix, where N is the number of layers.τ z is the Pauli matrix acting on the sublattice index.M is an N×N diagonal matrix in which M ij = (−1) i−1 ( i, j = 1, 2, ..., N ) if i = j and 0 otherwise.In bilayers, M equals σ z , which is the Pauli matrix acting on the layer index.M⊗I corresponds to the layer-pseudospin, where I is the 2 × 2 identity matrix, and the eigenvalues ±1 correspond to the two states in which the electrons are in the odd and the even layers, respectively.Following Prodan's approach, the occupied states can be decomposed into two layer-pseudospin sectors by using the projected layer-pseudospin operator 19,20,32,33 .Matrix elements of the projected layer-pseudospin operator are given by < ϕ m (k)|M⊗I|ϕ n (k) > , where ϕ m (k) and ϕ n (k) are the wavefunctions for the occupied state with m and n running over all the occupied states.After diagonalizing the matrix, the eigenvalues M z (k) give the (projected) layer-pseudospin spectrum, which can be decomposed into two sectors if the spectrum has a gap.The wavefunctions ψ ± in the two layer-pseudospin sectors are given by linear superpositions of the occupied states with the eigenvector of the projected layer-pseudospin operator as the coefficient of the superposition.The layer-pseudospin-resolved Berry curvature can be defined as A zigzag-edge ribbon, a one-dimensional system periodic in a zigzag direction, is referred to as n-ZNR, where the number n of the A-B pairs on a layer in the unit cell corresponds to the ribbon width.A domain wall was introduced by changing the sign of at the middle of the ribbon.In Fig. 2, the AB ′ α and AB ′ β stackings correspond to the two domains of the AB ′ stacking, which can be obtained by changing the sign of .
(1) H = −t 0 c † p,i c q,i − t 1 c † p,i c q,j + p,i c † p,i c p,i .

Figure 1 .
Figure 1.Schematic for layer-valley coupling.(a) Two topological domains of a honeycomb monolayer.(b) Two stacking types of honeycomb bilayers.Only the edge states corresponding to one of the two valleys are shown.The edge states corresponding to the opposite valley propagate in the opposite direction.Red and blue arrows correspond to the opposite sublattice-pseudospin polarization.

Figure 2 .
Figure 2. Schematic for high-symmetry stacking orders.Solid and dashed lines correspond to the nearest neighbor hopping within and between layers, respectively.Ai and Bi indicate A-and B-sublattices in the i-th layer, respectively.Black and grey discs indicate negative and positive on-site potentials, respectively.Red and blue arrows correspond to sublattice-pseudospin polarizations in the opposite directions.

Figure 3 .
Figure 3. TB model calculations for honeycomb bilayers.The first and second columns correspond to the AB and AA ′ stackings, respectively.(a, e) Band structures along high-symmetry points.(b, f) Berry curvatures, (c, g) layer-pseudospin spectra, and (d, h) layer-pseudospin-resolved Berry curvatures, projected on the k x axis.The red and blue colours correspond to the two layer-pseudospin sectors.

Figure 4 .
Figure 4. TB model calculations for honeycomb bilayers.(a) Band gap E g , (b) minimal gap S g in the layer- pseudospin spectrum, (c) valley Chern number C v , and (d) layer-valley Chern number C pv as a function of the sublattice potential .The black, red, green, and blue colours correspond to the AA, AA ′ , AB, and AB ′ stackings, respectively.

Figure 6 .
Figure 6.TB model calculations for trilayer 80-ZNR with a domain wall.The first to third columns show the band structure, | | 2 at E = 0 , and layer-pseudospin spectrum, respectively.The first to fourth rows correspond to the AAA, ABA, AA ′ A, and AB ′ A stackings, respectively.In (b, d), the red lines represent gapless edge states predominantly localized to the odd layers.The broken curve in (g) represents the gap in the layer-pseudospin spectrum.The same convention as in Fig. 5 was used.
± = ×A ± , where A ± = i < ψ ± | | ψ ± > is the layer-pseudospin-resolved Berry connection.The valley-and layer-resolved Chern number C ±,ξ can be obtained by integrating the layer-pseudospin-resolved Berry curvature over the half Brillouin zone.Thus, C v = C +,K − C +,K ′ + C −,K − C −,K ′ is the valley Chen number and C pv = C +,K − C +,K ′ − C −,K + C −,K ′ is the layer-valley Chern number, where K and K ′ correspond to two valleys.