Introduction

Topological surface states have been studied intensively for over a decade, and in particular surface spectroscopic methods such as angle-resolved photoemission spectroscopy (ARPES) have been instrumental in visualising their properties. However, topological modes form whenever there is an interface between a topological insulator and ordinary insulator, and not exclusively when the latter is air/vacuum. Embedded topological states (ETSs), i.e., states that form at the interface between topological and non-topological solid materials, offer interesting possibilities such as the controlled rendering of other topological phases1, whilst also providing a simple means to shield topological states from environmental perturbations (humidity, air pressure, dust etc.). Clearly, ETSs cannot be probed using ARPES, and in order to understand and subsequently manipulate them, one needs to devise alternate methods.

The GeTe-Sb\(_2\)Te\(_3\) system is a TI-NI system that offers clear advantages towards studying ETSs. Here, GeTe is a normal insulator (NI)2 that becomes superconducting below \(\approx ~1\) K3,4,5, and Sb\(_2\)Te\(_3\) is a topological insulator (TI)6,7. From a materials perspective, the GeTe-Sb\(_2\)Te\(_3\) system is well-known due to its phase-change properties8,9,10,11,12,13, ferroelectric characteristics14,15, and potential thermoelectric properties16. In addition, superlattices of alternating Sb\(_2\)Te\(_3\) and GeTe layers are known to have a non-trivial band topology that is governed by (i) the coupling of topological modes1,17 that appear at each Sb\(_2\)Te\(_3\)-GeTe interface in the superlattice, and (ii) the physical intermixing of adjacent layers, which is significant owing to the strong chemical affinity between the materials. However, to date the majority of investigations, theoretical and experimental, focuses on monolayer/few monolayer superlattice units14,15,18,19,20,21 for which intermixing easily destroys the layered heterostructure22. In this paper, by contrast, we will be interested in relatively less explored structures where the individual GeTe and Sb\(_2\)Te\(_3\) layers are more bulk-like, and consequently, in which individual GeTe-rich and Sb\(_2\)Te\(_3\)-rich regions are well-defined. This approach allows us to study individual ETSs which are hard to delineate in superlattices.

In this manuscript we present a comprehensive analysis of the band structure of bulk-like GeTe-Sb\(_2\)Te\(_3\) heterostructures and the ETSs that form at the heterointerface. On the experimental side, we present a detailed materials characterisation of the heterostructures including X-ray photoemission spectroscopy (XPS) to evaluate the band offset, and Atom Probe Tomography (APT) to visualise the spatial atomic distribution. On the theoretical front, we first calculate the band-offset between GeTe and Sb\(_2\)Te\(_3\) from first principles and verify our result against the XPS measurements. We then develop a continuum model of GeTe-Sb\(_2\)Te\(_3\) heterostructures based on parameters extracted from the first-principles calculation, which allows us to model thick heterostructure for which first-principles calculations are not possible. The continuum model shows that the physical structure at the Sb\(_2\)Te\(_3\)-GeTe interface strongly influences the extent over which ETS are localised and, therefore, in mediating inter-ETS coupling in multi-layer heterostructures. More specifically, we find that under conditions of a sharp, well-defined interface, interactions between topological modes separated by more than a few nm are relatively weak, but under more realistic conditions where the interface has some degree of intermixing, topological modes separated by as much as 10 nm couple and develop a gap of several meV. Finally, we use first-principles calculations to understand the impact of chemical disorder on the ETSs and establish that these show a striking degree of resilience to structural disorder arising from the intermixing.

This paper is structured as follows: In the first section “Band offset”, we present experimental measurements of the band offset between Sb\(_2\)Te\(_3\) and GeTe on molecular-beam-epitaxy (MBE)-grown samples using X-ray photoemission spectroscopy (XPS). We also present atom probe tomography (APT) of a Sb\(_2\)Te\(_3\)-GeTe-Sb\(_2\)Te\(_3\) (SGS) heterostructure which shows well-defined Sb\(_2\)Te\(_3\) and GeTe regions separated by narrow but finite intermixed regions. In the section “First-principles calculations” we use first-principles calculations to obtain an independent estimate of the band offset at the SG interface for a range of experimentally relevant microstructures. Our estimates compare favorably with the XPS data, thereby validating the first-principles result. Next, in the section “Four-band continuum model”, we develop a continuum model of an SGS tri-layer within a four-band framework, the parameters of which are obtained from our first-principles calculations. The continuum model can be used to study large systems that would be computationally too expensive to study directly using first-principles methods. Finally, in the section “First-principles calculations of interfacial states in SGS tri-layer”, we analyze the robustness of topological modes to chemical interactions and examine how the microscopic structure of the interfacial region may impact the interlayer coupling of topological modes.

Band offset

In this section, we discuss the band offset between the GeTe and Sb\(_2\)Te\(_3\) layers in the GST heterostructure. First, in “Experimental evaluation”, we present experimental measurements of the band offset at the interface of GeTe and Sb\(_2\)Te\(_3\) using XPS depth profiles. In “First-principles calculations”, we then use first-principles calculations to evaluate the bulk crystal structure of GeTe and Sb\(_2\)Te\(_3\), from which we obtain an independent evaluation of the band offset, which agrees with the experimental results. The agreement between experiment and theory validates the theoretical calculation, and provides a basis for the continuum model developed in “Four-band continuum model” to describe larger systems.

Experimental evaluation

We use XPS measurements to determine the band offset between bulk GeTe and bulk Sb\(_2\)Te\(_3\). Following Refs.23,24,25, this is done by evaluating the difference of the core electron energy levels in bulk samples, and comparing this to the difference in the core energy levels in a heterostructure. Conventionally, this would require XPS spectra of three separate samples: a bulk GeTe film, a bulk Sb\(_2\)Te\(_3\) film, and a heterostructure of the two in which the top layer is sufficiently thin (\(\sim 5\) nm) that the X-rays can penetrate it fully and sample both materials. However, such a procedure will have unknown systematic errors when considering Sb\(_2\)Te\(_3\)-GeTe heterostructures as the two compounds have a strong chemical affinity and will undergo significant intermixing, especially in the vicinity of the interface. Here, we obtain instead a “depth profile” of a single GeTe-Sb\(_2\)Te\(_3\) heterostructure where XPS spectra are taken between successive Ar-ion etches of the sample, which successively remove the top layers of the heterostructure. The measurements are continued for the entire depth of the sample, i.e., until the Ar-ion etch fully depletes the material. This approach eliminates variations due to the different growth conditions for separate samples. The samples considered here are MBE-grown heterostructures in which the GeTe (top) layer has a thickness of 11 nm and the Sb\(_2\)Te\(_3\) layer is 25 nm thick. The samples are grown on a Si(111) substrate as described in Ref.26.

Figure 1
figure 1

XPS spectra of the GeTe-Sb\(_2\)Te\(_3\) heterostructure after consecutive ion milling steps for three different energy ranges near the (a) Ge(2p3), (b) Sb(3d), and (c) Te(3d) peaks. Spectra are obtained for eight different etch times of (bottom to top) \(t=25, 65, 75, 90, 120, 130, 140\), and 170s, and we include an arbitrary offset between spectra to guide the eye. (a) Shows an initial pronounced Ge(2p3) peak at 1220eV that nearly vanishes after 170s of milling, indicating that the GeTe is fully eroded at that time. Consistent with this, (b) shows a clear peak at 540eV, corresponding to the Sb (3d) transition, that emerges after 120s of etching, indicating the absence of Sb in the top layers of the original unetched sample. (c) Shows a consistent peak at 573eV corresponding to the Te(3d) transition, indicating that the Te is present throughout the heterostructure. Panels (a) and (b) also show residual Sb(Ge) peaks in the Ge(Sb)-rich regions, which suggests an intermixing between the GeTe and Sb\(_2\)Te\(_3\) layers.

Figure 2
figure 2

Absolute position of the (a) Sb(3d) and (b) Ge(2p3) core levels with respect to the valence band energy \(E_v\). (a) Shows an XPS spectrum measured in the Ge-rich phase and (b) shows an XPS spectrum in the Sb-rich phase. The valence band energy \(E_v\) is defined as the minimum energy at which emissions are observed, i.e., at which the XPS spectrum just becomes non-zero (indicated by arrows in (a) and (b)). This is obtained as the intersection of the binding energy curve with the background level in the respective left panels of (a,b). (c) XPS spectrum after 120 s of etching which contains signals from both Ge and Sb core levels, from which the difference in core energy levels is evaluated as shown.

Figure 1 shows XPS depth profiles for different etching times \(t=25, 65, 75, 90, 120, 130, 140\), and 170s in which the Ge(2p3) transition (Fig. 1a), the Sb(3d) transition (Fig. 1b), and the Te(3d) transition (Fig. 1c) is monitored. As expected, initially there is a pronounced Ge peak which begins to diminish at the same time the Sb peak appears, indicating that the top GeTe layer is completely eroded after 130s of etching. The Te(3d) transition (Fig. 1c) shows little depth dependence, which is expected. The traces taken between 120s and 140s show features corresponding to both Ge and Sb, suggesting that the X-rays probe both layers in this range, which points to an intermixing between the layers. These traces also reflect the diffuse nature of interface between the two materials. The band offset \(\Delta E_v\) is obtained using the following formula23,24,25:

$$\begin{aligned} \Delta E_v = (E_{\text{CL}}^{Ge} - E_v^{Ge}) - (E_\text{CL}^{Sb} - E_v^{Sb}) + \Delta E_{\text{CL}} . \end{aligned}$$
(1)

Here, the first two terms on the right-hand side represent the difference in energy between the core level (CL) and valence band edge (\(E_v\)) of Ge and Sb, respectively. These are obtained from the bulk spectra as shown in Fig. 2a,b. The third term is the difference in energy between the core levels of Ge and Sb obtained from the combined spectrum shown in Fig. 2c. The result for the band offset is \(\Delta E_v = 0.4 \pm 0.2\) eV.

Further evidence for intermixing region is obtained in Fig. 3 where we show atom probe tomography (APT) of a Sb\(_2\)Te\(_3\)/GeTe/Sb\(_2\)Te\(_3\) sample. APT is based on the evaporation of atoms in the form of ions from a single tip-shaped sample by means of an electric field. During the analyses, ions are projected from the apex of the tip onto a position-sensitive single ion detector27 by the electric field. On the basis of the measured positions and the time-of-flight between the tip apex and the detector surface a 3D reconstruction of the analyzed volume is created28. Further details on the APT can be found in the Supplementary Material.

Figure 3
figure 3

Atom probe tomography (APT) of Sb\(_2\)Te\(_3\)/GeTe/Sb\(_2\)Te\(_3\) samples show a depth profile of the concentrations of Ge, Sb and Te through the sample thickness (see Supplementary Material for APT methods). One clearly observes distinct Sb-rich, Ge-rich and Sb-rich regions separated by intermixed regions located at 8 nm and 22 nm. Within the Sb (Ge)-rich region there is a relative concentration of \(<~20\)% Ge (Sb) whereas in the interfacial regions (centred at \(\approx ~8\) nm and \(\approx ~21\) nm) the relative concentrations of Ge and Sb are equal.

First-principles calculations

Table 1 Fermi energy, valence band maximum (VBM), and conduction band minimum (CBM).

In this section, we present results for the band offset obtained from electronic structure calculations of bulk Sb\(_2\)Te\(_3\) and GeTe via density functional theory (DFT) and non-equilibrium Green function (NEGF) theory. In our calculations, we use the SIESTA29 and Smeagol30 program packages, details of which can be found in the Supplementary Material.

The band offset is calculated as follows: first, the Fermi level \(E_F\) is obtained from DFT calculations of bulk systems that include spin-orbit interactions. Then, we define an extended cell C by taking \(1 \times 1 \times 3\) unit cells and apply the self-consistent NEGF-DFT. The left and right sides of C are connected to the bulk semi-infinitely by the self-energy terms, such that our calculations give the Green’s function projected on C. Using the resulting Green’s functions, we analyze the spectral density and evaluate the conduction band minima (CBM) and valence band maxima (VBM). Next, we carried out NEGF-DFT for the same C while the right side of the cell C is now terminated by vacuum. Practically, we took a vaccuum region of \(z_{\text{vac}} = 15.0~\AA\) in the z-direction. Now, we can introduce the unique definition of the Fermi level \(E_F^0\) using the vacuum level, i.e.,

$$\begin{aligned} E_F^0 = E_F - V_H(z=z_{\text{vac}}) , \end{aligned}$$
(2)

where \(V_H\) is the Hartree potential averaged over the xy plane. As the last step, we corrected the values of VBM and CBM by Eq. (2), which are denoted as \(E_v^0\) and \(E_c^0\), respectively. We applied the above procedures to Sb\(_2\)Te\(_3\) (Rh) and GeTe (dRk), and as a reference, also to the rock salt (Rk) structures of Sb\(_2\)Te\(_3\) and GeTe which are possible crystal phases representing vacancy states or at high temperature31,32,33. The value of \(E_F^0\) of Sb\(_2\)Te\(_3\) (Rh) is \(-4.61\) eV.

In Table 1, we set \(E_F^0\) of Sb\(_2\)Te\(_3\) (Rh) to zero and list the values of the VBM, the CMB and the Fermi level of different structures relative to this. The band offset between Sb\(_2\)Te\(_3\) (Rh) and GeTe (dRk), given by the difference between the respective VBM is \(\Delta E_v \approx 0.36\)eV, which agrees well with our experimental value reported in the section “Band offset”. The validity of the calculations is also confirmed by noting that the calculated band gap of Sb\(_2\)Te\(_3\) is 0.1eV, which is consistent with its narrow gap, p-type semiconductor character. Likewise, the calculated band gap of GeTe (dRk)\(=0.47\)eV is close to the experimental value of 0.6eV2. We note here that our calculated band gap of GeTe (RK/dRK) is slightly lower compared to the experimental results. We attribute this to the disorder considered in the GeTe models which is known to underestimate the bandgap34,35,36,37.

The present results suggest the validity of our computational model in order to analyze the topological modes of an SGS tri-layer quantitatively. In the next section, in order to treat large heterostructures beyond the range of numerical DFT simulations, we construct an effective four-band model with parameters derived from our first-principles calculations.

Four-band continuum model

In the previous section, we have both experimentally and theoretically determined the structure of the SG interface. Experimental measurements of the band gap obtained using XPS were shown to be consistent with theoretical results from ab-initio DFT calculations of a semi-infinite slab structure, which indicates that our computational DFT model is predictive for these systems. The aim of this section is to extend the theoretical model to tri-layer structures and to address the recent experiments by Nguyen et al.26 by considering the qualitative effect of a thick, bulk-like GeTe intermediate layer on the embedded interface states.

While the numerical DFT method is in principle exact, i.e., it will accurately describe the inter- and intralayer coupling as well as the chemical intermixing at the interfaces, modeling very thick bulk-like heterostructures comes with a prohibitive numerical cost. In practice, we are restricted to very thin structures of typically less than ten layers. In order to make contact with the experiments on bulk-like structures of Ref.26, in this section, we introduce an effective four-band model using parameter values derived from the bulk calculations presented in the previous section. This model allows us to describe tri-layer structures of arbitrary thickness. Indeed, as a main result of this paper, our findings indicate a significant interlayer-coupling of surface states across the GeTe layer, which is consistent with the experiment26.

The effective four-band model is predictive for inter- and intralayer coupling effects, but it neglects the physical intermixing of the GeTe and Sb\(_2\)Te\(_3\) phases at the interface, i.e., a reconfiguration of atomic positions. In order to take this into account, we consider additionally a model in which the GeTe film is replaced by a Ge\(_2\)Sb\(_2\)Te\(_5\) (GST225) crystal phase, which is one of the most standard compositions of the GST alloy. We adopt the Kooi structure of GST22538, the crystal structure of which is shown in Fig. 3 of the supplemental material. To further support our effective model, in the subsequent section “First-principles calculations of interfacial states in SGS tri-layer”, we present ab-initio results for thin heterostructures that are consistent with the results obtained by the four-band model.

Table 2 Band parameters of the four-band continuum model.

A simple description of the electronic spectrum in semiconductor heterostructures is obtained using the envelope function formalism39, and details of this calculation are presented in the supplemental material29. The formalism describes separate layers in terms of effective bulk band models, which for topological insulators in the Bi\(_2\)Se\(_3\) and Sb\(_2\)Te\(_3\) family capture the band structure near the \(\Gamma\) point7,40:

$$\begin{aligned} H = \begin{pmatrix} \varepsilon (\mathbf{k}) + M(\mathbf{k}) &{} B(k_z) k_z &{} 0 &{} A(k_\parallel ) k_- \\ B(k_z) k_z &{} \varepsilon (\mathbf{k}) - M(\mathbf{k})&{} A(k_\parallel ) k_- &{} 0\\ 0 &{} A(k_\parallel ) k_+ &{} \varepsilon (\mathbf{k}) + M(\mathbf{k})&{} - B(k_z) k_z \\ A(k_\parallel ) k_+ &{} 0 &{} - B(k_z) k_z &{} \varepsilon (\mathbf{k}) - M(\mathbf{k}) \end{pmatrix} . \end{aligned}$$
(3)

Here, adopting the notation of Ref.40, \(\varepsilon (\mathbf{k}) = C_0 + C_1 k_z^2 + C_2 k_\parallel ^2\), \(M(\mathbf{k}) = M_0 + M_1 k_z^2 + M_2 k_\parallel ^2\), \(A(k_\parallel ) = A_0\), \(B(k_z) = B_0\), \(k_\parallel ^2 = k_x^2 + k_y^2\), and \(k_\pm = k_x \pm i k_y\). The interface is aligned in the xy-plane, and the z-direction is perpendicular to the interface.

The parameters of Sb\(_2\)Te\(_3\) were derived in Ref.7 and are summarised in Table 2, where we define the zero of the energy scale at the Fermi level of Sb\(_2\)Te\(_3\). For calculations of multilayer structures, we also require parameter values for an effective model of the GeTe phase. Due to band inversion, the band gap of GeTe (dRk) increases at the L point in the rock salt structure18,41. The four-band model (3) is still applicable, where the L point of the rock salt cell relates to the \(\Gamma\) point in the conventional hexagonal cell. We construct an effective Hamiltonian of GeTe (dRk) by fitting the first-principles data of the bulk unit cell presented in the section “First-principles calculations” near the \(\Gamma\) point to our effective model. The parameter set is given in Table 2. Related parameters for a Hamiltonian that describes the GST225 (Kooi) phase are also given in Table 2.

Figure 4
figure 4

Calculated results for the band structure of an SGS tri-layer in the continuum model. (a) Band structure of an Sb\(_2\)Te\(_3\)/GeTe/Sb\(_2\)Te\(_3\) tri-layer. (b) Band structure of an Sb\(_2\)Te\(_3\)/GST225/Sb\(_2\)Te\(_3\) tri-layer.

We now discuss the band structure as obtained from the envelope function formalism of an SGS tri-layer that consists of two outer Sb\(_2\)Te\(_3\) layers and an embedded GeTe middle layer. Figure 4a shows results for the band structure near the \(\Gamma\) point, where we choose an outer bulk-like Sb\(_2\)Te\(_3\) layer of thickness \(L_s=15\) nm, and an inner GeTe layer of thickness \(L_g=5\) nm. While the surface states at the outer edges of the tri-layer are still localised at the slab energy of \(-0.14\) eV, the inner Dirac states are shifted in energy, but no band gap opens at the \(\Gamma\) point. Our numerical result is essentially unchanged for different values of the thickness \(L_g\) of the inner GeTe layer, and a gap opening at the \(\Gamma\) point is only observed for very small values \(L_g < 2\) nm. This result suggests that (i) the interlayer coupling is completely suppressed even for moderate GeTe layers and (ii) that the perturbation of the GeTe wave function is sufficiently weak that the band structure of SG interfacial state is unchanged. This is a very reasonable result considering the large offset of the GeTe valence band maximum and conduction band minimum compared to the Sb\(_2\)Te\(_3\) layer: The VBM and CBM of Sb\(_2\)Te\(_3\) are located within the band gap of GeTe and there are no GeTe states in the energy range of surface states. The interlayer coupling of the topological mode is thus suppressed.

As an alternative model, we examine an SGS structure in which the GeTe layer is replaced by GST225 (Kooi) with a thickness of \(L_k=5\) nm. The results are shown in Fig. 4b. Interestingly, and in contrast to the previous case, we find a band gap opening at the \(\Gamma\) point. The band gap appears to close at points away from the \(\Gamma\) point, but this closed gap should be considered as an accidental degeneracy of the wave functions originating from the band structure of Sb\(_2\)Te\(_3\) and GST225(Kooi), rather than any topological modes. However, the behaviour at the \(\Gamma\) point indicates clearly that the microscopic structure of the NI region strongly influences the overall band structure. Within the continuum model we understand this as being a consequence of the reduced band offset between Sb\(_2\)Te\(_3\) and GST225 compared to Sb\(_2\)Te\(_3\) and simply GeTe which facilitates interlayer coupling of the topological modes. Further insights into the nature of the ETSs are obtained in the following section.

First-principles calculations of interfacial states in SGS tri-layer

The continuum model assumes that the Hamiltonian derived from the (homogeneous) bulk structure is applicable to the heterostructure junction. Although we mimicked the effect of physical intermixing of chemical species by introducing the GST225 layer, effects due to local electronic states and/or vacancies, which we refer to as “chemical interactions”, are not represented. While charge transfer or charge accumulation by impurities at the SG interface is expected to be sufficiently small (as the electron affinity of both Sb\(_2\)Te\(_3\) and GeTe is strong), the robustness of any topological mode against local fields arising from chemical interactions effect is not clear. In order to address this, in the following, we extract interfacial states of various thin SGS tri-layers directly by using NEGF-DFT.

In our computational model of the tri-layer, the left- and right-hand side of Sb\(_2\)Te\(_3\) is represented explicitly by \(1\times 1\times 3\) unit cells, where the outermost cells are connected to the bulk by self-energy terms as discussed in the section “First-principles calculations”. Hence, different from the previous section, we can only consider the embedded interfacial state on the SG interface sides to analyze the topological mode, i.e., the intralayer-coupling is automatically eliminated. We consider three separate models, A, B, and C, with three separate structures for the NI part. Model A represents a sharp SG interface, Model B depicts a disordered SG interface, and Model C represents the situation where there is an intermixed region (GST225) at the SG interface. Based on our APT measurements in the section “Experimental evaluation”, Model C is the closest approximation of the system we consider here.

Model A: [(Sb\(_2\)Te\(_3\))\(_9\)] /(GeTe)\(_{3n}\)/[(Sb\(_2\)Te\(_3\))\(_9\)] (ideal interface).

Model B: [(Sb\(_2\)Te\(_3\))\(_9\)]/(Ge\(_2\)Te\(_2\))(GeTe)\(_{3(n-2)}\)(Ge\(_2\)Te\(_2\)) /[(Sb\(_2\)Te\(_3\))\(_9\)] (disordered interface).

Model C: [(Sb\(_2\)Te\(_3\))\(_9\)] /(GST225)\(_m\)(GeTe)\(_{3(n-2)}\)(GST225)\(_m\)[(Sb\(_2\)Te\(_3\))\(_9\)] (realistic interface).

In our notation, [(Sb\(_2\)Te\(_3\))\(_3\)]\(_2\), for example, denotes a staking of the two (conventional hexagonal) unit cells of the Sb\(_2\)Te\(_3\) crystal, i.e., it is a stacking of six Sb\(_2\)Te\(_3\) quintuple monolayers (QLs). The intermediate layer in Model A is a GeTe (dRk) layer. Here, the stacked numbers of (GeTe)\(_3\) units, n, is taken as \(n = 6\) (recall that a stacking of three GeTe monolayers is also the unit cell of the GeTe (dRK) bulk crystal in the conventional hexagonal cell). A change in the interface structure is taken into account in model B, which contains a vacancy layer at the boundary of a GeTe (dRk) phase. Here, the label (Ge\(_2\)Te\(_2\)) represents a vacancy layer that consists of a single (GeTe)\(_3\) block. Finally, in Model C we introduce two unit cells of GST225 on either side of GeTe as an intermixing region, i.e., \(m=2\). The outermost regions are connected to bulk Sb\(_2\)Te\(_3\) (Rh). The 2D band dispersion of the interface is extracted by projecting the density of states (DOS) on the interface, and is exactly calculated from the Green’s function as a function of energy E and wave vector \(k_{||}\). We take k to point along M-\(\Gamma\)-K line and the DOS was projected on each QL in the junction. We present the projected band structure at three separate positions: (i) the Sb\(_2\)Te\(_3\)-QL closest to the SG interface plane, (ii) the secondary neighboring QL, and (iii) the third QL in order to analyze the localization of topological mode. We labeled the above the three QLs as QL\(^{(i)}\), QL\(^{(ii)}\), and QL\(^{(iii)}\), respectively.

In Model A (Fig. 5a) and Model B (Fig. 5b), the extracted 2D band structure on QL\(^{(iii)}\) is very similar to that of bulk Sb\(_2\)Te\(_3\) (Rh) as given in Fig. 4a. Although a very weak spectral density coming from the DOS of QL\(^{(ii)}\) is found, the electronic state in QL\(^{(iii)}\) is essentially a bulk state for both Model A and Model B. In contrast, the projected band dispersion of QL\(^{(i)}\) is more complicated and shows strong hybridization with the states of GeTe, i.e., the Sb\(_2\)Te\(_3\) layer immediately adjacent to GeTe is strongly perturbed by chemical interactions. In the QL\(^{(ii)}\) of Model B, we find a clear Dirac cone with a much stronger spectral density than in QL\(^{(i)}\), and similar to the clean Sb\(_2\)Te\(_3\) surface state. This is consistent with the results of Schubert et al.42 wherein it is found that strong disorder can shift the topological state away from the surface and into the material. Interestingly, for Model A we observe a Rashba-type split band rather than topological mode. While it is hard to gauge this unexpected result against experimental measurements as it represents an ideal interface, unlikely to occur in real systems, we note that this finding is comparable to that seen in Ref.43 where ETSs appear to have a Rashba-like character.

These results lead to the following conclusions: first, the interfacial state characterised as the “surface” state of Sb\(_2\)Te\(_3\) can be localised narrowly in the secondary neighboring Sb\(_2\)Te\(_3\) monolayer in the junction; and second, the topological mode is not robust to chemical interaction even when the band offset is sufficiently large. By comparing models A and B, the existence of a vacancy layer at the SG interface induces a significant chemical interaction effect. We speculate that the local electric field due to a (GeTe)\(_3\) block in the SG interface works as a built-in asymmetric external field and gives rise to a Rashba-type interfacial state even though the topological mode is protected by GeTe block containing a vacancy layer, i.e., Ge\(_2\)Te\(_2\).

Interestingly, in contrast to model A and model B, we found that 2D band on QL\(^{(iii)}\) of model C (Fig. 5c) does not converge to that of bulk. The presence of GST225 opens large band gap and the band dispersion is more NI-like near the \(\Gamma\) point. This result is also found in the continuum model (Fig. 4b), although Model C represents a somewhat more realistic intermixing at the SG interface in which GST225 is only narrow sublayer of NI rather than the entire NI slab. The absence of a bandgap in Model A and B suggests that the GeTe block is sufficiently thick to suppress interlayer coupling of topological modes from the two Sb\(_2\)Te\(_3\) slabs. However, this changes dramatically in Model C wherein, for the same thickness of the NI slab, one finds a band gap. In other words, the large band gap at \(\Gamma\) is a direct consequence of the existence of GST225.

The origin of the band gap in Model C can be due to disorder42 or it may be due to local interactions and/or hybridization of each topological mode with the electronic state narrowly localised on GST225. A third possibility is that it is due to long-range coupling of ETSs on either side of the NI region as suggested by Nguyen et al.26 We can rule out disorder since the first principles calculations are performed for ideal crystalline systems of which only Model B incorporates disorder. Based on the first principles results we cannot categorically rule out or posit either of the other options, i.e., local or long-range interactions. However, we can use the continuum model to conclude the following: it follows from Fig. 4b that if the NI region is fully GST225 then the band gap must be a consequence of long-range coupling of ETSs as disorder is not included in the model. The intermixed regions in our samples are \(\approx 6 - 8\) nm thick (Fig. 3), which is comparable to the NI thickness considered in Models A–C. Moreover, Fig. 3 also indicates a finite intermixing across the entire NI slab which indicates that the experimental system corresponds closely to the continuum model considered in Fig. 4b. This would then lead to the conclusion that if the intermixed GST225 region is sufficiently thick in the NI part, the topological modes can be made to couple by tuning “bulk-like” thickness of the NI in the SGS tri-layer. Thus, these results strongly suggest the role of long-ranged coupling of ETSs in the results observed by Nguyen et al.26.

Figure 5
figure 5

The extracted 2D band structures projected on a Sb\(_2\)Te\(_3\) quintuple layer in the SGS tri-layer system by NEGF-DFT calculation. The Fermi level of bulk Sb\(_2\)Te\(_3\) is set to zero. The zero of \(k_{||}\) is at the \(\Gamma\) point, and \(k_{||}\) is positive along the \(\Gamma\)-K line and negative along the \(\Gamma\)-M line. Three SGS tri-layer structures denoted as model A, B, and C were examined (see section “First-principles calculations of interfacial states in SGS tri-layer” of the text for the definition), corresponding to (ac), respectively. The left column is the 2D band structure projected on QL\(^{(i)}\), i.e., the Sb\(_2\)Te\(_3\) quintuple layer nearest to the GeTe (or GST225) block. The middle and right columns show band structures projected on the second (QL\(^{(ii)}\)) and third (QL\(^{(iii)}\)) nearest quintuple layer, respectively.

Conclusions

In this paper, we have studied the band structure of bulk-like heterostructures of Sb\(_2\)Te\(_3\) and GeTe as a prototypical TI-NI system. We have focused on the interfacial region between these materials and discuss the conditions under which the topological mode may or may not be present. A particular focus of our study were realistic structures in which the interface is not perfect, but rather contains an intermixed phase of the parent compounds Sb\(_2\)Te\(_3\) and GeTe. Our principal finding is that the presence of this intermediate phase serves to enhance the length over which topological modes may interact with each other. Importantly, we confirm experimentally that SGS heterostructures show a degree of intermixing, thereby underlining the relevance of our findings to the existing experimental literature on SGS systems. Of particular importance in this context is the experimental work by Nguyen et al.26 which observes unexpectedly long-ranged interactions between topological modes in an SGS heterostructure. Our findings provide a natural explanation for these results and lay the foundation for future work wherein superlattices of bulk-like Sb\(_2\)Te\(_3\) and GeTe layers can be used to deterministically produce different topological phases1.