Introduction

The discoveries of topological insulator (TI) phases1,2,3,4 in HgTe quantum wells5 and Bi-based materials (Bi1−xSbx, Bi2Se3, Bi2Te3, etc.)6,7,8,9,10,11,12,13 have inspired extensive research efforts on exploiting their exotic quantum properties14,15,16,17,18 and potential applications in e.g. quantum computing19, spintronics20 and catalysis21,22. The uniqueness of TI lies in the robust metallic topological surface states (TSS) with linear Dirac-cone-like energy-momentum dispersion. The TSS arises from the strong intrinsic spin-orbit coupling (SOC) that drives the electronic structures of the TIs to a topologically nontrivial phase and is expected to appear at an interface between topologically nonequivalent regions, such as between a TI and a conventional insulator (CI). The requirement of strong SOC suggests that heavy-element, small-band-gap semiconductors are the most promising candidate materials11,12,13,23 to reach the TI phase. Recent research efforts have also revealed that a trivial insulator can be twisted into a topological state by manipulating the spin-orbit interaction24,25, the crystal lattice11,13, or with the application of an external electric field26, driving the system through a topological phase transition.

The present study offers a conceptually intriguing alternative approach to achieving such quantum phase transitions, via dual topological proximity effects, characterized by controlled shift of the spatial location of the TSS in TI-based heterostructures. Recently, a number of peculiar quantum phenomena have been discovered in various TI-based heterostructures14,15,16,17,18,19,20,21,27,28,29,30,31,32,33,34,35, such as Majorana fermions induced by the superconducting proximity effect19, topological magnetoelectric effect4, quantized anomalous Hall effect14 and electron reservoir effect of TSS in surface catalysis21. Although the TSS was confirmed to be robust as the TI surface is intruded under diverse conditions provided that time reversal symmetry is preserved21,27,28,29,30,31, its properties can be largely modified due to the tunability in the magnitude of interfacial coupling within the heterostructures.

Specifically, the present work reveals a new degree of freedom in the tunability of the TSS properties, namely, its location perpendicular to the surface, as a CI overlayer is placed onto a TI substrate. We establish this intriguing and highly desirable tunability using ultrathin (single stoichiometric layer) films of CIs (ZnM, M = S, Se and Te) on Bi2Se3 or Bi2Te3 as prototypical examples and the concepts established are expected to be applicable to other related heterostructural systems or when thicker overlayers are used. We show that, because of the strong influence of the interaction with the TI on the electronic properties of the ultrathin CI films, the latter may undergo a nontrivial topological phase transition, signified by the relocation of the TSS from the TI/CI interface to the top of the CI film (Fig. 1a). Furthermore, the TSS can also be manipulated to stay put at the TI/CI interface (Fig. 1b) by tuning one or more key physical properties of the CI to be out of the window for topological phase transition. Most strikingly, we discover a reverse-proximity effect: The top quintuple layer (QL) of the TI may even be driven by the CI film to a topologically trivial phase, signified by a spatial shift of the TSS toward the inside of the TI (Fig. 1c). Collectively, these results help to enrich our understanding of possible topological quantum phase transitions in the hybrid systems and also lay the foundation for precise manipulation of the real space properties of TSS in TI heterostructures of diverse technological significance. Here we also wish to emphasize one important aspect associated with the introduction of “topological proximity effect”: It does not involve symmetry breaking associated with a local order parameter, as the topological phase is a nonlocal, global property of the system36. It is qualitatively different from traditional proximity effects19,37,38, which typically invoke some symmetry breaking processes measured by the corresponding local order parameters.

Figure 1
figure 1

Schematic illustration on tuning the vertical location of the topological surface states (TSS) as a topological insulator (TI) is covered with a layer of conventional insulator (CI).

(a), TSS floating to the top of the CI. (b), Staying put at the CI/TI interface. (c), Diving into the TI. (d), The atomic structure of ZnM/Bi2Se3 (M = S, Se, Te). The red lines denote the TSS; the arrows indicate the resulting directions of the topological phase transition.

Full size image

Results

A bulk ZnM (M = S, Se and Te) material crystallizes in a zincblende structure (space group Fd-3m), with its (111) surface geometry matching well with the (0001) surface of Bi2Se3 or Bi2Te3. The lattice mismatch between ZnM (111) and Bi2Se3 (0001) is +8.4%, +3.5% and −3.9% for M = S, Se and Te, respectively, where “+” or “−” denotes tensile or compressive strain. Another advantage of using the ZnM family is that they span a considerable range of the key property parameters such as the SOC strength and band gap34 (see Table I and Supplementary Section S1). As proof of principles, here we limit ourselves to the cases where a single stoichiometric ZnM layer is deposited onto a TI substrate (Fig. 1d).

Table 1 Characteristic properties and vertical locations of the TSS in different CI/TI heterostructures. The values for the band gap (Vg), spin-orbit splitting (ΔSO) and work function (Φ) are obtained for free-standing single-layered CIs; the bulk lattice constants (a) of the CI are from experiments. The last three columns indicate the lattice mismatch (Δa), binding energy (Eb) between the CI and TI and the location of the TSS (Z). The values in the square brackets are the corresponding values for the TI substrates
Full size table

Topological proximity effects in ZnS/Bi2Se3 and ZnSe/Bi2Se3

Figures 2a–c display the atom-resolved band structures of ZnS/Bi2Se3. The Dirac points of the TSSs are still located at the Fermi level, indicating a negligible charge transfer (or transfer doping39) between the ZnS film and Bi2Se3. Although the existence of the Dirac points is guaranteed by time reversal symmetry, away from the Γ point, the degeneracy of the two TSSs from the two surfaces of the TI slab is lifted, due to the interaction between the ZnS film and the upper surface of the Bi2Se3. To find out the spatial location of the TSS, we plot the atom-specific character of each band, as indicated by the dots superposed onto the band structure. It is apparent from Figs. 2b and 2c that the Dirac-cone like TSS bands are localized on the 1st QL (upper surface) and 6th QL (lower surface) of the Bi2Se3, whereas from Fig. 2a we see that the TSS has negligible electron weight from the ZnS film. The real-space density of states of the upper-surface TSS at the Dirac point (Fig. 2d) also confirms this identification. Therefore, the ZnS/Bi2Se3 system can be regarded as the same as a bare Bi2Se3 film placed in the vacuum, in the sense that the ZnS film remains a topologically trivial CI just like the vacuum and hence the TSS in essence stays put at the surface of the Bi2Se3.

Figure 2
figure 2

Band structures of ZnS/Bi2Se3 (upper row) and ZnSe/Bi2Se3 (lower row) along the K–Γ–M direction.

The dots indicate the electronic bands contributed by the CI (a) and (e), the 1st QL of the TI (b) and (f) and the 6th QL (c) and (g), respectively; the sizes and colors of the dots also indicate different spectral weights and contributions from different atoms, respectively. (d) and (h) show the charge density distribution of the upper-surface TSS at the Γ point marked by the circle and indicated by DPU. The DPU/L stands for the Dirac point at the upper/lower surface. The grey and cyan bars denote the locations of the different QLs and the CI, respectively.

Full size image

The same analysis yields very different behaviors for the system of ZnSe/Bi2Se3, shown in Figs. 2e–h. Compared to the former case of ZnS/Bi2Se3, the upper-surface TSS now shifts downward in energy due to enhanced charge transfer from ZnSe to Bi2Se3, which can be qualitatively understood from the smaller work function of the ZnSe film than that of ZnS (see Table I and also Supplementary Section S2). However, the most interesting difference between the present system and the former one is that, from Fig. 2e, a considerable weight of the TSS is carried by the ZnSe film, which is also unambiguously confirmed by the real space density of states of the TSS in Fig. 2h. Because a TSS marks the boundary between topologically inequivalent phases of matter, the upward shift of the TSS to the surface of the ZnSe overlayer indicates that the (ZnSe/TI) heterostructure as an entirety becomes an expanded TI via the topological proximity effect. More specifically, the topologization of the CI overlayer is signified by the whole (CI/TI) system possessing the same and only TSS located on top of the CI surface, namely, at the global boundary between the topologically trivial vacuum and the topologically nontrivial and combined (CI/TI) system.

We now discuss the underlying physical origin of the contrasting behaviors of the TSS in ZnS/Bi2Se3 and ZnSe/Bi2Se3. Since the nontrivial topology of the electronic structure of Bi2Se3 is ascribed to its strong SOC and small bulk band gap, it is intuitive that these factors associated with the CI materials are also essential for the different topological proximity effects we just observed. Indeed, the bulk phase of ZnSe has a much larger SOC (0.12 eV compared to 0.02 eV) and smaller band gap (1.20 eV compared to 2.09 eV) than ZnS and the trend is preserved for the single-layer films as well (see Table I and Supplementary Section S1), making ZnSe “more proximate” to a TI material property-wise. Thus the term “proximity effect” here carries a dual meaning of both spatial and property-wise closeness. To gain a deeper understanding of the roles played by these key parameters of the ZnM, we look into the band structures of the two heterostructures (Figs. 2a, e). In the ZnSe/Bi2Se3 case (Fig. 2e), one can see that the SOC-induced gap (ΔSO) within the valence band of the ZnSe is aligned with the energy range of the TSS and the TSS is mainly derived from the lower branch of the SOC-split bands. Charge transfer or transfer doping39 from the ZnSe to the Bi2Se3 down-shifts the TSS, thereby facilitating its hybridization with the lower branch of the SOC-split bands. In particular, when the energy range of the TSS of the TI substrate falls into the SOC-induced gap region of the ZnSe, the coupling between the initial TSS and the SOC-split bands of the CI will be strong, leading to the newly-derived TSS of the resultant (CI/TI) system that contains high electron weight from the ZnSe overlayer. Moreover, because the dispersion of the lower branch of the SOC-split band is similar to the linear dispersion of the TSS, these two bands are more likely to hybridize in a wider range of the k-space.

In contrast, for ZnS/Bi2Se3, the bulk gap (and correspondingly the TSS) of the TI falls completely within the gap of the CI film (see Figs. 2a–c); consequently, the bands from the ZnS and Bi2Se3 are only mutually weakly perturbed. This “bad” alignment in the sense of absent proximity effect is partly due to the much larger band gap of ZnS than that of ZnSe and partly because of the smaller mismatch between the work functions of the ZnS film and Bi2Se3 substrate than that in the ZnSe/Bi2Se3 case (Table I). As an alternative view, the seemingly trivial case of ZnS/Bi2Se3 in essence represents a balanced outcome of more accurate tuning, where one or more of the key CI parameters are pushed out of the range for the topological phase transition to take place.

Reverse topological proximity effect in ZnTe/Bi2Se3

With these competing factors in mind, we next look into the system of ZnTe/Bi2Se3. Since ZnTe has an even larger SOC and a smaller bulk band gap than that of ZnSe (see Supplementary Figs. 2)40, the first intuition is to see an even more pronounced relocation of the TSS to the top of the ZnTe film. However, as a counterintuitive surprise, the upper-surface TSS now has a reverse shift toward the inner of Bi2Se3, with its peaked density located on the top of the 2nd QL (Figs. 3a–e).

Figure 3
figure 3

Band structures of ZnTe/Bi2Se3 and ZnTe/Bi2Se3 along the K–Γ–M direction.

Band structures of the relaxed ZnTe/Bi2Se3 system (top row), the ZnTe/Bi2Se3 system with an increased separation of 3 Å from 2.5 Å between the 1st QL and 2nd QL of the TI substrate (second row), the ZnTe/Bi2Se3 system with an increased separation of 3 Å from 2.6 Å between the CI and 1st QL (third row) and the relaxed ZnTe/Bi2Se3 system (bottom row). All other symbols are the same as in Fig. 2.

Full size image

To reveal the physical origin of this unexpected reverse-proximity effect, we first note that, from Figs. 3a–d, both the bulk bands of Bi2Se3 and the TSS have a more noticeable downward shift in energy compared to the case of ZnSe/Bi2Se3, while the ZnTe film becomes strongly p-doped, indicating the most pronounced charge transfer from the CI overlayer to the TI substrate among the three cases. This behavior can be understood from the much smaller work function of the ZnTe film in comparison to that of ZnSe. The concomitant misalignment between the TSS and valence band top of ZnTe prevents the ZnTe film from being topologized by the Bi2Se3 substrate, as we have discussed before. Moreover, because of the significant charge transfer, the binding energy between the ZnTe film and the Bi2Se3 substrate (Table I) is much larger (0.29 eV) than the other two systems (<0.1 eV). On an atomic scale, for the heterostructures investigated in the present study, the cation (Zn) in a given ZnM overlayer is located directly above the anion (Se) of the Bi2Se3 substrate; therefore, the competition between the in-plane Zn-M bonding and the out-of-plane Zn-Se bonding will significantly influence the interfacial coupling strength. Specifically, as all the out-of-plane bonds in the systems of ZnM/Bi2Se3 are given by the same type of Zn-Se bond, variations in the interfacial coupling strength for the three different systems lie in the different in-plane Zn-M bonds. Qualitatively, we further observe that the magnitudes of the band gaps for ZnM (M = S, Se, Te) satisfy Vg(ZnS) > Vg(ZnSe) > Vg(ZnTe), suggesting that the bond strengths should also satisfy Vb(Zn-S) > Vb(Zn-Se) > Vb(Zn-Te). Therefore, the in-plane bonding in ZnTe/Bi2Se3 is the weakest, making the interfacial bonding in ZnTe/Bi2Se3 the strongest. Since the QLs of Bi2Se3 are mutually coupled through weak van der Waals-like interactions, such a strong coupling between the ZnTe film and the 1st QL will compete with and weaken the interaction between the 1st and 2nd QL. As a consequence, the ZnTe film forces the 1st QL of the TI to be electronically partially decoupled from the remaining QLs. Furthermore, because one QL of Bi2Se3 does not have gapless TSS (see Supplementary Section S3)41, the upper-surface TSS will naturally be relocated to the top of the 2nd QL. In other words, the CI has prevailed by topologically “trivializing” the 1st QL of the TI via the reverse-proximity effect.

To provide further support of the physical picture revealed above, we have performed two additional comparative studies. In the first one, we kept the relative positions of the ZnTe film and the 1st QL of Bi2Se3 fixed, but increased the distance between the 1st and 2nd QL from 2.5 Å to 3 Å, hence further decreasing their coupling. As shown in Figs. 3f–j, the TSS now becomes even more localized on top of the 2nd QL. In the second study, we fixed all the QLs of Bi2Se3 and slightly increased the distance between the ZnTe film and the 1st QL from 2.6 to 3 Å. As expected, the TSS now moves back to the top of the 1st QL, as shown in Figs. 3k–n.

As yet another strong and complementary support of the above picture, here we study the system of ZnTe/Bi2Te3, i.e., replacing Bi2Se3 with the structurally nearly identical Bi2Te3, which is also a TI. The charge transfer is less dramatic than that in ZnTe/Bi2Se3, as shown in Figs. 3p–s. The reason lies in the smaller work function mismatch between the tensilely strained ZnTe and Bi2Te3 than that between the compressively strained ZnTe and Bi2Se3 (Table I). The band gap of the ZnTe film is also decreased from 1.85 eV in the ZnTe/Bi2Se3 system to 1.39 eV in ZnTe/Bi2Te3, because of the different strains in the two systems (Table I). As a consequence, both the band misalignment (Fig. 3p) and binding energy (Table I) between the ZnTe film and the Bi2Te3 substrate become smaller compared to that in the former case. Naturally, the TSS now floats to the top of the ZnTe film, indicating that the ZnTe is now topologized by the Bi2Te3 substrate.

Discussion

For the specific system of ZnTe/Bi2Se3, the interfacial coupling strength is fixed under normal contact conditions. Nevertheless, when an electric bias potential is applied in the direction perpendicular to the interface, the resultant charge transfer across the interface can be altered, potentially weakening the overall interfacial coupling to be comparable to the case of ZnTe/Bi2Te3 or ZnSe/Bi2Se3. If so, the reverse topological proximity effect is likely to be tuned into a normal proximity effect. This intriguing feasibility will be explored systematically in a future study.

It is also instrumental to elaborate more on the delicate roles of the different physical parameters influencing the dual proximity effects and the corresponding topological phase transitions. First, the overall coupling strength at the CI/TI interface, measured by the interfacial binding energy (Eb), is naturally a crucial parameter. This strength is largely determined by the relative work functions of the CI and TI (which in turn depend on their relative band gaps), the resultant charge transfer across the interface and the nature of the local interfacial bonding. In particular, for the systems considered here, we observe that, (1) the direction of charge transfer is always from the cation (Zn) of the CI to the anion (Se) of the TI and, (2) stronger interfacial coupling exists for heterostructures in which the CI has a smaller work function than the TI. To see how the magnitude of Eb is closely correlated with the topological proximity effects, we further observe that, when the CI/TI interfacial coupling is extremely weak (i.e., Eb = 0.06 eV for the given supercell size adopted) the location of the TSS will stay put at the interface, indicating absence of the topological proximity effect (Figs. 2a–d and Fig. 4a). In contrast, for ZnSe/Bi2Se3, the work functions of the CI and TI are nearly equivalent, the interfacial coupling strength is moderate (i.e., 0.06 eV < Eb < 0.20 eV, see Figs. 2e–h, 3p–t and 4b,c) and the topological proximity effect is sufficiently effective in inducing the topological phase transition as observed. Furthermore, for ZnTe/Bi2Se3, the work function of the CI is smaller than that of the TI and the strong interfacial coupling is jointly reflected by the large binding energy (i.e., Eb = 0.29 eV) and the contaminant alignment of the TSS of the TI and the valence band of the CI. As a consequence, the substantial electronic hybridization at the interface will pull the first QL of the TI substrate to be even more weakly coupled with the second QL and the rest of the TI substrate, thereby topologically trivializing the first QL and shifting the TSS to be on top of the second QL (Figs. 3a–e and 4d).

Figure 4
figure 4

Illustrations on the effects of the key physical parameters on topological phase transitions.

(a), Weak interfacial coupling. (b & c), Moderate interfacial coupling, but with different spin-orbit coupling strengths. (d), Strong interfacial coupling. The red triangles denote the TSS; the solid and dotted black curves correspond to the valence bands of the CI and the Fermi level of the heterostructures, respectively.

Full size image

Aside from the interfacial coupling strength Eb, another delicate physical parameter in influencing the topological proximity effect is the spin-orbit coupling strength within the CI, ΔSO. This aspect can be qualitatively understood by recalling that the global topological property of the TI substrate is intimately tied to the spin-orbit coupling within the substrate1. Therefore, proper spin texture30 of the CI overlayer due to its own spin-orbit coupling is also required in order for the topological proximity effect to be operative in the form of advancing the TSS from initially on the top of the TI to the top of the CI. This is qualitatively illustrated in Figs. 4b and 4c, showing that, when ΔSO is weaker than some critical value ΔC, the TSS is still located at the interface (Figs. 2a–d, Fig. 4b); but when ΔSO > ΔC, the TSS will move to a lower energy due to stronger interfacial coupling (Fig. 4c) and spatially shift to the top of the CI (Figs. 2e–h, 3p–t).

Finally, we note that the dual-proximity effects revealed in TI-based heterostructures imply an inherent property of topological quantum phase transitions that do not invoke symmetry breaking measured by a local order parameter. This new type of quantum phase transition induced by topological proximity effects may find broad applicability. Motivated by these findings from density functional theory (DFT) calculations, an ongoing effort is to develop a more generic theory to describe the rich topological phase transitions in various CI/TI heterostructures based on consideration of the dominant physical factors controlling the transitions. Furthermore, experimental efforts demonstrating such tunability in the vertical location of the TSS and other related manifestations and opportunities enabled by this tunability, are expected to materialize. Appealing examples include exploiting the robust nature of the TSS on top of a conventional catalyst for enhanced surface catalysis, spatially separating the TSS from the bulk state of a TI for potential definitive conductivity measurement of the TSS, protecting the TSS from gas contamination in air33, etc. Such advances will also likely enrich the functionality and potential technological applications of the TI-based heterostructures.

Methods

All the density functional calculations were carried out using the Vienna ab initio simulation package (VASP)42 with projector-augmented-wave potentials43 and the Perdew-Burke-Ernzerhof generalized gradient approximation44 for exchange-correlation functional. The lattice constants of Bi2Se3 and Bi2Te3 were adopted from experiments. The generic Bi2Se3 and Bi2Te3 substrates were modeled by a slab of 32 atomic layers or 6 QLs. The semiconducting overlayers ZnM (111) (M = S, Se, Te) were epitaxially placed on the (0001) surface of the TIs and the energetically most stable configuration is obtained by putting Zn directly above Se and M in the hcp sites, with the Zn face adjacent to the Bi2Se3 surface (Fig. 1d). The vacuum layers used are over 20 Å thick to ensure decoupling between neighboring slabs. During structural relaxation, atoms in the TI substrate were fixed in their respective bulk positions and all the other atoms in the semiconducting overlayer were allowed to relax until the forces on them are smaller than 0.01 eV/Å. Test calculations with the first QL of the TI substrates also fully relaxed show that the relaxation effects are minimal and do not change the main physical pictures emphasized in the present study. A 7 × 7 × 1 k-point mesh was used for the 1 × 1 surface unit cell45. The SOC splitting of a semiconductor is defined as ΔSO = ε(Γ8υ) − ε(Γ7υ) at the top of the valence band40. For the single-layered ZnM films, we use the magnitude of the SOC-split gaps in the valence band as a measure of the SOC.