Introduction

The study of the effects of spin-orbit coupling (SOC) has been one of the central themes in physics research in the past decade. A significant discovery is that a strong SOC can convert a normal insulator into a topological insulating phase, in which metallic surface states appear in the bulk gap and they are protected by a nontrivial bulk topological order1,2. Another important consequence of SOC is the Rashba effect in inversion asymmetric structures3,4. It has been extensively studied in the context of spintronics, which aims to utilize the electron's spin degree of freedom for device applications. It is of great interest to search for a material hosting both robust TSS and large RSS which could be used as a playground to study the interplay between these two interesting effects and also as a base material for technological applications.

A nonzero RSS requires structural inversion symmetry breaking. It occurs naturally at sample surfaces or interfaces. For example, RSS has been observed at metallic surfaces5,6,7, ultrathin metal films8,9 and semiconductor heterostructures10. In particular, ARPES experiments on the topological insulator Bi2Se3 have shown that RSS exists at surfaces with artificially deposited adsorbates11,12,13,14. Such RSS, however, are typically small and sensitive to the surface environment. On the other hand, a large RSS of the bulk bands is possible in polar compounds with heavy elements, such as the noncentrosymmetric layered semiconductor BiTeI which was recently reported to have the largest known RSS strength15. However, BiTeI is a trivial insulator at ambient condition. It was proposed that under pressure BiTeI might be turned into a topological insulator16. But pressure induced structural transition might also occur as suggested by experiments17,18.

Motivated by the recent progress in the experimental fabrication of layered heterostructures19,20, here we predict that the coexistence of robust TSS and giant RSS can be realized in the [(BiTeI)m(Bi2Te3)n] heterostructure in its native state without any need of strain or pressure (m and n represent the number of layers of each constituent in a unit cell). Moreover, we find that both the topological order and the RSS energy can be controlled by varying the number of layers. In particular, the topological invariants of heterostructure with even (odd) m are given by [1;000] ([1;001]). And the RSS strength is tunable by varying the ratio between m and n. Especially the structures [(BiTeI)2(Bi2Te3)1] and [(BiTeI)3(Bi2Te3)1] have Rashba energies even larger than that of BiTeI. The TSS at different crystalline surfaces are analyzed in detail, which demonstrate characteristic features of a strongly inversion asymmetric topological insulator. Our findings thus identify a promising topological material for future spintronics and quantum computation applications.

Results

Basic crystal structure

The well known topological insulator material Bi2Te3 has a layered structure with five atomic layers (Te-Bi-Te-Bi-Te) as a quintuple layer (QL) unit21. BiTeI has a similar layered structure with three atomic layers (Te-Bi-I) as a triple layer (TL) unit15. The interactions between QLs (or TLs) are of weak van-der-Waals (vdW) type, while the interactions within one QL (or TL) are of strong covalent or ionic type. Since in-plane lattice constants of Bi2Te3 (4.383 Å) and BiTeI (4.339 Å) are very close, their heterostructure [(BiTeI)m(Bi2Te3)n], which consists of alternating m TLs of BiTeI and n QLs of Bi2Te3, is possible to be epitaxially grown along the crystallographic c-axis22. The simplest structure [(BiTeI)1(Bi2Te3)1] (hereafter referred to as Bi3Te4I) is shown in Fig. 1(a). In the following, we shall choose it as a representative example for the discussion of general structural and electronic properties of this family of heterostructures.

Figure 1
figure 1

(a) Crystal structure of the Bi3Te4I superlattice consisting of one QL of Bi2Te3 and one TL of BiTeI along the crystallographic c-axis. There are eight nonequivalent atoms in one unit cell with the stacking order of Bi1-I-Te1-Bi2-Te2-Bi3-Te3-Te4 from bottom to top, as labeled in the figure. (b) Hexagonal Brillouin zone with the high-symmetry points. The projected surface 2D Brillouin zones are also shown. (c) The calculated phonon spectrum of Bi3Te4I with SOC included.

The optimized lattice parameters and bond lengths without and with vdW corrections are listed in Table 1. As can be seen from the table, the inter-layer bond lengths (I-Te1 and Te3-Te4) are much larger than the intra-layer bond lengths in both cases. With vdW correction, the intra-layer bond lengths are hardly changed, while the inter-layer bond lengths are significantly reduced, confirming that the interactions between TL and its adjacent QLs are of weak and non-local vdW type. The Te3-Te4 and I-Te1 bond lengths relaxed by vdW-DF method are similar to the inter-layer Te-Te bond length in Bi2Te3 (3.748 Å) and the I-Te bond length in BiTeI (3.924 Å) respectively, which is reasonable. The dynamic stability of the structure is further investigated through the phonon spectrum calculations. The calculated phonon spectrum with SOC is shown in Fig. 1(c). The absence of imaginary frequencies demonstrates that our structure of Bi3Te4I is dynamically stable. In the following electronic calculations, the optimized structure with vdW corrections is adopted.

Table 1 Lattice parameters and bond lengths of crystal structures optimized using GGA-PBE and vdW-DF methods (see Fig. 1(a) for the atomic labels)

Giant Rashba spin splitting

The calculated band structure of Bi3Te4I without SOC is shown in Fig. 2(a). As we can see, Bi3Te4I is a direct band gap semiconductor with Eg ≈ 0.52 eV, much smaller than that of BiTeI (≈1.2 eV)23. The direct band gap is located at A-point of the Brillouin zone (labeled in Fig. 1(b)). The states near the band edges are dominated by Bi-6pz orbitals for the conduction band bottom and by Te-5pz orbitals for the valence band top.

Figure 2
figure 2

The band structure of Bi3Te4I without (a) and with (b) SOC. For (a), the Bi1-pz (Bi in TLs) and (Bi in QLs) orbital projected character of bands in the dashed box is shown on the right side. The size of circle denotes the weight of projection. (c) Band dispersion in the vicinity of Γ points showing the Rashba splitting. The parameters of Rashba splitting are also illustrated in the figure. (d) The local band gap at kz = 0.5 plane (around A points) and kz = 0.0 plane (around Γ point) as a function of the strength of SOC, where λSO = 1 represents the intrinsic value of SOC. (e) Surface energy spectrum of side-cut surface (ac plane) of Bi3Te4I.

When SOC is turned on, as shown in Fig. 2(b), the energy bands are spin-splitted and at the same time the conduction band (local) minima are shifted away from the high symmetry points Γ and A, which is similar to the case of bulk BiTeI. This is the signature of a Rashba type spin splitting15. The RSS is characterized by the Rashba energy ER, the momentum offset k0 and the Rashba coupling parameter αR = 2ER/k0. These parameters are illustrated in Fig. 2(c) for the conduction band bottom near Γ point. The RSS at Γ point is slightly anisotropic. Along Γ-K direction, we have ER ≈ 76 meV and αR ≈ 2.54 eV·Å, while along the Γ-M direction, the corresponding values are smaller, with ER ≈ 74 meV and αR ≈ 2.48 eV·Å. The RSS of conduction band near A point has a slightly smaller Rashba energy (ER ≈ 56 meV), but the value of αR (≈2.51 eV·Å) is close to that of the Γ point. The value of RSS in the heterostructure Bi3Te4I is much larger than the RSS of conventional semiconductor heterostructures (on the order of meV). This giant RSS can be viewed as being inherited from the BiTeI component. The bulk BiTeI has a RSS of 116 meV (k0 ≈ 0.05 Å−1) in our calculation, which is in good agreement with earlier calculation (113 meV)23 and experimental ARPES data (100 meV)15.

Topological insulating phase

In addition to RSS, the band order is also inverted after the introduction of SOC. In the presence of SOC, spin and orbital angular momentum are mixed and the pz states are transformed into j = 1/2 states. As a consequence, the conduction band bottom, which is mainly occupied by the Bi-pz orbital, is pushed downward. Meanwhile the valence band top, which is occupied by the Te-pz orbital, is pushed upward. Due to the large atomic SOC from Bi and Te, the energy gap around A point is inverted. To facilitate our understanding of this band inversion process, we gradually increase the strength of SOC (λSO) from zero to its full value and monitor the evolution of the band structure. The local band gap at kz = 0.5 plane (around A points) and at kz = 0.0 plane (around Γ point) is plotted in Fig. 2(d). As λSO increases, the local gap around Γ points decreases from 0.85 eV to 0.12 eV without closing. In contrast, the local gap around A points decreases, closes, reopens and finally reaches a value of 0.22 eV. This band inversion induced by SOC signals a topological phase transition in the current system.

To pin down the topological phase of the heterostructure more precisely, we calculate the topological invariants. Since the inversion symmetry is explicitly broken in Bi3Te4I, the parity criterion can not apply24. Instead, we evaluate the invariants ν0; (ν1ν2ν3) in terms of the Berry gauge potential and Berry curvature by using our recently developed general method within the full-potential linearized augmented plane wave framework25. The calculated for Bi3Te4I is consistent with the previous band inversion analysis. At first sight, this topological insulating phase seems to be inherited from the Bi2Te3 component. However, it should be noted that the invariants of Bi3Te4I is different from that of Bi2Te3. Instead it is more like bulk BiTeI under pressure16, which has the same topological invariants. The difference in the index ν3 reflects the different band inversion point in the Brillouin zone. For Bi2Te3, band inversion occurs at Γ point, while for Bi3Te4I band inversion occurs at A point. The nonzero value of ν3 can have interesting physical consequences. For example, it has been shown that a system with ν3 = 1 may offer a topologically protected one-dimensional conducting channel along the c-axis26.

The nontrivial value is tied to the existence of TSS27. As another verification, we calculate the surface energy spectrum directly. Fig. 2(e) shows the local density of states (LDOS) of a side-cut surface (the ac plane of Bi3Te4I). A Dirac cone in the bulk energy gap can be clearly seen, which is located at point of the projected two-dimensional Brillouin zone. This again confirms the topological insulating phase of Bi3Te4I.

Controllability of properties

Thus far, we have shown that the simplest heterostructure Bi3Te4I was a strong topological insulator with giant bulk RSS. Now we extend our investigation to the family of heterostructures [(BiTeI)m(Bi2Te3)n] with different number of layers m and n and see how the topological invariants and the RSS depend on the layer configuration. The calculated results are summarized in Table 2. We observe that the coexistence of strong topological insulating phase and giant bulk RSS is common to all these heterostructures. Quite interestingly, the ν3 index of has a dependence on the parity of m. Namely ν3 = 1 when m is odd and ν3 = 0 when m is even. For odd m, band inversion occurs at A point, while for even m, it occurs at Γ point. The location of the surface Dirac cone is associated with the band inversion point, hence can also be controlled by the parity of m. (Note that this is meaningful for side surfaces where the projections of A and Γ are distinguishable.) From the results, we see that the Rashba energy is also tunable to a great extent by varying m and n. ER generally increases with larger m and smaller n. It should be noted that heterostructures [(BiTeI)2(Bi2Te3)1] and [(BiTeI)3(Bi2Te3)1] have extremely large ER of 130 meV (αR ≈ 5.23 eV·Å) and 139 meV (αR ≈ 5.45 eV·Å) respectively, which are both larger than that of bulk BiTeI.

Table 2 Energy gap (Eg in meV), Rashba splitting energy (ER in meV) and invariants in heterostructures with different layer configurations. (TLmQLn refers to the structure [(BiTeI)m (Bi2Te3)n])

Surface states

A salient feature of an inversion asymmetric topological insulator is that different crystal surfaces have different TSS. In our heterostructure system, due to the giant RSS and the freedom to control the properties by layer configurations, more interesting physics are expected to appear on the sample surface. As two representative examples with different invariants, we shall discuss Bi3Te4I with and Bi4Te5I2 ([(BiTeI)2(Bi2Te3)1]) with in the following.

Due to the lack of inversion symmetry, there are two different surfaces of Bi3Te4I (or Bi4Te5I2) perpendicular to the c-axis, namely the I-terminated surface and the Te-terminated surface. They are constructed by cleaving the adjacent QL and TL (TL and TL for Bi4Te5I2) through the weakly coupled Te-I gap. As can be seen in Fig. 3(a,c), although with different ν3 index in , the two heterostructures have similar surface spectra. There are two different kinds of electronic states in the energy window around the bulk band gap. The states with dispersions identical for both I-terminated and Te-terminated surfaces are the bulk RSS states whose wavefunctions spread over the bulk region, thus are insensitive to the surface environment. The other states with linear dispersions that cross the bulk gap are the TSS. Owing to the strong inversion asymmetry, TSS are very different at different atom-terminated surfaces. For both heterostructures, the Dirac point of TSS stays in bulk band gap at I-terminated surface [Fig. 3(a, c)], while it is buried in bulk valence bands at Te-terminated surface [Fig. 3(b, d)]. If Fermi level lies in the bulk gap below the Dirac point of I-terminated surface, the two surfaces of a sample would have different types of charge carriers, which can be used to construct a topological p-n junction28.

Figure 3
figure 3

The surface energy spectrums of I-terminated (a), Te-terminated (b) surfaces of Bi3Te4I; and I-terminated (c), Te-terminated (d) surfaces of Bi4Te5I2.The Fermi level is set to the bulk valence band maximum and dotted lines in (a) and (b) denote various constant energy levels where spin textures are imaged. In (e)–(g) the spin-resolved constant energy contours are plotted for the I-terminated surface of Bi3Te4I, while those for the Te-terminated surface are shown in (h)–(j). The red and blue colors denote the positive and negative values of spin projections of Sx, Sy and Sz, respectively. The arrows label the spin polarizations of the outermost contour.

Due to the coexistence of two types of spin-momentum locking—TSS and RSS, the surface spin texture is expected to be more complicated in our system. In Fig. 3(e–g), we show the spin projections of Sx, Sy and Sz for the I-terminated surface of Bi3Te4I at different constant energies. In Fig. 3(e1) at energy 250 meV, there are three contours: inner, middle and outer ones. All of them have the largest spin distributions along kx direction while suppressed values along ky direction. The outer and inner ones are from the RSS states which have opposite spin polarizations, while the middle one is from the TSS. When shifting the energy level below the RSS crossing point [Fig. 3(e2)], the inner spin contour will change its polarization and then switches place with the middle TSS contour and merges with the outer one [Fig. 3(e3)], leaving only two spin contours. Below the conduction band edge, only the TSS contour remains and its spin distribution changes sign when the energy level crosses the Dirac point [Fig. 3(e4)]. The spin projection of Sy shown in Fig. 3(f) shows similar features. Combining results in Fig. 3(e) and 3(f), one can easily obtain the in-plane spin chiralities of TSS and RSS states. For TSS, the upper and lower Dirac cones have the left- and right-handed spin textures respectively. For RSS states, the inner-branch has the left- (right-) handed spin textures above (below) the RSS crossing point, while the outer-branch always has the right-handed spin texture. The Sz projection shown in Fig. 3(g) represents the out-of-plane spin texture. It can be observed that the outer RSS branch has the largest out-of-plane spin component than TSS and the inner branch of RSS. For the Te-terminated surface, only the spin projections for the upper Dirac cone are shown [Fig. 3(h)–3(j)] because the lower Dirac cone is deeply buried in the bulk valence bands. The spin textures of RSS states are identical to those of I-terminated surface, hence are not shown here.

Apart from the spin-momentum locking discussed above, recent theoretical and experimental works found that in topological insulator Bi2Se3 there was an additional locking between spin and orbital for TSS, leading to a spin-orbital texture29,30,31. Following the method of Zhang et al.30, we calculate the in-plane (px and py) and out-of-plane (pz) orbital projections together with the orbital-selective spin textures for TSS of both I-terminated and Te-terminated surfaces of Bi3Te4I. First let's examine the I-terminated surface. As shown in Fig. 4, for the in-plane orbitals, the right-handed spin texture is coupled to tangential (radial) orbital texture on the upper (lower) Dirac cone. (tangential and radial in-plane orbitals refer to linear combinations of px and py orbitals with their orbital axis along tangential or radial directions30.) For the out-of-plane pz orbital, it is coupled to the left-(right-) handed spin texture on the upper (lower) Dirac cone. As for the Te-terminated surface, the orbital-selective spin textures are opposite to the I-terminated ones. We have therefore demonstrated that the interesting spin-orbital texture of TSS, initially observed in Bi2Se331, also appears in our inversion asymmetric heterostructure systems.

Figure 4
figure 4

The px and py orbital-selective spin textures of Bi3Te4I at the upper (a) and lower (b) Dirac cone for I-terminated surface and (c) the upper Dirac cone for Te-terminated surface, respectively.The corresponding pz orbital-selective spin textures are shown in (d), (e) and (f). More red and blue colors in (a)–(c) denote more px and py character respectively, while more red colors in (d)–(f) denote more pz character. The surface states in the energy range of −0.1 eV ~ 0.2 eV are used for the orbital projections. The black arrows represent the total in-plane spin textures related to the corresponding orbitals.

Discussion

As we mentioned before, in several aspects, the BiTeI/Bi2Te3 heterostructure retains the properties of the bulk BiTeI. However, the bulk BiTeI is a trivial insulator. It can only be converted to a topological insulator under pressure. In contrast, for the system which we study, no external constraint is required. It is the intercalation by layers of Bi2Te3 that changes the topological order of the structure. To better understand this result, we start from the orbital analysis of Bi2Te3 and BiTeI. Let's focus on the conduction band edge. The states there are mainly from Bi p orbitals. For a QL of Bi2Te3, the p orbitals from the two Bi atoms can be combined to form bonding and antibonding orbitals. When crystal field is taken into account, pz orbital is split from px,y orbitals, with the bonding orbital having the lower energy21, as shown in the left shadowed area of Fig. 5. Similar energy splitting due to crystal field also occurs for BiTeI23.

Figure 5
figure 5

Schematic diagram of the evolution from atomic orbitals of Bi, Te and I into the states near the band edges of the BiTeI/Bi2Te3 heterostructure in the absence of SOC.

The evolution stages are explained in the main text.

When the layers of two materials are stacked together to form a heterostructure, due to the overlap between orbitals, the levels with close energies and the same symmetry character are coupled and repel each other. Using Bi3Te4I as an example, on the right hand side of Fig. 2(a), we show the Bi1-pz (Bi in TLs) and (Bi in QLs) orbital projected character of the low energy conduction bands. At Γ point, the and Bi1-pz orbital are mainly concentrated in the lowest and the second lowest conduction band respectively. As moving towards A point, the coupling between and Bi1-pz orbitals is getting stronger, which pushes the lowest conduction band downward. Therefore the intercalated Bi2Te3 not only provides the orbital which has a lower energy than the Bi1-pz orbital, but also through the coupling between the two orbitals it helps to further reduce the band gap (from 1.2 eV in bulk BiTeI to 0.52 eV in Bi3Te4I). Hence it becomes easier to achieve the band gap inversion when SOC is turned on.

This band gap reduction induced by intercalation also helps to explain the observed giant RSS. As been pointed out in the study of bulk BiTeI23, one of the several factors contributing to a large RSS is that the band gap should be small. Approximately, the RSS strength is inversely proportional to the gap size. The reduced gap size of the heterostructure helps to realize a giant RSS. This point is reflected from the results in Table 2. The two structures ([(BiTeI)2(Bi2Te3)1] and [(BiTeI)3(Bi2Te3)1]) with RSS energy greater than that of bulk BiTeI indeed have very small band gaps.

To conclude, we have investigated the crystal structure and electronic properties of the heterostructures [(BiTeI)m(Bi2Te3)n] by first principles calculations. We discover that these heterostructures host both strong topological insulating phase and giant RSS in their native states. Moreover, both the invariants that characterize the topological properties and the Rashba energy that characterizes strength of RSS can be controlled by varying the layer numbers m and n. The surface electronic structure and spin texture present unique features of the coexistence of TSS and RSS and of strongly inversion asymmetric topological insulators32. Our result indicates a rich physics in this class of heterostructures which provide an ideal platform for future development of spintronics and quantum computation devices, for example designing the topological p-n junctions, creating majorana fermions via combining with superconductors33.

Methods

Structure optimizations

The lattice parameters and the atomic positions of the heterostructures are optimized using the projector augmented wave method34 implemented in VASP code35,36. Perdew-Burke-Ernzerhof parametrization of the generalized gradient approximation (GGA-PBE) is used for the exchange correlation potential37. Since the interaction between TLs and QLs is of the vdW-type similar to bulk BiTeI and Bi2Te3, the vdW corrections by the approach of Dion et al.38 (vdW-DF) as implemented in VASP is employed to further relax the out-of-plane lattice parameter c and atomic positions39. The plane wave energy cutoff is set to 300 eV and the Brillouin zone is sampled by a 16 × 16 × 4 mesh. Both the crystal's shape and the atomic positions are relaxed until force on each ion becomes less than 0.01 eV·Å−1. The phonon spectrum is calculated using the PHONOPY code40 through the DFPT approach41.

Bulk electronic structure calculations

The electronic ground-state calculations are performed using full-potential linearized augmented plane-wave (FP-LAPW) method42, implemented in the package WIEN2K43. The convergence criteria are more stringent than those of structural optimizations, with KmaxRMT = 8.0 and a 20 × 20 × 4 k-mesh, where RMT represents the smallest muffin-tin radius (2.5 Bohr for all atoms) and Kmax is the maximum size of reciprocal-lattice vectors. Spin-orbit coupling is included by a second-variational procedure42.

Surface electronic structure calculations

Surface electronic structures are calculated using the combination of maximally localized wannier function (MLWF)44,45 and surface Green's function methods46,47. The construction of MLWF using package wannier9048 is a non-self-consistent process on a 8 × 8 × 3 k-mesh with formerly converged self-consistent charge potential. The surfaces with different atom-terminations are considered as semi-infinite systems, from which the iterative method46,47 is used to obtain the surface Green's function. The LDOS is extracted from the imaginary part of the surface Green's function, which is used to analyze surface electronic structures. With the advantage of addressing an individual surface, the surface Green's function methods is more suitable for the analysis of surface-dependent TSS.