Introduction

The fields of metamaterials and plasmonics have seen an extraordinary evolution in recent years, from the initial theoretical predictions of artificial negative refractive indices and cloaking1, 2 to the experimental realization of photonic metadevices with various functionalities that can be engineered and obtained on demand.3, 4 However, the practical implementations of plasmonic and metamaterial devices have long been hampered by energy dissipation in plasmonic media, especially in the visible to ultraviolet (UV) range, where even the best plasmonic metals (such as gold, silver and aluminum) suffer from strong dissipation due to interband electronic transitions and Drude losses.5 This has inspired a search for alternative low-loss plasmonic materials for optical devices operating at high frequencies.6, 7 Candidates that are currently being investigated include highly doped semiconductors, metallic alloys, nitrides and oxides8 and, more recently, two-dimensional materials and topological insulator (TI) materials.

TIs represent a new quantum phase of matter, which originates from the topological character of the bulk electronic bands in certain materials. The strong spin–orbit coupling in such materials leads to a band inversion and the appearance of Dirac surface states in the band gap. These topological surface states are chiral and protected from back-scattering by time-reversal symmetry. As a consequence, charge carriers from topologically protected surface states can carry current and are free to move parallel to the surface. Thus, TIs are ideal candidates to realize exotic plasmonic phenomena. Localized plasmons have recently been observed in TIs at THz frequencies (in Bi2Se3)9, 10 as well as visible-to-UV frequencies (in single crystal Bi1.5Sb0.5Te1.8Se1.2, or BSTS).11 These experiments observed large bulk resistances and surface-dominated transport.12, 13 In addition, multiple plasmon modes were recently reported in solution-synthesized Bi2Te3 nanoplates14 and nano-discs and flakes of BSTS.15 Ellipsometric data indicated the presence of high optical conductivities, possibly linked to the existence of topologically protected surface states. If true, BSTS or other TIs would be a radically new material platform for broadband nanoplasmonic devices that can be modulated optically,16 through injection of electrons,17 or by applied magnetic fields.18

In this work, we use density functional theory (DFT) calculations to investigate the electronic band structures and optical properties of seven compounds in the BixSb1-xTeySe1-y (BSTS) family of TIs, in bulk crystal and thin film forms, with the overall goal of determining their suitability for nanoplasmonics (similar ab-initio studies have proven effective with metals,19, 20, 21 alkali-noble intermetallics22 and gallium-doped zinc oxide23). Our results show that plasmonic behavior of quaternary trichalcogenide compounds in the optical part of the spectrum has three origins: (a) bulk interband transitions contributing primarily in the visible spectral region; (b) intraband transition within topologically protected surface bands contributing in the mid-infrared region; (c) interband transition between topologically protected surface states and bulk states, dominating in the UV–near-infrared (NIR) range.

Materials and methods

DFT calculations were used to study the electronic structure and optical response of TI materials employing the local-density approximation, using the Quantum ESPRESSO code.24 Experimental lattice parameters of bulk TI materials12, 25 were used as the initial structure, and ground states geometries of the TI systems were obtained by the total energy minimization method upon relaxing their crystal framework and atomic coordinates. Electron-ion interactions (plane waves and core electrons) were generated by norm-conserving non-relativistic and relativistic pseudopotentials with electrons for Bi (6s2, 6p3, 5d10); Sb (5s2, 5p3); Se (4s2, 4p2); and Te (5s2, 5p2, 4d10). The relativistic effects and spin–orbital coupling (SOC) have significant effects on the band structure due to heavy elements, such as Bi, Sb and Te. Supplementary Figure 1 shows the small energy differences (<0.005 Ry) between all-electron and our generated pseudopotentials, confirming the good transferability properties of these pseudopotentials. Single-particle wavefunctions (charges) were expanded on a plane-wave basis set up to a kinetic energy cutoff of 80 Ry (500 Ry) for both TI bulk and slabs. The bulk and thin film layers were relaxed until forces on the atoms were lower than 0.01 eV/Å. The k-space grid of 6 × 6 × 6 in the Brillouin zone was chosen for the calculation of the band structures.

The optical response calculations were performed by the Bethe-Salpeter equations (BSE) method with the YAMBO code, using ground state wavefunctions from Quantum ESPRESSO:26, 27

where Eck and Evk are the quasiparticle energies of the conduction and valence states, respectively; are the expansion coefficients of the excitons, and ΩS are the eigenenergies.

The imaginary part of the permittivity was calculated by evaluating direct electronic transitions between occupied and higher-energy unoccupied electronic states as obtained from

where are the dipole matrix elements for electronic transitions from valence to conduction states. The real part can then be calculated via the Kramers-Kronig relation . SOC interactions and the spinor wave functions were included as input for the electronic and optical calculations on the level of many-body perturbation theory. Previously this approach has been used to successfully study the optical properties of systems with strong spin–orbital interaction.28 For optical permittivity calculations, a k-point grid corresponding to 12 × 12 × 12 and 40 conduction bands and 40 valence bands both for bulk and slabs were chosen to ensure a sufficient description of the optical permittivity.

For the crystal slabs, the optical permittivity consists of two parts: (i) the interband contribution was calculated in the framework of the BSE method involving transitions between two surface bands and bulk bands, as well as transitions from surface valence band to bulk conduction bands and (ii) the Drude-like free-electron intraband contribution from the metallic surface, which can be described by a two-dimensional Drude model determined by the Fermi energy, carrier scattering time and thickness of the surface layer as described in the main text.

Results and Discussion

Before studying the optical properties of the BSTS crystals, we optimized their crystallographic structure parameters and calculated their band structures. The three-dimensional (3D) bi-chalcogenide compounds have a rhombohedral crystal structure arranged in the order Bi(Sb)–Se(Te)–Bi(Sb)–Se(Te), and all of them belong to space group .12 As shown in Figure 1a, this is a layered structure forming the quintuple layer (QL) unit cell. The intra-layer interaction within a QL can be either covalent or ionic, whereas the weak inter-layer interaction has van der Waals character. We performed DFT calculations of the equilibrium lattice constants within the local-density approximation, with and without accounting for SOC. As shown in Supplementary Figure 2, the calculated lattice parameters are in good agreement with available experimental data12, 25 Due to the larger ionic radius of Te lattice parameters, ahex and chex increase with Te content (for example, Bi2Te3 vs Bi2Se3). The agreement with experimental data improves when SOC is accounted for, which indicates the validity of our chosen pseudopotentials (Supplementary Figure 1).

Figure 1
figure 1

(a) Crystal structure of BixSb1-xTeySe1-y (BSTS) compounds exhibiting topological insulator (TI) behavior (rhombohedral and equivalent hexagonal lattice cell, with a quintuple layer (QL) shown by the green rectangle); (b) band structures of bulk BSTS compounds calculated with and without account of spin–orbit coupling.

Full size image

The calculated band structures of the seven bulk BSTS compounds are shown in Figure 1b, both with (solid blue lines) and without (black dashes) accounting for SOC. Without SOC, all the compounds exhibit parabolic band dispersions with small (<0.5 eV) direct band gaps at the Γ-point. The direct gaps for the binary compounds Bi2Se3 (54 meV), Bi2Te3 (159 meV) and Sb2Te3 (123 meV), as well as for the ternary compounds Bi2Se2Te (477 meV) and Bi2Te2Se (84 meV), compare well with previous calculations.29, 30 The characteristic effect of SOC on the electronic band structures of TIs is to produce a band inversion, inducing topologically nontrivial band gaps.31, 32 Such gaps are indeed observed at the Г-point for the trichalcogenide compounds (Figure 1b). The conduction band minima and valence band maxima shift away from the Γ-point, along the symmetric Z-F direction, turning the direct band gaps into indirect band gaps of width 0.10–0.35 eV (Supplementary Figure 3). Note that it is possible to estimate the gap energies more accurately by introducing self-energy corrections in the quasiparticle energy approximation, using many-body perturbation theory (for example, the GW method).33, 34 However, this would not have noticeable effects on the UV to near-infrared optical properties, which are dominated by high-energy interband transitions.

To quantify the plasmonic properties of BSTS compounds, we separately evaluated their bulk and surface dielectric functions. In the bulk, the complex dielectric function (ɛ) originates from interband transitions, which we calculated from the band structures using the Bethe-Salpeter and Kramers-Kronig equations (see MATERIALS AND METHODS). In Figure 2, we plot the real part of the dielectric function (ɛ′), which describes the strength of the polarization induced by an external electric field, and the imaginary part (ɛ′′), which describes the losses encountered in polarizing the material.5 The strongest peak in the spectral dispersion of ɛ′′ moves from the near-infrared (0.5–1.5 eV) to the visible spectral region (1.5–2.5 eV) when tellurium content is reduced. This peak arises from interband transitions occurring along the symmetry line near the Γ-point (0, 0.16666, −0.16666). Once SOC effects are included in the optical response calculations, we find that there are two main absorption peaks in ɛ′′ (Supplementary Figure 5). The first peak in the visible region (1.5–2.0 eV) is red-shifted as the tellurium content is reduced, which is attributable to the resonant character of interband transitions between the spin–orbit split valence band and the lowest conduction band near the Γ valley in momentum space. As a result, the dual interband transitions at high symmetry point around band inversion lead to the much stronger dielectric function compared to the case without SOC. The other strong peaks appear at ~0.6 eV due to interband transitions around the (0.33333, −0.5, −0.5) point. The transition details for the optical permittivity are listed in Supplementary Table 1. The calculated values of bulk optical permittivity agree well with experimental data reporting prominent peaks of the imaginary part of the permittivity between 1.5 and 2 eV for Bi2Te3, Sb2Te3 and Bi1.5Sb0.5Te1.8Se1.2 (Figure 2b),11, 14, 35 although calculated permittivities at peaks are somewhat higher than the experimentally measured ones. For the real part of the optical permittivity, the main features are broad peaks centered around 1.0–2.0 eV, followed by a steep decrease between 1.5 and 3.0 eV, after which ɛ′ becomes negative and eventually increases slowly toward zero at higher energies. The epsilon-near-zero behavior, interesting for many applications36, 37, 38 appears in the UV region, above 5 eV.

Figure 2
figure 2

Interband transition contribution to the real (ɛ′) and imaginary part (ɛ′′) of the permittivity of bulk BixSb1-xTeySe1-y (BSTS) materials: (a) density functional theory (DFT) results without spin–orbital coupling (SOC) and (b) experimental data obtained from refs 11, 14, 35.

Full size image

Thus, for the bulk materials, plasmonic behavior emerges when ɛ′ becomes negative in a frequency range of around 2 eV or higher, depending on the compound. We note that negative permittivities due to interband transitions have also been reported in previous DFT calculations on Bi2Se3.39 Our results show that increasing the tellurium content gives plasmonic behavior over a wider frequency range, with much larger negative values of ɛ′ (for example, for Bi2Te3).

The bulk dielectric functions indicate that the BSTS compounds can outperform plasmonic materials in the visible-to-UV spectral range, including traditional plasmonic metals. To quantify this, in Figure 3, we determine the plasmonic figure of merit,40 , where is the complex wave vector of the surface plasmon polariton mode. Such FOM can be thought of as the number of wavelengths a surface plasmon polariton mode can travel before dissipation. In Figure 3 we plot the FOM of bulk BSTS compounds obtained from our DFT calculations as well as experimental data available in the literature.11, 14, 35 The FOM of aluminum, silver and gold are also included for comparison. Both sets of data indicate that the BSTS materials have higher FOM than Au below ~600 nm, and higher FOM than Ag below ~450 nm. Most strikingly, in the UV-blue wavelengths of ~400 nm, surface plasmon polariton modes in bulk BSTS chalcogenide crystals are expected to propagate 3–6 wavelengths more than in Au or Ag. Although Al has a higher plasmonic FOM in this spectral region, its plasmonic properties are strongly dependent on the surface oxidation status.41 Note that the intrinsic electron–phonon coupling in the topological surface state of TIs is strong enough to activate the phonon-assisted intraband processes in the plasmonic response,20, 42 which could slightly reduce the FOM values of these TIs.

Figure 3
figure 3

Plasmonic figures of merit for bulk BixSb1-xTeySe1-y (BSTS) compounds and noble metals determined using experimental dielectric constants from the literature (top panel) and theoretical dielectric constants obtained from density functional theory (DFT) calculations without spin–orbital coupling (SOC; bottom panel).

Full size image

We now turn our attention to the surface properties of BSTS compounds. To study the optical response of topological surface states, we performed DFT calculations considering thin slabs. The emergence of topological effects and charge accumulation at surface states in TIs is expected to be clearly seen in slabs with thickness of a few QLs, as indicated by previous theoretical studies32, 43 and experimental observations.44 Based on this, we carried out DFT calculations for a (111) crystal slab with five QLs (5QL) of approximate thickness of ~5 nm, and looked for the emergence of surface states upon activation of spin–orbital coupling. The contribution of surface states to the optical response was then obtained considering two types of transitions: (i) intraband transitions described by 2D Drude-like response of surface free carriers,9, 10, 45 and (ii) interband transitions from topological surface states to higher energy bands.

Here we take Bi2Se3 slabs as an example and present their electronic structure, charge distribution of the surface state and optical permittivity (Figure 4). Similar data for the other BSTS compounds are provided in Supplementary Figures 6. Without accounting for SOC effects, the band structure of 5QL-slab (orange solid line in Figure 4a) is similar to those of the bulk crystals obtained using periodic boundary conditions (Figure 1b). Increasing the slab thickness decreases the energy band gap because of the reduction in quantum confinement, with the energy gap approaching the bulk value at about 19QL thickness (Supplementary Figure 4). Upon including SOC effects, a single non-degenerate Dirac point appears at the Γ-point. At the Dirac cone below the Fermi energy level, the electronic charge density distribution is mainly localized at the surface and penetrates ~1–2 nm into the bulk (see surface layer thickness in Table 1 and integrated charge density function in Supplementary Figure 8). The optical response of the 5QL Bi2Se3 slab shows similar spectral features as the bulk: the dominant peak of the imaginary part of the permittivity appears at 1.6 eV, and a low-energy peak emerges around 0.6 eV in agreement with the experimental dispersion curves shown in Figure 2b.

Figure 4
figure 4

(a) Band structure of 5QL Bi2Se3 slab without and with inclusion of spin–orbit coupling effects (the horizontal dashed line indicates the Fermi energy level and K′–M′ the region of occupied surface states). (b) Three- and one-dimensional charge density distribution of conducting surface states; all occupied orbitals below the Fermi level and their degeneracy (Supplementary Figure 7) were considered to calculate the average charge density. (c) Comparison between real and imaginary parts of the dielectric functions of bulk and 5QL Bi2Se3 slab with spin–orbital coupling (SOC).

Full size image
Table 1 Surface state thickness s, effective mass m, two-dimensional carrier concentration n2D of surface states, carrier scattering time τ, damping factor Γ and conductivity σ of 5QL-TI slabs
Full size table

Knowledge of the band structure and surface charge distribution of the TI slabs allows extraction of other relevant parameters of the TI, such as effective mass m of the carriers, surface layer thickness d and the surface carrier density n2D used to determine the DC conductivity, . These parameters are listed in Table 1. For the 5QL-slabs, the effective mass of electron was estimated by fitting the linear dispersion curve of the lowest conduction band with the relationship where νF is the Fermi velocity,46 while total surface carrier concentration was obtained by integrating the occupied surface states charge below Fermi energy level (the region K′–M′, see Supplementary Figures 7 and 8). Note that the predicted carrier densities are systematically higher than those derived from electrical and optical measurements, since we consider all the intrinsic surface charge carriers of TI materials, which in actual experiments could undergo several scattering pathways (for example, charge carrier-optical phonon scattering or charge carrier-surface scattering). The intraband surface contribution to the dielectric response was determined using the methodology developed for other two-dimensional systems, such as graphene.47 The DC conductivity due to the TI surface states is given by , where Ef is the Fermi energy, τ is the carrier scattering time and d is the thickness of the surface layer. At higher frequencies, the dispersion of the optical conductivity is given by the two-dimensional (2D) Drude model:10, 45, 47, 48 . The contribution of surface free carriers to the optical response can then be evaluated from the expression: . This expression coincides with the dispersion of the dielectric constant of conventional plasmonic metals, provided that the 3D static conductivity, , is replaced by the 2D static conductivity, , of the surface states. Unlike the case of conventional plasmonic metals, however, this dispersion relationship has no cutoff (plasma) frequency, thus yielding negative real permittivity even in the optical spectral range, irrespectively of the carrier density (Fermi energy). The contribution of surface state intraband transitions to the dielectric constant (Figure 5b–h, dotted lines) was quantified using the parameters determined from first-principles DFT calculations, except for the free carrier damping factor, assumed to be Γ=0.07 based on the typical experimental carrier lifetime of ~60 fs (Table 1).49, 50, 51, 52, 53 The plasmonic contribution of surface free carriers is rather small in the visible part of the spectrum but becomes prominent at longer wavelengths, starting from the near IR region. This is reflected in the extremely high plasmonic figures of merit of BSTS TI slabs at the longer wavelengths (refer to Supplementary Figure 10a).

Figure 5
figure 5

Inter- and intra-band contributions to the real and imaginary parts of the optical constants of BixSb1-xTeySe1-y (BSTS) topological insulator (TI) slabs originating from topological surface states. (a) Experimental values of Bi1.5Sb0.5Te1.8Se1.2 surface contribution were derived in ref. 11 using a two-layer model. (b–h) Intraband contributions calculated according to a 2D Drude model (dotted lines) and interband contributions due to transitions from the surface state band to higher energy bands (solid lines).

Full size image

The dielectric response arising from interband surface transitions, that is optical transitions from topologically protected surface state to higher energy bands, is shown in Figure 5 (solid lines). The real and imaginary part of the dielectric constants were evaluated from the DFT band structure of the TI slabs using similar methodology employed for estimating the optical response of the bulk (as described in MATERIALS AND METHODS). These transitions give rise to plasmonic response (negative real permittivity) in the optical part of the spectrum with strong absorption features around 1.5 and 0.5 eV. In contrast to the bulk, these surface interband transitions yield plasmonic behavior over an extremely broad spectral range, from the visible through the NIR, with lower losses and appreciably high plasmonic figures of merit (see Supplementary Figure 10b).

Our estimate of the overall optical response arising from intra- and inter-band transitions from topological surface states of BSTS compounds is in good qualitative agreement with the optical permittivity of the surface of Bi1.5Sb0.5Te1.8Se1.2 determined from ellipsometric measurements (Figure 5a), assuming a multilayer model with a thin Drude surface layer.11 This confirms that topologically protected surface charge carriers have appreciable contribution to the plasmonic response of the TI crystals at optical frequencies, with strong Drude response in the infrared region and significant interband surface contribution throughout the optical spectrum, up to the UV. The good agreement with experiments gives us confidence that the proposed TI slab model, in which the contributions of the conducting surface layer and the bulk are treated independently, can adequately describe the optical and plasmonic response of this family of TI materials. Most importantly, the cumulative plasmonic response of the TI brought about by the combination of inter- and intra-band surface transitions is extremely broadband in nature. Both types of transitions originate from topological surface states that, besides being immune to scattering from defects, are spin-polarised and strongly coupled to the bulk. This opens up exciting opportunities for broadband plasmono-spintronic devices that can be controlled by charge injection, external magnetic fields, or light helicity.

In summary, we presented a systematic study of the optical and plasmonic properties of BSTS TI crystals using first-principles DFT calculations. In addition to their negative bulk permittivity, TIs show robust, spin-polarized surface states, which are capable of supporting surface plasmons over a very broad wavelength range. Comparison of the calculated dielectric functions of bulk crystals revealed the dependence of the optical band gap and permittivity on composition, indicating plasmonic behavior with figures of merit higher than noble metals in the UV-blue spectral region. Increase of Te versus Se content can extend the overall plasmonic response deeper into the NIR region and reduce losses. Furthermore, knowledge of surface charge distribution in thin TI slabs has allowed isolating the plasmonic response of the surface. The combination of interband and intraband (2D Drude) transitions involving topologically protected surface states yields plasmonic response over an extremely broad spectral range. This new understanding of bulk and surface state contributions to the overall plasmonic response of TIs will enable targeted strategies to exploit these materials in nanophotonic applications, for instance tailoring the growth of thin films by epitaxial growth methods to maximize the contribution of topological surface states and lower plasmonic losses at desired optical frequencies.