Theoretical studies of magneto-optical Kerr and Faraday effects in two-dimensional second-order topological insulators

Optical approaches are useful for studying the electronic and spin structure of materials. Here, based on the tight-binding model and linear response theory, we investigate the magneto-optical Kerr and Faraday effects in two-dimensional second-order topological insulators (SOTI) with external magnetization. We find that orbital-dependent Zeeman term induces band crossings for SOTI phase, which are absent for trivial phase. In the weak-magnetization regime, these crossings give rise to giant jumps (peaks) of Kerr and Faraday angles (ellipticity) for SOTI phase. In the strong-magnetization regime, we find that two nearly flat bands are formed at the high-symmetry point of Brillouin zone of SOTI phase. These flat bands give rise to two successive giant jumps (peaks) of Kerr and Faraday angles (ellipticity). These phenomena provide new possibilities to characterize and detect the two-dimensional SOTI phase.

insulators 2,3,42,43 with some symmetry-breaking terms.The advantage of the model is that we can switch SOTI phase on and off by tuning parameters.This provides opportunities to compare the results of SOTI with trivial insulators.The light is normally incident into 2D SOTI and magnetic substrate from the vacuum, whose electromagnetic field (also that of reflected or transmitted light) follows the standard Maxwell's equations 31 .We relate the electromagnetic fields in the vacuum and substrate region by the modified boundary conditions incorporating the conductivities contributed by 2D SOTI.By solving these equations, the Kerr and Faraday angles are then directly obtained from the reflection and transmission coefficients of electric field.On the other hand, the finite-frequency longitudinal and Hall conductivities of 2D SOTI are derived by using the Kubo formula based on linear response theory 44 .Particularly, the Hall conductivity tensor is a consequence of out-of-plane magnetization in 2D SOTI.
For the treatment of magnetization, we consider the Zeeman effect in multi-orbital systems 2,43 , which can be decomposed into orbital-independent and orbital-dependent terms.By symmetry analysis, we find that only orbital-dependent Zeeman term contributes to the Kerr and Faraday effects in such systems.We also find that the magnetization induces band crossings in conduction and valence bands only for the SOTI phase.In the regime of weak magnetization, these crossings lead to giant jumps (peaks) of Kerr and Faraday angles (ellipticity).In the regime of strong magnetization, two nearly flat bands are formed at the high-symmetry X point of Brillouin zone of SOTI.These give rise to two successive giant jumps (peaks) of Kerr and Faraday angles (ellipticity) for the SOTI phase.By the quantitative analysis, we find that the model parameters and order of magnitude of rotation angles are all within experimental reach for realistic materials.Therefore these phenomena provide new features to characterize the SOTI phase, which may have practical applications in distinguishing SOTI from trivial insulators.

Model
We consider a generic tight-binding model of two-dimensional chiral second-order topological insulators H(k) = H 0 (k) + H Λ (k) + H z 7,9,13 , with Here m(k) = M − 2B[2 − α=x,y cos(k α a)] and Λ(k) = Λ[cos(k x a) − cos(k y a)] .k x , k y are the wave vectors and a is the lattice spacing (set to be unity).Pauli matrices σ α and s α ( α = 0, x, y, z ) act on orbital and spin degree of freedoms, respectively.H 0 (k) is the minimal tight-binding model for topological insulators 3,42 .H 0 (k) describes topological insulating phase with gapless edge states when 0 < M/B < 8 ; otherwise it describes trivial phase.
Here we choose the hopping parameters t = 0.06 eV adopted from the HgTe quantum well systems 2,43 .Other parameters such as M and B are discussed in the units of t in the following.H Λ (k) represents a T (time-reversal) symmetry breaking term which gaps the edge states and destroys the topological insulating phase.This leads to the formation of second-order topological insulators.
In this paper, we consider the external magnetization-induced rather than magnetic-field-induced Kerr and Faraday effects.Thus there are no Landau levels and the only consequence of magnetization is the Zeeman energy.For the HgTe quantum wells 2,43 , the Zeeman term reads Figure 1.Schematic illustration of magneto-optical Kerr and Faraday effects in 2D second-order topological insulators (SOTI) on a magnetic substrate.θ K and θ F are Kerr and Faraday angles, respectively.The reflected light is shifted a bit for better visibility.
g E and g H originate from different effective g factors of electronic orbitals |E1� and |H1� .Zeeman term H zeeman can be decomposed into orbital-independent part σ 0 s z and orbital-dependent part σ z s z .Here orbital-independent Zeeman term σ 0 s z can be neglected since it leads to a zero Hall response.Later we will demonstrate it by sym- metry analysis.As a result, we only need to consider the orbital-dependent Zeeman term, which is relabeled by H z in Eq. (1).
The symmetry properties of the Hamiltonian H(k) are summarized in Table 1.In the absence of Zeeman term H z , the Hamiltonian H(k) preserves the combined Ŝ4 = Ĉ4 Î and Ĉ4 T symmetries, whereas breaking Î , T and Ĉ4 symmetries, respectively.Either H z or σ 0 s z term will break the Ĉ4 T symmetry.Additonally, there is a "hidden" symmetry operation P = σ x s y K relating the states with momentum (k x , ±k y ) , which can be broken by H z rather than σ 0 s z .The effect of P on the Hall conductivity σ xy will be discussed in the following.
The optical conductivity tensor can be given by using the Kubo formula 44 where ǫ kµ and |k, µ� refer to the eigenvalue and eigenstate of Hamiltonian H(k) from Eq.
(1). µ, µ ′ = {1, 2, 3, 4} are band indices.At zero temperature, the Fermi-Dirac distribution , where ǫ F is the Fermi energy and �(...) is the Heavi- side function.ω is the photon energy and τ s is the relaxation time of bulk states.The contribution of edge states to τ s can be safely neglected when the light is shined away from the edge regions.Current operator reads j α = (e/ )∂H(k)/∂k α , with α, β = {x, y}.
When the orbital-independent Zeeman term gσ 0 s z is taken into account, the Hamiltonian H(k) pre- serves the P symmetry (see Table 1).As a result, the eigenstates with momentum (k x , ±k y ) satisfy the rela- tions ǫ kµ = −ǫ (k x ,−k y ) μ and |k, µ� = e iφ P|k x , −k y , μ� , where φ is arbitrary phase factor.Moreover, the sys- tem has the particle-hole symmetry ǫ kµ = −ǫ k μ , where µ and μ label a pair of particle-hole-symmetric bands.As an anti-unitary operator, P establishes the following relations between current matrix elements: As a consequence, we find that σ αα (k, ω) = σ αα (k x , −k y , ω) for α = x, y , and σ xy (k, ω) = −σ xy (k x , −k y , ω) .This indicates that the Hall conductivity σ xy (ω) = 0 when only the orbital-independent Zeeman term is considered.By contrast, the orbital-dependent Zeeman term H z breaks the P symmetry, thus gives rise to a nonzero σ xy (ω).When a light is propagating along −z direction into 2D second-order topological insulators deposited on a magnetic substrate (see Fig. 1), the Kerr and Faraday angles are defined as the relative rotations between left-and right-handed circularly polarized light: 30,31 where the electric field E (l) y and l = r, t refer to the reflected and transmitted light, respectively.The reflection (transmission) coefficients read (2) (3) Table 1.Symmetry of the Hamiltonian H(k) satisfying Ô−1 H(k) Ô = ηH( Ô−1 k) with operator Ô and η = ± .
For hidden symmetry operator P , P−1 where σ ± = σ xx ± iσ xy and Z 0 = cµ 0 = √ µ 0 /ǫ 0 = 376.7 is the impedance of vacuum.ǫ r and µ r are the dielectric constant and magnetic permeability, respectively.Then θ K and θ F can be obtained.See Methods for details of these calculations.Additionally, we can introduce the Kerr and Faraday ellipticity γ K , γ F : 45 By combining θ K , θ F and γ K , γ F , complex Kerr and Faraday angles can be introduced 40,46

Results
Here we show numerical results of optical conductivities, Kerr and Faraday angles and ellipticity for 2D chiral SOTI in the presence of out-of-plane magnetization.In the absence of magnetization, model Hamiltonian (1) describes a second-order topological phase ( ν = 1) 9,13,47,48 when 0 < M < 8B and a topologically trivial phase ( ν = 0 ) otherwise.An introduction of magnetization may affect the topological behaviors of the system, thereby inducing a Chern insulating phase 49,50 .The energy dispersions of Hamiltonian (1) are given by with the band index µ = 1, 2, 3, 4 .The bulk band gap between two middle bands closes at the high-symmetry momentum Ŵ = (0, 0) when g = |M| ; at M = (π, π) when g = |M − 8B| ; at X = (π, 0) and Y = (0, π) when g = (M − 4B) 2 + 4Λ 2 .As a result, diverse topological phases with different Chern number C can be realized by tuning the parameters.The phase diagrams of model Hamiltonian (1) are shown in Fig. 2, where both the Chern number C and second-order topological invariant ν are provided.For different regimes of parameters, the phase diagrams can be quite different.To make the discussion explicit, we mainly focus on two regimes of parameters: weak and strong magnetization case, corresponding to case (a) and (e) of Fig. 2.
A. Weak magnetization.First we consider the case with weak magnetization, corresponding to Fig. 2a.In this case, X = (π, 0) and Y = (0, π) are no longer gap closing points for any given parameters.As a result, the Chern number becomes C = 1 when −g < M < g or 8B − g < M < 8B + g , and C = 0 otherwise.To check the topological properties, we plot the energy spectrum and wave function distribution of finite-size samples in Fig. 3 for parameters: M/t = 1 , 0 and −1 with B/t = 0.25 .In the absence of magnetization, these parameters (5) www.nature.com/scientificreports/correspond to the SOTI, semimetal and trivial phase, respectively.When magnetization is induced, according to Fig. 2a, these parameters correspond to the SOTI, Chern insulating and trivial phase, respectively.In Fig. 3a and b, we can see the existence of zero-energy corner states.In Fig. 3c and d, we can see the existence of gapless edge states.Such real-space calculations prove our results of phase diagram.The band dispersions along the high-symmetry lines of Brillouin zone are shown in Fig. 4. Different values of M are considered, and Chern numbers are also labeled.Note that the model shows symmetric behaviors between parameters M > 4B and M < 4B , thus we only choose parameters with M ≤ 4B , including M/t = 1 , 0 and −1 .T i ( T o ) labels the optical transitions for two inner (outer) branches of bands.Remarkably, there are new crossings in both conduction and valence bands of SOTI in the Ŵ − M direction (see Fig. 4a), which are absent in the trivial phase.The topological protection of band crossings can be understood by noting that in the Ŵ − M direction (i.e., k x = k y ), H Λ (k) = 0 for Hamiltonian (1).Thus the model reduces to that of topological insulators.For topological insulating phase ( 0 < M < 8B ), the bands are inverted at the Ŵ or M point, leading to the band crossings between them.For trivial phase, there are no band inversions or crossings.
The real and imaginary part of optical conductivities σ xx and σ xy are plotted in Fig. 5, where for convenience we set the Fermi energy E F = 0 .A striking difference between SOTI ( M/t = 1 ) and trivial insulators ( M/t = −1 ) lies in their order of magnitude.In SOTI, σ xx and σ xy are enhanced due to the existence of additional channels of interband transitions.Threshold photon energies for the interband transitions T i and T o are indicated by arrows in Fig. 5.At these transitions, Re[σ xx ] and Im[σ xy ] show sudden jumps while Re[σ xy ] and Im[σ xx ] show positive or negative peaks.For example, in Fig. 5a, Re[σ xx ] show sudden jumps for M/t = ±1 at ω/t = 1.2 due to the activation of inner interband transitions T i .At ω/t = 2.8 , another jumps occur due to the activation of outer interband transitions T o .For moderate photon energy ω , the magnitude of Re[σ xx ] for SOTI ( M/t = 1 ) becomes much larger than trivial insulators ( M/t = −1 ).This is attributed to the crossing points along the ŴM line of the Brillouin zone of SOTI (see Fig. 4a), which induces new channels of interband transitions at some non-high-symmetry momentum along the ŴM line.Moreover, the states at the high-symmetry point M = (π, π) have non-negligible contributions due to the band degeneracy between Ŵ and M.These together contribute to the large magnitude of Re[σ xx ] in SOTI.Similar arguments can be given to Im[σ xy ] (see Fig. 5d).On the other hand, Im[σ xx ] and Re[σ xy ] are proportional to the slope of Re[σ xx ] and Im[σ xy ] , respectively, thereby exhibiting giant jumps near the small peaks at ω/t = 2.8 (see Fig. 5b and c).At even higher photon energy, the magnitude of σ xx and σ xy is greatly reduced due to the closure of optical interband transitions.
According to Eq. ( 8), the conditions for the occurrence of crossing points are given by m(k and the giant jump of Im[σ xx ] , Re[σ xy ] may potentially be used to characterize the SOTI phase.However, such argument is not applicable for the critical value M = 0 , in which case the magnetization drives the system into Chern insulators with Chern number C = 1 (see Fig. 4b).In this situation, the Chern insulating phase can be distinguished by the integer Hall conductivity in the low-energy limit, that is, Re[σ xy ] = e 2 /h as highlighted in Fig. 5c.
The Kerr and Faraday angles θ K , θ F and ellipticity γ K , γ F are plotted in Fig. 6.It is manifest that θ K and θ F (also γ K and γ F ) are complementary to each other.Basically, θ F ( θ K ) shows the same (opposite) behaviors as Re[σ xy ] in Fig. 5c.This can be understood from Eq. ( 5), where Z 0 σ ± ≪ 1 can be treated as perturbations.After some algebra, we have θ F ∝ −θ K ∝ Re[σ xy ] .This means that Kerr and Faraday angles inherit the properties from Hall conductivity Re[σ xy ] , and hence can also be used to characterize the SOTI.γ K and γ F seem more likely to inherit  the properties from Im[σ xy ] , which, together with the Kerr and Faraday angles, can be adopted to distinguish SOTI from trivial insulators.
B. Strong magnetization.Now we consider the case with strong magnetization, corresponding to Fig. 2e.We consider two representative parameters: M/t = 4 and M/t = −6 .In the absence of magnetization, they cor- respond to the SOTI and trivial phase, respectively.When strong magnetization is induced, the band structure is modified greatly, and M/t = 4 now reduces to trivial insulating phase.However, we reveal in the following that M/t = 4 and M/t = −6 have distinct optical features as they originate from different topological phases in the absence of magnetization.
The band dispersions along the high-symmetry lines of Brillouin zone are shown in Fig. 7, where two representative parameters are considered: M/t = 4 (SOTI) and M/t = −6 (trivial).We find that similar to the case with weak magnetization, there are crossings in both conduction and valence bands of SOTI, which are absent for trivial phase.The threshold optical transitions for two inner (outer) branches of bands are labeled as T i/o at Ŵ point, X i/o at X and R i/o at the crossing points.At the Ŵ or M point of Brillouin zone, transitions T i/o are allowed since the initial and final states share the same spin angular momentum.At the X point, H Λ (k) from Eq. ( 1) mixes states with different spins, despite the fact that inner and outer states are orthogonal to each other.As a result, only transitions X i/o within inner or outer states are allowed.
The optical conductivities σ xx and σ xy are plotted in Fig. 8, where the interband transitions contributing to the peaks and jumps are indicated by arrows.We find that SOTI show larger peaks of Re[σ xx ] and jumps of Im[σ xx ] from optical transitions R i/o than trivial insulators.Nevertheless, their differences are much smaller than those in the weak-magnetization case.This is due to the loss of degenerate channels of optical transitions driven by strong magnetization.On the other hand, strong magnetization induces nearly flat bands at X point for SOTI (see Fig. 7a), which still gives rise to giant peaks and jumps of optical conductivities as a result of the enhanced joint density of states for optical transitions X i/o (see Fig. 7c).This may provide another way to distinguish SOTI from trivial insulators.
The resulting Kerr and Faraday angles and ellipticities are plotted in Fig. 9.There are two successive giant jumps (peaks) in both θ K and θ F ( γ K and γ F ) originating from optical transitions X i/o for SOTI.By contrast, there is only one small jump or peak from optical transitions X i for trivial insulators.Compared with weak magneti- zation, strong magnetization tends to suppress the magnitude of Kerr and Faraday angles and ellipticity.The reduction of Kerr and Faraday angles and ellipticity is due to the enhancement of band gaps modified by strong magnetization.This leads to the suppression of optical Hall conductivities, thus the reduction of Kerr and Faraday rotations.This reduction under strong magnetization is different from the general view of magneto-optical effects due to strong magnetic field, where Landau levels are formed.Here the magnetization does not induce Landau levels, but just modifies the band structure.However, even for the reduced Kerr and Faraday angles and ellipticity, they are still within the experimental reach.

Discussions and conclusions
Numerical results are mainly based on the model parameters of HgTe quantum wells.t = 0.06 eV and M/t = 1 are within experimental reach by tuning the quantum well thickness 2,43 .The strong magnetization regime requires that g/B > 4 , suggesting that g > 0.1 eV.This can be realized in Mn-doped HgTe quantum wells under strong magnetic field 51 , Cr-doped (BiSb) 2 Te 3 thin film 52 or monolayer MoTe 2 on EuO substrate 53 .The photon energy ranges from 0.01 eV to 0.6 eV, corresponding to the terahertz and far infrared frequencies [32][33][34]54 . In he weak www.nature.com/scientificreports/magnetization regime, the rotation angles are tens of mrad, which share the same order of magnitude with experimental results of Bi 2 Se 3 on Al 2 O 3 substrate 32 .In the strong magnetization regime, the rotation angles become a few mrad, in the same order of magnitude with experimental results of strained HgTe and Bi 2 Se 3 on InP substrate 33,34 .Our studies can also be applied to other proposed 2D SOTI, such as graphdiyne 26 , Bi on EuO substrate 27 and monolayer FeSe 28 .
To realize SOTI from magnetic doped TI, the existence of H Λ (k) term in Eq. ( 1) is essential.According to Table 1, H Λ (k) term breaks T and Î symmetries while preserving the Ĉ4 T and Ĉ4 Î symmetries.Physically, H Λ (k) term can be realized in two possible ways.One is to induce orbital currents that break time-reversal sym- metry oppositely in the x and y direction.The other is to induce ( π , π , 0) noncollinear antiferromagnetic order in the system.Details of this issue are worthy further study.b), (d) M/t = −6 .The optically-induced inner (outer) interband transitions T i/o at Ŵ , X i/o at X and R i/o at the crossing points are depicted by double arrows.In the absence of magnetization, M/t = 4 and −6 correspond to SOTI and trivial insulator, respectively.In the presence of magnetization, the Chern number in each case is indicated.Parameters: t = 0.06 eV 43 , B/t = 1 , Λ/t = 0.5 , g/t = 5.  and (c-d) σ xy (in units of e 2 /h ) as functions of photon energy ω (in units of t) for 2D SOTI with strong magnetization.The arrows label the energies of optically-induced inner (outer) interband transitions T i/o , X i/o and R i/o .Parameters: t = 0.06 eV 43 , B/t = 1 , Λ/t = 0.5 , g/t = 5 , ǫ r = 4 31,45 , µ r = 1 , /τ s = 0.05 , E F = 0.
Note that in our setup (see Fig. 1), semi-infinite magnetic substrate is assumed for simplicity.For realistic samples, the role of substrate thickness should be taken into account.Following the previous work 31 , we know that the results are independent of the substrate properties when the substrate thickness is much smaller than the light wavelength in the low-frequency limit.By increasing the thickness, the magnitude of Kerr and Faraday angles is suppressed.Particularly, when the resonance conditions are satisfied, that is, the substrate thickness contains an integer number of half wavelengths, Kerr and Faraday angles show Fabry-Perot-type oscillations and again become independent of the substrate properties.
Since ellipticity is a dispersive effect and sensitive to the distortions of dielectric tensor, it may not be suitable to detect 2D SOTI.Ellipticity may serve as a supplementary to Kerr and Faraday angles, which together can be used to characterize the 2D SOTI phase.Additionally, the comparison between ellipticity and rotation angles can provide information of distortions or inhomogeneities of the system.
Magneto-optical Kerr and Faraday effects have also been studied in topological insulators 30 and Floquet topological insulators 55 .By contrast, we are concentrated on the SOTI with proximity magnetization, rather than introducing the Landau levels 55 .Rashba spin-orbit interaction may further be introduced at the interface as a result of inversion symmetry breaking.This may modify the spin and pseudospin structures of electronic bands, inducing additional channels for optical interband transitions.The presence of impurity scattering affects the relaxation time τ s in Eq. ( 3), which gives rise to a broadening of peaks and jumps for Kerr and Faraday angles and ellipticity 31 .For surface states of topological insulators Bi 2 Se 3 and Bi 2 Te 3 56,57 , hexagonal warping term is present.This term may modify the interband transitions, Fermi velocity and density of states, leading to a quasilinear shape of Re[σ xx ] with a concave upward bent 58,59 .Our discussion focuses on the zero-temperature limit, and the increasing temperature tends to suppress the magnitude of peaks and jumps of Kerr and Faraday rotations 33 .However, as long as the temperature is not high enough, the main features should still be observable.
To conclude, we have studied the magneto-optical Kerr and Faraday effects in two-dimensional secondorder topological insulators.By symmetry analysis, we find that to observe the Kerr and Faraday effects in such systems, Zeeman term must be orbital dependent, rather than orbital independent.The magnetization induces new crossings in conduction and valence bands only in the SOTI phase.In the regime of weak magnetization, these crossings lead to giant peaks of Re[σ xx ] , Im[σ xy ] and giant jumps of Im[σ xx ] , Re[σ xy ] .As a result, Kerr and Faraday angles (ellipticity) θ K and θ F ( γ K and γ F ) show giant jumps (peaks) only in the SOTI phase.In the regime of strong magnetization, nearly flat bands are formed at X point for SOTI.These give rise to two successive giant peaks of Re[σ xx ] , Im[σ xy ] and giant jumps of Im[σ xx ] , Re[σ xy ] .In this sense, Kerr and Faraday angles (ellipticity) θ K and θ F ( γ K and γ F ) show giant jumps (peaks) only in the SOTI phase.These phenomena may potentially be used to distinguish the SOTI from trivial insulators.Note that our proposal may not be applicable to the regime close to the topological phase boundary, such as M = 0 , which may be driven into Chern insulating phase under magnetization.

Method
We consider a light propagating along −z direction from the vacuum into a 2D material (at z = 0 ) deposited on a magnetic substrate (see Fig. 1).In the vacuum ( z > 0 ), the electric field of incident light reads ( 9) , µ r = 1 , /τ s = 0.05 , E F = 0.
where ω and c refer to the energy and speed of light in the vaccum, respectively.For the reflected light, the electric field reads In the magnetic substrate ( z < 0 ), the electric field of transmitted light reads where the refractive index n r = √ ǫ r µ r .ǫ r and µ r are the dielectric constant and magnetic permeability, respec- tively.According to the Faraday's law ∇ × E = −∂B/∂t , the magnetic field of light follows Based on the Maxwell's equations, the boundary conditions at z = 0 are given by where the current density in the 2D material satisfies the relations j α = β=x,y σ αβ E β .ǫ 0 and µ 0 are vacuum permittivity and permeability, respectively.By substituting the forms of electric and magnetic field into above equations, we can obtain the relations of coefficients from Eqs. ( 9)- (11).Now we introduce a scattering matrix between incoming and outgoing electric fields by where and similarly for R ′ , T ′ .The detailed form of R, T can be determined by using the boundary conditions (13).As a result, we find that where D = ( 1 xy and Z 0 = cµ 0 = √ µ 0 /ǫ 0 = 376.7� is the impedance of vacuum.In the derivation, we have used the relations σ xx (ω) = σ yy (ω) and σ xy (ω) = −σ yx (ω) , which are appropriate for our system.According to the definition of Kerr and Faraday angle from Eq. ( 4), we need where σ ± = σ xx ± iσ xy .This reproduces the results in Eq. (5).

Data availability
On reasonable request, the corresponding author will provide all relevant data in this paper.

Figure 2 .
Figure 2. Phase diagram of model Hamiltonian (1) versus M for different regimes of parameters.C and ν are the Chern number and second-order topological invariant, respectively.g Λ = g 2 − 4Λ 2 .

Figure 7 .
Figure 7. Band dispersions and density of states (DOS) of 2D second-order topological insulators (SOTI) with strong magnetization for parameters (a), (c) M/t = 4 and (b), (d) M/t = −6 .The optically-induced inner (outer) interband transitions T i/o at Ŵ , X i/o at X and R i/o at the crossing points are depicted by double arrows.In the absence of magnetization, M/t = 4 and −6 correspond to SOTI and trivial insulator, respectively.In the presence of magnetization, the Chern number in each case is indicated.Parameters: t = 0.06 eV43 , B/t = 1 , Λ/t = 0.5 , g/t = 5.