Proximity Effect induced transport Properties between MBE grown (Bi1−xSbx)2Se3 Topological Insulators and Magnetic Insulator CoFe2O4

In this study, we investigate the proximity effect in topological insulator (TI) and magnetic insulator bilayer system. (Bi1−xSbx)2Se3/CoFe2O4 (CFO) heterostructure was fabricated using molecular beam epitaxy and pulsed laser deposition system respectively. As revealed from the magnetoresistance measurement, the weak anti-localization (WAL) is strongly suppressed by proximity effect in (Bi1−xSbx)2Se3/CFO interface. Modified Hikama-Larkin-Nagaoka equation was used to fit the WAL results so that the size of surface state gap can be extracted successfully. The temperature-dependent resistance of the heterostructures at small and large perpendicular magnetic fields were also measured and analyzed. The results indicate that the surface band gap can be induced in TI and continuously enlarged up to 9 T, indicating the gradual alignment of the magnetic moment in CFO under perpendicular magnetic field. The approaches and results accommodated in this work show that CFO can effectively magnetize (Bi1−xSbx)2Se3 and the heterostructures are promising for TI-based spintronic device applications.

interface between TI and magnetic insulator layers. For this reason, the angle-resolved photoemission spectroscopy is not an appropriate way to observe the proximity effect since only the electronic structure of top surface (a few nanometers thickness) can be measured.
Another method to observe the surface gap opening is magnetoresistance (MR) measurement. During the past few years, numerous literature have focused on MR transport and observed the suppression of weak anti-localization 11,12 However, those studies did not deliberate the gap size of surface state directly. To our knowledge, only limited studies discussed and used the modified Hikama-Larkin-Nagaoka (HLN) [20][21][22][23] to obtain the gap size. Therefore, in proximity effect of TI/MI, it's a challenge to research on the gap opening in the TI surface state. Meanwhile, in low temperature range, both the quantum interference (contributed by the WAL and the weak localization (WL)) and electron-electron interaction strongly influence the temperature-dependent conductivity under external magnetic fields (σ (T, B)). In other words, analysis on σ(T, B) may also provide information regarding the surface gap opening, when massive Dirac fermions are considered. However, to our knowledge, there is still no experimental work clearly identifies this issue about the relationship between the slope change of σ(T, B) and the induced surface gap.
In this paper, we report the transport properties of (Bi 1−x Sb x ) 2 Se 3 /CoFe 2 O 4 heterostructure, which fabricated using molecular beam epitaxy and pulsed laser deposition system, respectively. CoFe 2 O 4 (CFO) has high Curie temperature and is an ideal insulating material that suitable to magnetize TI layer and use for MR measurement of the bilayer structure. Bi 2 Se 3 is a prototype three-dimensional topological insulator and has been the subject of extensive research. However, most Bi 2 Se 3 films suffer a serious problem, which has high n-type bulk carrier concentration due to the presence of Se vacancies that act as donors. Sb doping into Bi 2 Se 3 can effectively reduce the vacancies of selenium and anti-site of bismuth and selenium. For this reason, Sb doping can greatly reduce carrier concentration of Bi 2 Se 3 24, 25 and here we choose Sb-doped Bi 2 Se 3 as TI layer. After all, we analyzed and fitted the MR using modified HLN equation. The results show that CFO can magnetize TI layer and open the gap of TI. The phase coherence length l φ of TI also decreases, which indicate that the electron of surface state strongly scattered by magnetic moment. Moreover, we measured and fitted the temperature dependence of conductivity at different magnetic fields in low temperature regime. The results indicate that the slopes κ reveal two different behaviors in small and large perpendicular magnetic field. In the small magnetic field, the κ will sharply increase with increasing magnetic field, which consistent with literature. However, in large magnetic fields, the κ decreases with increasing magnetic field. These results are significantly different with single layer results, which can be attributed to the continual enhancement of the surface gap size in the large magnetic field. Our findings should be useful for the future studies of magnetic TI and TI-based spintronic devices.

Results and Discussion
For the structural analysis, Fig. 1(a) shows the X-ray diffraction of the single layer Bi 2 Se 3 , CoFe 2 O 4 , and bilayer (Bi 1−x Sb x ) 2 Se 3 /CFO samples. All peaks can be identified with (00n) diffraction peak of (Bi 1−x Sb x ) 2 Se 3 , while CFO shows the (111) series diffraction peak. No other phases were observed in the heterostructure samples, which indicates that Sb was perfectly doped into Bi 2 Se 3 in all the samples. Meanwhile, magnification of the diffraction peaks around 57.5° (show on the right inside the dashed rectangle) also reveal that the ideal interface and layer structure can be formed in (Bi 1−x Sb x ) 2 Se 3 /CFO heterostructures. Figure 1(b) shows the RHEED pattern of (Bi 1− x Sb x ) 2 Se 3 layer, further confirming the high crystalline quality of the (Bi 1−x Sb x ) 2 Se 3 /CFO heterostructures samples. The structural analysis reveals that all (Bi 1−x Sb x ) 2 Se 3 /CFO heterostructures are ideal for electrical transport measurement.
Next we focus on transport properties of (Bi 1−x Sb x ) 2 Se 3 /CoFe 2 O 4 heterostructures. Figure 1(c) schematically illustrates the device configuration of (Bi 1−x Sb x ) 2 Se 3 /CoFe 2 O 4 heterostructures for MR measurement. The Hall bar was patterned using photolithography, which is observed by optical microscope as shown inside the dashed circle. Figure 2(a) presents the normalized magnetoconductance (MC) of the single layer (Bi 2 Se 3 ) sample (squares) and bilayer ((Bi 1−x Sb x ) 2 Se 3 /CFO) sample (circles) at 2 K in the low magnetic field region. With perpendicular magnetic field, the MC of Bi 2 Se 3 single layer exhibits weak anti-localization (WAL) behavior with a sharp cusp at low field region. In contrast to single layer Bi 2 Se 3 , the bilayer samples illustrate that the MC cusp feature disappears completely which means WAL was suppressed by being placed in proximity with the CFO. One of the origin for the suppression of WAL in TI is the enhancement of sheet resistance R s , where disorder could drive the crossover from quantum diffusive into strong localization regime 26,27 . Here we estimate the disorder level by taking dimensionless conductivity, g ≡ σ/(e 2 /h) where σ = 1/R s . Even at intermediate disorder regime (1<g<3), deviation of α from the ordinary value (≥|0.5|) could be observed 26,27 . However, our transport data reveals a g ≥16 (see Supplementary Figure S1c), which fulfills the condition for quantum diffusive transport (where g>>1). We thus attribute the observed MC to the quantum interference effect 28 . To understand the origin of weakened WAL cusp in bilayer samples, we carry out a quantitative analysis of low-field MC data. The modified HLN equation 22,23 has been manipulated to fit the MC data, where Ψ is digamma function, the magnetic length = l e B /4 2 and l φ is phase coherence length. The modified HLN equation has two terms, in general, one is for weak localization (WL, i = 0) and the other is for weak anti-localization (WAL, i = 1), which contributes two groups parameter α 0 , l 0 and α 1 , l 1 . In the WAL limit, α 0 = 0, α 1 = −0.5 while in the WL limit, α 0 = 0.5, α 1 = 0. α 0 , α 1 , l 0 , l 1 are all related to cos θ = Δ/2E F , where Δ is the gap size of the surface state and E F is the Fermi level. Figure 2( between ±0.5 T. First, we can easily observe that all the extracted Δ/2E F are above zero, which means that (Bi 1−x Sb x ) 2 Se 3 layer has been magnetized by CoFe 2 O 4 and a surface gap was induced in (Bi 1−x Sb x ) 2 Se 3 . To estimate the surface gap size, we use (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO for the calculation since the surface E F (n 2D ∼ 7.8 × 10 12 cm −2 ) was below the conduction band edge (Fig. S1). By taking 2D Fermi wave vector of k F = (2πn 2D ) 1/2 , we estimate E F 29, 30 and calculate the surface gap where ∆ = .
≈ . E m eV 2 /0 132 51 F Interestingly, the Δ/2E F increases at the beginning, but decreases with continuously increasing the Sb ratio. As shown in the Hall measurement (Fig. S1), the E F was actually shifted down in Bi 2 Se 3 due to the Sb doping (with reduced carrier concentration). Therefore, the reduction of Δ/2E F implies that the Δ should become smaller with increasing the Sb doping level.
To further elucidate the Sb doping effect to the MR behavior, the fitted phase coherence length l φ vs. Sb ratio is examined. As shown in Figure 2(c), l φ reduces with increasing the Sb ratio, but the tendency has been slowed down at high Sb ratios. The decreasing of l φ may indicate strong magnetic scattering occurred at the interface of (Bi 1−x Sb x ) 2 Se 3 /CoFe 2 O 4 that enhanced as a result of Sb doping. This observation suggests that the magnetic proximity effect has been enhanced in these heterostructures. Nevertheless, the Δ shows completely different trends as described above. This sounds counter-intuitive, since Sb is a non-magnetic dopant that is not expected to have influence on Δ. Here we propose a scenario as follow: in 3D TI layer, not only the bottom but also top surface state (SS) can contribute in MR behaviors. Owing to the proximity effect, the bottom SS is magnetized, leading to the TRS breaking induced surface gap, as indicated in the WL behavior. However, the top SS remains gapless, presenting spin-momentum locked (SML) massless Dirac fermions immune to backscattering (that resulting in WAL). Therefore, we deduce that the two transport mechanisms at different surface states are competitive. In our previous Sb doped Bi 2 Se 3 studies, we have observed the phenomena where WAL increases with increasing Sb doping ratio due to the reduction of carrier concentration 31 . We thus suggest the slight decrease of the Δ with further increasing Sb doping ratio was attributed to the enhanced dominance of the WAL (top SS) over WL behavior (bottom SS). This finding indicates that there might be an interplay between the SML spin carriers and the magnetized interfacial spins. We summarize the extracted α 0 and α 1 as a function of Sb doping ratio as shown in Fig. 2(d,e) respectively. In our results, we can observe α 1 ∼ −0.44 and α 0 ∼ 0.08, revealing again that TI has been magnetized by CFO because α 1 and α 0 are more than −0.5 and 0, respectively. These results indicate that the proximity effect can slightly suppress WAL, which is consistent with the above-proposed scenario. Therefore, we deduce that the induced magnetic moments of (Bi 1−x Sb x ) 2 Se 3 may not contribute in the perpendicular direction. The dependence of Δ/2E F on Sb doping ratio further implies CFO may induce more in-plane magnetic moments in TI layer when increasing Sb content. which is the critical magnetic field characterized by the phase coherent length l φ . Therefore, the modified HLN equation can only describe low magnetic field range. Following, we employ the temperature-dependent resistance of the heterostructures at small and large perpendicular magnetic fields to further investigate the proximity effect [32][33][34] . Figure 3(a,b) show the temperature-dependent conductivity of Bi 2 Se 3 /CFO and (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO respectively, where normalized conductivity vs. lnT at different perpendicular magnetic fields (from 0 T to 9 T) are plotted. We fitted the low temperature range data to get slope κ, which defines as κ = (πh/e 2 )∂σ/∂lnT and plotted as a function of B shown in Fig. 3(c,d) for Bi 2 Se 3 /CFO and (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO, respectively. Figure 3(c,d) shows the similar tendency, where κ at first increases in low magnetic fields, but decreases with further increasing the magnetic fields. Lu et al. 32 had pointed out that κ are dominated by two different effects. One is the electron-electron interaction (EEI) and the other is quantum interference (QI) effect. In low field range, both QI and EEI contribute to κ. However, QI can be destroyed by further increasing perpendicular magnetic field and leads to increasing of κ. In our result, κ of Bi 2 Se 3 /CFO and (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO are saturated in 0.5 T and 1 T respectively, which is determined by the magnitude of l φ . In high field range, only the EEI contribute to κ and the slope of EEI can be described as κ EEI = 1−η Λ F, where η Λ = (1 + cos 2 θ)/2 and F is electron screening factor. In general, the κ EEI should be constant in magnetic fields. However, in our data, the κ decreases at high magnetic fields, which means that the η Λ and the F must be influenced by magnetic fields. The theoretical studies indicate that η Λ and F are functions of the surface gap 32 . Therefore, the decreasing of κ at high magnetic fields is due to the proximity-induced surface gap, revealing that the CFO has induced more magnetic moments in TI layer at high magnetic field. Considering the magnetic anisotropy of CFO, here we propose a mechanism for the high field enhanced proximity effect, as explained in the following section. Figure 4 schematically illustrates the relationship between surface gap size and magnitude of perpendicular magnetic fields of (Bi 1−x Sb x ) 2 Se 3 /CoFe 2 O 4 heterostructures. Because the CFO easy axis is at in-plane direction ( Figure S3), the magnetic moments are more likely to lie in plane at small external perpendicular magnetic fields. Consequently, only weak magnetization of topological insulator and smaller band gap is induced. While the perpendicular external magnetic field further increases, the magnetic moments of CFO align to the external B field gradually. Due to the larger perpendicular component of magnetic moments, the magnetization of topological insulator would be stronger and the surface gap opening would be extensive. Therefore, we can observe the phenomenon of surface band gap opening through analysis of temperature-dependent conductivity at different magnetic field.
Controlling the TI surface state gap 35 is a prominent process for spintronic and related devices [36][37][38] . For instance, by varying the surface gap using perpendicular magnetic fields, spin texture and the spin polarization of TI's surface state can be effectively tuned. This tunability can be utilized in spintronic devices because the spin polarization rate is the key for spin transport. In addition, heterojunctions of magnetic-doped TI and magnetic insulator are promising candidates for proximity-induced high-temperature magnetic order in TI channel. Our results and methodology of fabricating the TI/MI heterostructures provide not only a platform for investigating the surface gap opening in TI, but a versatile method to explore the practical applications based on magnetic topological insulator.

Conclusion
In conclusion, we successfully grew (Bi 1−x Sb x ) 2 Se 3 /CFO heterostructures on the c-plane sapphire substrate and the crystal orientation is well defined by XRD. In the bilayer system, the proximity effect can be observed clearly via magnetoresistance measurements. The gap size per E F of surface state can be obtained using modified HLN equation fitting for all bilayer samples, where the surface gap is estimated about 51 meV for (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO. In large perpendicular magnetic fields, analysis on the temperature-dependent conductivity reveals that the gap size can be enlarged by increasing external B field, which is caused by the magnetic moments of CFO rotating from in-plane to out-of-plane. This work offers a method to control the gap size of TI surface state in (Bi 1−x Sb x ) 2 Se 3 /CFO heterostructures by magnetic proximity effect.

Materials and Methods
The CFO films were grown on c-plane sapphire in a 1.5 × 10 −5 torr O 2 ambient at 750 °C by pulse laser deposition (PLD) system and the thickness of CFO films were around 50 nm. The CFO film were then in situ transferred from PLD to molecular beam epitaxy (MBE) system in UHV condition. 15 QL-thick topological insulator (Bi 1−x Sb x ) 2 Se 3 layers were grown on the CFO layer at 290 °C with base pressure of ∼1 × 10 −10 torr. Surface morphology was monitored in-situ by reflection high energy electron diffraction (RHEED). After growth, the samples were capped with 2 nm Se for protecting the TI layer in MBE chamber. We varied the Sb doping ratio of (Bi 1−x Sb x ) 2 Se 3 /CFO samples with five different Sb fractions, namely x = 0, 0.11, 0.22, 0.28, 0.35. The crystal structure of all samples was confirmed by X-ray diffraction (XRD). To investigate the magnetic response and electric properties of (Bi 1−x Sb x ) 2 Se 3 /CFO bilayer structure, the samples were patterned into standard Hall bar devices with 100 μm length and 50 μm width by photolithography. The Hall effect measurements at 2 K indicate that the carrier concentration of the Bi 2 Se 3 /CFO and (Bi 0.65 Sb 0.35 ) 2 Se 3 /CFO samples were ∼2.97 × 10 13 cm −2 and 7.8 × 10 12 cm −2 respectively ( Figure S1). Four-probe magneto-transport and temperature-dependent resistance measurements were conducted using Physical Properties Measurement System (PPMS).