## Introduction

The quantum anomalous Hall effect (QAHE), i.e., quantum Hall effect (QHE) without external magnetic field, is of great fundamental interest and of potential application in metrology.110 Similar to QHE, the microscopic mechanism of QAHE is due to the presence of dissipationless chiral edge states. The realization of QAHE requires two criteria: the breaking of time reversal symmetry and topologically nontrivial band structure. In principle, there are several ways to realize the QAH state. So far, introducing ferromagnetism into a topological insulator (TI) is the only viable route to realize the fascinating QAH state. The ferromagnetic behavior has been observed by doping TIs with various transition metal elements, such as Cr, V, and Mn.1117 Yet, it is challenging to control the Fermi level in the exchange gap region in these materials so that the transport is dominated by the chiral edge states for observing clear quantized Hall conductance. The breakthrough was accomplished by Chang et al., who demonstrated QAHE (at 30 mK) in Cr-doped BixSb2−xTe3 (BST) thin films synthesized by molecular beam epitaxy,18 soon confirmed by several other groups.1922 However, the QAH state is very fragile in the Cr-doped BST thin films, where quantized Hall conductance was observed at mK temperature. It was shown that modulation doping of Cr can enhance QAH temperature.23 On the other hand, a high precision robust QAH state was observed in V-doped BST thin films24 at a higher temperature (~130 mK) with the same doping level. V-doped BST is a hard ferromagnet with a larger coercive field (HC~1.0 T) and higher Curie temperature (TC).24 These features make the V-doped BST thin films, if combined with modulation doping, more preferable to realizing QAHE at higher temperature.

In addition to the robust QAH effect, a remarkable spontaneous self-magnetization effect has been reported in V-doped BST thin films.24 The non-zero net magnetization develops in the virgin state without the assistance of external magnetic fields, which is not commonly seen in conventional ferromagnetic materials with some exceptions in soft magnetic thin films.25,26 In contrast, slow relaxation behaviors were observed in transport studies, indicating superparamagnetism due to inhomogeneity.27,28 The superparamagnetic dynamic behavior of the quantum phase transition between the two anomalous Hall plateaus was visualized in Cr-doped BST thin films using scanning nanoSQUID (nano superconducting quantum interference device).29 However, recent transport measurements at lower temperatures revealed large discrete jumps near the coercive field, suggesting domain-like behavior.30 It is not clear whether this superparamagnetic behavior is universally favorable in other magnetic TI thin films such as V-doped BST. In order to resolve these mysteries, we utilized magnetic force microscopy (MFM) technique to investigate the domain behavior of V-doped Sb2Te3 thin films.

In this Article, we report cryogenic MFM studies of the domain states of Sb1.89V0.11Te3 thin films. A zero magnetization state with equal population of up and down domains was observed after zero field cooling (ZFC). Our MFM results reveal that a small external magnetic field (approximately 5–10 Oe) is sufficient to polarize the magnetization of the film, which provides a physical explanation of the ‘self-magnetization’ mystery.24 The observed domain states are consistent with in situ magnetoresistance measurements, suggesting that our MFM observation is representative of the bulk properties. In contrast to the superparamagnetic behavior in previous reports of Cr-doped BST, our MFM results reveal a typical ferromagnetic domain behavior of the magnetization reversal process via nucleation and domain wall propagation in V-doped Sb2Te3 thin films. Since the ferromagnetism in V doped BST varies little with Bi doping, our MFM results provide supporting evidence of the correlation between improved ferromagnetism and enhanced QAHE.24

## Results and Discussion

Figure 1 shows topographic and MFM images of the Sb1.89V0.11Te3 thin film at 5 K after ZFC and field cooling (FC). The topographic image shows a flat surface with roughness ~1.9 nm. The topographic features are likely surface contamination after the film was exposed to air. The virgin state after ZFC, as shown in Figure 1b, exhibits a multi-domain state which is unlike the reported self-magnetization behavior. Figures 1c–g show MFM images of the same region at 5 K after FC. The sample was thermally cycled to 60 K, then cooled down to 5 K with the superconducting magnet set to the field values labeled on the upper left corner of each panel in Figure 1. As shown in Figure 1c, the nominal 0 Oe FC data shows that the sample is slightly negatively polarized, indicating the existence of negative trapped flux in the superconducting magnet. This is consistent with the history of the superconducting magnet usage, since the field was ramped down from a positive value to zero, i.e., negative trapped flux due to a slight overshot of the power supply. Applying a small field (e.g., 5 Oe) during cooling can significantly polarize the film (see Figure 1d) and the film saturates at 100 Oe (see Figure 1g), indicating divergent correlation length at TC due to strong ferromagnetic exchange interactions. The systematic variation of domain population is a strong evidence of long-range ferromagnetic order in V-doped Sb2Te3.

In order to quantitatively characterize the domain states after different field cooling, the population of positive and negative domains was estimated by conducting a histogram analysis of these MFM images. The histogram curves exhibit two peaks which correspond to up and down domains (Supplementary Figure S2). We used the peak height to estimate the population of up and down domains. The normalized magnetization of the thin film M/MS (MS: saturation magnetization) therefore can be estimated from: (NN)/(N+N), where N and N are populations of up and down domains, respectively. The normalized magnetization as a function of cooling field is plotted in Figure 1h. The trapped flux of our superconducting magnet inferred from linear extrapolation is ~3 Oe. Therefore, the previously observed self-magnetization effect is likely an extrinsic effect due to non-zero trapped flux in the superconducting magnet.24

The ferromagnetic domain behavior was also investigated by performing field-dependent MFM imaging of the domain state after ZFC, as shown in Figure 2. The ZFC domain state is essentially unchanged up to 0.4 T, indicating a strong domain wall pinning effect of a hard ferromagnet.24 The domain contrast enhancement between 0 and 0.4 T is due to the enhancement of the MFM tip moment by the external magnetic field. Higher magnetic fields induce more red regions (up domains), where spins are parallel with external fields. The film reaches saturation at 1.5 T, as no blue region (down domain) remains. There are some weak MFM contrasts in the saturation state, which anti-correlate with topographic features shown in Figure 1a. So they are non-magnetic features, possibly originating from residual electrostatic interactions due to the variation of contact potential caused by surface contamination. The film remained in the saturation state after the field was ramped down to zero, suggesting a strong uniaxial anisotropy that prevents domain nucleation at zero field. This is in sharp contrast with the superparamagnetic behavior observed in prior studies of Cr-doped BST, where substantial relaxation was observed at zero and low fields.29 For our sample, significant relaxation was observed near the coercive field.

The down domains started to nucleate at −0.5 T. Further decrease in H induced more down domains (blue region). A fifty–fifty domain state was recovered at approximately −0.7 to −0.8 T, which is defined as the coercive field (HC). Significant domain relaxation was observed near HC, as shown by sharp domain switching along the slow scan axis in MFM images near coercive fields, e.g., as indicated by arrows in Figures 2h–j. At −1.5 T, the sample was fully polarized (Figure 2l). The magnetization reversal process observed in the 10 QL Sb1.89V0.11Te3 thin film is more consistent with typical ferromagnetic domain reversal behavior, including domain nucleation and domain wall propagation.26 The M(H) hysteresis loop deduced from MFM images is plotted in Figure 2m. Note that we only measured the virgin curve and half of the hysteresis loop. The other half (dashed line) was replicated from measured results via a symmetric operation, assuming the hysteresis loop is symmetric about zero field. The coercive field (HC) deduced from MFM images is ~0.76 T.

The in situ two-probe resistance (R) measurement was performed simultaneously with MFM measurement. The R(H) butterfly loop is qualitatively consistent with the M(H) loop (Figure 2m) though the HC from transport (~0.92 T) is slightly higher. The difference likely originates from the stray field from the MFM tip, which locally adds to the external magnetic field.17 So the scanned region (probed by MFM) experiences a higher magnetic field than the rest of the sample (probed by R). Note that it took some time (approximately 5–10 min) to capture each MFM image. The steps in the R(H) loop indicate a substantial time relaxation effect around the coercive field. Although similar relaxation behavior was also observed in Cr-doped magnetic TIs,2729 the relaxation in V-doped films is more consistent with typical ferromagnetic domain behavior, i.e., domain nucleation and domain wall propagation (Supplementary Figure S3). It is interesting to note that a recent transport study at ultralow temperature (10–200 mK) also revealed domain-like behavior in Cr doped BST,30 indicating much softer ferromagnetism in Cr-doped systems.

Further evidence of ferromagnetic domain behavior comes from the robust domain pattern against thermal fluctuation at elevated temperature up to 20 K (Supplementary Figure S4). The domain contrast quickly diminished upon warming the film to 35 K. This is consistent with TC~32 K deduced from R(T) data (Supplementary Figure S4).17 The agreement between bulk transport measurement and local MFM measurement suggests that our MFM results reflect a representative domain behavior of the Sb1.89V0.11Te3 thin film. Although V doped Sb2Te3 is not insulating because of substantial bulk carrier concentration, introducing a small amount of Bi can tune the Fermi level to the Dirac point.24 Since the ferromagnetism (TC and HC) of V doped BST weakly depends on the concentration of Bi,24 we expect similar ferromagnetic domain behavior in V-doped BST that shows QAHE at low temperature.

Figure 3 shows in situ magnetoresistance (MR) measurement of different FC domain states. The initial MR curves of different domain states show a systematic behavior. First, the single domain state is more conducting than the multidomain state, as the zero-field resistance R(0) of 100 Oe FC is significantly less than ZFC, while R(0) of 0 and 5 Oe FC are very close to ZFC. The difference may be related to the density of domain walls in different domain states. The more polarized film has smaller domain wall density. It is known that the domain walls enhance electron scattering in ferromagnetic metals,31,32 thus, the multi-domain state is usually more resistive. Second, the slope of the MR curve at zero field is proportional to the net magnetization, as shown in Figure 3b. This can be understood in terms of averaged MR of the multi-domain state. For domains with magnetization along the direction of the external magnetic field, the intensity of thermal excitation, i.e., spin wave excitation (magnon), will be suppressed by the external field. Therefore magnon-electron scattering will be suppressed, and thus resistivity will be lowered. Conversely, the magnon-electron scattering will be enhanced for domains with magnetization opposite to the direction of the external magnetic field because of enhancement of spin wave intensity. Assuming a linear response at the zero-field limit, the change in resistance due to the external magnetic field for positive or negative domains is ΔR±=k·H (k>0). Assuming the fraction of up (down) domains is N (N) and a simple linear resistance model, the net resistance change would be, ΔR=N·ΔR+N·ΔR=k·(NNH, i.e., $\frac{\mathrm{d}R}{\mathrm{d}H}{|}_{H=0}=k\left({N}_{↓}-{N}_{↑}\right)\propto -M/{M}_{\mathrm{S}}$, in good agreement with results in Figure 3b. Note that the transport properties of V-doped Sb2Te3 are dominated by bulk (valence band) carriers.24 Such MR behavior due to bulk carriers would diminish in insulating V doped BST at low temperature where QAH chiral edge states dominate the transport.

## Conclusion

In conclusion, we present a systematic characterization of ferromagnetic domain behavior of Sb1.89V0.11Te3 molecular beam epitaxy thin film using cryogenic MFM. A zero net magnetization domain state was observed in 10 QL Sb1.89V0.11Te3 thin film when cooled in zero H. The magnetization reversal process at HC shows typical domain behavior of a hard ferromagnet with strong out-of-plane uniaxial anisotropy. In contrast to Cr-doped TI thin films, which exhibit strong inhomogeneity and superparamagnetic behavior of nano-islands,29 the V-doped magnetic TI thin films are more homogeneous with less disorder. Furthermore, the film can be significantly polarized with a small field (approximately 5–10 Oe) applied during cooling through TC, which explains the ‘self-magnetization’ behaviors in the previous report.24 Compared with Cr-doped TIs, the V-doped magnetic TIs are more stable in the single domain state due to their high anisotropy and coercivity as well as lower magnetization.24 This property helps V-doped magnetic TI films enter the single domain state at relatively higher temperature with the presence of a small magnetic field (e.g., trapped flux in a superconducting magnet, earth field, and so on), which is a necessary condition for observing QAH effect. Our MFM studies of V-doped magnetic TI thin films shed light on the path to high-temperature QAH materials.

## Materials and methods

### Sample preparation

Epitaxial thin film Sb1.89V0.11Te3 capped with 10 nm Te was grown on heat treated SrTiO3(111) substrate by co-evaporation in a molecular beam epitaxy system. The nominal thickness of the film is 10 quintuple layers (QL).

### MFM measurement

The MFM experiments were carried out in a homemade cryogenic atomic force microscope (AFM) using commercial piezoresistive cantilevers (spring constant k≈3 N/m, resonant frequency f0≈42 kHz). The homemade AFM is interfaced with a Nanonis SPM Controller (SPECS) and a commercial phase-lock loop (SPECS).17 MFM tips were prepared by depositing nominally 100 nm Co film onto bare tips using e-beam evaporation. MFM images were taken in a constant mode with the scanning plane ~40 nm above the sample surface. MFM signal, the change of cantilever resonant frequency, is proportional to out-of-plane stray field gradient.33 Electrostatic interaction was minimized by nulling the tip-surface contact potential difference. Red (blue) regions in MFM images represent up (down) ferromagnetic domains, where magnetizations are parallel (anti-parallel) with the positive external field.

### MR measurement

In-situ 2-probe resistance of the thin-film samples was measured with a digital multimeter (Keithley 2100).

### ZFC experiments

ZFC MFM measurements were carried out after warming the superconducting magnet to room temperature to eliminate trapped magnetic flux. During ZFC, the MFM tip was >100 μm above sample surface to minimize tip stray magnetic field on the surface.