Introduction

Anomalous Hall (AH) effect, which exists in ferromagnetic metals1,2, means that the electric Hall conductance remains to be finite even at zero external magnetic field. Furthermore, if the system becomes insulating, the Hall conductance is quantized to an integer C times e2h−1 with C being the Chern number of its band structure. This effect is called the QAH effect. QAH insulators, i.e., magnetic insulators exhibiting QAH effect (C ≠ 0), as important members in the family of quantum Hall systems, have attracted intensive interest from both theoretical and experimental physicists3,4,5,6,7,8. In 1988, Haldane proposed a lattice model of spinless fermions to realize integer Hall effect in a staggered magnetic flux with zero net magnetic field9. Later it was proposed that ferromagnetic insulators with spin-orbit coupling (SOC) may exhibit QAH effect4,10. And there was also an attempt to derive AFM Chern insulator from the Kane-Mele Hubbard model11. Meanwhile, many materials have been predicted to be QAH insulators12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27. So far, QAH effect has been experimentally observed in four different classes of two-dimensional (2D) systems: thin films of magnetically doped topological insulators28, thin films of the intrinsic magnetic topological insulators29, moiré materials formed from graphene30, and transition metal dichalcogenides31. In all of these materials, the QAH plateau only shows up at very low temperatures (of order of 1 Kevin or lower).

It was believed that ferromagnetism is necessary for experimental realization of QAH effect4. However, ferromagnetism can be found easily in metals but rarely in insulators. For this reason, the critical temperature of the observed QAH phase is very low. On the other hand, there are plenty of AFM insulating materials in nature even above room temperatures. However, so far, the QAH effect has never been found in collinear AFM materials, even the corresponding mechanism has never been proposed. Supposing that it can be realized in collinear AFM insulators, then the realization of QAH effect will be much cheaper. Attractively, the critical temperature for observing the QAH effect may be greatly increased since the critical temperature of the AFM order can be very high. The resultant AFM QAH insulators can have extensive applications in designing devices. Therefore, the realization of QAH effect in AFM materials is a very important issue both academically and practically. Accordingly, we first need to clarify the mechanism underlying the QAH effect in AFM systems.

Non-collinear AFM order generally breaks all symmetries of the corresponding crystal, the realizing of AH32,33,34 or QAH35 effect is thus relatively natural. In contrast, collinear antiferromagnetic (AFM) systems usually possess an ‘prohibiting symmetry’ such as {T | τ} or IT, where T stands for the time reversal operation, τ denotes the associated fractional translation, and I represents the spatial inversion. Such an ‘prohibiting symmetry’ restricts the Chern number of a gapped ground state to be C=0, resulting in a trivial band insulator. This limits the study of AH and QAH effect in collinear AFM systems. Recently, the AH effect in three-dimensional collinear AFM metals36 was theoretically studied, but the realizing of QAH effect in two-dimensional collinear AFM insulators is still a challenging open question.

In this work, we propose a minimal lattice model to show such a realization and then provide a guidance to searching for a potential collinear AFM material to realize the QAH effect. Now we stress thatThe monolayer CrO is then adopted as a prototype material to demonstrate the mechanism underlying the QAH in collinear AFM systems.

Results

Lattice model

We firstly consider a square lattice with two sites (labeled by sublattice index α = 1,2) in each unit cell which contains local anti-parallel magnetic moments \({{{\boldsymbol{m}}}}_1\) = \(- {{{\boldsymbol{m}}}}_2\) = \({{{\boldsymbol{m}}}}\). The tight-binding model for spin-orbit coupled electrons in the lattice reads

$$\begin{array}{l}H = \mathop {\sum }\limits_{j,\;d_i} \left[ {t^{d_i}C_{1,\;j}^{\dagger} C_{2,\;j + d_i} + C_{1,\;j}^{\dagger} \left( {i{{{\boldsymbol{\lambda }}}}^{d_i} \cdot {{{\boldsymbol{\sigma }}}}} \right)C_{2,\;j + d_i} + h.c.} \right]\\ \qquad\quad+ \mathop {\sum }\limits_{\alpha ,\;j} \left[ {t_\alpha ^xC_{\alpha ,\;j}^{\dagger} C_{\alpha ,\;j + x} + t_\alpha ^yC_{\alpha ,\;j}^{\dagger} C_{\alpha ,\;j + y} + h.c.} \right]\\ \qquad\quad+ \mathop {\sum }\limits_{\alpha ,\;j} C_{\alpha ,\;j}^{\dagger} \left[ {\mu _\alpha \sigma _0 + {{{\boldsymbol{m}}}}_\alpha \cdot {{{\boldsymbol{\sigma }}}}} \right]C_{\alpha ,\;j}\end{array}$$
(1)

Where \(C_{\alpha ,\;j}^{\dagger} = ( {C_{\alpha ,\;j \uparrow }^{\dagger} ,C_{\alpha ,\;j \downarrow }^{\dagger} })\) are electron creation operators, \(t^{d_i}\) and \(t_\alpha ^x\), \(t_\alpha ^y\) are the kinetic hopping terms along d1,2,3,4 and x, y directions respectively, with x = \(a_1{{{\hat{\boldsymbol x}}}}\), y = \(a_2{{{\hat{\boldsymbol y}}}}\), \(a_1\), \(a_2\) being the lattice constants and \({{{\hat{\boldsymbol x}}}}\), \({{{\hat{\boldsymbol y}}}}\) being unit vectors, and \(d_{1,3} = \pm \frac{1}{2}\left( {x + y} \right),\;d_{2,4} = \frac{1}{2}\left( {x - y} \right)\). The \(i{{{\boldsymbol{\lambda }}}}^{d_i} \cdot {{{\boldsymbol{\sigma }}}}\) term denotes spin-orbit coupling, \({{{\boldsymbol{m}}}}_\alpha = \left( { - 1} \right)^\alpha {{{\boldsymbol{m}}}}\) stands for the static collinear AFM order which couples to the electron spin as a Zeeman field, and µα represents the chemical potential on the α-sublattice. We further define \(\mu _ \pm = \frac{1}{2}(\mu _1 \pm \mu _2)\), where µ- is the staggered potential energy and µ+ is adopted such that the energy bands are half-filled. Here we ignore the correlation between the electrons by simply assuming that the AFM moment couples to the electron spin as a Zeeman field. The effect of electron correlations, such as the Anderson or the Kondo coupling terms, will be left for future study.

Since inversion symmetry is crucial to guarantee a nonzero Chern number, it requires that \(t^{d_1} = t^{d_3}\), \(t^{d_2} = t^{d_4}\) and \({{{\boldsymbol{\lambda }}}}^{d_1} = {{{\boldsymbol{\lambda }}}}^{d_3}\), \({{{\boldsymbol{\lambda }}}}^{d_2} = {{{\boldsymbol{\lambda }}}}^{d_4}\). For simplicity, we set \(t^{d_1} = t^{d_2} = t^d\) and \({{{\boldsymbol{\lambda }}}}^{d_1} = {{{\boldsymbol{\lambda }}}}^{d_2} = {{{\boldsymbol{\lambda }}}}\). Furthermore, to ensure that the energy bands have a full gap, we let \(t_\alpha ^{x,y} = \left( { - 1} \right)^\alpha t^{x,y}\).

After the Fourier transformation, the Hamiltonian can be rewritten in momentum space as

$$H = \mathop {\sum }\limits_k \left( {C_{1k}^{\dagger} {{\Gamma }}_k^{12}C_{2k} + h.c.} \right) + \mathop {\sum }\limits_{\alpha ,\;k} C_{\alpha k}^{\dagger} {{\Gamma }}_k^\alpha C_{\alpha k},$$
(2)

with \(C_{\alpha k}^{\dagger} = ( {C_{\alpha ,\;k \uparrow }^{\dagger} ,\;C_{\alpha ,\;k \downarrow }^{\dagger} })\) the fermion operators, \({{\Gamma }}_k^{12} = 2\left( {t^d + i\lambda \cdot \sigma } \right)\cos ( {\frac{{k_x}}{2} + \frac{{k_y}}{2}}) + 2\left( {t^d - i\lambda \cdot \sigma } \right)\cos ( {\frac{{k_x}}{2} - \frac{{k_y}}{2}})\) the inter-sublattice terms and \({{\Gamma }}_k^{\left( \alpha \right)} = \left( { - 1} \right)^\alpha 2\left( {t^x\cos k_x + t^y\cos k_y} \right) + \left( {\mu _\alpha + {{{\boldsymbol{m}}}}_\alpha \cdot {{{\boldsymbol{\sigma }}}}} \right)\) the intra-sublattice terms.

We firstly turn off the SOC by setting \({{{\boldsymbol{\lambda }}}}^{d_1} = {{{\boldsymbol{\lambda }}}}^{d_2} = 0\), then the system has a spin point group symmetry37,38,39,40 generated by \(\left( {E||C_2^z} \right)\),\(\left( {E||C_2^x} \right)\),\(\left( {E||I} \right)\) and \(\left( {C_2^ \bot T||T} \right)\), \(\left( {C_2^m||E} \right)\) where the notation \(\left( {g||h} \right)\) denotes a combined operation of the spin operation g and the lattice operation h, and \(C_2^ \bot / C_2^m\) respectively stand for the twofold spin rotation along the axis perpendicular/parallel to the m-direction. In this case the \({{\Gamma }}_k^{12}\) term vanishes on the boundary of the BZ. In other words, the two species of fermions C1k and C2k do not hybridize if kx = π or ky = π. It can be shown that on the boundary line (kx, π), the C1k, C2k bands carry quantum numbers 1 and −1 of the symmetry operation \(\left( {E||C_2^x} \right)\) [or \(\left( {E||M_y} \right)\)], respectively. Similarly, on the line (π, ky) the C1k, C2k bands respectively carry quantum numbers −1 and 1 of the symmetry operation \(\left( {E||C_2^y} \right)\) [or \(\left( {E||M_x} \right)\)]. Owing to these different quantum numbers, if the C1k and C2k bands are inverted, the band crossing will be protected from opening a gap. Resultantly, on the boundary lines (kx, π) and/or (π, ky) there will be pairs of Weyl cones (see Fig. 1b as an example) among which the two in each pair are related by the inversion I operation. It can be further shown that when the condition (4) given below is satisfied, there will be odd pairs of Weyl cones on the BZ boundary [see the Supplementary Note 1]. In the Supplementary Note 1 we also show that the Weyl cones are robust under perturbations as long as the symmetry \(\left( {E||I} \right) \times \left( {C_2^ \bot T||T} \right) = \left( {C_2^ \bot T||IT} \right)\) is unbroken.

Fig. 1: Model for the AFM QAH insulator.
figure 1

a Illustration of the Hamiltonian in Eq. (1); b Without SOC the ground state being a Weyl semimetal with a pair of Weyl cones on the (π, ky) boundary of the BZ; c With SOC the band structure being gapped with Chern number C = 1; d the Berry curvature which is inversion symmetric.

Then we turn on the SOC. In this case, the physical symmetry is described by a magnetic point group owing to the locking of the lattice rotation g and the spin rotation g (Simply denote \(\left( {g||h} \right)\) as g in the following). Generally, if \({{{\boldsymbol{\lambda }}}}\cdot {{{\boldsymbol{m}}}} \,\ne\, 0\) then all the Weyl cones obtain a mass and the energy band is fully gapped (see Fig. 1c). Noticing that the SOC does not break the inversion symmetry I, and that the pattern of the Berry curvature of the occupied bands is invariant under I (see Fig. 1d), each pair of Weyl cones (related by I) together contribute either 1 or −1 to the total Chern number (recalling that a single cone carries a π Berry phase or \(\pm \frac{1}{2}\) Chern number). Therefore, if there are an odd number of pairs of Weyl cones, the Chern number in the gapped state must be nonzero and the system will exhibit QAH effect.

The conditions for the nontrivial total Chern number \(( \pm 1)\) can be summarized as the following inequalities,

$$t^d \,\ne\, 0,\;{{{\boldsymbol{\lambda }}}}\cdot {{{\boldsymbol{m}}}} \,\ne\, 0$$
(3)
$$\left| {\left| {{{\boldsymbol{m}}}} \right| - 2\left| {t^y - t^x} \right|} \right| < \left| {\mu _ - } \right| < \left| {{{\boldsymbol{m}}}} \right| + 2\left| {t^y - t^x} \right|$$
(4)

These conditions restrict the crystalline symmetry of the potential materials (see Supplementary Note 1 for detailed discussions). If the AFM order lies in the lattice plane (without losing of generality we assume \(m||\hat y\)), then the QAH effect can be realized in triclinic lattice with symmetry \(\bar 1 = \left\{ {E,\;I} \right\}\) or monoclinic lattice with magnetic point group \(2^\prime / m^\prime = \left\{ {E,\;I,\;C_2^xT,\;M_xT} \right\}\). On the other hand, if the AFM order is perpendicular to the lattice plane (namely \(m||\hat z\)), then the conditions (3) and (4) can be satisfied in orthogonal lattice with magnetic point group symmetry \(m^\prime m^\prime m = \left\{ {E,\;C_2^xT,\;C_2^yT,\;C_2^z,\;I,\;M_xT,\;M_yT,\;M_z} \right\}\) (see Supplementary Note 2 for discussions about QAHE with large Chern numbers and systems with nonsymmorphic magnetic space groups).

If \({{{\boldsymbol{\lambda }}}}\cdot {{{\boldsymbol{m}}}} = 0\), and if m lies in the lattice plane, e.g., \({{{\boldsymbol{m}}}}||{{{\hat{\boldsymbol y}}}}\), then the \(\left( {C_2^y||C_2^y} \right)\) or the \(\left( {C_2^y||M_y} \right)\) = \(\left( {C_2^y||C_2^y} \right) \times \left( {E||I} \right)\) symmetry can protect the Weyl cones on the BZ boundary from being gapped out (see SM). This means that if the AFM order is parallel to an in-plane C2 axis, then the QAH effect cannot be realized through our mechanism.

Now we stress that the spin-orbit coupling term in model (1) is inversion symmetric. When \({{{\boldsymbol{\lambda }}}}^{d_1} = \lambda {{{\hat{\boldsymbol x}}}}\), \({{{\boldsymbol{\lambda }}}}^{d_2} = \lambda {{{\hat{\boldsymbol y}}}}\), it looks like a Rashba SOC term, but the difference is that the Rashba term breaks I symmetry while our model (1) does not. In this case, the QAH effect can be realized if conditions \({{{\boldsymbol{m}}}}\cdot ({{{\hat{\boldsymbol x}}}} + {{{\hat{\boldsymbol y}}}}) \,\ne\, 0\) and (4) are satisfied.

Compared with the Haldane model9 and the FM QAH model10, our model (1) requires neither staggered flux nor ferromagnetism. The nonzero Chern number is resultant from a pair of massive Weyl cones related to each other by inversion symmetry, where the pair of Weyl cones result from band inversion (triggered by AFM order and stagger potential), and the mass is generated from spin-orbit coupling (SOC). The advantage is that the AFM critical temperature can be very high (even with an order of 1000 Kelvin). Therefore, if the band gap is not too small, then hopefully the QAH effect can be realized in experiments close to room temperatures.

So far AFM QAH insulators are never observed experimentally. In the following, by studying a concrete material (monolayer CrO) as a prototype, we illustrate that our mechanism can be used to search for QAH insulators in AFM materials.

Candidate materials

A monolayer CrO has a square lattice structure whose symmetry is described by the symmorphic space group P4-mmm. The point group of P4-mmm is D4h generated by \(C_4^z\), \(C_2^x\) and I. The unit cell of a monolayer CrO contains two O atoms plus two Cr atoms (see Fig. 2a, b) and the corresponding BZ is shown in Fig. 2c. The Cr2+ ions have a fairly strong correlation and they interact with each other via super-exchange interactions mediated by the O atoms. It was suggested in ref. 41 that monolayer CrO is in Neel antiferromagnetic phase, which is further confirmed by our calculations [see the Supplementary Note 3]. Since the two Cr atoms with opposite magnetic moments locate at space-inversion invariant points in the Neel antiferromagnetic order, the IT symmetry is broken. Meanwhile, the two O atoms break the {T | τ} symmetry. Therefore, the Neel antiferromagnetic order has no ‘prohibiting’ symmetry.

Fig. 2: Crystal structure and Brillion zone of monolayer CrO.
figure 2

a, b Illustration of the crystal structure of monolayer CrO viewed along [110] and and (b) [001] directions respectively. The red and blue balls respectively represent the O and Cr atoms. c The Brillouin zone and the corresponding high-symmetry points. d Structure of CrO under external strain (the deformation is exaggerated to guide the eye). The tensile strain removes the \(C_4^z\)T symmetry (such that there are odd pairs of Weyl cones), while the sheared strain breaks the \(C_2^z\)T symmetry (such that the Weyl cones are gapped out under SOC).

Since the electronic band structure is sensitive to the Hubbard U, we need to determine the proper value of U. By comparing the electronic band structure of Heyd-Scuseria-Ernzerhof (HSE) hybrid functional within the framework of HSE0642, we find that U = 3.2 eV, which is slightly <3.5 eV as proposed in ref. 41. This is due to the fact that our relaxed lattice parameter is 1.4 percent less than that adopted in ref. 41.

When ignoring the SOC, a monolayer CrO has a spin point group symmetry whose generators are \(\left( {T||TC_4^z} \right)\), \(\left( {E||M_z} \right)\), \(\left( {E||M_x} \right)\), \(\left( {T||M_{x + y}} \right)\) and \(\left( {C_2^ \bot T||T} \right)\), \(\left( {C_2^m||E} \right)\). According to the \(\left( {T||TC_2^{x \pm y}} \right) = \left( {E||M_z} \right)\left( {T||M_{x \mp y}} \right)\) symmetry, the spin-up and spin-down are degenerate on the high-symmetry lines Γ-M along the x + y or xy direction. Away from these two high-symmetry lines, the spin degeneracy is generally lifted as illustrated in Fig. 3a, where the red and blue lines represent spin-up and spin-down bands, respectively. Furthermore, the band inversion occurs around the X and Y points which are related to each other by the \(\left( {T||TC_4^z} \right)\) symmetry and hence carry opposite spins. Since the two inverted bands carry different quantum numbers of \(\left( {E||C_2^x} \right)\) or \(\left( {E||C_2^y} \right)\), the band inversion results in a band crossing with two pairs of Weyl points41. The two pairs of Weyl cones around the X, Y points can also appear in the model (1) when \(t^x = t^y\) and \(\lambda = 0\).

Fig. 3: The band structure of monolayer CrO along the high-symmetry directions.
figure 3

a No SOC, b no SOC, but tensile strain 10 percent along b-direction. The spin-up and spin-down bands in (a) and (b) are marked in red and blue, respectively. The ∆1 and ∆3 represent different irreducible representations of the little co-group C2v.

If the \(\left( {T||TC_4^z} \right)\) symmetry is preserved, then after the two pairs of Weyl cones being gapped out, the Berry curvature contributed from these cones will exactly cancel each other, resulting in a trivial Chern number. Therefore, in order to realize the QAH effect in a monolayer CrO, the \(\left( {T||TC_4^z} \right)\) symmetry must be removed. To this end, we apply a tensile strain 10 percent along the b-direction. Then the crystalline point group of the strained monolayer CrO reduces to D2h which is generated by \(C_2^z\), \(C_2^x\) and I.

Under the above tensile strain, the calculated electronic band structure is shown in Fig. 3b, where the pair of Weyl points on the X-M axis merge and gap out with the disappearing of band inversion. But the pair of Weyl points on the Y-M axis remain stable. When considering the SOC, our calculations indicate that the magnetic moments are along the y-direction in the lattice plane. Then the monolayer CrO has a magnetic point group symmetry \(m^\prime m^\prime m = \left\{ {E,\;C_2^xT,\;C_2^y,\;C_2^zT,\;I,\;M_xT,\;M_y,\;M_zT} \right\}\). However, due to the \(C_2^zT \equiv (C_2^zT||C_2^zT)\) symmetry, the Berry curvature is always zero in the whole BZ and the Weyl cones are still robust. Therefore, the \(C_2^zT\) symmetry has to be removed to gap out the Weyl cones and then to realize the QAH effect.

To break the \(C_2^zT\) symmetry, we further apply shear strain (up to 3 percent) such that the two out-of-plane O atoms obtain a slight displacement along the in-plane diagonal direction (the inversion symmetry is always kept). At this time, the monolayer CrO has only space-inversion symmetry I. Finally, the pair of Weyl cones open a gap for amount ~1 meV (see Fig. 4d for the gapped cone) and the resultant gapped ground state has a Chern number C = −1, which makes the strained monolayer CrO a QAH insulator. As shown in Fig. 4c, the Berry curvature mainly concentrates around the gapped Weyl points.

Fig. 4: The Weyl cones of a monolayer CrO shift their positions from the M-Y high symmetry line to the inside of the BZ by the tensile and shear strain, and are further gaped out if SOC is turned on.
figure 4

a, b Show the electronic band structure (with strain and without SOC) along the M-Y line and the line \(k_y = 0.49747 \times \frac{{2\pi }}{{a_2}}\) respectively, it can be seen that the Weyl points shift toward the inside of the BZ. c, d Respectively show the Berry curvature and the dispersion (around the Weyl point) of the gapped bands when SOC is turned on.

It should be emphasized that SOC plays an important role in giving mass to the Weyl cones. If SOC is absent, the Weyl cones are robust against the strains [see Fig. 4a, b where the Weyl points on the Y-M line shift to (0.09025, 0.49747) and (−0.09025, −0.49747)] owing to the protection of the spin point group symmetry element \(\left( {C_2^ \bot T||IT} \right)\). So, it is not a wonder that the size of the gap is rather small since the SOC in CrO is very weak. For this reason, the QAH effect can only be observed at about 10 Kelvin even though the Neel temperature of a monolayer CrO is estimated to be above the room temperature41.

Discussion

Before concluding, it is illustrative to compare the QAH effect with the AH effect in collinear AFM systems. Both require nonzero Berry curvature and have similar symmetry conditions for potential materials. But they differ at least in the following two aspects: (i) AH effect exists in either three-dimensional or two-dimensional metals, but QAH effect only exists in two-dimensional insulators; (ii) QAH insulators are topological states where the integration of the Berry curvature over the occupied bands is quantized and the corresponding non-zero Chern number is generally resultant from band inversion, but AH effect is not necessarily related to topological electronic structures (although the Weyl cones can enhance the AH effect).

We come up with a four-band lattice model to illustrate an interesting mechanism to realize AFM QAH insulators. The mechanism provides important guidelines for experimental studies, it indicates that AFM materials satisfying the following conditions are promising to be QAH insulators: (1) respecting the required symmetries (see the main text for the specific magnetic point groups); (2) having nonzero SOC; (3) containing band inversion near the Fermi level. Based on the mechanism and from the first-principles calculations, we predict that the strained monolayer CrO is a candidate AFM material to realize the QAH effect. Once QAH insulator is realized in AFM systems with strong SOC, it may allow people to observe QAH effect close to room temperatures.

Method

First-principles calculations

The electronic structures of a monolayer CrO were studied with the projector augmented wave method43,44 as implemented in the VASP package45,46. The Perdew-Burke-Ernzerhof exchange-correlation functional at the generalized gradient approximation level was adopted to describe the interaction between the ionic cores and the valence electrons47. For the electronic correlation in the d orbitals of Cr2+, we adopted the Hubbard U formalism. The kinetic energy cutoff of the plane-wave basis was set to 700 eV and a 20 × 20 × 1 was adopted for the k-point mesh. The Gaussian smearing method with a width of 0.01 eV was utilized for the Fermi surface broadening. Both cell parameters and internal atomic positions were fully relaxed until the forces on all atoms were smaller than 0.001 eV/Å. The calculated lattice parameters were 1.4 percent less than the previous values41. A 20 Å vacuum layer was used to avoid the residual interaction between adjacent layers. The Berry curvature of monolayer CrO was calculated by using the Wannier90 package48,49. The topological invariants of monolayer CrO were studied by WannierTools package50.