Prediction of Weyl semimetal, AFM topological insulator, nodal line semimetal, and Chern insulator phases in Bi2MnSe4

Three dimensional materials with strong spin-orbit coupling and magnetic interactions represent an opportunity to realize a variety of rare and potentially useful topological phases. In this work, we use first principles calculations to show that the recently synthesized material Bi2MnSe4 displays a combination of band inversion and magnetic interactions, leading to several topological phases. Bi2PbSe4, also studied, also displays band inversion and is a topological insulator. In bulk form, the ferromagnetic phase of Bi2MnSe4 is either a nodal line or Weyl semimetal, depending on the direction of the spins. When the spins are arranged in a layered antiferromagnetic configuration, the combination of time reversal plus a partial translation is a new symmetry, and the material instead becomes an antiferromagnetic topological insulator. However, the intrinsic TRS breaking at the surface of Bi2MnSe4 removes the typical Dirac cone feature, allowing the observation of the half-integer quantum anomalous Hall effect (AHC). Furthermore, we show that in thin film form, for some thicknesses, Bi2MnSe4 becomes a Chern insulator with a band gap of up to 58 meV. This combination of properties in a stoichiometric magnetic material makes Bi2MnSe4 an excellent candidate for displaying robust topological behavior.


Introduction
Since the discovery of time-reversal invariant (Z2) topological insulators 1-4 a decade ago, there has been a major increase in interest in topological condensed matter systems, due to both scientific interest and potential applications. [4][5][6][7][8][9][10] Despite these efforts, it remains challenging to find robust materials with realizations of many phases, especially those with broken time reversal symmetry (TRS). For example, Chern insulators, 11 which are two-dimensional insulators that display the quantum anomalous Hall effect, were first demonstrated by Haldane in 1988, 12 but their first material realizations were not created until recently. 13 While current quantum anomalous Hall materials, based on magnetically doped topological insulators. [14][15][16][17][18] are limited to very low temperatures (~1K), 13 there is no intrinsic reason for this limit, and there remains significant interest [19][20][21][22][23][24] in finding Chern insulators with larger band gaps and higher magnetic ordering temperatures. [24][25][26][27][28][29] There are many additional related topological phases with broken TRS that are also difficult to realize experimentally. For example, Weyl semimetals (WSM) [30][31][32] are materials with topologically protected linearly dispersing band crossings, known as Weyl points, near the Fermi level. WSM require the breaking of at least one among inversion symmetry and TRS. [33][34][35][36][37][38] Broken inversion WSM have been studied experimentally, [39][40][41][42][43][44][45][46][47] while TRS-broken WSM have been more difficult to verify. 37,[48][49][50][51] They are, however, very interesting as they have different and in some cases simpler properties, such as potentially a minimum two Weyl points in the Brillioun zone, as well as nonzero anomalous Hall conductivity. 34,35 Furthermore, the interplay between TRS invariant topological insulators and broken TRS materials can lead to a variety of interesting behaviors related to the quantized topological contribution of the magneto-electric effect: = 2 2 ℎ ∫ 3 . ⃗⃗⃗ ⃗ . 2,19,52,53 where θ is a dimensionless phase that ranges from 0 to 2 . θ=0 in trivial insulators with TRS, but θ= in Z2 topological insulators with TRS, making Z2 materials potentially useful magnetoelectrics. [54][55][56] However, to observe this effect, it is necessary to remove the topologically protected metallic surface state of the Z2 material by breaking TRS on the surface. [57][58][59] In this work, we investigate the electronic properties of Bi2MSe4 (BMS, M = Pb, Mn) using density functional theory (DFT) and Wannier-based tight-binding models, and we predict a series of topologically non-trivial phases. Recently, we have reported the growth of a thin film of magnetic systems with stoichiometric composition Bi2MnSe4 by molecular beam epitaxy (MBE). 60 We demonstrated that the introduction of an elemental beam of Mn during the molecular beam epitaxial growth of Bi2Se3 results in the formation of layers of Bi2MnSe4 that intersperse layers of pure Bi2Se3. Here, we concentrate on the topological properties of this crystal structure. First, we look at Bi2PbSe4, which we find to be a Z2 topological insulator due to band inversion at the Z point. In the rest of this letter, we focus on combining this band inversion with the magnetic properties of Bi2MnSe4. We find that depending on the magnetic ordering and thickness, Bi2MnSe4 can be a nodal line system, magnetic Weyl semimetal, antiferromagnetic topological insulator, or Chern insulator, in addition to displaying the half-integer quantum anomalous Hall effect. In this work we have considered three different materials and two (Bi3Se4 and Bi2MnSe4) of them have been synthesized, whereas Bi2PbSe4 hasn't been synthesized yet. All these materials are stoichiometric and have significant band gaps, which should help make these topological phases robust and possible to observe at increased temperature.

Computational
Calculations were carried out using density-functional theory (DFT) 61,62 as implemented in QUANTUM ESPRESSO code. 63 We used the PBEsol generalized gradient approximation as exchange and correlation potential. 64 We have used fully relativistic norm-conserving pseudopotentials. 65

Results
We begin our discussion of the Bi2MSe4 structure by considering Bi3Se4. 73 Each septuplet building block of Bi3Se4 is comprised of seven atomic (Se-Bi1-Se-Bi2-Se-Bi1-Se) layers along the c axis and separated by a van der Waals gap. The optimized lattice constants (aBMS = 4.28 Å and cBMS = 40.9 Å) are in good agreement with experimental results 73 . Relaxed structural parameters, such as bond length and intralayer distance, have been tabulated in Table S1 (Supplementary). As expected by electron counting, our band structure calculations reveal that Bi3Se4 is a metallic system, 74 as shown in Fig. 1. However, we expect that substituting a (+2) ion for one Bi will exactly fill the Se states, possibly opening a gap.
Accordingly, we replace the Bi2 atom by with Pb, creating Bi2PbSe4 (BPS). The calculated bulk band structure, shown in Fig. 2a, demonstrates that BPS has a bulk band gap of 0.15 eV. We calculate the topological invariants of BPS, and we find that, like the closely related Bi2Se3, BPS is a strong (Z2) topological insulator, although the topological band inversion happens at the Z point instead of the Γ point in Bi2Se3. This is confirmed by the surface local density of states (LDOS) (Fig. 2b), calculated using our Wannier tight-binding parameters, which clearly shows a single Dirac cone at the Г point in the surface band structure and confirms that BPS is topologically non-trivial. Encouraged by these results, we next investigate the magnetic material Bi2MnSe4 (BMS), which we will concentrate on for the rest of the paper. Consistently with Otrokov et. al., 75 who studied Bi2Se3/Bi2MnSe4 interfaces, we find that the spins prefer to orient ferromagnetically in the case of a single BMS layer. We focus on the topological properties of two spin orderings, ferromagnetic and antiferromagnetic, with alternating layers along the (001) direction (Fig. S3). We find that antiferromagnetic BMS is 1.5 meV per formula unit lower in energy than the ferromagnetic phase, but given the small energy difference it may be possible to align the spins in an external field to observe the ferromagnetic state.
We first consider the ferromagnetic band structure with spins oriented in the (001) direction, as shown in Fig. 3a. By looking at the atomic character of the bands, shown by the colors in Fig. 2a, we can see that there is band in inversion at Г, with a Bi band well below the Fermi level. We also note that there is spin splitting on the order of 0. has the minimum two Weyl points, and they are both exactly at Fermi level due to inversion symmetry 34 . As shown in Fig. S2, we confirm the topological character of the bulk by calculating the surface band structure using our Wannier tight-binding model, finding the expected surface Fermi arcs connecting the two Weyl points. Now, we will discuss the antiferromagnetic (AFM) phase of Bi2MnSe4, the band structure of which is shown in Fig. 4a. Unlike the ferromagnetic phases, AFM Bi2MnSe4 has a symmetry operation that combines TRS with a partial translation along the z direction, and therefore allows for a classification by a Z2 invariant for layered antiferromagnetic topological insulators. 76 Due to band inversion at Г, we find that the AFM phase is a non-trivial antiferromagnetic topological insulator.
However, we note that, unlike a typical non-magnetic topological insulator, the (001) surface of AFM BMS will break time-reversal, due to the loss of translation symmetry along z. This change leads to surface properties that differ from a typical Z2 material, as the Dirac cone feature that is characteristic of topological insulators, including Bi2PbSe4 as shown in Fig. 2b, is protected by TRS, and is absent in this material. This intrinsically broken TRS on the surface makes AFM Bi2MnSe4 an ideal system to observe some of the more exotic magneto-electric behaviors of topological insulators, that are usually hidden by the metallic surface states. Here we can observe these behaviors without needing to engineer an interface between a topological insulator and a magnetic insulator. For instance, a thick slab of AFM BMS will display the half-integer quantum anomalous Hall effect, where both the top and bottom surfaces will contribute +/-(e 2 /2h) to the Hall conductivity, depending on the direction of spins at each surface. 19,76 If there are an even number of layers, the total AHC of the slab will be zero, but each surface will separately contribute |e 2 /2h|. We demonstrate this effect in Fig. 4b, where we use our Wannier tight-binding model to separate the total AHC of a 20 layers AFM BMS slab into layer-by-layer contributions, using the method for calculating the geometric contribution to the AHC. 77 Each surface contributes nearly +/-(e 2 /2h), although the convergence to bulk-like behavior is rather slow. If, instead, there were an odd number of layers, then there would be a net magnetic moment in the system, and the two surfaces would give equal contributions to the AHC, e 2 /2h + e 2 /2h = e 2 /h, making the slab a 2D Chern insulator. In order to investigate the topological behavior of Bi2MnSe4 further, we perform first principles slab calculations with a variety of thicknesses and magnetic orders, with (001) surfaces exposed. In the case of a ferromagnetic slab with spins along the z direction, we find that one and two layer thick slabs are topologically trivial, due to confinement increasing the band gap and preventing band inversion. For three layer and thicker slabs, a band inversion occurs at Г, and the material becomes a Chern insulator with C=-1. In Fig. 5a, we show the band structure of the 3 layers case, which has a band gap of 29 meV. We show other band structures with different thickness of FM and AFM systems in the supplementary materials (Fig S3 and S4). In Fig. 5b, we calculate the edge state of the slab, finding a single spin-polarized edge channel, as expected for a Chern insulator. We also find a Chern insulator in the 4 layers case, with a gap of 58 meV. As the material converges to bulk-like behavior, the gap should eventually close and the AHC will be proportional to the distance between the Weyl points.
For AFM slabs, we find that all slabs from one to five layers are topologically trivial, but with decreasing band gaps from 240 meV for two layers down to 22 meV for five layers, as shown in Fig. 5c. To match the bulk calculations, we expect this trend to continue, and thicker slabs to show a band inversion and a Chern insulating phase, if there are an odd number of layers, as discussed above. Using our Wannier model, we confirm that any perturbation that closes the band gap of the five layers slab, for instance lowering the Bi on-site energies, results in a Chern insulator.

Conclusion
We study the topological behavior of Bi2MnSe4. We find that due to the interplay of band inversion at the Gamma point, strong spin-orbit coupling, and significant spin splitting, Bi2MnSe4 displays a variety of rare topological phases. These phases include Weyl semimetal, nodal line semimental, AFM topological insulator, and Chern insulator. In thin film form, Bi2MnSe4 has a non-zero Chern number starting at three layers in the FM phase. We hope that due to its strong magnetic interactions and the fact that it is stochiometric and has significant band gaps, Bi2MnSe4 will prove a fruitful material for studying many different TRS-broken topological phases that are usually difficult to realize.
The purpose of identifying the computer software in this article is to specify the computational procedure. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology.