Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Realization of topological Mott insulator in a twisted bilayer graphene lattice model

## Abstract

Magic-angle twisted bilayer graphene has recently become a thriving material platform realizing correlated electron phenomena taking place within its topological flat bands. Several numerical and analytical methods have been applied to understand the correlated phases therein, revealing some similarity with the quantum Hall physics. In this work, we provide a Mott-Hubbard perspective for the TBG system. Employing the large-scale density matrix renormalization group on the lattice model containing the projected Coulomb interactions only, we identify a first-order quantum phase transition between the insulating stripe phase and the quantum anomalous Hall state with the Chern number of ±1. Our results not only shed light on the mechanism of the quantum anomalous Hall state discovered at three-quarters filling, but also provide an example of the topological Mott insulator, i.e., the quantum anomalous Hall state in the strong coupling limit.

## Introduction

Twisted bilayer graphene (TBG) burst on the scene as a tunable two carbon-atom layers thick system realizing a remarkable multitude of interaction-driven macroscopic quantum phenomena1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20. Although significant progress has been achieved in understanding the nontrivial topology of the narrow bands, as well as the correlated electron states in the magic-angle TBG21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42, many important questions remain open. One of the most fascinating question is the origin and the mechanism of the quantum anomalous Hall (QAH) state with Chern number C = ±16,7 at three-quarters filling of the system, aligned with the hexagonal boron nitride (hBN), and the insulating state which replaces the QAH in devices without the hBN alignment.

Currently, the prevailing opinion is that the QAH can be obtained from narrow band models with large Coulomb interactions28,43,44,45,46, but that the nontrivial topology of the narrow bands prevents a faithful construction of local “Hubbard-like” tight-binding models that locally respect all the symmetries23. Although there exists no a priori Wannier obstruction, as the narrow bands’ total Chern number vanishes, there is yet no clear understanding of how the QAH could arise within such correlated lattice model, even in principle, in the limit where the Coulomb interactions dominate the kinetic energy.

Precisely such a state was sought by Raghu et al. in an entirely different context47, coining the term topological Mott insulator (TMI), which we define to be a QAH in a strong coupling limit of a local lattice model with a vanishing ratio of the bandwidth to Coulomb interaction. However, the original proposal47 was subsequently shown not to host a QAH, and therefore not TMI either48,49. More recent works have found the interaction-induced QAH state in a different model, but it is stabilized by the kinetic energy and necessitates sizable bandwidth50,51,52. Because it gives way to more conventional Mott insulators in the strong coupling regime52, these models do not host a TMI.

Here we show that the TMI is realized in a simple lattice model introduced by two of the authors as a local description of the correlations within the TBG narrow bands26,53,54. The key new ingredients are the off-site terms appearing alongside the usual on-site terms in the projected density operator. Physically, such terms originate in the extended multi-peak nature of the maximally localized Wannier states22,24 arising from the nontrivial topology25,25,35,55,56,57,58 of the narrow bands, and, importantly, remain finite even when the bandwidth vanishes.

## Results

### Honeycomb moiré lattice model

In the strong coupling limit, the aforementioned model (as illustrated in the upper panels of Fig. 1) is

$$H = U_{0}{\mathop{\sum}\limits_{\hexagon}}(Q_{\hexagon}+\alpha T_{\hexagon}-1)^{2},$$
(1)

where U0 constitutes the overall energy scale in the problem (≈40 meV in TBG and set to unity henceforth). $$Q_{\hexagon}\equiv \,\frac{1}{3}\mathop{\sum }\nolimits_{l = 1}^{6}{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l}}^{{{{\dagger}}} } {c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l}}^{\,}$$ represents the cluster charge term22,25,53,54,59,60 (c.f. Fig. 1c), and $$T_{\hexagon}\equiv\, \mathop{\sum }\nolimits_{l = 1}^{6}[{(-1)}^{l}{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l+1}}^{{{{\dagger}}} }{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l}}^{\,}+h.c.]$$ represents the Coulomb induced hopping with alternating sign (c.f. Fig. 1d). Fermion annihilation and creation operators $${c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l}}^{\ }$$ and $${c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{l}}^{{{{\dagger}}} }$$ are defined at the sites of the honeycomb lattice R + δl, where R = m1L1 + m2L2 with integer m1,2 spans the triangular Bravais lattice. The hexagon centers, over which we sum in Eq. (1), are connected to the six nearest honeycomb lattice sites l = 1, 2, 6 through δl (c.f. Fig. 1e). As we focus on the three-quarters filling of the TBG, where the spin and orbital degrees of freedom are assumed to be polarized, Eq. (1) thus constitutes a simplification to the full Hamiltonian of ref. 26. The parameter α controls the relative strength of charging and assisted-hopping of the projected Coulomb interaction. It originates from the overlap of two neighboring Wannier states in the continuum model and thus depends on the lattice relaxation. Due to the background charge from the remote bands, which is approximated to be uniform in Eq. (1), the projected Coulomb interaction is in the form of density-density repulsion43,61,62, instead of being normal ordered. Although the projected interaction contains other terms such as next-nearest neighbor interaction, the more detailed calculations at the chiral limit have shown that the interaction-induced dispersion of the charged excitation at the charge neutrality point is dominated by α, the nearest neighbor assisted hopping63.

The original bandwidth W ~ 8 meV24 is much smaller than U0, suggesting the system is in the strong coupling regime. Furthermore, after the states on the remote bands are integrated out, the superexchange interaction (5 × 10−3e2/(ϵLm)) is found to be negligible compared with the projected Coulomb interaction61; this justifies neglecting additional fermion bilinear (kinetic) terms in Eq. (1). The kinetic term, as well as the further-range assisted hopping terms, may shift the critical value αc of the phase transition but do not qualitatively change the phase diagram in Fig. 1f. In addition, we do not include the additional symmetry breaking term produced by the possible hBN alignment that favors the QAH phase64, but focus on the topological phase transitions purely driven by interactions.

It is worth emphasizing that Eq. (1) corresponds to the leading order terms when the distance to the gates lg is about the same as the moiré lattice constant L1, and thus the electron-electron repulsion decays exponentially when the inter-electron separation is larger than L126. With larger lg, the longer range aspect of the Coulomb repulsion will have to be included, but because currently there is no experimental indication that there are significant changes in the nature the insulating states for different lg1,3,65, it is reasonable to neglect the longer range terms in Eq. (1). We should note that terms in Eq. (1) are purely real, and because the two QAH states with opposite Chern numbers transform into each other under complex conjugation, the QAH state is not a priori favored by this model. In what follows, we will demonstrate that, for a range of α, Eq. (1) naturally leads to the TMI ground state via spontaneous symmetry breaking without including any other interactions or kinetic terms.

### Phase diagram

We solve the TBG lattice model in Eq. (1) using DMRG on long cylinders of XC (zigzag, Fig. 1a) and YC (armchair, Fig. 1b) geometries, with widths W up to 6 and lengths L up to 24. The details of DMRG implementation and finite-size analysis are given in the Methods and Supplementary Note 1. The obtained ground state phase diagram, as a function of α, is shown in Fig. 1f. We identify two gapped insulating phases: a stripe phase with charge density wave (CDW) for small α, and a TMI phase for α > αc ≈ 0.12. These two ground states are separated by a first-order quantum phase transition (QPT). In Fig. 2, we show results for various quantities, including the ground state energy eg, entanglement entropy SE, charge structure factor Cn, and the imaginary part of the equal time correlation $$\langle J\rangle \equiv \frac{{i}}{2}\langle ({c}_{l}^{{{{\dagger}}} }{c}_{l^{\prime} }^{\ }-{c}_{l^{\prime} }^{{{{\dagger}}} }{c}_{l}^{\ })\rangle$$. As shown in Fig. 2a, the eg curve exhibits a discontinuity in the slope (a kink) at αc, indicating the first-order QPT. In Fig. 2b, we calculate the entanglement entropy $${S}_{E}(x)\equiv -{{{{{{{\rm{Tr}}}}}}}}[{\rho }_{{{{{{{{\mathcal{A}}}}}}}}}(x){{{{{{\mathrm{ln}}}}}}}\,({\rho }_{{{{{{{{\mathcal{A}}}}}}}}}(x))]$$, with $${\rho }_{{{{{{{{\mathcal{A}}}}}}}}}(x)$$ the reduced density matrix of the subsystem $${{{{{{{\mathcal{A}}}}}}}}$$ consisting of the first x columns (c.f. Fig. 1a, b). By setting x = L/2 (for even L), i.e., cutting at the very center of the system, we compute SE(L/2) and show it vs. α in Fig. 2b, where an evident “jump” takes place right at the QPT. In addition, for α < αc, the negligibly small SE(L/2) indicates the existence of a nearly direct product state with virtually no charge fluctuations in the CDW pattern. On the other hand, the sizable SE(L/2) for α > αc indicates a finite amount of quantum entanglement in the ground state. In the insets of Fig. 2b, SE(x) vs. subsystem length x shows a flat plateau in the bulk of the system, indicating that both phases in Fig. 1f are gapped, consistent with the exponentially decaying single-particle Green’s functions also obtained by our DMRG (see the Supplementary Note 1).

### Stripe and QAH insulators

The emergence of the stripe phase at small α can be understood from a perturbative analysis26. Up to second-order corrections (c.f. Supplementary Note 2), we find the ground-state energy eg/U0α2, and plot it together with the DMRG results in Fig. 2a, where the high accuracy of this analytical calculation can be clearly seen. The CDW order can be characterized by the structure factor, $${C}_{n}({{{{{{{\bf{k}}}}}}}})\equiv \frac{1}{N}\mathop{\sum }\nolimits_{\lambda = 1}^{2}{\sum }_{{{{{{{{\bf{R}}}}}}}}}{e}^{-i{{{{{{{\bf{k}}}}}}}}\cdot ({{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda })}{\tilde{n}}_{{{{{{{{\bf{R,\lambda }}}}}}}}}$$, where the quantity $${\tilde{n}}_{{{{{{{{\bf{R,\lambda }}}}}}}}}=\langle {c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}^{{{{\dagger}}} }{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}^{\ }\rangle -1/2$$ counts the number of electrons (with respect to the half filling) on the honeycomb site R + δλ. In Fig. 2c, we find that Cn(k) peaks at $${{{{{{{\bf{M}}}}}}}}=(0,\frac{2\pi }{\sqrt{3}| {{{{{{{{\bf{L}}}}}}}}}_{1}| })$$ for α < αc, and drops abruptly to 0 for α > αc, confirming that the small-α regime has a CDW order, while for α > αc the insulating phase has no charge order. Remarkably, this α > αc regime turns out to be a topological phase with spontaneous time-reversal symmetry (TRS) breaking and a quantized Hall conductance, i.e., a QAH phase.

To reveal the TRS breaking in the large-α QAH phase, in Fig. 2d we show the correlation $$\langle J\rangle$$ on both the nearest-neighbor (NN) and next-nearest-neighbor (NNN) $$(l,l^{\prime} )$$ pairs. We find a finite value of $$\langle J\rangle_{{{\mathrm{NN}}}}$$ ~ 0.22 and $$\langle J\rangle_{{{\mathrm{NNN}}}}$$ ~ 0.1 in the bulk of the cylinder for large-α phase, while they vanish in the stripe phase. In the QAH phase, the real part of $$\langle {c}_{l}^{{{{\dagger}}} }{c}_{l^{\prime} }^{}\rangle$$ is negligibly [O(10−7~−8)] smaller compared to its imaginary part, and thus $$\langle {c}_{l}^{{{{\dagger}}} }{c}_{l^{\prime} }^{}\rangle$$ emerging from interactions is virtually purely imaginary. The corresponding hopping process thus acquires a π/2 phase (labeled as i in Fig. 1g), rendering a 3π/2 flux for a circulating triangular loop current, which resembles the Haldane model66. The difference is that the TRS breaking NNN hopping term is introduced explicitly in the Haldane model, while here it emerges spontaneously due to electron interactions, a typical feature of TMIs. We also note that in a recent quantum Monte Carlo simulation applied at charge neutrality53 (i.e. even integer filling), a quantum valley Hall state is found at intermediate coupling for a specific choice of kinetic energy terms. Such a state is different from the QAH found at odd integer filling here as it preserves the TRS with helical valley edge modes and undergoes a first-order phase transition into intervalley coherent insulator at strong coupling, consistent with the exact results obtained in ref. 26.

### Quantized Hall conductance

To reveal the topological properties in the large-α phase, we perform a flux insertion experiment on the cylindrical geometry (c.f. the inset of Fig. 3a) and compute the Hall conductance. We thread a ϕ-flux along the cylinder by modifying the boundary condition $${c}_{{{{{{{{\bf{R}}}}}}}}+W({{{{{{{{\bf{L}}}}}}}}}_{1}-{{{{{{{{\bf{L}}}}}}}}}_{2})+{\delta }_{\lambda }}\equiv {c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}$$ to $${c}_{{{{{{{{\bf{R}}}}}}}}+W({{{{{{{{\bf{L}}}}}}}}}_{1}-{{{{{{{{\bf{L}}}}}}}}}_{2})+{\delta }_{\lambda }}\equiv {e}^{-i\phi }{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}$$ for XC geometry and $${c}_{{{{{{{{\bf{R}}}}}}}}+W({{{{{{{{\bf{L}}}}}}}}}_{1}-{{{{{{{{\bf{L}}}}}}}}}_{2}/2)+{\delta }_{\lambda }}\equiv {c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}$$ to $${c}_{{{{{{{{\bf{R}}}}}}}}+W({{{{{{{{\bf{L}}}}}}}}}_{1}-{{{{{{{{\bf{L}}}}}}}}}_{2}/2)+{\delta }_{\lambda }}\equiv {e}^{-i\phi }{c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}$$ for YC geometry. During the process of the flux insertion, ϕ is adiabatically increased from 0 to 2π in the DMRG calculations. One thereafter obtains the Hall conductance $${\sigma }_{H}=\frac{{e}^{2}}{h}{{\Delta }}Q$$ by measuring the net charge pumping ΔQ from one edge of the cylinder to the other. In DMRG, we calculate the net charge transfer as $${{\Delta }}Q=\mathop{\sum }\nolimits_{x = L-l+1}^{L}[{\tilde{n}}_{x}^{{{{{{{{\rm{col}}}}}}}}}(\phi )-{\tilde{n}}_{x}^{{{{{{{{\rm{col}}}}}}}}}(0)]$$, i.e. the pumped charge to the rightmost l columns (chosen as l = 3–4 in practice) where $${\tilde{n}}_{x}^{{{{{{{{\rm{col}}}}}}}}}(\phi )$$ is the deviation of the charge number of the x-th column measured in the ϕ-flux inserted ground state $$|{\psi }_{\phi }\rangle$$ from the half filling. For instance, we have $${\tilde{n}}_{x}^{{{{{{{{\rm{col}}}}}}}}}(\phi )=\mathop{\sum }\nolimits_{y = 1}^{W}\mathop{\sum }\nolimits_{\lambda = 1}^{2}\langle {\psi }_{\phi }|{\hat{n}}_{(x-1){{{{{{{{\bf{L}}}}}}}}}_{{{{{{{{\bf{1}}}}}}}}}+y({{{{{{{{\bf{L}}}}}}}}}_{{{{{{{{\bf{1}}}}}}}}}-{{{{{{{{\bf{L}}}}}}}}}_{2})+{\delta }_{\lambda }}-\frac{1}{2}|{\psi }_{\phi }\rangle$$ for the XC geometry, and similar expressions for YC.

As shown in Fig. 3a, for both XC and YC systems (with widths W = 4 and 6) in the QAH phase (e.g., α = 0.15), we find a net charge transfer ΔQ = 1 through a 2π flux insertion, showing that the Chern number C = ±1. In addition, Fig. 3b shows the column charge distribution $${\tilde{n}}_{x}^{{{{{{{{\rm{col}}}}}}}}}$$, where a half-charge $$\pm\kern-2pt \frac{1}{2}$$ appears in two edges in $$|{\psi }_{\phi = 0}\rangle$$. As ϕ gradually increases, the left/right-end charge smoothly reduces/increases from $$\pm \kern-2pt\frac{1}{2}$$  to $$\mp \frac{1}{2}$$, which corresponds to an end-to-end pumping of a unit charge ΔQ = 1, without “disturbing” the charge distribution in the bulk. We note that there is two-fold degenerate QAH ground state (apart from the additional degeneracy due to half-charge zero edge modes, see discussion below), and the charge pumping could be ΔQ = ±1, corresponding to the spontaneous TRS breaking states with C = ±1.

### Understanding the TMI phase

With DMRG calculations, we can also calculate the single-particle Green’s function $${G}_{\lambda ,\lambda ^{\prime} }({{{{{{{\bf{R}}}}}}}}-{{{{{{{\bf{R}}}}}}}}^{\prime} )=\langle {c}_{{{{{{{{\bf{R}}}}}}}}+{\delta }_{\lambda }}^{{{{\dagger}}} }{c}_{{{{{{{{\bf{R}}}}}}}}^{\prime} +{\delta }_{\lambda ^{\prime} }}^{\ }\rangle$$, from which we can find the electron occupation $${n}_{\lambda ,\lambda ^{\prime} }({{{{{{{\bf{k}}}}}}}})$$ in the momentum space. Due to the two-sublattice structure, $${G}_{\lambda ,\lambda ^{\prime} }({{{{{{{\bf{R}}}}}}}}-{{{{{{{\bf{R}}}}}}}}^{\prime} )$$ and its Fourier transformation $${\tilde{G}}_{\lambda ,\lambda ^{\prime} }({{{{{{{\bf{k}}}}}}}})$$ are both 2 × 2 matrices (cf., Supplementary Note 1). The two eigenvalues {n1(k), n2(k)} of $$\tilde{G}({{{{{{{\bf{k}}}}}}}})$$ are shown in Fig. 3d. We find for all allowed k points, the larger eigenvalue n2(k)  1 and the smaller value n1(k)  0, representing the “two-orbit” electronic structure with one orbit filled while the other left empty. Albeit small, charge fluctuations between the two orbits are still present. We compute the von Neumann entropy $${S}_{{{{{{{{\rm{vN}}}}}}}}}({{{{{{{\bf{k}}}}}}}})\equiv -\mathop{\sum }\nolimits_{\lambda = 1}^{2}{n}_{\lambda }({{{{{{{\bf{k}}}}}}}}){{{{{{\mathrm{ln}}}}}}}\,{n}_{\lambda }({{{{{{{\bf{k}}}}}}}})$$ that measures the deviation of the DMRG ground state from a Slater determinant of Bloch states. In Fig. 3e, we show the calculated SvN averaged over the first BZ, which decreases as α increases, and becomes very small for large α cases. For example, we show the detailed k-dependent profile for the α = 0.25 case, in the inset of Fig. 3e. The relatively small SvN values suggest the QAH state, emerging in the interacting TBG model as revealed by DMRG calculations, actually very much resembles the Slater determinant ground state of the Haldane model and thus can be captured by a mean-field description.

To be specific, for small α, a second-order perturbation shows the charging term $${\mathop{\sum}\limits_{\hexagon}}(Q_{\hexagon}-1)^{2}$$ favors the insulating phases in which each hexagon of the honeycomb lattice contains exactly one electron, i.e. $$Q_{\hexagon}=1$$ for every hexagon. Among all the states satisfying this requirement, the first- and second-order corrections from the cross terms $$T_{\hexagon}(Q_{\hexagon}-1)$$ vanish. The stripe phase is selected from such states because it minimizes the contribution of $$\langle{\mathop{\sum}\limits_{\hexagon}} T_{\hexagon}^{2}\rangle$$, with the energy $$\langle H\rangle_{{{\mathrm{stripe}}}}$$ ≈ α2U0 (c.f. Supplementary Note 2).

For large α, motivated by the resemblance of the DMRG ground state to the Slater determinant, we perform a variational mean-field calculation that approximates the true ground state with the ground state of a tight-binding model containing various hoppings (see Methods and Supplementary Note 3). In particular the Fig. 1g demonstrates the emergence of NNN currents which constitute a loop in each hexagon, spontaneously choosing either the left- or right-chiral direction (here the right chirality). We find that the cross terms, i.e. $$\langle T_{\hexagon} (Q_{\hexagon} -1)\rangle_{{{{{\mathrm{QAH}}}}}}$$ become negative and thus favor the QAH phase. Including both the charging terms and $${\mathop{\sum}\limits_{\hexagon}} T_{\hexagon}^{2}$$, the variational mean-field analysis results in $$\langle H\rangle_{{{\mathrm{QAH}}}}$$ ≈ U0(0.037 − 0.27α + 0.71α2). Therefore, as α continuously increases from 0, the mean-field theory also finds the first-order phase transition from the stripe phase to the QAH, in agreement with the DMRG result mentioned earlier. The mean-field energy is shown in Fig. 2a as indicated by the blue and red dashed line for the stripe and QAH phases respectively. Both lines provide a good approximation to the DMRG energy curve, and the intersection of two mean-field energies also provides a very good estimate of the QPT value $${\alpha }_{c}^{{{{{{{{\rm{MF}}}}}}}}}\simeq 0.125$$. Interestingly, the energy difference between the mean-field approximation and the DMRG calculation decreases as α moves away from the QPT, reflecting the suppression of the quantum fluctuations for large α − αc, also illustrated by the SvN in Fig. 3e.

Moreover, as shown in Fig. 3b, there exist half-charge zero modes on both edges of the cylinder with even W, which also coincide with the Haldane model wrapped on the cylinder (for more details, see the Supplementary Note 4). We also compute the entanglement spectrum (ES), defined as $${\xi }_{i}\equiv -{{{{{{\mathrm{ln}}}}}}}\,({\rho }_{i})$$ with ρi the eigenvalues of the reduced density matrix. As shown in Fig. 3c, when we cut at the center of the system, a two-fold degeneracy in the ES is evident, which accounts for the half-charge zero modes in the edge (c.f. Fig. 3b), through the bulk-edge correspondence.

## Discussion

As we mentioned, the QAH can be obtained from narrow band models of TBG with large Coulomb interactions, but these models are built in the basis of extended states27,28,43,44,45 making the interaction potential rather unwieldy. The results indeed show that several phases: QAH, strongly correlated topological semimetal, and insulating stripe phases, are energetically competitive for the ground states at odd integer fillings28,35,44,46,67.

The common belief, however, is that the nontrivial symmetry-protected topology of the narrow bands prevents a faithful construction of models within exponentially localized basis even when the bands’ total Chern number vanishes23. On the other hand, as first shown in the context of the Z2 topological insulators68, the obstruction is not as severe as in the case of a nonzero Chern band (or band composite). If the total Chern number vanishes, the exponentially localized Wannier states can be constructed69, but some of the protecting symmetries do not have a simple on-site implementation68,70,71. Because the transformation from the Bloch to Wannier basis is unitary and no information is lost in the process, it is therefore expected that the lattice tight-binding description should also result in the same ground state as found in unobstructed, extended states, basis. However, any practical implementation of this program needs to truncate the expansion of the interaction to on-site and few nearest neigbour sites. What is not obvious, therefore, is whether all the terms need to be included in the expansion or whether it can be truncated to recover the ground state.

The results presented here show that the truncation at just the nearest neighbor, parameterized by α in Eq. (1), is sufficient to recover the insulating and the topologically nontrivial phases. In addition, the main features of the single-particle excitation dispersion of the strong coupling correlated ground states at the charge neutrality point53 from the model in Eq. (1) match those computed exactly in the extended basis61,67. This demonstrates the practicality of Wannier description even for such symmetry-obstructed bands. Our real-space interaction-only model therefore establishes the microscopic mechanism of the evolution between the insulating stripe and QAH phases. Our effective model and its unbiased numerical solution therefore revealed the essence of the physics in this particular regime, and is also consistent with other theoretical calculations27,28,44,46.

As for relevance of our model towards the real system, it is understood that other than the $$Q_{\hexagon}$$ and $$T_{\hexagon}$$ terms, we do not include all the other projected interactions nor the small kinetic terms, i.e., the detailed feature of the TBG material, which will surely modify the specific value of αc. Apart from that they should not qualitatively alter the two phases and thus also the main conclusion of the present work. In addition to the ground states given above, the dispersion of the charged excitations produced by Eq. (1) is also found to be qualitatively consistent with more detailed calculation by two of the authors in refs. 61,63. Reference63 has also explicitly shown that the dispersion at the charge neutrality point is dominated by the α term in the chiral limit. For systems away from the chiral limit, it is expected that the inclusion of other terms may only quantitatively change the dispersion.

## Methods

### Density matrix renormalization group

We employ the DMRG method, realized in the matrix product state form and with U(1) charge symmetry implemented, to accurately find the ground state of the TBG model. Following standard 2D DMRG calculations, we map the cylindrical geometries through a snake-like path, i.e., a quasi-1D structure, where highly controllable and efficient simulations can be performed. In practice, we retain up to D = 512(1024) for W = 4(6) cylinders, with truncation errors ϵ < 5 × 10−5, for an accurate large-scale calculations. The detailed convergence check of the TBG model calculations can be seen in the Supplementary Note 1.

### Mean-field analysis

We also applied the mean-field theory to approximate the interactions by a tight-binding model with variational hopping constants. The hopping amplitudes are obtained by minimizing the expectation value of the interactions in Eq. (1) for the state produced by the tight-binding model. In practice, the tight-binding model includes hopping amplitudes up to the 5th nearest neighbor. The details are presented in the Supplementary Note 3.

## Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

## Code availability

All numerical codes in this paper are available upon request to the authors.

## References

1. 1.

Cao, Y. et al. Correlated insulator behaviour at half-filling in magic-angle graphene superlattices. Nature 556, 80 (2018).

2. 2.

Cao, Y. et al. Unconventional superconductivity in magic-angle graphene superlattices. Nature 556, 43 (2018b).

3. 3.

Yankowitz, M. et al. Tuning superconductivity in twisted bilayer graphene. Science 363, 1059 (2019).

4. 4.

Lu, X. et al. Superconductors, orbital magnets and correlated states in magic-angle bilayer graphene. Nature 574, 653 (2019).

5. 5.

Xie, Y. et al. Spectroscopic signatures of many-body correlations in magic-angle twisted bilayer graphene. Nature 572, 101 (2019).

6. 6.

Sharpe, A. L. et al. Emergent ferromagnetism near three-quarters filling in twisted bilayer graphene. Science 365, 605 (2019).

7. 7.

Serlin, M. et al. Intrinsic quantized anomalous Hall effect in a moiré heterostructure. Science 367, 900 (2020).

8. 8.

Stepanov, P. et al. Untying the insulating and superconducting orders in magic-angle graphene. Nature 583, 375 (2020).

9. 9.

Kerelsky, A. et al. Maximized electron interactions at the magic angle in twisted bilayer graphene. Nature 572, 95 (2019).

10. 10.

Jiang, Y. et al. Andrei, Charge order and broken rotational symmetry in magic-angle twisted bilayer graphene. Nature 573, 91 (2019).

11. 11.

Choi, Y. et al. Electronic correlations in twisted bilayer graphene near the magic angle. Nat. Phys. 15, 1174 (2019).

12. 12.

Nuckolls, K. P. et al. Strongly correlated Chern insulators in magic-angle twisted bilayer graphene. Nature 588, 610 (2020).

13. 13.

Uri, A. et al. Mapping the twist-angle disorder and Landau levels in magic-angle graphene. Nature 581, 47 (2020).

14. 14.

Choi, Y. et al. Tracing out correlated chern insulators in magic angle twisted bilayer graphene. arXiv:2008.11746 (2020).

15. 15.

Wu, S., Zhang, Z., Watanabe, K., Taniguchi, T. & Andrei, E. Y. Chern insulators and topological flat-bands in magic-angle twisted bilayer graphene. arXiv:2007.03735 (2020).

16. 16.

Saito, Y., Ge, J., Watanabe, K., Taniguchi, T. & Young, A. F. Independent superconductors and correlated insulators in twisted bilayer graphene. Nat. Phys. 16, 926 (2020).

17. 17.

Das, I. et al. Symmetry broken Chern insulators and magic series of Rashba-like Landau level crossings in magic angle bilayer graphene. arXiv:2007.13390 (2020).

18. 18.

Park, J. M., Cao, Y., Watanabe, K., Taniguchi, T. & Jarillo-Herrero, P. Flavour Hund’s coupling, Chern gaps and charge diffusivity in moiré graphene. Nature 592, 43 (2021).

19. 19.

Zondiner, U. et al. Cascade of phase transitions and Dirac revivals in magic-angle graphene. Nature 582, 203 (2020).

20. 20.

Wong, D. et al. Cascade of electronic transitions in magic-angle twisted bilayer graphene. Nature 582, 198 (2020).

21. 21.

Bistritzer, R. & MacDonald, A. H. Moiré bands in twisted double-layer graphene. Proc. Natl Acad. Sci. 108, 12233 (2011).

22. 22.

Kang, J. & Vafek, O. Symmetry, maximally localized wannier states, and a low-energy model for twisted bilayer graphene narrow bands. Phys. Rev. X. 8, 031088 (2018).

23. 23.

Po, H. C., Zou, L., Senthil, T. & Vishwanath, A. Faithful tight-binding models and fragile topology of magic-angle bilayer graphene. Phys. Rev. B 99, 195455 (2019).

24. 24.

Koshino, M. et al. Maximally Localized Wannier Orbitals and the Extended Hubbard Model for Twisted Bilayer Graphene. Phys. Rev. X 8, 031087 (2018).

25. 25.

Po, H. C., Zou, L., Vishwanath, A. & Senthil, T. Origin of Mott insulating behavior and superconductivity in twisted bilayer graphene. Phys. Rev. X 8, 031089 (2018).

26. 26.

Kang, J. & Vafek, O. Strong coupling phases of partially filled twisted bilayer graphene narrow bands. Phys. Rev. Lett. 122, 246401 (2019).

27. 27.

Liu, J. & Dai, X. Theories for the correlated insulating states and quantum anomalous Hall effect phenomena in twisted bilayer graphene. Phys. Rev. B 103, 035427 (2021).

28. 28.

Kang, J. & Vafek, O. Non-Abelian Dirac node braiding and near-degeneracy of correlated phases at odd integer filling in magic-angle twisted bilayer graphene. Phys. Rev. B 102, 035161 (2020).

29. 29.

Xie, M. & MacDonald, A. H. Nature of the correlated insulator states in twisted bilayer graphene. Phys. Rev. Lett. 124, 097601 (2020).

30. 30.

Ahn, J., Park, S. & Yang, B.-J. Failure of Nielsen-Ninomiya theorem and fragile topology in two-dimensional systems with space-time inversion symmetry: application to twisted bilayer graphene at magic angle. Phys. Rev. X. 9, 021013 (2019).

31. 31.

Po, H. C., Watanabe, H. & Vishwanath, A. Fragile topology and Wannier obstructions. Phys. Rev. Lett. 121, 126402 (2018).

32. 32.

Liu, J., Ma, Z., Gao, J. & Dai, X. Quantum valley Hall effect, orbital magnetism, and anomalous Hall effect in twisted multilayer graphene systems. Phys. Rev. X. 9, 031021 (2019).

33. 33.

Wu, F. & DasSarma, S. Collective excitations of quantum anomalous Hall ferromagnets in twisted bilayer graphene. Phys. Rev. Lett. 124, 046403 (2020).

34. 34.

Zhang, Y., Jiang, K., Wang, Z. & Zhang, F. Correlated insulating phases of twisted bilayer graphene at commensurate filling fractions: A Hartree-Fock study. Phys. Rev. B 102, 035136 (2020).

35. 35.

Liu, J., Liu, J. & Dai, X. Pseudo Landau level representation of twisted bilayer graphene: Band topology and implications on the correlated insulating phase. Phys. Rev. B 99, 155415 (2019).

36. 36.

Tarnopolsky, G., Kruchkov, A. J. & Vishwanath, A. Origin of magic angles in twisted bilayer graphene. Phys. Rev. Lett. 122, 106405 (2019).

37. 37.

Carr, S., Fang, S., Zhu, Z. & Kaxiras, E. Exact continuum model for low-energy electronic states of twisted bilayer graphene. Phys. Rev. Res. 1, 013001 (2019).

38. 38.

Ren, Y., Gao, Q., MacDonald, A. H. & Niu, Q. WKB estimate of bilayer graphene’s magic twist angles. Phys. Rev. Lett. 126, 016404 (2021).

39. 39.

Guinea, F. & Walet, N. R. Electrostatic effects, band distortions, and superconductivity in twisted graphene bilayers. Proc. Natl Acad. Sci. 115, 13174 (2018).

40. 40.

Guinea, F. & Walet, N. R. Continuum models for twisted bilayer graphene: effect of lattice deformation and hopping parameters. Phys. Rev. B 99, 205134 (2019).

41. 41.

Yuan, N. F. Q. & Fu, L. Model for the metal-insulator transition in graphene superlattices and beyond. Phys. Rev. B 98, 045103 (2018).

42. 42.

Xu, C. & Balents, L. Topological superconductivity in twisted multilayer graphene. Phys. Rev. Lett. 121, 087001 (2018).

43. 43.

Bultinck, N. et al. Ground state and hidden symmetry of magic-angle graphene at even integer filling. Phys. Rev. X 10, 031034 (2020).

44. 44.

Soejima, T., Parker, D. E., Bultinck, N., Hauschild, J. & Zaletel, M. P. Efficient simulation of Moiré materials using the density matrix renormalization group. Phys. Rev. B 102, 205111 (2020).

45. 45.

Lian, B. et al. TBG IV: Exact insulator ground states and phase diagram of twisted bilayer graphene. arXiv:2009.13530 (2020).

46. 46.

Kwan, Y. H. et al. Kekulé spiral order at all nonzero integer fillings in twisted bilayer graphene. arXiv:2105.05857 (2021).

47. 47.

Raghu, S., Qi, X.-L., Honerkamp, C. & Zhang, S.-C. Topological Mott Insulators. Phys. Rev. Lett. 100, 156401 (2008).

48. 48.

Jia, Y., Guo, H., Chen, Z., Shen, S.-Q. & Feng, S. Effect of interactions on two-dimensional Dirac fermions. Phys. Rev. B 88, 075101 (2013).

49. 49.

Capponi, S. & Läuchli, A. M. Phase diagram of interacting spinless fermions on the honeycomb lattice: a comprehensive exact diagonalization study. Phys. Rev. B 92, 085146 (2015).

50. 50.

Sun, K., Yao, H., Fradkin, E. & Kivelson, S. A. Topological insulators and nematic phases from spontaneous symmetry breaking in 2D Fermi systems with a quadratic band crossing. Phys. Rev. Lett. 103, 046811 (2009).

51. 51.

Zhu, W., Gong, S.-S., Zeng, T.-S., Fu, L. & Sheng, D. N. Interaction-Driven Spontaneous Quantum Hall Effect on a Kagome Lattice. Phys. Rev. Lett. 117, 096402 (2016).

52. 52.

Sur, S., Gong, S.-S., Yang, K. & Vafek, O. Quantum anomalous Hall insulator stabilized by competing interactions. Phys. Rev. B 98, 125144 (2018).

53. 53.

Liao, Y.-D. et al. Correlation-induced insulating topological phases at charge neutrality in twisted bilayer graphene. Phys. Rev. X 11, 011014 (2021).

54. 54.

Liao, Y.-D., Xu, X.-Y., Meng, Z.-Y. & Kang, J. Correlated insulating phases in the twisted bilayer graphene. Chin. Phys. B 30, 017305 (2021).

55. 55.

Song, Z. et al. All magic angles in twisted bilayer graphene are topological. Phys. Rev. Lett. 123, 036401 (2019).

56. 56.

Khalaf, E., Chatterjee, S., Bultinck, N., Zaletel, M. P. & Vishwanath, A. Charged skyrmions and topological origin of superconductivity in magic angle graphene. arXiv:2004.00638 (2021).

57. 57.

Zou, L., Po, H. C., Vishwanath, A. & Senthil, T. Band structure of twisted bilayer graphene: emergent symmetries, commensurate approximants, and Wannier obstructions. Phys. Rev. B 98, 085435 (2018).

58. 58.

Xie, F., Song, Z., Lian, B. & Bernevig, B. A. Topology-bounded superfluid weight in twisted bilayer graphene. Phys. Rev. Lett. 124, 167002 (2020).

59. 59.

Xu, X. Y., Law, K. T. & Lee, P. A. Kekulé valence bond order in an extended Hubbard model on the honeycomb lattice with possible applications to twisted bilayer graphene. Phys. Rev. B 98, 121406 (2018).

60. 60.

DaLiao, Y., Meng, Z. Y. & Xu, X. Y. Valence bond orders at charge neutrality in a possible two-orbital extended Hubbard model for twisted bilayer graphene. Phys. Rev. Lett. 123, 157601 (2019).

61. 61.

Vafek, O. & Kang, J. Renormalization group study of hidden symmetry in twisted bilayer graphene with Coulomb interactions. Phys. Rev. Lett. 125, 257602 (2020).

62. 62.

Bernevig, B. A., Song, Z., Regnault, N. & Lian, B. TBG III: Interacting Hamiltonian and exact symmetries of twisted bilayer graphene. arXiv:2009.12376 (2020).

63. 63.

Vafek, O. & Kang, J. Lattice model for the Coulomb interacting chiral limit of the magic angle twisted bilayer graphene: symmetries, obstructions and excitations. arXiv:2106.05670 (2021).

64. 64.

Bultinck, N., Chatterjee, S. & Zaletel, M. P. Mechanism for anomalous Hall ferromagnetism in twisted bilayer graphene. Phys. Rev. Lett. 124, 166601 (2020).

65. 65.

Liu, X. et al. Tuning electron correlation in magic-angle twisted bilayer graphene using Coulomb screening. Science 371, 1261 (2021).

66. 66.

Haldane, F. D. M. Model for a Quantum Hall Effect without Landau Levels: Condensed-Matter Realization of the “Parity Anomaly”. Phys. Rev. Lett. 61, 2015 (1988).

67. 67.

Bernevig, B. A. et al. TBG V: Exact analytic many-body excitations in twisted bilayer graphene Coulomb Hamiltonians: charge gap, Goldstone modes and absence of Cooper pairing. arXiv:2009.14200 (2020).

68. 68.

Soluyanov, A. A. & Vanderbilt, D. Wannier representation of Z2 topological insulators. Phys. Rev. B 83, 035108 (2011).

69. 69.

Brouder, C., Panati, G., Calandra, M., Mourougane, C. & Marzari, N. Exponential localization of Wannier functions in insulators. Phys. Rev. Lett. 98, 046402 (2007).

70. 70.

Marzari, N., Mostofi, A. A., Yates, J. R., Souza, I. & Vanderbilt, D. Maximally localized Wannier functions: theory and applications. Rev. Mod. Phys. 84, 1419 (2012).

71. 71.

Wang, X. & Vafek, O. Diagnosis of explicit symmetry breaking in the tight-binding constructions for symmetry-protected topological systems. Phys. Rev. B 102, 075142 (2020).

## Acknowledgements

B.-B.C. and W.L. are indebted to Shou-Shu Gong, Xian-Lei Sheng, Xu-Tao Zeng, and Tao Shi for stimulating discussions. Y.D.L. and Z.Y.M. acknowledge the RGC of Hong Kong SAR of China (Grant Nos. 17303019, 17301420 and AoE/P-701/20), MOST through the National Key Research and Development Program (Grant No. 2016YFA0300502) and the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB33000000). B.-B.C., W.L., and Z.C. acknowledge the support from the NSFC through Grant Nos. 11974036, 11834014, 12074024, and 11774018. O.V. was supported by NSF DMR-1916958, and by the National High Magnetic Field Laboratory through NSF Grant No. DMR-1157490 and the State of Florida. J.K. acknowledges the support from the NSFC Grant No. 12074276, and Priority Academic Program Development (PAPD) of Jiangsu Higher Education Institutions. We thank the Center for Quantum Simulation Sciences at Institute of Physics, Chinese Academy of Sciences, the Computational Initiative at the Faculty of Science and Information Technology Service at the University of Hong Kong, the HPC Cluster of ITP-CAS, and the Tianhe platforms at the National Supercomputer Centers in Tianjin and Guangzhou for their technical support and generous allocation of CPU time.

## Author information

Authors

### Contributions

Z.Y.M., W.L., and J.K. initiated the work. B.-B.C. and Y.D.L. performed the many-body calculations. J.K. and O.V. conducted the theoretical analysis and mean-field calculations. All authors contributed to the analysis of the results. W.L., Z.C., and Z.Y.M. supervised the project.

### Corresponding authors

Correspondence to Jian Kang, Wei Li or Zi Yang Meng.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Peer review informationNature Communications thanks Nick Bultinck, Eslam Khalaf and the other, anonymous, reviewer for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Chen, BB., Liao, Y.D., Chen, Z. et al. Realization of topological Mott insulator in a twisted bilayer graphene lattice model. Nat Commun 12, 5480 (2021). https://doi.org/10.1038/s41467-021-25438-1

• Accepted:

• Published: