Abstract
Optical lattices have an important role in advancing our understandingof correlated quantum matter. The recent implementation of orbital degrees of freedom in chequerboard^{1,2} and hexagonal^{3} optical lattices opens up a new avenue towards discovering novel quantum states of matter that have no prior analogues in solidstate electronic materials. Here, we predict that an exotic topological semimetal emerges as a parityprotected gapless state in the orbital bands of a twodimensional fermionic optical lattice. This new quantum state is characterized by a parabolic banddegeneracy point with Berry flux 2π, in sharp contrast to the π flux of Dirac points as in graphene. We show that the appearance of this topological liquid is universal for all lattices with D_{4} pointgroup symmetry, as long as orbitals with opposite parities hybridize strongly with each other and the band degeneracy is protected by odd parity. Turning on interparticle repulsive interactions, the system undergoes a phase transition to a topological insulator whose experimental signature includes chiral gapless domainwall modes, reminiscent of quantum Hall edge states.
Main
The search for topological states of matter has been a focus of theoretical and experimental studies since the discovery of the quantum Hall effect^{4}. This problem was brought to the forefront again recently by the theoretical prediction and experimental discovery of the timereversal invariant Z_{2} topological insulators in semiconductors with strong spin–orbit couplings^{5,6,7,8,9,10,11}. (For recent reviews see refs 12 and 13.) For noninteracting particles, the topological properties of insulators as well as topological superconductors have recently been classified on the basis of the antiunitary symmetries of the systems^{14,15}. However, this elegant topological classification does not apply to Fermi liquid (metal or semimetal) states owing to the existence of fermionic lowenergy modes in gapless systems. In this paper, we shall show, however, that a novel type of topologically nontrivial semimetal unexpectedly arises as a universality class for arbitrary twodimensional lattices with D_{4} point group symmetry owing to the mixing of orbitals of opposite parity. We believe that our discovery should be rather easily realizable in fermionic coldatom optical lattices.
The physics of higher orbtials in optical lattices has recently emerged as an exciting new front in both theoretical^{16,17,18} and experimental (for example, early^{19,20,21,22} and recent^{1,2,3}) studies. We specifically examine a model system that resembles the D_{4} symmetric doublewell lattice reported in refs 1 and 2, but our conclusions apply generally to other lattices with the same pointgroup symmetry. Consider the optical lattice shown in Fig. 1a with the potential Here, k=2π/a and a is the lattice constant. x and y are the coordinates in 2D configuration space. The parameters V _{1} and V _{2} are chosen to be positive. This optical lattice can be formed using a single chromatic light field following the experimental setup shown in Fig. 1b for V _{2}/V _{1}≥1/2. For completeness, we will first consider the general situation with V _{2}/V _{1}≥0. Then, we will show that the parameter range of interest in our work is V _{2}/V _{1}∼2/3>1/2, which can be realized using the proposed experimental setup shown in Fig. 1b and is discussed in the Methods section.
For V _{2}=0, the V _{1} term induces a square lattice with lattice constant a. As V _{2} increases, the potential energy at the centre of a unit cell (with coordinates (0,0)) is increased whereas the potentials near the bond centres (with coordinates (±a/2,0) and (0,±a/2)) are reduced. For V _{2}>V _{1}/2, each unit cell contains potential minima located at (±a/2,0) and (0,±a/2), as shown in Fig. 1a.
We numerically solve the band structure of this lattice by means of planewave expansions and find that band degeneracy points appear between higher orbital bands at the Γ and M points (the centre and corner of the Brillouin zone). For the lowest four bands, as shown in Fig. 2, in the small V _{2} limit, the second and third bands cross at both Γ and M points. For larger V _{2}, there are still two banddegeneracy points for the lowest four bands, but now the second and third bands only cross at M, whereas the third and fourth bands become degenerate at Γ. For even larger V _{2} (not shown), the first and second bands become degenerate at M, whereas the third and fourth bands touch at Γ. This large V _{2} limit is dominated by the same physics as in the intermediate V _{2} regime and, thus, we will only focus on the small and intermediate V _{2} in this paper.
The banddegeneracy phenomenon described above is generic and stable. In fact, as shown in the Supplementary Information, for noninteracting particles these banddegeneracy points are topologically protected and remain stable when system parameters are tuned adiabatically, as long as the lattice pointgroup symmetry is maintained (although a banddegeneracy point may move from between the n and n+1 bands to the m and m+1 bands, as shown in the examples above). As detailed in the Methods section and the Supplementary Information, near the banddegeneracy point, a 2D vector field (h_{k}) in momentum space can be defined using the Hamiltonian of the system. At momentum k, the length of this 2D vector (h_{k}) gives (half of) the energy splitting between two energy bands. For the banddegeneracy points in our model, this vector field possesses a topological defect, a vortex with winding number 2. At the vortex core, the length of the vector vanishes (h_{k}=0), indicating that the bandgap vanishes here (that is, a banddegeneracy point appears). It is this topological property that dictates the stability of the band degeneracy against any adiabatic deformation. From a mathematical point of view, this nontrivial topology can be described rigorously using the topological index of the Berry flux, which is 2π for this case.
In addition, the banddegeneracy point is also protected by the parity of the Bloch wavefunctions under space inversion. In fact, as shown in the Methods section, it turns out that all the essential physics of the topological semimetal can be understood within a simple tightbinding picture, without considering the full bandstructure theory, and the key ingredient for this phenomenon is the mixing between the orbitals of opposite parity. In the particular model we consider here, the semimetal is formed by the hybridization between the d orbital and the two p (p_{x} and p_{y}) orbitals at each lattice site.
We now discuss the instability of the topological semimetal in the presence of interaction; the details are presented in Supplementary Information. We start with the tightbinding Hamiltonian and derive an effective lowenergy theory around the Fermi point, which in this case is the degeneracy point of the third and fourth band (Fig. 2b). It turns out that this effective theory in the presence of interaction can be mapped onto a general theoretical model of dwave symmetry which was analysed in refs 23, 24 by means of the renormalizationgroup technique. Therefore, by mapping the results back from that dwave model, we obtain the universal property for the banddegeneracy point of the topological semimetal we present here. Below, we summarize the main results.
As the temperature is lowered below a critical value, T_{c}, the system undergoes a secondorder phase transition, where T_{c}∼We^{−α/N(0)V} with N(0) the density of states at the chemical potential, V the interaction strength and Wthe bandwidth. The parameter α is a dimensionless constant whose value is determined by the band structure. In our model, the order parameter describing this lowtemperature ordered phase is the zcomponent of the angular momentum , where p_{x,r} and p_{y,r} are the fermion annihilation operators of the p_{x} and p_{y} orbitals on site r. (This order parameter can be mapped to the order parameter ϕ in the general theory studied in ref. 23.)
In our system, the repulsive interaction can be reformulated as where V >0 is the interaction strength. This interaction term favours a state with nonzero angular momentum 〈L^{z}〉≠0. In an ordinary metal or insulator (or graphene^{25}), the formation of nonzero angular momentum costs kinetic energy, which usually dominates over the energy gain from interaction unless the interaction strength is very large. However, for the topological semimetal at low enough temperature, we find the energy cost for nonzero 〈L^{z}〉 from the kinetic part is always subleading compared with the energy gain from interaction. This results in the spontaneous generation of angular momentum, which is a key theoretical insight of our work.
From a symmetry point of view, this lowtemperature phase spontaneously breaks the D_{4} point group symmetry down to C_{4}, and also breaks the timereversal symmetry. This symmetrybreaking pattern belongs to the Ising universality class, resulting in two degenerate ground states with opposite angular momentum.
As for the band structure, the band degeneracy at Γ is lifted in the symmetrybroken phase. (The degeneracy at M is also lifted. However, this is not relevant to our study as that degeneracy point is located far below the chemical potential.) Hence the topological semimetal becomes a fully gapped insulator in the presence of interaction, with the gap being V 〈L^{z}〉.
This insulator turns out to be topologically nontrivial, characterized by the nontrivial value of the topological index, known as the first Chern number. The Chern number for this state is 1, which indicates that this system shares the same topological properties as the quantum Hall state with filling 1. However, in contrast to the quantum Hall effect, where the nontrivial topological state is induced by a strong external magnetic field, here the same quantum topological state of matter originates from manybody effects in the absence of any external magnetic field. In general, states with nonzero Chern number in the absence of an external magnetic field are known as the anomalous quantum effect states, first proposed in a toy model on a honeycomb lattice by Haldane^{26}. Recently, several different possible realizations of the Haldane model in cold gases have been discussed using lattice rotations^{27} or lightinduced vector potentials^{28}. In our predicted topological phase, however, interaction plays a decisive role, in sharp contrast to the noninteracting situation prevailing in the quantum Hall effect or anomalous quantum Hall effect. To the best of our knowledge, our work is the only theoretical prediction in the literature of an interactiondriven anomalous quantum Hall state.
Furthermore, if two spin components are both present in the atomic gases, the same interaction effect may lead to a timereversal invariant Z_{2} topological insulator. This phenomenon can be partially understood as an interactiondriven 2Dversion of HgTe. As pointed out in ref. 8, the combined effect of spin–orbit coupling and strain opens a gap at a 3D quadratic banddegeneracy point and leads to a 3D topological insulator. By contrast, in our 2D system, topological states arise purely because of manybody interaction effects.
To further demonstrate the topological nature of this insulating phase, we computed the band structure of this state on a cylinder, as shown in Fig. 3. Here, although the bulk modes are all gapped, there is a gapless topological chiral edge state on each of the two edges of the system, which is the direct signature of a topologically nontrivial insulator.
The phase transition being discussed in our work has a strong analogue in the Bardeen–Cooper–Schrieffer (BCS) theory of superconductivity. In particular, the two classes have similar scaling formula for the meanfield transition temperature (T_{C}∼e^{−α/N(0)V}). However, the phase transition here breaks only a discrete symmetry (time reversal) and thus belongs to the Ising universality class. In 2D, the fluctuation effect is weak for an Ising transition and longrange order is sustained at finite temperature. In contrast, the BCS transition breaks the continuous U(1)symmetry and belongs to the XY universality class. As a result, the BCS transition in 2D is a Kosterlitz–Thouless transition, whose transition temperature is strongly suppressed by phase fluctuations and is much lower than the meanfield prediction. Thus the transition temperature for our problem should be much higher than the BCS transition, if all other parameters (N(0), V, and so on) have the same value. Therefore, under equivalent conditions the phase transition predicted by us should be much easier to observe in 2D optical lattices than the corresponding BCS Kosterlitz–Thouless transition.
Beyond its theoretical significance, the topological semimetal state also has robust and unique experimental signatures. For example, the energyband structure of the unique bandcrossing degeneracy point can be detected directly using experimental techniques such as Bragg scattering^{29}, as discussed in the Supplementary Information.
At low temperatures, the system remains a topological semimetal for attractive interactions but becomes an insulator for repulsive interactions. As both the values and the signs of interaction can be tuned in ultracold gases, this phase transition, between a compressible liquid and an incompressible insulator, can be studied experimentally by measuring the compressibility for different interactions. In addition, Bragg scattering can also be used to detect the insulating gap induced by the repulsive interactions. Because the lowtemperature topological insulating state spontaneously breaks the timereversal symmetry, any experimental measurements sensitive to the timereversal symmetry, such as the Hall effect, can also be used to identify this phase.
The direct experimental evidence for a topological insulator is the gapless chiral edge state, which is a metallic state localized on the edge of a topologically nontrivial insulator. However, it is worthwhile to note that the sharp edge in the condensed matter system is absent in coldatom gases. Owing to the existence of the slowly varying trap potential, one expects the density to decrease away from the centre of the trap. Therefore, the system is a liquid near the edge because of the low filling fraction. This liquid state from incommensurate filling will hybridize with the topological edge state, which makes the observation of the topological edge states challenging in atomic systems. This difficulty can be avoided if two domains of topological insulating phases with opposite angular momenta are induced. At the domain wall between these two areas, compressible chiral domainwall states should exist. As this domain wall can be chosen to locate near the centre of the trap, far away from the trivial liquid state near the edge of the system, it should in principle provide a clean signature for the topological edge states. These domainwall modes can also be detected using Bragg scattering, where one finds that the insulating gap is reduced to zero near the domain wall. In each real experimental system, owing to the finite number of particles on a particular optical lattice, the vanishing of the insulating gap at the domain wall is in fact prohibited by finitesize effects. For topological insulators, such finitesize effects have been systematically studied and the metallic edge states are found to be detectable even for a system of about ten particles^{30}. An alternative experimental way of seeing the topological edge state would be to have a sharp trap boundary, as in a squarewell potential, which would suppress the hybridization between the trivial liquid phase and the edge topological state. In such a squarewell trap, the topological edge state should manifest itself directly.
Methods
Creation of the optical lattice.
In the experimental setup shown in Fig. 1b of the Letter, by superimposing two monochromatic optical standing waves oscillating in phase, we implement the electric field The corresponding lightshift potential is U(x,y)=−χE(x,y)^{2}, with χ denoting the real part of the polarizability. It is straightforward to check that this potential is identical to the potential we propose in the main text, up to a trivial constant: with By choosing blue detuning, that is, χ<0, we obtain V _{1}>0 and V _{2}>0. When the polarization direction, α, is changed, the ratio V _{2}/V _{1}can be tuned to any value above 1/2. For example, using fermionic potassium ^{40}K with a principal fluorescence line at 767 nm, a standard green frequencydoubled Nd:YAGlaser (532 nm) would be a suitable light source for implementing the desired optical potential.
Hybridization between orbitals of opposite parity.
It turns out that all the essential physics of the topological semimetal can be understood within a simple tightbinding picture and the key ingredient for this phenomenon is the mixing between orbitals with opposite parities under space inversion. Here, we outline the main procedures and results of the calculation for the mixing of parityeven ${d}_{{x}^{2}{y}^{2}}$ and parityodd p_{x} and p_{y} orbitals, and defer the details (for example, model Hamiltonian, band structure, and so on) to Section S6 of the Supplementary Information. In this study, these three orbital bands are considered next to the chemical potential level and all other orbitals are assumed to be separated far from them (such that their effects can be dynamically ignored). When the mixing between the two types of orbitals is weak, the parityodd orbitals form two bands, which cross each other at the Γ and M points, wheras the band formed by the parityeven orbitals shows no degeneracy (similar to the situation shown in Fig. 2a, which was obtained by numerical diagonalization). In contrast, as the mixing between different types of orbitals is enhanced, the three bands formed by these three orbitals hybridize together, and now the middle band crosses with both the other two bands, one at Γ and another at M, similar to Fig. 2b. In fact, the top three bands shown in Fig. 2 are mainly contributed by the p_{x}, p_{y} and d orbitals. In the Supplementary Information, a full comparison is provided between the band structure of the optical lattice model defined by the potential equation (1) and that of the effective threeorbital (p_{x},p_{y},${d}_{{x}^{2}{y}^{2}}$) tightbinding model.
Instability under infinitesimal repulsion.
Using the conclusions from ref. 23, we found that under the renormalization group, the repulsive interaction shown in equation (2) is a marginally relevant perturbation and it is also the only relevant perturbation for spinless fermions with shortrange interactions. Therefore, at low temperature, this interaction term dominates the lowenergy physics and will stabilize a state with nonzero angular momentum 〈L^{z}〉≠0. This state is a topological insulator with Chern number 1, in agreement with the general study shown in ref. 23. This conclusion is further verified in Fig. 3, where we examined the meanfield singleparticle spectrum for a cylindrical geometry and observed the gapless chiral edge states.
References
 1.
Wirth, G., Ölschläger, M. & Hemmerich, A. Evidence for orbital superfluidity in the pband of a bipartite optical square lattice. Nature Phys. 7, 147–153 (2011).
 2.
Ölschläger, M., Wirth, G. & Hemmerich, A. Unconventional superfluid order in the f band of a bipartite optical square lattice. Phys. Rev. Lett. 106, 015302 (2011).
 3.
SoltanPanahi, P., Lühmann, DS., Struck, J., Windpassinger, P. & Sengstock, K. Quantum phase transition to unconventional multiorbital superfluidity in optical lattices. Preprint at http://arxiv.org/abs/1104.3456 (2010).
 4.
Nayak, C., Simon, S. H., Stern, A., Freedman, M. & Das Sarma, S. NonAbelian anyons and topological quantum computation. Rev. Mod. Phys. 80, 1083–1159 (2008).
 5.
Kane, C. & Mele, E. Z_{2} topological order and the quantum spin Hall effect. Phys. Rev. Lett. 95, 146802 (2005).
 6.
Bernevig, B. A., Hughes, T. L. & Zhang, SC. Quantum spin Hall effect and topological phase transition in HgTe quantum wells. Science 314, 1757–1761 (2006).
 7.
Konig, M. et al. Quantum spin Hall insulator state in HgTe quantum wells. Science 318, 766–770 (2007).
 8.
Fu, L. & Kane, C. L. Topological insulators with inversion symmetry. Phys. Rev. B 76, 045302 (2007).
 9.
Moore, J. E. & Balents, L. Topological invariants of timereversalinvariant band structures. Phys. Rev. B 75, 121306 (2007).
 10.
Roy, R. Z_{2} classification of quantum spin Hall systems: An approach using timereversal invariance. Phys. Rev. B 79, 195321 (2009).
 11.
Hsieh, D. et al. A topological Dirac insulator in a quantum spin Hall phase. Nature 452, 970–974 (2008).
 12.
Hasan, M. Z. & Kane, C. L. Colloquium: Topological insulators. Rev. Mod. Phys. 82, 3045–3067 (2010).
 13.
Qi, XL. & Zhang, SC. Topological insulators and superconductors. Rev. Mod. Phys. 83, 1057–1110 (2011).
 14.
Kitaev, A. Periodic table for topological insulators and superconductors. Preprint at http://arxiv.org/abs/0901.2686 (2009).
 15.
Schnyder, A. P., Ryu, S., Furusaki, A. & Ludwig, A. W. W. Classification of topological insulators and superconductors in three spatial dimensions. Phys. Rev. B 78, 195125 (2008).
 16.
Isacsson, A. & Girvin, S. M. Multiflavor bosonic Hubbard models in the first excited Bloch band of an optical lattice. Phys. Rev. A 72, 053604 (2005).
 17.
Liu, W. V. & Wu, C. Atomic matter of nonzero momentum Bose–Einstein condensation and orbital current order. Phys. Rev. A 74, 013607 (2006).
 18.
Kuklov, A. B. Unconventional strongly interacting Bose–Einstein condensates in optical lattices. Phys. Rev. Lett. 97, 110405 (2006).
 19.
Köhl, M., Moritz, H., Stöferle, T., Günter, K. & Esslinger, T. Fermionic atoms in a three dimensional optical lattice: Observing Fermi surfaces, dynamics, and interactions. Phys. Rev. Lett. 94, 080403 (2005).
 20.
Browaeys, A. et al. Transport of atoms in a quantum conveyor belt. Phys. Rev. A 72, 053605 (2005).
 21.
Lee, P. J. et al. Sublattice addressing and spindependent motion of atoms in a doublewell lattice. Phys. Rev. Lett. 99, 020402 (2007).
 22.
Müller, T., Fölling, S., Widera, A. & Bloch, I. State preparation and dynamics of ultracold atoms in higher lattice orbitals. Phys. Rev. Lett. 99, 200405 (2007).
 23.
Sun, K., Yao, H., Fradkin, E. & Kivelson, S. A. Topological insulators and nematic phases from spontaneous symmetry breaking in 2d Fermi systems with a quadratic band crossing. Phys. Rev. Lett. 103, 046811 (2009).
 24.
Sun, K. & Fradkin, E. Timereversal symmetry breaking and spontaneous anomalous Hall effect in Fermi fluids. Phys. Rev. B 78, 245122 (2008).
 25.
Das Sarma, S., Adam, S., Hwang, E. H. & Rossi, E. Electronic transport in twodimensional graphene. Rev. Mod. Phys. 83, 407–470 (2011).
 26.
Haldane, F. D. M. Model for a quantum Hall effect without Landau levels: Condensedmatter realization of the parity anomaly. Phys. Rev. Lett. 61, 2015–2018 (1988).
 27.
Wu, C. Orbital analogue of the quantum anomalous Hall effect in pband systems. Phys. Rev. Lett. 101, 186807 (2008).
 28.
Stanescu, T. D., Galitski, V. & Das Sarma, S. Topological states in twodimensional optical lattices. Phys. Rev. A 82, 013608 (2010).
 29.
Ernst, P. T. et al. Probing superfluids in optical lattices by momentumresolved Bragg spectroscopy. Nature Phys. 6, 56–61 (2010).
 30.
Varney, C. N., Sun, K., Rigol, M. & Galitski, V. Interaction effects and quantum phase transitions in topological insulators. Phys. Rev. B 82, 115125 (2010).
Acknowledgements
We appreciate the very helpful discussions with L. Fu, C. L. Kane and XL. Qi. The work of K.S. and S.D.S. is supported by JQINSFPFC, AFOSRMURI, ARODARPAOLE, and AROMURI. W.V.L. is supported by ARO (W911NF0710293 and 1110230), ARODARPAOLE (W911NF0710464) and the National Basic Research Program of China (Grant No 2012CB922101). A.H. acknlowledges support by DFGSFB925. We thank the Kavli Institute for Theoretical Physics at UCSB for its hospitality where this research is supported in part by National Science Foundation Grant No. PHY0551164.
Author information
Affiliations
Condensed Matter Theory Center and Joint Quantum Institute, Department of Physics, University of Maryland, College Park, Maryland 20742, USA
 Kai Sun
 & S. Das Sarma
Department of Physics and Astronomy, University of Pittsburgh, Pittsburgh, Pennsylvania 15260, USA
 W. Vincent Liu
Kavli Institute for Theoretical Physics, University of California, Santa Barbara, California 93106, USA
 W. Vincent Liu
Center for Cold Atom Physics, Chinese Academy of Sciences, Wuhan 430071, China
 W. Vincent Liu
Institut für LaserPhysik, Universität Hamburg, Luruper Chaussee 149, 22761 Hamburg, Germany
 Andreas Hemmerich
Authors
Search for Kai Sun in:
Search for W. Vincent Liu in:
Search for Andreas Hemmerich in:
Search for S. Das Sarma in:
Contributions
W.V.L., K.S., and S.D.S. planned the work. K.S. and W.V.L. carried out most of the calculations with input from S.D.S. A.H. provided the experimental protocol. All authors contributed to the writing of the manuscript.
Competing interests
The authors declare no competing financial interests.
Corresponding author
Correspondence to W. Vincent Liu.
Supplementary information
PDF files
 1.
Supplementary Information
Supplementary Information
Rights and permissions
To obtain permission to reuse content from this article visit RightsLink.
About this article
Further reading

1.
npj Quantum Materials (2018)

2.
Scientific Reports (2018)

3.
Dirac Magnon Nodal Loops in Quasi2D Quantum Magnets
Scientific Reports (2017)

4.
New family of Dirac and Weyl semimetals in XAuTe (X = Na, K, Rb) ternary honeycomb compounds
Science China Physics, Mechanics & Astronomy (2016)

5.
Tightbinding models for ultracold atoms in optical lattices: general formulation and applications
Science China Physics, Mechanics & Astronomy (2016)