Orbital-selective Dirac fermions and extremely flat bands in frustrated kagome-lattice metal CoSn

Layered kagome-lattice 3d transition metals are emerging as an exciting platform to explore the frustrated lattice geometry and quantum topology. However, the typical kagome electronic bands, characterized by sets of the Dirac-like band capped by a phase-destructive flat band, have not been clearly observed, and their orbital physics are even less well investigated. Here, we present close-to-textbook kagome bands with orbital differentiation physics in CoSn, which can be well described by a minimal tight-binding model with single-orbital hopping in Co kagome lattice. The capping flat bands with bandwidth less than 0.2 eV run through the whole Brillouin zone, especially the bandwidth of the flat band of out-of-plane orbitals is less than 0.02 eV along Γ−M. The energy gap induced by spin-orbit interaction at the Dirac cone of out-of-plane orbitals is much smaller than that of in-plane orbitals, suggesting orbital-selective character of the Dirac fermions.

A typical electronic band structure of kagome lattices is characterized by a Dirac-like band capped by a flat band, which can be produced by using the tight-binding method with single-orbital nearest-neighbor hopping. Recently, the linearly dispersive energy bands and dispersionless flat bands have been partially reported in two-dimensional (2D) layered kagomelattice 3d transition metals Fe 3 Sn 2 1-3 , Fe 3 GeTe 2 4,5 , Co 3 Sn 2 S 2 6-10 , and FeSn 11,12 . Topologically protected linearly dispersive bands show distinctly different behaviors from the traditional parabolic bands. When such bands are tuned to the Fermi energy (E F ), the low-energy quasiparticle excitations would be drastically different from that of the conventional parabolic-band fermions and thus lead to novel transport properties [13][14][15] . For instance, large intrinsic anomalous Hall effects associated with Dirac/Weyl nodes near E F in antiferromagnetic (AFM) 16 and ferromagnetic (FM) [8][9][10]17 kagome metals have been reported. When taking the spin-orbit coupling (SOC) into account, a small band gap can open, adding a mass term to the linearly dispersive band, and a massive Dirac fermion thus can be formed 1 . Contrasting with the linear band hosting massless or lightweight quasiparticles, flat band is dispersionless over a finite range of momentum, usually with super-heavy localized electrons, extremely singular density of states (DOS), and correlated states with broken symmetries. Because of the similarity with 2D continuum Landau levels, flat bands can induce novel quantum behaviors, like Mott insulators, magnetism, fractional quantum Hall states, even at high temperatures and superconductivity [18][19][20] , as predicted in twisted bilayer graphene 21,22 . In kagome metals, the extremely light band (Dirac fermions) and extremely heavy band (flat bands) coexist. Unconventional phenomena would happen if either of the two bands is tuned close to E F . Though the local DOS reflected by scanning tunneling spectra indicate the existence of the flat bands which have also been found in rather limited momentum space by angle-resolved photoemission spectroscopy (ARPES) in previous reported kagome materials 2,6,7,10 , the extremely capping flat band with a much smaller bandwidth runs through the entire Brillouin zone (BZ) have not been clearly found in realistic kagome lattices.
Moreover, five 3d orbitals dominate the low-lying electronic states in these 3d transition kagome materials, resulting in the presence of multiple orbital physics. Along with the increased filling of the electronic 3d shell, orbital characters of the lowenergy bands near E F can be tuned. Crystal field can also affect the orbital energy-levels in kagome lattice. The Dirac-like band and the flat band with certain orbital characters at E F should be associated with novel quantum phases, when the chemical potential were tuned properly. Furthermore, when SOC opens band gaps at the Dirac points, orbitals with different SOC strengths will cause various sizes of the band gaps. If the sizes of the band gaps have obvious distinctions among the orbitals, the Dirac cones with small gaps would be largely responsible for the novel electronic excitations, leading to orbital-selective Dirac fermions. The bands with specified orbital characters have been found in paramagnetic kagome lattice YCr 6 Ge 6 (d 4 ) 23 , as well as in helically coupled FM kagome layers of YMn 6 Sn 6 (d 5 ) and antiferromagnetically coupled FM kagome layers of FeSn (d 6 ) 11 . Mainly due to complicated magnetism, correlation effect, interlayer coupling, and even the inappropriate E F , typical kagome bands with orbital differentiation physics near E F are key features that have long been theorized but still remain elusive in real kagome materials.
In this work, we clearly demonstrate the existence of sets of typical kagome bands with their orbital characters near E F in the frustrated kagome-lattice CoSn by means of ARPES, scanning tunneling microscopy/spectroscopy (STM/S), and in combination with theoretical calculations. Because of representation of the group D 6h , five 3d orbitals are divided into three groups: in-plane d xy /d x2−y2 , out-of-plane d xz /d yz , and d z2 orbitals, with individual sets of feature for each group. We find that the SOC induced energy gap at the Dirac cone with d xy /d x2−y2 orbitals characters is much larger than that with d xz /d yz orbitals characters, suggesting that the latter can be predominant in the Dirac-like fermionic excitations. While, the flat bands near E F are mainly dominated by d xy /d x2−y2 and d xz /d yz orbitals, respectively. The bandwidth of the former is less than 0.2 eV along Γ−K and <0.1 eV along Γ−M. The latter, with dispersion <0.12 eV along Γ−K and <0.02 eV which is limited by energy resolution along Γ−M, features in touch with a quadratic band at the BZ center, which is a key feature to distinguish from the flat band in heavy fermions and Morié lattice. The d z2 orbital sinks downward to higher binding energy and mainly contributes dispersions in a narrow range of momentum space around k z~π plane, indicating 2D character of the flat bands of CoSn.

Results
The crystal structure and transport property of CoSn. CoSn is isostructural to FeSn, crystalizing in a hexagonal structure with P6/mmm (No. 191) space group, in which Co kagome lattice in single Co 3 Sn layer is the one closest to the 2D limit, as shown in Fig. 1a. High-quality single crystals were synthesized by the Sn flux method (see Supplementary Note 1). The samples were characterized by X-ray diffraction (XRD) (Supplementary Fig. 1) and the XRD pattern of ground crystals can be well fitted by using the reported structure of CoSn 24 . The c-axial electrical resistivity ρ cc (T) of CoSn as a function of temperature is shown in Supplementary Fig. 2, from which a clear metallic behavior is observed as the ρ cc (T) decreases rapidly upon cooling the sample. Further, Supplementary Fig. 3 demonstrates that CoSn is a Pauli paramagnet with very small positive magnetic moment. To directly observe the cleaved surface and kagome lattice, we performed STM experiments on the (001) surfaces of CoSn. The high-resolution STM topograph taken on the cleaved surface ( Fig. 1b) shows flat Co 3 Sn plane decorated with islands of Sn. The atomic-resolved STM topograph taken on Co 3 Sn surface (Fig. 1c) shows perfect Co kagome lattice. With an ideal 2D kagome lattice and without complicated magnetism, the electronic bands of CoSn would be advantageous for clearly addressing the fundamental questions of kagome physics both theoretically and experimentally.
Clear visualization of typical kagome bands of CoSn. The typical kagome band, characterized by a Dirac-like band capped by a flat band, is produced by using the tight-binding method with single-orbital nearest-neighbor hopping 6 , as shown in Fig. 1d. When taking SOC into account, a small band gap can open at the Dirac point, and the dispersion of the flat band is modified. The bandwidth of the flat band can be further broadened by orbital and interlayer coupling in real materials. To directly detecting the features, we performed polarization-and photon-energy-dependent ARPES measurements on (001) surfaces of CoSn. The corresponding BZ with high-symmetry points and two mirror planes (σ h and σ v ) are shown in Fig. 1e. The experimental setup shows the high-symmetry directions and the normal of the sample surface defined a mirror plane (Supplementary Fig. 4). The π (σ) geometry refers to the electric fields of the incident photons within (normal to) the mirror plane. The even (odd) orbitals with respect to the mirror plane are detected in π (σ) geometry. With selected experimental symmetries, certain orbitals combinations, e.g., d xy (odd) and d x2−y2 (even) [d xz (even) and d yz (odd)], could be enhanced or suppressed individually, thus the Dirac cones and the flat bands features could be more prominent under certain conditions. Figure 2a-d shows ARPES intensity plots along cut1, which is perpendicular to the Γ À K line, as marked on the Fermi surface (FS) plot (Fig. 1f). One can clearly see the feature of kagome bands characterized by the Dirac-like dispersion capped by the phasedestructive flat band, in spite of the intensity contrast caused by the matrix element effect under σ and π geometries. At K point, two close Dirac cones at around −0.6 eV below E F are shown in both geometries, as shown in Fig. 2a, b, e, and f. The two close cones can be much clearly seen from the momentum distribution curves (MDCs) in Fig. 2i. The two flat bands at about −0.07 and −0.3 eV below E F are revealed in σ (Fig. 2a, e) and π geometries (Fig. 2b, f), respectively. The former is an extremely flat top of a hole-like band, which contributes a hotspot around the K point at E F , as shown in Fig. 1f. The latter entangles with the Dirac-like bands and looks like a break at the center of the K point. At H point, two Dirac cones locate at about −0.42 and −0.68 eV below E F , respectively, as shown in Fig. 2c, g. The two cones can also be determined from the MDCs in Fig. 2j. With the same binding energies as at the K point, the two flat bands at the H point are also revealed in σ (Fig. 2c, g) and π geometries (Fig. 2d, h), respectively, indicating that the flat bands have negligible k z dispersions. We applied the projected 2D BZ ( Γ À K À M) in our studies of the flat bands unless further stated, due to the weak k z dispersions and the limitation of k z resolution in ARPES experiments.  Fig. 4a). With the inclusion of SOC, the two bands further hybridize and open an energy gap of~40 meV, which can be estimated from the EDCs data. We note that the flat band in touch with the quadratic band at BZ center is a key feature to distinguish from the flat band in heavy fermions and Morié lattices, but is critically missing in previous experiments. Along the Γ À M direction, the two flat bands with dispersion <0.1 eV and <0.02 eV, respectively, are much narrower than that along Γ À K. Figure 3d shows EDCs of Fig. 3b, clearly exhibiting the extremely flat band (FB 2) at the binding energy of~0.27 eV. As mentioned above, the clear capping flat band with such a small bandwidth through the entire BZ has not yet been reported. Complementary to ARPES measurements on the flat band of CoSn, we performed STS experiments. Figure 3e displays the scanning tunneling spectra (dI/dV), which reflects the local DOS. Two peaks were observed around −0.07 and −0.3 eV on both Co 3 Sn and Sn plane. The peak at −0.07 eV corresponds well to the flat band top at the K point measured by ARPES. The other peak reflects the flat band at about −0.3 eV below E F .
The band calculations. To study the kagome bands with their orbital characters computationally, we performed DFT and dynamical mean-field theory (DMFT) calculations. Results of orbital-projected DFT calculation without and with the inclusion of SOC along high-symmetry directions are shown in Fig. 4a, b, respectively, revealing several sets of orbital-selective kagome bands. Compared with calculated and observed low-energy band structures in other 3d transition AFM or FM kagome Intensity (Arb. unit)    Fig. 3 The flat bands of CoSn. a, b Intensity plots and corresponding second derivative plots along the Γ À K and Γ À M directions in σ geometry, respectively. Flat bands are indicated by the red dashed lines in (a, b). The red lines on the corresponding second derivative plots are DFT calculated bands renormalized by a factor of 1.5. The bandwidths of the flat bands are marked by the white squares and indicated by the black arrows. c EDCs of (a). d EDCs of (b). The flat bands (FB 1, and FB 2) and the parabolic band (QB) are indicated by the black dashed lines and open circles, respectively. e Scanning tunneling spectra (dI/dV) taken on the Co 3 Sn surface (red) and the Sn surface (yellow). Setpoint condition: V = 1 V, I = 100 pA. electron-like band touching with the flat bands at the Γ point extends to the K point and forms Dirac cone above E F , as discussed in Fig. 3. On the A−H−L plane, the cone with d xy /d x2−y2 orbitals characters ascends, and the cone with d xz /d yz orbitals characters descends to higher binding energy. Under the protection of time-reversal and spatial-inversion symmetry (T ⋅ P symmetry), the two cones at the K point form two nodal lines along k z direction when the SOC is neglected (Supplementary Fig. 5). With the inclusion of SOC, the two cones open gaps with energy of~96.5 meV (d xy /d x2−y2 ) and~8 meV (d xz /d yz ) in the DFT calculations (Fig. 4b), respectively. The strengths of SOC of in-plane orbitals are much stronger than those of out-ofplane orbitals, indicating the orbital-selective character of the SOC strength. The d z2 (blue) orbital mainly contributes a set of kagome band at a higher binding energy around k z~π plane, showing a strong three-dimensional character.
We quantitatively compare the features between experiments and DFT calculations with SOC on the Γ−K−M plane, details can be seen in the Supplementary Table 1. The calculated velocity (V D ) of the Dirac cones is~1.3 times more than that in the experiment, and the relative energy shift between Dirac points and flat bands (E Δ ) in calculations is~1.6 times of the experimental results. Since the flat band contributes the high DOS, the two positions of the momentum-integrated EDCs of the flat bands correspond well to DFT calculated DOS with renormalization of 1.5 in energy range, as displayed in Fig. 4c. The overall width of the peaks around −0.3 eV can be estimated as~0.2 eV. In addition, the calculated bandwidth of in-plane orbitals is almost the same with experimental value. While, the calculated bandwidth of out-of-plane orbitals needs to be renormalized about 1.5 times along Γ−K and 2 times along Γ−M possibly due to the correlation effects induced by interlayer coupling. Compared to the DFT calculation, the DFT+DMFT calculated bands (Fig. 4d and Supplementary Fig. 6) are narrower by a factor of~1.5, in good agreement with experiment. The quantitative comparisons the features between DFT+DMFT results and experimental data are even much better (Supplementary Table 1

Discussion
We have unambiguously demonstrated the existence of sets of typical kagome bands with different orbitals characters in CoSn. The flat bands with bandwidth less than 0.2 eV run through the whole BZ, especially the bandwidth of the flat band of d xz /d yz orbitals is less than 0.02 eV along Γ−M which have yet been found in pervious reports. The flat band top at E F could induce instabilities such as superconductivity or ferromagnetism even at high temperatures, if the chemical potential were tuned properly. The band gap induced by SOC at the Dirac cone with d xz /d yz orbital characters is much smaller than that with d xy /d x2−y2 orbital characters, suggesting orbital-selective Dirac fermions in low-energy excitations. In addition, experimental observation of the extremely flat band over the whole BZ and in touch with the quadratic band at the BZ center is crucial in the fundamental kagome model. With an ideal 2D kagome lattice and without complication from magnetism, CoSn is a close-to-textbook example of kagome bands with interesting orbital differentiation physics, can be described by the long-sought minimal kagome tight-binding model.  The mixture was sealed in a quartz ampoule under partial argon atmosphere and heated up to 1173 K, then cooled down to 873 K with 2 K/h. The CoSn single crystals were separated from the Sn flux by using centrifuge. The samples were characterized by powder XRD and transport measurements, including temperature dependence of resistivity and magnetic susceptibility, and field dependence of magnetization, as shown in Supplementary Figs. 1-3.
ARPES experiments. ARPES measurements were performed at Dreamline and 03U beamlines of Shanghai Synchrotron Radiation Facility (SSRF). Experimental setup is shown in Supplementary Fig. 4. Samples were cleaved in situ, yielding a flat mirror-like (001) surface. During measurements, the temperature was kept at T = 20 K. The pressure was maintained <5 × 10 −11 Torr.
STM/S experiments. STM experiments were carried out with a Unisoku lowtemperature STM at the base temperature of 4.3 K. CoSn single crystal samples were cleaved in situ under ultra-high vacuum and transferred immediately into STM head under the vacuum of 2 × 10 −10 Torr. STS measurements were done by using standard lock-in technique with 5 mV modulation at the frequency of 914 Hz.
Band calculations. DFT calculation was done in the nonmagnetic phase, whereas DFT+DMFT calculation was done in the paramagnetic phase. For DFT calculations, we used the full-potential linear augmented plane wave method implemented in Wien2K 26 in conjunction with Perdew-Burke-Ernzerhof generalized gradient approximation 27 of the exchange correlation functional. DFT+DMFT was implemented on top of Wien2K and documented in ref. 28 . In the DFT+DMFT calculations, the electronic charge was computed self-consistently on DFT+DMFT density matrix. The quantum impurity problem was solved by the continuous time quantum Monte Carlo method 29,30 , at a temperature of 116 K and with a Hubbard U = 5.0 eV and Hund's rule coupling J = 0.8 eV in the paramagnetic state. The same values as we used for many iron-based compounds [31][32][33] . The experimental crystal structure (space group P6/mmm, No. 191) of CoSn with lattice constants a = b = 5.275 Å and c = 4.263 Å was used in the calculations.
Data availability