Fermion–boson many-body interplay in a frustrated kagome paramagnet

Kagome-nets, appearing in electronic, photonic and cold-atom systems, host frustrated fermionic and bosonic excitations. However, it is rare to find a system to study their fermion–boson many-body interplay. Here we use state-of-the-art scanning tunneling microscopy/spectroscopy to discover unusual electronic coupling to flat-band phonons in a layered kagome paramagnet, CoSn. We image the kagome structure with unprecedented atomic resolution and observe the striking bosonic mode interacting with dispersive kagome electrons near the Fermi surface. At this mode energy, the fermionic quasi-particle dispersion exhibits a pronounced renormalization, signaling a giant coupling to bosons. Through the self-energy analysis, first-principles calculation, and a lattice vibration model, we present evidence that this mode arises from the geometrically frustrated phonon flat-band, which is the lattice bosonic analog of the kagome electron flat-band. Our findings provide the first example of kagome bosonic mode (flat-band phonon) in electronic excitations and its strong interaction with fermionic degrees of freedom in kagome-net materials.

T he kagome-net, a pattern of corner-sharing triangular plaquettes, has been a fundamental model platform for exotic states of matter, including quantum spin liquids and topological band structures [1][2][3] . Recently, the transition metal-based kagome metals [4][5][6][7][8][9][10][11][12][13] are emerging as a new class of topological quantum materials to explore the interplay between frustrated lattice geometry, nontrivial band topology, symmetry-breaking order, and many-body interaction. A kagome lattice tight-binding model generically features a Dirac crossing and a flat-band, which are the fundamental sources of nontrivial topology and strong correlation. Such topological fermionic structures arising from the correlated 3d electrons in the kagome lattice have been widely reported in several quantum materials [4][5][6][7][8][9][10][11][12][13] , including Mn 3 Sn, Fe 3 Sn 2 , Co 3 Sn 2 S 2 , TbMn 6 Sn 6 , FeSn, and CoSn. In parallel, the band dispersion of bosonic excitations on a kagome lattice also features Dirac crossings and flat-bands, as demonstrated in photonic crystals 14,15 . A question naturally arising when studying kagome lattice electrons is the possibility of a nontrivial many-body interplay between the bosonic kagome lattice phonons and the fermionic quasiparticles.
Such fermion-boson interactions often manifest as a perturbation of the bare band structures at very low-energy scales. However, most kagome lattice materials exhibit complicated multi-bands near the Fermi energy [4][5][6][7][8][9][10][11][12] , severely challenging the clear identification of the many-body effect by spectroscopic methods. Among all known kagome lattice materials, the paramagnetic CoSn is recently highlighted as an outstanding kagome topological metal with much cleaner bands and simpler Fermi surface 13 , making it an ideal platform to search for the geometrical frustrated fermion-boson interaction. Here we report the discovery of fermion-boson many-body interplay in kagome lattice of CoSn, which arises from the coupling of the phonon flat-band with the kagome electrons, utilizing the low-temperature (T = 4.2 K), high energy-resolution (ΔE < 0.3 meV), atomic layer-resolved scanning tunneling microscopy.

Results
Atomic-scale visualization of kagome lattice. CoSn has a hexagonal structure (space group P6/mmm) with lattice constants 13,16 a = 5.3 Å and c = 4.4 Å. It consists of a Co 3 Sn kagome layer and an Sn 2 honeycomb layer ( Fig. 1a) with alternating stacking. The side-plane atomically resolved map of the crystal measured by transmission electron microscopy ( Fig. 1b) directly demonstrates this atomic stacking sequence along the c axis. Upon cryogenic cleaving, the surface yields either the Co 3 Sn-or Sn 2 -terminated atomic layer. The lower panel in Fig. 1c shows a highly rare topographic image of the cleaving surface that contains both terminations. It consists of a Co 3 Sn surface and islands of Sn 2 layer sitting on top. The simultaneously obtained differential conductance map directly reveals their difference in the electronic structure shown in the upper panel of Fig. 1c, where the Co 3 Sn surface has a higher density of states at the bias voltage of 100 mV. From this map, we also find that the Co 3 Sn surface has detectable impurities induced quasi-particle interferences, which is the basis for our further space-momentum investigation. Scanning the Sn 2 ( Fig. 1d) and Co 3 Sn (Fig. 1e) surfaces with higher magnification, we directly reveal their honeycomb and kagome lattice symmetry, respectively. Remarkably, our topographic image was able to resolve the fine corner-sharing triangle structure of the Co kagome lattice and the Sn atom in the kagome center (Fig. 1e). Such ultrahigh atomic resolution has not been achieved in the previous scanning tunneling studies of kagome lattice materials.
Bosonic mode coupling from kagome electrons. With the lattice structure of CoSn visualized at the atomic scale, we now study their electronic structure by measuring the differential conductance as shown in Fig. 2a. According to the first-principle calculations and photoemission measurement, the Fermi surface is dominated by a fairly simple electron-like band 13 . Strikingly, we find pronounced low-energy modulations for the spectra taken on the Co 3 Sn layer, while this feature is absent on the Sn 2 layer as shown in Fig. 2b. The observed peak-dip-hump modulation can indicate the strong bosonic mode coupling with the coherent state at the Fermi level, as similar spectroscopic features have been found in many strong coupling superconductors, including lead 17 , cuprates 18 , and iron-pnictides 19 . Such a bosonic mode often arises from phonons or spin resonances. As this material is nonmagnetic and has no detectable magnetic field dependence of tunneling spectra up to 8T (Fig. 2c) mode is more likely to arise from the coupling to phonons. Moreover, the electronic coupling to the bosonic mode can be described within the Eliashberg theory 20,21 with α 2 F(ω) where α is the coupling matrix element and F(ω) is the bosonic density of states, and this can be studied in the tunneling experiments. α 2 F(ω) is intimately related to the second differentiation of the tunneling spectra [17][18][19][20][21][22] . Analyzing the measured tunneling spectra (Fig. 2d, e), we find a Gaussian-like state in the second derivative centered at E M = 15 meV with a full width at half maximum of 9 meV (Fig. 2e), and identify it as a candidate signal related to the Eliashberg function (Fig. 2e, blue curves). The other peak features at lower energies of the second differential spectra can be expected from a coherent state at the Fermi energy as shown by the simulation curve in Fig. 2d, e.
Many-body kink in the kagome dispersion. To gain deeper insight into the bosonic mode coupling on the kagome lattice, we perform systematic spectroscopic imaging on a large Co 3 Sn area with only a few Sn 2 adatoms (Fig. 3a). By taking the Fourier transform of the differential conductance map (Fig. 3b), we obtain the quasi-particle interference (QPI) data. The QPI data at 0 meV (Fig. 3c) shows a clear ring-like signal, consistent with the intraband scattering of the dominant electron-like Fermi surface 13 centered at Γ. Thus, the low-energy QPI dispersion reflects the behavior of the electron-like band crossing the Fermi surface (Q = 2k). Analyzing the QPI dispersion along Γ-M direction in Fig. 3d, we observe a pronounced double kink feature, different from its bare band dispersion calculated by the first-principles (dashed line). The spectroscopic kink feature has been identified as a fingerprint of the bosonic mode coupling [23][24][25][26][27][28][29] and indicates a giant mode coupling strength. The energy of the QPI kink is around ±15 meV, well consistent with the mode energy E M in the second differential conductance signal. The coupling strength can be estimated from Fermi velocity renormalization where v f0 and v f are the Fermi velocities of the bare QPI band and renormalized QPI band, respectively. We also explored the QPI on the Sn 2 honeycomb lattice but did not find any clear kink. Hence the unique kagome lattice resolving capability combined with low temperature and high energy-resolution of our advanced spectroscopic technique can be the key for the kink discovery in this material.

Discussion
The pronounced kink signal from the kagome lattice allows us to analyze the electron many-body self-energy Σ(ω) and further quantify the Eliashberg function α 2 F(ω). Σ(ω) is intimately related to α 2 F(ω), the Fermi-Dirac distribution f(E) and the Bose-Einstein distribution n(ω) (see Eq. (4) in "Methods" section). The real part of the self-energy Re(Σ) is related to the energy difference between the observed kink dispersion and the bare QPI dispersion, while the imaginary part of the self-energy Im(Σ) is related to the electron band broadening, which is inversely proportional to the QPI intensity. Re(Σ) and Im(Σ) are tied to one another through the Kramers-Kronig relation, and we can more accurately acquire Re(Σ) from the QPI data in Fig. 3e. We take the shape of α 2 F(ω) in reference to the second differential conductance (Fig. 2), and tune the coupling strength λ ¼ R α 2 F ω ð Þ=ω dω and calculate the real part of the self-energy Re (Σ). We find that with λ = 1.9 with s.d. error bar of 0.3, the calculated Re(Σ) can account for that determined by the experiment (Fig. 3e), which agrees with the estimated λ from Fermi velocity renormalization. We further simulate the QPI signal with this α 2 F(ω) under the Green function formalism in Fig. 3f, which also shows reasonable consistency with the experimental data both in dispersion and intensity evolution, providing key support to our many-body analysis of the kagome fermion-boson interaction.
Having characterized the many-body fermion-boson interaction in the kagome lattice, we perform first-principles calculations of the phonon band to understand the nature of the bosonic mode. Firstly, the calculated phonon density of states exhibits a pronounced peak at 15 meV (Fig. 4a), coincide with the mode energy in experiments. Secondly, this phonon mode mainly arises from the Co 3 Sn kagome layer, consistent with the experiments. Thirdly, the calculation provides momentum space insight into the origin of this peak, in that it arises from a flat-band in phonon  Fig. 4b. Lastly, through the atomic displacement resolved calculation, we identify that the flat-band phonon is mainly associated with the Co kagome lattice vibrations confined to the line connecting the centers of two neighboring triangles (Fig. 4b, inset).
In light of the first-principle calculation, we build a kagome lattice vibration model to elucidate the striking physics. The essential momentum features of the flat-band can be well captured by this model (Fig. 4c), with the flat-band toughing a parabolic band bottom. This model is highly analytical and offers a heuristic understanding of the non-propagating nature of the kagome flat-band phonon mode. We find that the collective lattice displacement shown in the inset of Fig. 4c, a deformation from a hexagonal ring (inner six atoms) rotating clockwise or anti-clockwise, would not exert any net force on the outer atoms. Hence, such geometrically frustrated vibrations can be localized forming the phonon flat-band. It is also clear now that this phonon flat-band is the lattice analog of kagome electron flatband 8,13 , whose quadratic band touching feature distinguishes them from the isolated flat-bands in heavy-fermion systems 30    research papers but is highly anticipated to explain the giant fermion-boson interaction observed here. Our findings suggest that the flat phonon dispersion can be probed by future momentum-resolved phonon-sensitive scattering experiments including inelastic X-ray scattering and neutron scattering. The correspondence between the kagome lattice, tunneling conductance, magnetic field response, double kink feature, selfenergy analysis, first-principles, and lattice vibration model provides strong evidence and conceptual framework for the fermion-boson interaction in a geometrically frustrated topological quantum material. The nontrivial kagome band structures have been widely observed in both fermionic and bosonic systems 15,32 , but their many-body interactions were rarely experimentally observed previously. We expect the latter to be quite general in many topological quantum materials with flatbands. Such fermion-boson interactions can be the driving force for future discovery of incipient density waves and superconductivity in kagome lattice materials through pressure tuning or chemical engineering. Although our research addresses the coupling of the dispersive electrons and the flat-band phonon, it would be interesting to explore in the future the intriguing possibility of coupling of the kagome flat electron band and flat phonon band when they are tuned to the similar energies.

Methods
Sample preparation. High-quality single crystals of CoSn were synthesized by the Sn flux method. The starting elements of Co (99.99%), Sn (99.99%) were put into an alumina crucible, with a molar ratio of Co:Sn = 3:20. The mixture was sealed in a quartz ampoule under a partial argon atmosphere and heated up to 1173 K, then cooled down to 873 K with 2 K/h. The CoSn single crystals were separated from the Sn flux by using a centrifuge.
Scanning tunneling microscopy characterization. Single crystals with size up to 2 mm × 2 mm × 1 mm were cleaved mechanically in situ at 77 K in ultra-high vacuum conditions, and then immediately inserted into the STM head, already at He4 base temperature (4.2 K). The magnetic field was applied with zero-field cooling. Tunneling conductance spectra were obtained with an Ir/Pt tip using standard lock-in amplifier techniques with a root mean square oscillation voltage of 0.3 meV and a lock-in frequency of 977 Hz. Topographic images were taken with tunneling junction set up: V = −100 mV, I = 2-0.05 nA. The conductance maps are taken with tunneling junction set up: V = −50 mV, I = 0.5 nA.
Transmission electron microscopy characterization. Thin lamellae were prepared by focused ion beam cutting using a ThermoFisher Helios NanoLab G3 UC DualBeam. All samples for experiments were polished by 2 kV Ga ion beam to minimize the surface damage caused by the high energy ion beam. Transmission electron microscopy imaging, atomic resolution high-angle annular dark-field scanning transmission electron microscopy imaging and atomic-level energy-dispersive X-ray spectroscopy mapping were performed on a double Cs corrected ThermoFisher Titan Cubed Themis 300 scanning/transmission electron microscope equipped with an extreme field emission gun (X-FEG) source operated at 300 kV and super-X energy-dispersive spectrometry system.
Many-body theory for bosonic mode coupling. Due to the scattering by bosonic modes, the electrons do not have a definite energy but rather a finite lifetime and a broadened spectral function. In many-body theory, this phenomenon is characterized by the electron self-energy Σ(ω), which can be regarded as a correction to the free-electron Green function. The electron Green function is where ϵ 0 k is the electron bare energy dispersion based on first-principle calculation. And the electron spectral function is The QPI dispersion is further described by With the Eliashberg function, the electron self-energy can be described by where α 2 F(ω) is Eliashberg coupling function describing the electron-bosonic mode interaction, f(E) is the Fermi-Dirac distribution, and n(ω) is Bose-Einstein distribution at temperature T. Since in most cases, the bosonic mode energy is far less than the Fermi energy ε F , we can make the approximation that the initial and final energies of scattered electrons are close to Fermi energy. In this way, we obtain an Eliashberg coupling function solely determined by the bosonic energy distribution. An effective coupling constant can be defined as In our experiment, we found a kink in the electron dispersion at E M = ±15 meV, which implies a dominant bosonic mode with ω E = 15 meV. In the calculation, we use the Einstein model and take α 2 F(ω) as a gaussian peak centered at ω E . The calculated Re Σ(k, ω) is given in Fig. 3e, with comparison to experimental data. With the electron Green function, we can simulate the QPI dispersion Q(q, ω) by Eq. (3) convoluted with the experimental resolution, which is given in Fig. 3f.
First-principles calculation. We perform electronic structure calculations within the framework of the density functional theory using norm-conserving pseudopotentials 33 as implemented in the Quantum Espresso simulation package 34 . The exchange-correlation effects are treated within the local density approximation with the Perdew-Zunger parametrization 35 . The electronic calculations used a plane-wave energy cutoff of 80 Ry and a 12 × 12 × 12 Γ-centered k-mesh to sample the Brillouin zone. Total energies were converged to 10 −7 Ry in combination with Methfessel-Paxton type broadening of 0.01 Ry. The phonon calculation is done by using a 2 × 2 × 2 q-mesh grid centered at Γ for the sampling of phonon momenta. Starting with the experimental structure, we have fully optimized both the ionic positions and lattice parameters until the Hellmann-Feynman force on each atom is <10 −3 Ry/a.u. (10 −4 Ry) and zero-stress tensors are obtained. We find that the flat-band phonon is mainly associated with the vibrations of Co atoms in Co 3 Sn lattice confined to the line connecting the centers of two neighboring triangles as shown by the dotted line in Fig. 5a. Figure 5b shows the atom displacement resolved phonon band structure.
Kagome lattice vibration model. We consider a kagome lattice vibration model, assuming the motion of the atoms is highly anisotropic and confined to the dotted line in Fig. 5a. The analysis reproduces the kagome band structure with a flat band quadratically touching another band. We choose the vectors in real space connecting nearest neighbor kagome atoms as The three atoms α = 1, 2, 3 in each kagome unit cell can move along the directions All atoms are then coupled with the same spring constant β, except for the sign. The atoms in one unit cell have the potential energy In contrast, the inter-unit cell coupling reads E inter ¼ β 4 X R U 1;R À U 2;RÀ2a 2 2 þ U 1;R À U 3;RÀ2a 1 2 þ U 2;R þ U 3;RÀ2a 3 2 þ U 2;R À U 1;Rþ2a 2 2 þ U 3;R À U 1;Rþ2a 1 2 þ U 3;R þ U 2;Rþ2a 3 2 : ð9Þ After Fourier transformation, the potential energy reads with v k ¼ β 2 4 À1 À e 2ikÁa 2 À1 À e 2ikÁa 1 À1 À e À2ikÁa 2 4 1þ e 2ikÁa 3 À1 À e 2ikÁa 1 1 þ e À2ikÁa 3 4 The spectrum of v k is given by where k 1 , k 2 stand for the inner product of k vector with a 1 , a 2 . We can observe the characteristic flat-band and two quadratic bands.