## Introduction

Fermi level multivalley states occurring around distinct wavevectors of some semimetal surfaces provide a basis for the occurrence of charge density waves (CDWs).1,2,3 The mechanism of multivalley CDWs at semiconductor surfaces, first described by Kelly and Falicov in the 70 s is conceptually similar to that of 1D Peierls CDWs.4 Both mechanisms are based on an almost perfect nesting between states with wavevectors k concentrated around well-defined values at the Fermi level. In the ordinary two (three)-dimensional case, where nesting is defined by stationary diameters across the Fermi contour (surface), fluctuations admix states with any Δk connecting the continuum of states at the Fermi level. In multivalley systems with narrow pockets, fluctuations can only mix states with Δk either very small or close to the intervalley nesting vectors, thus making intervalley CDWs more robust against fluctuations. Notably, in semiconductors and semimetals the Fermi level shifts substantially with temperature, so that the critical behaviour of a multivalley CDW is not just a matter of fluctuations as for ordinary CDWs. In addition, multivalley CDWs are generally, albeit not necessarily, stabilised by electron–phonon (e–ph) interaction.3,5,6,7

Helium atom scattering (HAS) has been recently recognised as a powerful tool to investigate the mode-selected e–ph interaction at the surface of conducting or highly polarisable surfaces.8,9,10 The ability relies on the fact that the exchange of energy and parallel momentum between the neutral probe atom and the atoms of the solid can only occur via the interposed surface electrons whose wave functions extend outward up to the scattering atom turning point. More precisely the inelastic-scattering amplitudes from surface phonons are proportional to the phonon-induced surface charge density oscillations, and therefore to the mode-selected e–ph interaction.9,11

Given this mechanism, a natural question is whether the energy and parallel momentum provided by the atom can be retained by low-energy surface electronic excitations, rather than being transmitted to phonons, and reveal their dispersion curves. Since the band structure at the Fermi level of semimetal surfaces like Bi(111) and Sb(111) is characterised by sets of narrow surface electron and hole pockets in the reciprocal space,12,13,14 multivalley CDWs are expected to be observed in HAS diffraction patterns.

The high sensitivity of HAS to CDWs as compared to other surface probes allowing for a precise determination of the transition temperatures and critical exponents was demonstrated long ago for layered chalcogenides15,16 as well as recently for the 1D topological metal Bi(114).17 The observation of phason dispersion curves with HAS was previously reported for the long-period reconstruction of the Au(111) $$23 \times \sqrt 3$$ surface18 and another hint comes from the non-adiabatic Kohn-anomalies observed for Mo(110):H and W(110):H19,20,21,22 which are associated with electron–hole pair excitations. However, a signature of collective surface electronic excitations in elastic atom–surface scattering spectra has remained missing up to date.

With regard to the ideal surfaces, there is some consensus about whether CDWs can exist in Bi(111),12,13 whereas they seemed to be excluded in Sb(111).13 On the other hand the spin–orbit (SO) coupling, which makes Sb(111) a weak topological insulator with spin-split bands around the zone centre,23 can in principle support a spin-density wave.14

Here we show that the spin-selected nesting between the states of narrow pockets at the Fermi level, peculiar of topological semimetals, yields sharp features in the HAS diffraction spectra corresponding to multivalley CDWs, and clear signatures of collective phase and amplitude CDW excitations in the THz spectral region. Inelastic HAS measurements on Sb(111) at two different temperatures reveal two additional branches in the gap below the Rayleigh wave branch. On the basis of density functional perturbation theory (DFPT) calculations of the surface phonon dispersion curves of Sb(111),23 it appears that the two low-energy branches cannot be attributed to any possible phonon branch of the ideal surface. Their dispersion curves appear to be compatible with the Brillouin zone (BZ) of the commensurate surface electron CDW lattice, which suggests their assignment to a phason/ampliton pair.

Sb(111) exhibits a rhombohedral crystal structure (A7, space group $${\mathrm{R}}\overline {\mathrm{3}} {\mathrm{m}}$$), where atoms are arranged in bilayers normal to the [111]-direction.24,25 Intralayer bonding is mostly covalent, whereas bilayers are held together by weaker interactions which permits an easy cleavage, giving rise to the hexagonal surface unit cell of Sb(111) with a lattice constant a = 4.308 Å26 (see Fig. 1b).

There are no significant changes in terms of the structure upon truncation of bulk Sb,26,27 in contrast to the electronic properties, which change significantly depending on the crystal thickness. While bulk Sb(111) is a topological semimetal,28,29 Sb thin films are predicted to become a topological insulator at a thickness under 22 layers, to exhibit a quantum spin Hall phase below 8 layers28,30 and finally to be a semiconductor at bilayer thickness.31,32 For a bismuth monolayer lattice on top of an insulator, a quantum spin hall phase at room temperature has also been recently predicted.33

A previous HAS analysis of the surface structure, step height, and terrace distribution of Sb(111) provided a surface corrugation of about 13% of the lattice constant, and confirmed the occurrence of bilayer steps, with 14% of specular reflection from surface portions including one-bilayer step, and only 2% from two-bilayer steps.34 For a HAS coherence length of the order of 50 Å the terrace width can be estimated based on the width of the specular peak, with 40–80 nm throughout several scans.35

## Results

### Elastic scattering

Figure 1c shows three HAS diffraction patterns of Sb(111) plotted versus the parallel momentum transfer ΔK. For elastic scattering ΔK is given by:

$$\Delta K = |{\mathbf{k}}_{\mathbf{i}}|\left( {\sin \vartheta _{\mathrm{f}} - \sin \vartheta _{\mathrm{i}}} \right)$$
(1)

with ki, the incident wave vector, and ϑi and ϑf the incident and final angle with respect to the surface normal, respectively (see Fig. 1a). Figure 1c shows scans along the $$\overline {\Gamma {\mathrm{K}}}$$ azimuth plotted versus ΔK, measured for an incident energy Ei varying between 11.85 and 22 meV and three different surface temperatures TS from 121 to 405 K. The intensity scale has been expanded to show additional features with smaller intensity. Besides some features assigned to bound-state resonances (e.g. the peak R) and kinematical focusing (KF), which are easily recognised due to the strong dependence of their position on the incident energy,36 there are features that occur at fixed values of ΔK, independently of the incident energy, and are therefore assigned to elastic diffraction processes, just as the two main diffraction peaks at ΔK = ±2.90 Å–1 corresponding to the periodicity of the surface atomic lattice. The additional weaker diffraction peaks occur at ΔK = ±1.45 Å–1 and ΔK = ±0.3 Å–1 (with an uncertainty of about ±0.04 Å–1).

The intensity of the sharp peaks at ΔK = ±1.45 Å–1 falling at one half of the G-vector along $$\overline {\Gamma {\mathrm{K}}}$$ decreases rapidly with temperature, faster than for the ordinary specular and diffraction peaks37 (see also the SI for an analysis of the temperature dependence). It might also be associated with a surface reconstruction disappearing above a certain temperature, however, the temperature dependence in Fig. 1c is inconsistent with that scenario. The intensity of the satellite diffraction peaks at ±0.3 Å–1 increases with temperature, contrary to the expected Debye–Waller attenuation. Since HAS is basically diffracted by the surface electron charge density, the interpretation of such superstructure should be searched for in the peculiar surface electronic structure of Sb(111) which is characterised by narrow electron and hole pockets at the Fermi level,13,38 potentially responsible for the presence of CDWs.

The diffraction patterns along the $$\overline {\Gamma {\mathrm{M}}}$$ direction36 (not reported here) show prominent features, besides the specular and diffraction peaks, which can all be assigned to bound-state resonances or kinematical focusing, from their dependence on the incident energy. Only a weak feature at –3G/2 ≈ –2.6 Å–1 (but not at +3G/2) appears to be persistent for varying incident energy, and is also vanishing for increasing temperature like that at G/2 along $$\overline {\Gamma {\mathrm{K}}}$$. The following analysis shows that the peculiar surface electronic structure of Sb(111) explains the HAS diffraction patterns measured at different surface temperatures in the $$\overline {\Gamma {\mathrm{K}}}$$ direction.

### Surface electronic structure of Sb(111)

The electronic surface states of Sb(111) have been analysed previously by Höchst and Ast38 and by Sugawara et al.13 with angle-resolved photoemission spectroscopy (ARPES) and later by Xie et al. with laser-based ARPES.39 Sb(111) exhibits a non-trivial surface electronic structure as seen in the DFT calculations including SO coupling of the band structure around the Fermi level depicted in Fig. 2a. It appears that, starting from a Dirac point at the zone centre ($$\overline \Gamma$$), two spin-split surface bands generate a narrow electron pocket at $$\overline \Gamma$$, a hole pocket in each of the six $$\overline {\Gamma {\mathrm{M}}}$$ directions and another electron pocket at the six $$\overline {\mathrm{M}}$$ symmetry points. The corresponding Fermi contours at the Fermi level are shown in Fig. 2c. They are found in excellent agreement with Xie et al. laser-based ARPES39 (Fig. 2b) and Sugawara et al. ARPES data13 (Fig. 2d).

As seen in Fig. 2d, in addition to the quasi-hexagonal Dirac ring around $$\overline \Gamma$$ (radius ≈0.07 Å–1), high-intensity spots from the pocket states crossing the Fermi level are observed in the $$\overline {\Gamma {\mathrm{M}}}$$ directions as described by DFT calculations, corresponding to hole pockets and electron pockets with maximum intensity at $$\overline {\mathrm{M}}$$. As appears in Fig. 2a the hole-pocket states have a prominent surface density of states (DOS) around the edge at Kh ≈ 0.15 Å–1 in good agreement with ARPES observation in Fig. 2d.

The presence of electron pockets at the $$\overline {\mathrm{M}}$$-points and of hole-pockets at $$\approx \overline {\Gamma {\mathrm{M}}} /4$$, and the close correspondence between the positions of the additional diffraction peaks and the corresponding intervalley separations ΔK in the $$\overline {\Gamma {\mathrm{K}}}$$ direction is a compelling evidence for multi-valley CDWs. As shown by Kelly and Falicov1, this is possible, provided the electron–phonon contribution to the intervalley exchange interaction is sufficiently strong. The authors remark that “the electron–electron interaction, via phonons is known to be the dominant intervalley exchange effect; it leads to superconductivity in some many-valley degenerate semiconductors,40 but is large and attractive even in materials which are not superconducting”. The antimony (111) surface clearly belongs to this family.

The basic mechanism for the formation of a multivalley CDW associated with the electron pockets at the $$\overline {\mathrm{M}}$$-points (K = G/2) can be understood by considering the three degenerate free-electron pocket states of equal spin at the Fermi level exp[iKx] and exp[iK$$(x/2 \pm \sqrt 3 y/2)$$] with K = G/2 and their three linear combinations A, E1, and E2 dictated by the crystal point symmetry. The density associated to each linear combination shows a CDW modulation, whereas their sum with equal weights gives a uniform density. Any perturbation, e.g. electron–phonon interaction causing a splitting of the states, and their different population at finite temperature yields a CDW. A totally symmetric ground state, would give a CDW shape:

$$|\psi _A|^2 \propto \sin ^2({\textstyle{{3Gx} \over 4}}) + \left[ {\cos ({\textstyle{{3Gx} \over 4}}) + 2\cos ({\textstyle{{\sqrt 3 Gy} \over 4}})} \right]^2.$$
(2)

Equation (2) defines a hexagonal surface CDW lattice rotated 30° with respect to the crystal lattice and a lattice constant $$a_{{\rm{CDW}}} = 2a/\sqrt 3 = 4.97$$ Å (Fig. 1b). A similar argument holds for a CDW associated with the hole states, where K = G/2 is replaced by Kh. The construction of a CDW originating from the superposition of free-electron waves from electron- and hole-pockets would lead of course to a more complex CDW lattice and a larger period. However, the temperature dependence of the diffraction spectra indicates that at low temperature the hole-pocket states play a negligible role.

The association of the additional diffraction peaks to the electron and hole-pockets stems from the peculiar temperature dependence of their intensities. On the one hand, the hole-pocket peaks increase with surface temperature, unlike ordinary diffraction peak intensities which decrease with increasing TS. On the other hand, the $$\overline {\mathrm{M}}$$ electron pocket peaks decrease faster than the ordinary diffraction peaks, possibly disappearing at ≈450 K. An analysis of the temperature dependence and the critical exponent based on the corresponding order parameter can be found in the SI. Such an anomalous temperature dependence may be attributed to a downward shift of the Fermi energy for increasing TS, which has the effect of increasing the DOS in the hole pocket and decreasing that of the $$\overline {\mathrm{M}}$$ electron pockets.

The DFT calculation tells that both pockets are already there at 0 K and the $$\overline {\mathrm{M}}$$ electron pocket minimum is at about –0.1 eV below the Fermi level (Fig. 2a). Thus a temperature increase of ≈450 K which gives rise to the disappearance of the $$\overline {\mathrm{M}}$$ electron pocket peaks, corresponds to a work function change rate of /dTS ≈ 0.22 meV K–1. The value of /dTS falls just in the range of other p-band metals, e.g. 0.22 meV K–1 for Sn and 0.20 meV K–1 for Bi41 and agrees with measurements of ϕ for Sb thin films at 77 and 300 K giving /dTS ≈ 0.18 meV K–1.42 The downward shift of the Fermi level is also responsible for the slight shift of the hole-pocket peaks to smaller absolute values of ΔK with increasing temperature (see Fig. 1c).

The above described analysis makes the CDW scenario the most plausible, despite previous indications that no CDW occurs in Sb(111). The fact that the presence of a CDW in Sb(111) has so far gone unnoticed in investigations with other experimental techniques may be explained by the higher sensitivity of HAS to charge density modulations. The smallest charge density modulation Δn detectable with HAS can be estimated based on an energy resolution ΔE ≈ 1 meV divided by the Esbjerg–Nørskov constant for low-energy He atoms, A = 750 eV/a.u.,43 yielding Δn ≈ 1.3 × 10–6 atomic units. Nevertheless, a very recent scanning tunnelling microscopy study44 has revealed for Sb(111) at a temperature of about 5 K a (2 × 2) surface structure, which is the minimum periodicity common to the $$(2/\sqrt 3 \times 2/\sqrt 3 )$$R30° CDW and the Sb(111)(1 × 1) lattice (for a short discussion see the SI, in particular Fig. S5). It follows from the above, that experimental techniques which distinctly probe the charge density over the topmost atomic layer seem to be better suited for the search of surface CDWs compared to techniques which imply some penetration depth into the bulk.

There is however, another possible explanation of the observed diffraction spectra not requiring CDWs: the quasi-elastic collisions of pocket electrons with the flying-by He atom and consequent spin-conserving intervalley transitions. A simple classical calculation conserving total energy and parallel momentum (the atom normal momentum change is as usual absorbed by the crystal center of mass) shows that for an incident He-atom energy of 15 meV, the electron energy for the ΔK = ±1.4 Å–1 transitions changes by –6.0 and +5.0 meV, respectively. These are small amounts as compared to the Fermi energy relative to the $$\overline {\mathrm{M}}$$-pocket bottom, at least at low temperature, but cause some asymmetry in the two diffraction amplitudes. Some asymmetry is noticed in Fig. 1c, but not at high temperature, where it should be more pronounced. The hole intervalley transitions would imply a much smaller energy transfer of the order of ±1 meV, whereas in the $$\overline {\Gamma {\mathrm{M}}}$$ direction the –3G/2 inter-pocket transition with larger energy transfer would cause large asymmetry, perhaps explaining the absence of the +3G/2 spot. It should be noted that in this direction the shortest inter-pocket transitions ΔK = ±G/2 would be spin forbidden and the same would hold for ΔK = ±3G/2 (the ΔK = ±G transition is covered by the lattice diffraction). Hence, since ΔK = ±3G/2 is just the shortest reciprocal vector GCDW of the CDW lattice in its [110]CDW direction the observation of a feature at –3G/2 supports the CDW scenario.

Moreover, for the alternative scenario without a CDW, inelastic transitions at special values of ΔK corresponding to single-particle inter-pocket transitions would be detected in time-of-flight (TOF) measurements. On the contrary, as shown in the following section, TOF spectra give, besides a complete set of surface phonon dispersion curves, two low-energy branches which can be assigned to the expected ampliton and phason excitations of the electron CDW lattice.45,46,47 The ability of inelastic HAS in its high-resolution 3He spin-echo version to observe phasons associated with misfit dislocations in the long-period $$23 \times \sqrt 3$$ reconstructed Au(111) surface was demonstrated by McIntosh et al.18 recently, while here CDW phasons and amplitons are observed with ordinary HAS spectroscopy.

### Dispersion curves

The dispersion curves of elementary excitations have been obtained by transforming TOF spectra of the scattered He atoms to energy transfer spectra (see Fig. S2 in the Supplementary Information). The experimental dispersion curves of surface phonons and the two additional low-energy branches assigned to phasons and amplitons measured at both 170 (only acoustic region) and 300 K (whole spectrum) are plotted in Fig. 3. The symbols represent the experimental data while the background lines correspond to the DFPT calculations. The colour scale of the DFPT calculations corresponds to the intensity of the phonon modes projected onto the first surface layer. The experimental data points are labelled according to their proximity to the calculated phonon modes, with either prominently shear vertical (SV) or longitudinal (L) polarisation on the first or third atomic layer, respectively.

As already observed in the surface phonon dispersion of Bi(111),10 the e–ph interaction on Sb(111) is sufficiently large to make vibrational modes that are mainly localised at the third layer (SV3, L3) visible for HAS. Due to the e–ph interaction, large modulations of the surface charge density are induced,10 which is also important in observing comparatively large inelastic intensities from surface optical modes.48 The property is common to other layered structures, as recently shown theoretically49 and experimentally50,51 for Bi2Te3(111) and Bi2Se3(111). In general the overall agreement of the measurements with the calculations is very good.

In addition to the data points which are reproduced by the DFPT calculations, there are data points plotted as grey stars in Fig. 3, which cannot be assigned to any surface vibrational mode. These data points form two separate branches, the aforementioned phasons and amplitons, which becomes particularly evident for the measurements at TS = 170 K along the $$\overline {\Gamma {\mathrm{KM}}}$$ direction. The features leading to these modes are small in intensity but consistently present over the whole measuring range as can be seen in the SI (Figs. S2 and S3). The broad maximum, falling in both branches at one half of the $$\overline {\Gamma {\mathrm{KM}}}$$ segment, is seen to occur at 0.7 Å–1 (green arrow in Fig. 3a), which is the $$\overline {\mathrm{M}} _{{\rm{CDW}}}$$ symmetry point of the CDW BZ (Fig. 3, inset). This correspondence and the approximate specularity of the branches with respect to $$\overline {\mathrm{M}} _{\rm{CDW}}$$ suggest to assign the two branches to CDW excitations, i.e. phasons and amplitons. Note that the commensurate excitations of a commensurate electron CDW have a finite energy at ΔK = 0 (here also at the $$\overline {\mathrm{M}} \equiv \overline \Gamma _{\rm{CDW}}$$ point) due to the pinning to the lattice.

For a phenomenological description of CDW excitations the multipole expansion (ME) model52 is applied to the M-electron CDW lattice, viewed as a 2D periodic lattice of electron shells subject to dynamical multipolar deformations. In the simplest picture the latter are restricted to dipolar deformations (rigid displacements) either normal (amplitons) or parallel longitudinal (phasons) to the surface. While the corresponding degrees of freedom are endowed with an effective inertial mass μ, the atomic lattice is assumed to be rigid (infinite mass) so as to ensure adiabaticity, and to exert on the electron shells an isotropic pinning force constant f0 (see “CDW structure and dynamics within the ME model” in the SI). The two dispersion curves can then be fairly well reproduced with just nearest neighbour (nn) and next nn (2n) force constants for normal (ampliton) and parallel longitudinal (phason) shell (dipolar) displacements, f1α and f1φ, and f2α and f2φ, respectively. The fit shown in Fig. 4 (effective masses = 1) is obtained (see SI) with f0 = 12, f1α = 3, f2α = –2 and f1φ = 1, f2φ = –1, all in meV2/μ units.

The deviations, notably the small dips at $$\overline {\mathrm{K}}$$ may be due to the coupling either with the hole pockets, which would introduce short-period oscillations in the dispersion curves, or with phonons. A possible coupling to phonons as the cause of dips at $$\overline {\mathrm{K}}$$ in the CDW branches (and apparently also in the lowest phonon branch) deserves further investigation. The physical meaning of the fitting parameter values does not seem to be trivial, especially the fact that the 2n force constants remain comparable to the nn ones, while having opposite signs. For a pure dipole–dipole interaction f1α and f2α should both be positive with a ratio $$f_{1\alpha }/f_{2\alpha } = \sqrt {27}$$, while f1φ and f2φ should be negative and in the same ratio. Pauli repulsion for in-plane shell displacements, many-body effects and dispersion forces, as well as the amplification/depression of interactions at different distances due to the interference of the electron CDW with that of the pocket holes (see “CDW structure and dynamics within the ME model” in the SI), are expected to play a role in the effective force constants of the ME model—an issue worth being investigated from first principles.

## Discussion

In summary, we have shown that additional elastic peaks which are observed in HAS diffraction spectra from Sb(111) can be assigned, on the basis of their peculiar temperature dependence, to electron and hole multi-valley CDWs. The multi-valley structure at the Fermi level consists of six electron pockets at the $$\overline {\mathrm{M}}$$ symmetry points of the surface BZ, and six hole pockets at about 1/5 of the $$\overline {{\mathrm{KM}}}$$ segments of alternate spins, as found in existing ARPES data and present DFT calculations including SO coupling.

Furthermore, inelastic HAS spectra, besides providing the surface phonon dispersion curves in agreement with DFPT + SOC calculations, reveal two additional dispersion curves of elementary excitations in the energy region well below the Rayleigh mode. They can be associated to phasons and amplitons of the multi-valley CDW associated with the electron pockets. The fact that inelastic HAS from surface phonons exclusively occurs, for a conducting surface, through the e–ph interaction8,9,10,11 implies that also the dispersion of low-energy surface electron collective excitations can be measured by HAS. It is expected that ordinary HAS, as well as its ultrahigh-resolution 3He spin-echo version,53,54 will be regarded as a choice spectroscopy for the investigation of surface electronic excitations in the THz domain.

## Methods

### Experimental details

The experiments were carried out on the helium-atom-scattering apparatus H.A.N.S which has been described in greater detail in earlier publications.55 Figure 1a shows a schematics of the experimental setup. The energetically narrow He-atom beam (ΔE/E ≈ 2%) is created by a supersonic expansion of He-gas and scattered off the surface in a fixed 91.5° source-sample-detector geometry. Energy dispersive measurements for inelastic scattering are performed using TOF measurements with a pseudo-random chopper disc.

The sample under investigation was an antimony (Sb) single crystal cleaved in the (111) direction in shape of a disc of 15 mm diameter and 2 mm thickness. The crystal was cleaned by several cycles of Ar+ sputtering and subsequent annealing at 475 K. The contamination and cleanness were checked by auger electron spectroscopy and the intensity of the diffuse elastic peak. The sample holder can be cooled using liquid nitrogen and heated using a button heater while the temperature is measured by a type K thermocouple. The used He beam energies range from 11.8 to 36.6 meV and the experiments were carried out at constant surface temperatures between 120 and 405 K.

### Computational details

Calculations have been performed within DFT by using a norm conserving fully relativistic pseudopotential and the Perdew–Burke–Ernzerhof generalised gradient approximation for the exchange and correlation functional.56 SO coupling effects are taken into account self-consistently with the formalism of non-collinear spin magnetisation as implemented in the QE distribution.57 The Kohn–Sham wave functions are expanded in plane waves up to a kinetic-energy cutoff of 45 Ry. A Gaussian smearing of 0.01 Ry is used to deal with the metallic character of Sb.

In accordance with Campi et al.,23 the Sb(111) surface is modelled by a slab geometry with a single atom per layer and 24 layers. The periodic images of the slab are separated by a vacuum region 12 Å wide. The band structure and the Fermi energy are calculated for the optimised slab geometry.23 To resolve the fine details of the electronic properties of interest, we first performed a self-consistent calculation with integration of the BZ over a (48 × 48 × 1) Monkhorst–Pack58 mesh followed by non self-consistent calculations over highly dense meshes.