Orbital-enhanced Warping Effect in P\textsubscript{x},P\textsubscript{y}-derived Rashba Spin Splitting of Monatomic Bismuth Surface Alloy Surface Alloy

Spin-split Rashba bands have been exploited to efficiently control the spin degree of freedom of moving electrons, which possesses a great potential in frontier applications of designing spintronic devices and processing spin-based information. Given that intrinsic breaking of inversion symmetry and sizeable spin-orbit interaction, two-dimensional (2D) surface alloys formed by heavy metal elements exhibit a pronounced Rashba-type spin splitting of the surface states. Here, we have revealed the essential role of atomic orbital symmetry in the hexagonally warped Rashba spin-split surface state of $\sqrt{3}\times\sqrt{3} R30^{\circ}$ BiCu$_{2}$ monatomic alloy by scanning tunneling spectroscopy (STS) and density functional theory (DFT). From $\mathrm{d}I/\mathrm{d}U$ spectra and calculated band structures, three hole-like Rashba-split bands hybridized from distinct orbital symmetries have been identified in the unoccupied energy region. Because of the hexagonally deformed Fermi surface, quasi-particle interference (QPI) mappings have resolved scattering channels opened from interband transitions of \textit{p$_{x},$p$_{y}$}($m_{j}=1/2$) band. In contrast to the \textit{s,p$_{z}$}-derived band, the hexagonal warping predominately is accompanied by substantial out-of-plane spin polarization $S_{z}$ up to 24\% in the dispersion of \textit{p$_{x}$,p$_{y}$}($m_{j}=1/2$) band with an in-plane orbital symmetry.


Introduction
Due to strong spin-orbit coupling, Dresselhaus pointed out that the spin degeneracy of electronic band structure can be lifted in bulk crystals lacking of inversion symmetry. [1] Same analogy can be applied to the surface or interface where the inversion asymmetry naturally exists, Rashba and Bychkov further conducted the pioneer work on the spin-orbit driven spin splitting of electronic band structure in two-dimensional (2D) systems. [2] Such spin splitting is intrinsic and particularly significant in elements with large atomic number, which also gives rise to an important feature of spin-momentum locking leading to an absence of backscattering in transport electrons. [3,4] Since an external magnetic field is no longer needed for this reduction of energy dissipating channel, Rashba-type spin splitting is then of great interest to technology application in this regard, one famous proposal belongs to the concept of Datta-Das spin transistor, which has promoted many efforts to design and pursuit Rashba effect on low-dimensional materials. [5] By means of evaporating heavy atoms onto the noble metal surfaces, the 2D binary Rashba alloys can be successfully prepared, for examples, Pb, Bi and Sb grown on Ag(111) and Cu(111). [6][7][8][9][10][11][12][13][14] With an amount of 1/3 monolayer (ML) Bi on Ag(111), the gigantic Rashba spin-splitting characterized by the Rashba parameter α R of 3.05 has been reported on BiAg 2 alloy with √3 × √3 30 0 superstructure. [7,15] Due to considerable in-plane potential gradient from each Bi atom surrounded by six Ag atoms in √3 × √3 30 0 lattice arrangement, BiAg 2 surface alloy also exhibits substantial out-of-plane spin polarization in addition to the spinmomentum locking of in-plane components. Furthermore, an important figure of merit for practical application, this 2D BiAg 2 alloy can be conveniently transferred to grow on semiconductor Si(111) substrate, i.e., the most common and compatible material used in industrial manufacturing, and a pronounced circular dichroic effect emerges as a result of the coupling between Rashba spin-split surface state and quantum-well states of Ag films. [8] These interesting phenomena and properties of one-layer-thick BiAg 2 binary alloy have mostly been investigated on the Rashba band with atomic orbital hybridization of Ag 5s and Bi 6p z , which typically can be accessed in energy region below Fermi level, i.e., occupied states of valence band, by employing the photoemission spectroscopy. Besides sp z hybridization, however, there are also Rashba bands hybridized from different atomic orbitals, e.g., the Bi 6p x p y (m j = 3/2; 1/2) and 6p z locating at energy region of unoccupied states above Fermi level. In order to study their Rashba characteristics, either depositing of various adsorbates, e.g., Na, Xe and Ar etc., to shift unoccupied Rashba bands downward to below Fermi energy or performing inverse photoemission experiment was required. [11] While a wealth of studies and understandings have been intensively focused on occupied sp z Rashba band, only few reports and relatively less are known for the unoccupied p x p y and p z -derived Rashba-split surface states in various binary surface alloys.
With high spatial and energy resolutions, tunneling spectroscopy measurement offers an alternative approach to explore Rashba physics in both occupied and unoccupied energy ranges. [6,7] According to energy-dependent quasiparticle interference (QPI) experiment, the sp z and p x p y (m j = 1/2)-derived Rashba bands in BiAg 2 alloy have been mapped out and the backscattering events above and below the Rashba energy E R are allowed under the assumption of the Bloch states. [16] From the dispersion of scattering vectors, the unoccupied p x p y (m j = 1/2) band shows the spin-orbit entangled spin texture and hybridizes with occupied sp z band leading to either a gap opening from strong spin-orbit coupling or inter-band spin orbit coupling between even and odd-orbital components. [16][17][18] Analogous to studies on BiAg 2 alloy, same experimental technique can also be applied to access the two unoccupied Rashbasplit bands of PbAg 2 surface alloy. [6] Two asymmetric peaks observed at unoccupied states in tunneling spectra clarify the larger inter-band separation, and quasiparticle interference mappings further confirm the absence of hybridization of these two unoccupied bands. [6] In this work, we study the unoccupied Rahba-split surface states of BiCu 2 surface alloy on Cu(111) by using low-temperature scanning tunneling spectroscopy (LT-STS) combined with density functional theory (DFT). Differential conductance dI/dU spectra on BiCu 2 show an asymmetric peak at 0.23 eV originated from the singularity of LDOS at band edge of Rashbasplit band formed by Cu 4s and Bi 6p z orbital hybridizations. Furthermore, dI/dU peak at 1.6 eV together with a shoulder at 1.4 eV are developed from the band crossing of four-fold degenerate p x p y (m j = 3/2) band right at ̅ point and the band edge of p x p y (m j = 1/2) band, respectively. From energy-dependent QPI mappings, we have clearly observed standing wave patterns, and two distinct scattering vectors along both ̅̅̅̅̅ and ̅̅̅̅ are able to be extracted at energy region covering the dispersion of p x p y (m j = 1/2) band. With substantial in-plane potential gradient from protrusion of Bi atoms on BiCu 2 surface, first-principle calculation reveals significant out-of-plane spin component S z in spin-dependent CECs. Such warping deformation results in opening scattering channels for possible transitions between timereversal partners from p x p y (m j = 1/2)-derived Rashba spin-split band.

Fabrication and Structure of Monatomic Bismuth Surface Alloy
According to structure models obtained from previous X-ray diffraction and STM studies, the surface alloys of Bi grown on Cu(111) have Bi coverages of 1/3 ML and 1/2 ML for √3 × √3 30 0 and [2012] phases, respectively. [19,20] This in principle provides the way how we can calibrate the amount of Bi in order to prepare the sample with well-defined and extended to study the dispersion of Rashba-split bands from energy-dependent QPI measurements. [13] From the atomic resolution image in Figure 1(c), the surface reconstruction exhibits a √3 × √3 periodicity with a 30 0 rotation to the high symmetry axis, e.g., [11 ̅ 0], of Cu (111) underneath, which is in agreement to corresponding structure model of BiCu2 alloy shown in the inset. [19,20] Since the Bi coverage of this sample is a bit higher than 1/3 ML, few isolated

Electronic Structures of Rashba Spin-split Bands
In order to determine the energy positions of different Rashba-split bands of Bi/Cu(111), the tunneling spectra have been acquired on both √3 × √3 30 0 BiCu 2 and Bi[2012] phases. derived Rashba band. [13] From the asymmetry of this peak, we can also identify that this meV is similar to ~ 20 meV obtained from photoemission studies, but our fitting of this dI/dU curve (not shown) gives E R ~ 65 meV, which might be due to the broadening of 20 meV modulation voltage used in our lock-in measurements. [7,22] In contrast to BiCu 2 , dI/dU curve measured at Bi[2012] (blue line) is rather featureless within this bias range.
To explore Rashba-split bands hybridized from p x p y orbitals on BiCu 2 , we have measured the dI/dU curve at larger bias range from 2.0 to -0.4 V and the results are shown in Figure 2(b).
Interestingly, a pronounced peak at 1.6 V as indicated by red arrow accompanied by a shoulder at 1.4 V (blue arrow) have been observed on BiCu 2 (black line), but no characteristic features can be observed from dI/dU curve taken at Bi[2012] (blue line). Since the intensity of conductance peak at 1.6 V is unusually higher than the Rashba peak at 0.23 V (black arrow), we have double checked the normalized dI/dU/(I/U) from simultaneously measured I/U curve.
The intensity of conductance peak at 1.6 V is indeed about 6 times higher than that of Rashba peak at 0.23 V (see Supplementary Figure S1 for more details), but less profound as compared to the result output directly from Lock-in amplifier.
Apart from tunneling spectroscopy measurements, we also performed first principle electronic structure calculations to reveal the origin of spectra peaks observed at different energy positions. The orbital-decomposed band structure of the surface BiCu 2 ions are shown in

QPI Mappings of Hexagonally Warped Rashba-split Surface States
To uncover the dispersion of p x p y -derived Rashba-split bands and compare their relations to sp z -derived Rashba band studied from previous work, we have carried out the QPI measurements covering both energy ranges on BiCu 2 . [13] As shown in STM topography image of Figure 3( scattering vectors q n (E). [23] Note that the energy-dependent scattering vector q n (E) is defined by q n (E) = k f (E) -k i (E) under the framework of elastic scattering, which links initial and final momentum eigenstates for mapping out the dispersion relation of surface bands. [13,16,17] Although spin-conserved scattering, i.e., k f (E) and k i (E) with the same direction of spinpolarization, is typically considered in analysis of Rashba-split band, we are also aware of spin-flip scattering which needs to be taken into account for the case of band structure with a spin-polarization inversion. [6,13,16,17,24] However, we denote that either spin-conserved or spinslip scattering only gives rise to a single scattering vector responsible for standing wave patterns from p x p y (m j = 1/2)-derived Rashba-split surface state.
Another feature worth being mentioned is the hexagonal shape of CEC on p x p y (m j = 1/2)derived Rashba band and this signature has been experimentally observed at both BiAg 2 and BiCu 2 surface alloys, which can be associated with a coupling between √3 × √3 30 0 crystalline structure and symmetry of p x p y atomic orbitals. [15,25] According to hexagonal warping effect on topological insulators (TIs), multiple pairs of stationary k points on warped CEC can contribute to additional scattering vectors for the emergence of standing wave patterns as verified by several QPI results before. [26][27][28][29]32] As shown in the schematic drawing at the bottom of Figure 3 can be used to identify ̅̅̅̅̅ and ̅̅̅̅ directions in reciprocal space. In addition to that, realspace standing waves produce separate diffraction patterns in FFT image as indicated by red and blue arrows, and they evolve as a function of bias voltage, e.g., the diffraction patterns shrink as the bias voltage increases, which enables us to trace the energy-dependent scattering vectors q n (E). and D 2 (green dots), they are formed from intra-and interband scattering processes of sp z and p x p y (m j = 1/2) Rashba-split bands. [13] Note that the empty black and greed dots are the data points we extracted from previous studies for a direct comparison. [13] The black and green lines are the parabolic fittings to obtain the effective masses ΓΜ ̅̅̅̅̅ * of (-0.34±0.02)m e and (-0.39±0.01)m e , respectively. They are slightly larger than previous QPI results due to a bit of deviations from some data points, but they are still within the error bars. We denote that these ΓΜ ̅̅̅̅̅ * values remain consistent with those reported from photoemission work. [22] Besides D 1 and D 2 , QPI results also reveal the band dispersions of D 3 (gray dots) and D 5 (blue dots) at occupied energy region. From the overlapping of some blue dots with extracted blue empty dots, we refer the D 5 dispersion to p x p y (m j = 1/2) Rashba band from intraband transitions, i.e., q 2 in previous work. [13] Note that there are not many data points visible from intraband transitions of p x p y (m j = 1/2) Rashba band, in particular at occupied states, we speculate that this is because the scattering vectors are quite large, such that they correspond to very short wavelengths of standing waves in real-space, which might be hard to detect and easily buried into background noises. As for D 3 dispersion, we associate it with the scattering between outer branch of sp z and p x p y (m j = 1/2) bands, since it starts appearing at about 0.1 eV well above the projected bulk band gap of Cu(111) substrate, whereas the strong hybridization between outer branch of sp z band and bulk states does not play significant role to forbid the emergence of this scattering vector q 3 . [13,14,25] Interestingly, two dispersions of D 4 and D 5 have been resolved for p x p y (m j = 1/2) Rashba-split band, and they are composed of scattering vectors q 4 and q 5 connecting to the scattering transitions between time-reversal partners as shown in Figure 4(b). Typically, the Rashba-split band consists of two concentric circles on CEC with an in-plane chiral spin structure perpendicularly locked to the momentum vector, leading to prohibited backscattering as a consequence of time-reversal symmetry. However, this scenario is not perfectly true if there is a warping distortion on CEC of Rashba-split band, where not only single pair of stationary points can be stabilized, but also new scattering channels can be opened, similar to hexagonal warping effect reported on TI materials. [27,[29][30][31]33] In order to verify the warping behaviour of the p x p y (m j = 1/2) Rashba-split band, we further calculate the corresponding CECs with S z component at some representative energies as shown in Figure 4

Conclusion
In conclusion, we have studied unoccupied Rashb-split surface bands on BiCu 2 binary alloy through tunneling spectroscopy experiment combined with DFT theory. According to dI/dU spectra and calculated band dispersions, we have identified an asymmetric peak at 0.23 eV from the band edge of sp z Rashba band, and a strong peak at 1.6 eV with a shoulder at 1. Theoretical Calculations: First-principles calculations are performed using the Vienna Ab initio Simulation Package (VASP) based on the density functional theory (DFT). [36][37][38] The projector-augmented-wave-type pseudopotential with the Ceperley-Alder and Perdew-Zunger (CA-PZ) type exchange-correlation functional are adopted in the local density approximation (LDA) calculations. [39][40][41][42][43] We consider the BiCu 2 monolayer on top of 9-layer Cu(111)

Supporting Information
Normalized tunneling conductance curve on BiCu 2 , dispersion of q n (E) along ̅̅̅̅ direction, energy and spin dependent CECs with S x and S y components, out-of-plane spin component S z of BiAg 2 and BiCu 2 surface alloys and FFT-QPI movie can be found in the Supporting Information, which is available from Wiley Online Library or from the author.  Hexagonally warped Rashba-split surface band develops considerable out-of-plane spin component on constant energy Fermi contour, leading to scattering transitions allowed in between time-reversal partners. The first time observation of warping effect reported here on single atomic layer of bismuth surface alloy offers a superior prospect for efficiently designing spintronic devices and processing spin-based information on low-dimensional Rashba materials.