Nematic pairing from orbital-selective spin fluctuations in FeSe

Abstract

FeSe is an intriguing iron-based superconductor. It presents an unusual nematic state without magnetism and can be tuned to increase the critical superconducting temperature. Recently it has been observed a noteworthy anisotropy of the superconducting gaps. Its explanation is intimately related to the understanding of the nematic transition itself. Here, we show that the spin-nematic scenario driven by orbital-selective spin fluctuations provides a simple scheme to understand both phenomena. The pairing mediated by anisotropic spin modes is not only orbital selective but also nematic, leading to stronger pair scattering across the hole and X electron pocket. The delicate balance between orbital ordering and nematic pairing points also to a marked k_z dependence of the hole–gap anisotropy.

Introduction

Soon after the discovery of superconductivity in iron-based systems, it has been proposed that pairing could be unconventional, i.e., based on a non-phononic mechanism.^1,2 This proposal has been triggered, from one side, by the small estimated value of the electron–phonon coupling, and, from the other side, by the proximity in the temperature-doping phase diagram of a magnetic instability nearby the superconducting (SC) one. Within an itinerant-electron picture pairing could be provided by repulsive spin fluctuations (SF) between hole and electron pockets, connected by the same wavevector characteristic of the spin modulations in the magnetic phase (see Fig. 1). This suggestion has been supported and confirmed by an extensive theoretical work, aimed from one side to establish why inter-pockets repulsion can overcome the intra-pocket one³ and from the other side to provide a quantitative estimate of the SC properties starting from the Random Phase Approximation (RPA)-based description of the SF susceptibility.^4,5

Fig. 1

FeSe Fermi surfaces at k_z = 0. a Paramagnetic phase. b Nematic phase. The colors represent the main orbital character of the Fermi surface. The green/red arrows denote the orbital-selective spin fluctuations (OSSF), connecting hole and electron pockets at different momenta, see Eqs. (6) and (7). The spin fluctuations along ΓX and ΓY are equivalent in the paramagnetic phase a and become anisotropic in the nematic one b

The success of the itinerant scenario as a unified description of Fe-based materials has been partly questioned by the discovery of superconductivity in the FeSe system. Recent experiments^6,7,8,9,10 detected sizeable SF in FeSe, however, a magnetic phase appears only upon doping. Superconductivity emerges below T_c ~ 9 K from the so-called nematic phase.¹¹ Here at temperatures below T_S = 90 K, the anisotropy of the electronic properties is far larger than what expected across a standard tetragonal-to-orthorhombic transition, suggesting that it is driven by electronic degrees of freedom.^11,12 In particular, Angle-Resolved Photoemission Spectroscopy (ARPES) experiments clearly show a dramatic change of the Fermi surface (FS) across T_S, that can be reproduced with an effective crystal-field splitting of the various orbitals.^{13,14,15,16,17,18,19,20,21}

In this situation, the explanation of the observed anisotropy of the SC gaps in FeSe becomes intimately related to the understanding of the nematic transition itself. Extensive experimental studies on the FeSe-based material, ranging from quasiparticle interference imaging^22,23 and ARPES measurements,^{24,25,26,27,28} to thermal probes,^29,30 suggest that the SC gap in FeSe is highly anisotropic on both hole and electron pockets. By defining θ the angle formed with the k_x axis measured with respect to the center of each pocket, one finds that the gap is larger at θ = 0 on the Γ pocket, where the predominant character in the nematic phase is xz,^19,26,27 and at θ = π/2 on the X pocket, where the dominant character is yz, Fig. 1b. Thus, accounting for an orbital-dependent SC order parameter does not reproduce the observed gap hierarchy, and additional phenomenological modifications of the pairing mechanism must be introduced^22,26,31 to describe the experiments.

Among the various attempts to theoretically understand the nematic phase from microscopic models, we have recently emphasized the outcomes of a theoretical approach which correctly incorporates the feedback between orbital degrees of freedom and SF.^19,32,33 From one side, the degree of orbital nesting between hole and electron pockets is crucial to determine the temperature scale where SF beyond RPA drive the spin-nematic instability,³² making SF at Q_X = (π/a, 0) and Q_Y = (0, π/a) anisotropic below T_S.³⁴ From the other side, SF renormalize the quasiparticle dispersion, so that the orbital ordering observed below T_S is a consequence of the spin nematicity, thanks to an orbital-selective shrinking mechanism.¹⁹ In this work, we show that such orbital-selective spin fluctuations (OSSF) provide also the key pairing mechanism needed to understand the SC properties of FeSe. Within an orbital-selective spin-nematic scenario, the C₄ symmetry breaking of the SF below T_S provides a pairing mechanism that is not only orbital selective but also nematic, in the sense that inter-pocket pair scattering along the ΓX and ΓY directions becomes anisotropic. As we show below, accounting only for the nematic band-structure reconstruction of the FS, the SC gap of the Γ pocket follows the modulation of the dominant xz orbital, with a weak relative maximum at θ = π/2, in striking disagreement with the experiments. The nematic pairing provided by OSSF is crucial to enhance the yz component of the SC order parameter, explaining why the anisotropy of the SC gap at Γ follows the subdominant yz orbital character of the underlying FS.^26,27 We also discuss its implications for the gap-structure measured at k_z = π (refs. ^24,25,28), where hole pocket retains a larger yz character even in the nematic phase, making the nematic pairing responsible for an enhancement of the moderate gap anisotropy triggered already by orbital-ordering effects.³⁵

Results

Model

To compute the SC properties of FeSe, we start from a low-energy model adapted from ref. ³⁶. The orbital content of each pocket is encoded via a rotation from the fermionic operators c_xz, c_yz, c_xy in the orbital basis to the ones describing the outer hole pocket (h) at Γ and the electronic pockets at X (e_X) and at Y (e_Y):

$$h_{\mathbf{k}} = u_{{\mathrm{\Gamma }},{\mathbf{k}}}c_{yz,{\mathbf{k}}} - v_{{\mathrm{\Gamma }},{\mathbf{k}}}c_{xz,{\mathbf{k}}},$$

(1)

$$e_{X,{\mathbf{k}}} = u_{X,{\mathbf{k}}}c_{yz,{\mathbf{k}}} - iv_{X,{\mathbf{k}}}c_{xy,{\mathbf{k}}},$$

(2)

$$e_{Y,{\mathbf{k}}} = u_{Y,{\mathbf{k}}}c_{xz,{\mathbf{k}}} - iv_{Y,{\mathbf{k}}}c_{xy,{\mathbf{k}}},$$

(3)

where the explicit definition of the coefficients $u_{\ell ,{\mathbf{k}}},v_{\ell ,{\mathbf{k}}}$ with $\ell = {\mathrm{\Gamma }},X,Y$ is given in Supplementary Note 1. For example, for the hole pocket in the tetragonal phase $u_{{\mathrm{\Gamma }},{\mathbf{k}}_F}\sim {\mathrm{cos}}\theta$ and $v_{{\mathrm{\Gamma }},k_F}\sim {\mathrm{sin}}\theta$, accounting for the predominant orbital character of the FS represented in Fig. 1a. By using the identities (1)–(3), one can establish^32,37 (see also Supplementary Note 2) a precise correspondence between the orbital character of the spin operator and the momenta Q_X or Q_Y connecting the hole and the X/Y pockets:

$${\mathbf{S}}{\kern 1pt} ({\mathbf{Q}}_X) \equiv {\mathbf{S}}_X^{yz} = \mathop {\sum}\limits_{\mathbf{k}} u_{{\mathrm{\Gamma }},{\mathbf{k}}}h_{\mathbf{k}}^\dagger {\kern 1pt} \vec \sigma u_{X,{\mathbf{k}} + {\mathbf{Q}}_X}e_{X,{\mathbf{k}} + {\mathbf{Q}}_X},$$

(4)

$${\mathbf{S}}{\kern 1pt} ({\mathbf{Q}}_Y) \equiv {\mathbf{S}}_Y^{xz} = \mathop {\sum}\limits_{\mathbf{k}} - v_{{\mathrm{\Gamma }},{\mathbf{k}}}h_{\mathbf{k}}^\dagger {\kern 1pt} \vec \sigma u_{Y,{\mathbf{k}} + {\mathbf{Q}}_Y}e_{Y,{\mathbf{k}} + {\mathbf{Q}}_Y}.$$

(5)

Since xz states are absent at X the $S_{\mathbf{q}}^{xz}$ operator has no component at the wavevector Q_X connecting the Γ and X pocket, and vice versa for the yz states. This leads to OSSF at different momenta, as depicted in Fig. 1:

$$\langle {\mathbf{S}} \cdot {\mathbf{S}}\rangle ({\mathbf{Q}}_X) \Rightarrow \langle {\mathbf{S}}_X^{yz} \cdot {\mathbf{S}}_X^{yz}\rangle ,$$

(6)

$$\langle {\mathbf{S}} \cdot {\mathbf{S}}\rangle ({\mathbf{Q}}_Y) \Rightarrow \langle {\mathbf{S}}_Y^{xz} \cdot {\mathbf{S}}_Y^{xz}\rangle .$$

(7)

The existence of OSSF provides a natural explanation of the orbital ordering observed in the nematic phase of FeSe. In fact, the self-energy corrections due to spin exchange imply a shift in the chemical potential with opposite sign for the hole and electron pockets, leading in both cases to a shrinking of the FS^19,38 that explains why experimentally they are always smaller than LDA predictions.^19,39,40 Within the OSSF model, due to the orbital-selective nature of SF, this mechanism is also orbital-dependent.¹⁹ As a consequence, within a spin-nematic scenario, the C₄ symmetry breaking of SF along ΓX and ΓY explains also the orbital ordering observed in the nematic phase. It has been shown¹⁹ that, by assuming stronger SF at Q_X below T_S, the self-energy difference ΔΣ between the xz and yz orbitals induces an orbital splitting being positive at Γ and negative at the electron pockets, leading to the observed deformations of the FS below T_S.^{11,15,17,19,20,21} Even though this orbital-selective shrinking mechanism is generic, its effect can be quantitatively different in the various family of iron-based superconductors. For example, in the 122 family the survival of the inner hole pocket enhances the degree of orbital nesting between hole and electron pockets favoring magnetism, this explains why in 122 the nematic transition is immediately followed by the magnetic one.³² The quantitative determination of the nematic splitting induced by the nematic spin modes requires a direct comparison with the low-energy band dispersion, as done explicitly for FeSe in ref. ¹⁹. Here, we take these results for granted and we start from a low-energy model that includes already the effective masses, isotropic shrinking, and nematic splittings needed to reproduce the ARPES FS measured in the nematic phase above T_c, and the k_z dependence of the hole pocket between the Γ (k_z = 0) and Z (k_z = π) point (see Supplementary Note 3). The resulting FS at k_z = 0 is shown in Fig. 1.

The effect of the nematic orbital splitting on the orbital factors below T_S is shown in Fig. 2. Here, ${\mathrm{\Delta }}\Sigma _h = (\Sigma _{xz}^{\mathrm{\Gamma }} - \Sigma _{yz}^{\mathrm{\Gamma }})/2$ denotes the nematic splitting at Γ and ${\mathrm{\Delta }}\Sigma _e = (\Sigma _{yz}^X - \Sigma _{xz}^Y)/2$ is the nematic splitting at M = (X, Y) with $\Sigma _{yz/xz}^\ell$ being the yz/xz orbital component of the real part of the self-energy for the $\ell$ pocket (see Supplementary Note 1). The maximum values are chosen to match the experimental ones,^11,19,20,21 i.e., ${\mathrm{\Delta }}\Sigma _{h/e} \simeq 15$ meV. The most dramatic changes due to the nematic order are found in the orbital occupation of the hole pocket (Fig. 2a, d). The presence of a relatively large spin–orbit coupling (≃20 meV) implies a mixing of the xz and yz orbitals on all the FS. However, below T_S the yz character of the hole pocket is strongly suppressed, and the pocket acquires a dominant xz character even at θ = 0, as observed by the polarization dependent ARPES measurements.^17,19,26,27 At the same time, the nematic splitting enhances the yz occupation at X (Fig. 2b, e), and suppresses the xz at Y (Fig. 2c, f). As a consequence, one easily understands that considering the orbital character of the SC order parameter is not enough to explain the observed gap hierarchy. In fact, on the X pocket the gap is maximum at θ = π/2, where the band has strong yz character, while on the Γ pocket it is larger at θ = 0, where a dominant xz character is found. The crucial ingredient required to account for the SC properties of FeSe comes indeed from the nematic pairing provided by OSSF, as we show below.

Fig. 2

Orbital content of the FS. a–c Color maps of the orbital content of the Γ (a), X (b), Y (c) pockets FS as a function of the angle and of the nematic splitting ΔΣ_h/e. The color code is the same as in Fig. 1. d–f Orbital content of the same pockets as a function of θ at ΔΣ_h/e = 0, i.e., in the tetragonal phase (dashed lines) and in the nematic phase ΔΣ_h/e = 15 meV (solid lines)

By building up the spin-singlet vertex mediated by the SF, (6) and (7), one obtains (see Supplementary Note 2) a pairing Hamiltonian involving only the xz/yz orbital sector:

$$H_{{\mathrm{pair}}}^{xz,yz} = - g_X\mathop {\sum}\limits_{{\mathbf{k}},{\mathbf{k}}\prime } u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2h_{\mathbf{k}}^\dagger h_{ - {\mathbf{k}}}^\dagger u_{X,{\mathbf{k}}\prime }^2e_{X, - {\mathbf{k}}\prime }e_{X,{\mathbf{k}}\prime } - g_Y\mathop {\sum}\limits_{k,{\mathbf{k}}\prime } v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2h_{\mathbf{k}}^\dagger h_{ - {\mathbf{k}}}^\dagger u_{Y,{\mathbf{k}}\prime }^2e_{Y, - {\mathbf{k}}\prime }e_{Y,{\mathbf{k}}\prime } + h.c.$$

(8)

The coefficients $u_{\ell ,{\mathbf{k}}}$, $v_{\ell ,{\mathbf{k}}}$, accounting for the pockets orbital character, preserve the C₄ band-structure symmetry above T_S and reproduce the nematic reconstruction below T_S. The g_X/Y couplings control the strength of the pair hopping between the Γ and X/Y pockets. Within a spin-nematic scenario, OSSF below T_S are stronger along ΓX than along ΓY leading to a nematic pairing anisotropy with g_X > g_Y. Within the present itinerant-fermions picture, the SF are peaked at the wavevectors connecting hole-like with electron-like pockets. Thus, due to the absence in FeSe of the hole-like xy band at Γ one can neglect the spin-mediated pairing in the xy channel. However, SF at RPA level was found³¹ to be most prominent at Q = (π, π). While this could be consistent with inelastic neutron scattering measurements at high temperatures, it does not account for the predominance of stripe-like SF at (π, 0) in the nematic phase.⁷ In addition, a predominant Q = (π, π) pairing channel implies a maximum gap value on the xy sector of the electron pocket, that is in sharp contrast with the experiments. This led the authors of refs. ^22,31 to phenomenologically introduce orbital-dependent spectral weights to suppress this channel (see Discussion section). In general, one can still expect that a smaller pair hopping between the X, Y pockets is present in the xy sector. For the sake of completeness, and with the aim of reducing the number of free parameters, we considered also in this case only an interband xy pairing term, acting between the two electron-like pockets:

$$H_{{\mathrm{pair}}}^{xy} = - g_{xy}\mathop {\sum}\limits_{{\mathbf{k}},{\mathbf{k}}\prime } v_{X,{\mathbf{k}}}^2e_{X,{\mathbf{k}}}^\dagger e_{X, - {\mathbf{k}}}^\dagger v_{Y,{\mathbf{k}}\prime }^2e_{Y, - {\mathbf{k}}\prime }e_{Y,{\mathbf{k}}\prime } + h.c.$$

(9)

The set of Eqs. (8) and (9) is solved in the mean-field approximation by defining the orbital-dependent SC order parameters for the hole $\left( {{\mathrm{\Delta }}_h^{yz},{\mathrm{\Delta }}_h^{xz}} \right)$ and electron $\left( {{\mathrm{\Delta }}_e^{yz},{\mathrm{\Delta }}_e^{xz},{\mathrm{\Delta }}_X^{xy},{\mathrm{\Delta }}_Y^{xy}} \right)$ pockets. The self-consistent equations at T = 0 reads:

$${\mathrm{\Delta }}_h^{yz} = - g_X\mathop {\sum}\limits_{\mathbf{k}} u_{X,{\mathbf{k}}}^2(u_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{yz} + v_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_X^{xy})/E_{X,{\mathbf{k}}}$$

(10)

$${\mathrm{\Delta }}_h^{xz} = - g_Y\mathop {\sum}\limits_{\mathbf{k}} u_{Y,{\mathbf{k}}}^2(u_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{xz} + v_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_Y^{xy})/E_{Y,{\mathbf{k}}}$$

(11)

$${\mathrm{\Delta }}_e^{yz} = - g_X\mathop {\sum}\limits_{\mathbf{k}} u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2(u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{yz} + v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{xz})/E_{{\mathrm{\Gamma }},{\mathbf{k}}}$$

(12)

$${\mathrm{\Delta }}_e^{xz} = - g_Y\mathop {\sum}\limits_k v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2(u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{yz} + v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{xz})/E_{{\mathrm{\Gamma }},{\mathbf{k}}},$$

(13)

$${\mathrm{\Delta }}_X^{xy} = - g_{xy}\mathop {\sum}\limits_{\mathbf{k}} v_{Y,{\mathbf{k}}}^2(u_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{xz} + v_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_Y^{xy})/E_{Y,{\mathbf{k}}}$$

(14)

$${\mathrm{\Delta }}_Y^{xy} = - g_{xy}\mathop {\sum}\limits_{\mathbf{k}} v_{X,{\mathbf{k}}}^2(u_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{xz} + v_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_X^{xy})/E_{X,{\mathbf{k}}}$$

(15)

Here, $E_{\ell ,{\mathbf{k}}} = \sqrt {\varepsilon _{\ell ,{\mathbf{k}}}^2 + {\mathrm{\Delta }}_{\ell ,{\mathbf{k}}}^2}$ is the dispersion in the SC state, where $\varepsilon _{\ell ,{\mathbf{k}}}$ is the band dispersion on each pocket $\ell = {\mathrm{\Gamma }},X,Y$ above T_c and ${\mathrm{\Delta }}_{\ell ,{\mathbf{k}}}$ is the band gap defined as:

$${\mathrm{\Delta }}_{{\mathrm{\Gamma }},{\mathbf{k}}} = u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{yz} + v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{xz},$$

(16)

$${\mathrm{\Delta }}_{X,{\mathbf{k}}} = u_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{yz} + v_{X,{\mathbf{k}}}^2{\mathrm{\Delta }}_X^{xy},$$

(17)

$${\mathrm{\Delta }}_{Y,{\mathbf{k}}} = u_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_e^{xz} + v_{Y,{\mathbf{k}}}^2{\mathrm{\Delta }}_Y^{xy}.$$

(18)

Superconducting gaps anisotropy

The overall momentum dependence of the band gaps is determined by the interplay between the momentum dependence of the orbital factors and the hierarchy of the orbital SC order parameters. In the absence of nematic order, Eqs. (10)–(18) preserve the symmetry in the exchange of the xz/yz orbitals. Thus, ${\mathrm{\Delta }}_h^{xz} = {\mathrm{\Delta }}_h^{yz}$ and the gap on the Γ pocket, Eq. (16), is constant, since $u_{\Gamma ,{\mathbf{k}}}^2 + v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2 = 1$. In the nematic state, the band structure breaks the C₄ symmetry, making $v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2 \gg u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2$ (see Fig. 2d), and also the SC orbital parameters ${\mathrm{\Delta }}_h^{xz}$ and ${\mathrm{\Delta }}_h^{yz}$ are in general different. However, as we shall see below, for isotropic pairing g_X = g_Y, the gaps anisotropy is the wrong one. The experimentally-observed anisotropy can only be achieved making ${\mathrm{\Delta }}_{yz}^h \gg {\mathrm{\Delta }}_{xz}^h$, that follows from the nematic pairing mechanism g_X > g_Y provided by spin-nematic OSSF.

To understand the effect of the band-structure nematic reconstruction on the SC gap anisotropy, we show in Fig. 3 the evolution of the orbital-factors overlaps appearing in Eqs. (10)–(13), where we define the angular average of a given function as $\langle f({\mathbf{k}})\rangle \equiv {\int} d\theta /(2\pi )f(k_F(\theta ))$, with k_F(θ) FS wavevector of a given pocket. We can in first approximation neglect the pairing in the subleading xy channel and consider only what happens in the xz/yz orbital sector. As mentioned above, the nematic splitting on the electron pockets leads to a moderate enhancement of the yz factor appearing in Eq. (10) with respect to the xz in Eq. (11), i.e., $\langle u_X^4\rangle >rsim \langle u_Y^4\rangle$, Fig. 3b. This effect, recently highlighted while discussing the k_z = π FS cut,³⁵ is however too small to account for the observed hole–gap anisotropy at k_z = 0. In fact, the strong modification of the hole-pocket orbital factors implies that $\langle u_{\mathrm{\Gamma }}^4\rangle \ll \langle u_{\mathrm{\Gamma }}^2v_{\mathrm{\Gamma }}^2\rangle < \langle v_{\mathrm{\Gamma }}^4\rangle$, Fig. 3a. Thus, by neglecting logarithmic corrections in the gap ratios, from Eqs. (10)–(13) one obtains that

$$\frac{{{\mathrm{\Delta }}_e^{yz}}}{{{\mathrm{\Delta }}_e^{xz}}} \simeq \frac{{g_X}}{{g_Y}}\frac{{\langle u_{\mathrm{\Gamma }}^2v_{\mathrm{\Gamma }}^2\rangle }}{{\langle v_{\mathrm{\Gamma }}^4\rangle }} \simeq 0.1\frac{{g_X}}{{g_Y}}$$

(19)

Fig. 3

Angular-averaged orbital-weight overlaps. a, b Nematic-splitting dependence of the angular-averaged orbital-weight overlaps appearing in the SC gap equations, i.e., Eqs. (10)–(13)

and

$$\frac{{{\mathrm{\Delta }}_h^{yz}}}{{{\mathrm{\Delta }}_h^{xz}}} \simeq \frac{{g_X}}{{g_Y}}\frac{{\langle u_X^4\rangle }}{{\langle u_Y^4\rangle }}\frac{{{\mathrm{\Delta }}_e^{yz}}}{{{\mathrm{\Delta }}_e^{xz}}} \simeq 1.8\frac{{g_X}}{{g_Y}}\frac{{{\mathrm{\Delta }}_e^{yz}}}{{{\mathrm{\Delta }}_e^{xz}}}.$$

(20)

Note that Eqs. (19) and (20) are almost unaffected once the xy pairing channel is taken into account. From Eqs. (19) and (20), it follows that an isotropic pairing interaction g_X = g_Y (as considered in ref. ³⁵) would lead to a suppression of the yz gap parameters. At the Γ pocket, where the yz orbital character is also strongly suppressed by nematicity ($u_{\mathrm{\Gamma }}^2 \ll v_{\mathrm{\Gamma }}^2$, Fig. 3a), the band gap would have only xz character, ${\mathrm{\Delta }}_{{\mathrm{\Gamma }},{\mathbf{k}}} \simeq {\mathrm{\Delta }}_h^{xz}v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2$, leading to a small modulation with a relative maximum at θ = π/2 (dashed line in Fig. 3a), in contrast with the experimental findings. On the other hand, the OSSF-mediated anisotropic pairing with $g_X/g_Y \gg 1$ gives a substantial enhancement of the ${\mathrm{\Delta }}_h^{yz}/{\mathrm{\Delta }}_h^{xz}$ ratio. This leads to ${\mathrm{\Delta }}_{{\mathrm{\Gamma }},{\mathbf{k}}} \simeq \Delta _h^{yz}u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2$, in agreement with the band-gap anisotropy observed experimentally as shown in Fig. 4a, where the numerical solutions of Eqs. (10)–(13) are reported along with the experimental data of ref. ²². Here the color code does not refer to the orbital content of the pocket, as in Fig. 1, but to the orbital content of the SC gap function, that is determined by the product of the SC order parameter times the orbital weight in each sector, Eqs. (16)–(18).

Fig. 4

Angular dependence of the SC band gaps. a SC gap on the hole, b on the X electron pocket, c on the Y electron pocket. Dashed lines are the results for the isotropic pairing g_X/g_Y = 1, while solid line for nematic pairing g_X/g_Y ~ 21. The color code accounts for the orbital component of the SC gap (green yz, red xz, blue xy), given by the product of the SC orbital parameter and the orbital weight, according to the definitions (16)–(18). For comparison, we reproduce the experimental gap values with standard deviations from ref. ²² (black circles)

The anisotropy g_X/g_Y = 21 extracted from this analysis is rather large, since one needs to overcome the strong suppression of the yz orbital due to nematic reconstruction at the hole pocket: one needs at least $g_X/g_Y \geq 2$ (not shown) to start to see the correct symmetry of the gap at Γ, i.e., a maximum at θ = 0. The value of g_X/g_Y obtained by the SC-gaps analysis is compatible with the anisotropy of the OSSF used to reproduce the orbital-selective shrinking of the FS in the nematic phase¹⁹ as discussed in Supplementary Note 3. In principle, the nematic-pairing anisotropy could also be estimated by the direct measurements of the SF. However, while it has been established that in the nematic phase SF are stronger at (π, 0) than at (π, π),^6,7,8 the different intensity expected at (π, 0) and (0, π) has not been measured yet in detwinned samples.

The gap obtained for the X pocket is shown in Fig. 4b. Its value is also in overall in agreement with the Scanning Tunneling Microscopy (STM) experimental data.²² To reproduce the experimental value of the xy component, we needed a small $\left( {|g_{xy}| \ll g_X} \right)$ attractive interband interaction between the two electron-like pockets. In fact, a negative g_xy guarantees, from Eqs. (14) and (15), that the SC xy order parameters on both electron pockets have the opposite sign with respect to the one at the hole pockets, as required by the dominant spin-mediated channel. In contrast, a repulsive g_xy induces a frustration that turns out in a gap with nodes along the FS.⁴¹ Even though this has been recently suggested by specific-heat measurements,⁴² the STM data²² shown for comparison exclude the presence of nodes and force us to consider a negative g_xy. It is important to stress that, even though the full set of Eqs. (10)–(15) must be solved self-consistently, adding or not the xy channel is not relevant for what concerns the understanding of the gap behavior in the xz/yz sector, especially for the gap anisotropy at the Γ pocket. For the sake of completeness, we report in Fig. 4c also the gap on the Y pocket, that has not been resolved so far in STM.²² As one can see, for the electronic pockets an isotropic pairing g_X = g_Y would lead to a strong difference between the absolute gap values at X and Y, due to the effect of nematic ordering at the electronic pockets, as one understands from Eq. (19) above. In contrast, nematic pairing leads to more similar gap values, which can be hardly disentangled experimentally, explaining why recent ARPES results claiming to resolve the Y pocket do not report appreciable significant gap differences on the two electron pockets.²⁸ The differences between the X and Y gaps due to the nematic pairing could however have implications for the thermal probes sensible to single-particle excitations. We leave the analysis of those effects for future work.

Recently, the k_z-dependence of the gap anisotropy on the hole pocket has been investigated,²⁸ and it has been shown that the Δ_Γ(θ = 0)/Δ_Γ(θ = π/2) anisotropy increases as one moves from the k_z = 0 to the k_z = π cut. Even though we did not consider a full 3D model, this effect can be understood by considering the variations of the hole-pocket orbital content when moving from k_z = 0 to k_z = π (Z point). The larger size of the hole pocket at Z makes its orbital content less sensitive to nematic ordering and spin–orbit mixing, so that it still preserves a marked yz character around θ = 0 (refs. ^24,25,35), with $u_{\mathrm{\Gamma }}\sim cos\theta$ and $v_{\mathrm{\Gamma }}\sim {\mathrm{sin}}\theta$ also in the nematic phase (Fig. 5a). In this situation, $\langle u_{\mathrm{\Gamma }}^4\rangle \sim \langle v_{\mathrm{\Gamma }}^4\rangle$ so that the enhancement $\langle u_X^4\rangle > \langle u_Y^4\rangle$ of the orbital factors in the electron pockets is enough to guarantee that ${\mathrm{\Delta }}_h^{yz} > {\mathrm{\Delta }}_h^{xz}$, leading to a hole-pocket gap anisotropy compatible with the measurements even when g_X = g_Y, as recently shown in ref. ³⁵ (dashed line Fig. 5b). On the other hand, by retaining the same ratio g_X/g_Y value extracted from the k_z = 0 gap fit (solid line Fig. 5b), we find an increase of the anisotropy when moving from the Γ to the Z pocket. While this is consistent with the observations in pure²⁸ and S-doped²⁴ FeSe, other groups^25,26 report instead an overall smaller gap at k_z = 0. The analysis of SC fluctuations above T_c, could provide an alternative experimental test to clarify the 3D behavior. As shown in ref. ⁴³, the crossover from 2D to 3D character of the fluctuation contribution to the paraconductivity is controlled by the k_z dependence of the pairing interactions. This effect, used to explain the measurements in 122 systems,⁴⁴ could be tested in FeSe as well.

Fig. 5

FS and SC gap for the hole pocket at k_z = π. a FS at k_z = 0 and k_z = π in the nematic phase. At k_z = π the hole pocket retains a full yz orbital character at θ = 0. b Angular dependence of the SC gap Δ_Z renormalized to its maximum value obtained using the same g_X/g_Y value extracted from the k_z = 0 gap fit (see Supplementary Note 3). Same color code of Fig. 4. The Δ_Z maximum at θ = 0 is obtained already for isotropic pairing g_X = g_Y (dashed lines), see also ref. ³⁵. The nematic pairing (solid lines) further enhances the gap anisotropy, leading to larger relative variations on the Z pocket, in agreement with ARPES experiments²⁸

Discussion

The C₄ symmetry breaking of paramagnetic SF is a consequence of SF interactions beyond RPA.^32,34 As a consequence, the effects of the nematic SF pairing g_X > g_Y highlighted in the present work cannot be captured by microscopic models where the SF are described at RPA level, even when RPA fluctuations are computed using the nematic reconstruction of the band structure.^31,45 An alternative route followed in refs. ^22,31 amounts to start from band dispersions fitted to ARPES data and to account phenomenologically for the role of correlations. The so-called orbital differentiation of the electronic mass renormalization due to local electronic interactions has been studied in DMFT-like calculations in the tetragonal phase,^46,47 which found in particular a larger renormalization of the xy orbital with respect to the xz/yz ones. In addition, correlations can also cooperate to enhance the xz/yz orbital differentiation induced by other nematic mechanisms.⁴⁸ Inspired by these results, the authors of refs. ^22,31 added phenomenologically orbital-dependent quasiparticle spectral weights, Z_orb, in the RPA-based calculation of the pairing interaction. By using $Z_{xy} \ll Z_{xz} < Z_{yz}$, they obtain the twofold result to make the Y pocket incoherent, explaining why it does not show up in the STM analysis,^22,49 and to move the maximum of SF from Q = (π, π) to Q = Q_X,⁴⁵ explaining the neutron-scattering experiments^7,8 and the observed gap hierarchy. However, this approach presents some inconsistencies. One issue is methodological: by using independent parameters to renormalize the band structure (that is fitted from the experiments) and to define the residua of the Green’s functions, one misses the strict relation between these two quantities. On the other hand, by implementing this relation self-consistently, as done for example in ref. ⁵⁰, it is not obvious how one can reconcile the large Fermi-velocity anisotropy implicit in the Z_xz < Z_yz relation with the experimental band structure, that is well reproduced accounting only for a crystal-field splitting of the tetragonal band structure having Z_xz = Z_yz.^11,26,27 A second issue arises by the comparison with experiments. The route followed in refs. ^22,31 is equivalent to rewrite the SC gap, e.g., on the Γ pocket as:

$${\mathrm{\Delta }}_{{\mathrm{\Gamma }},{\mathbf{k}}} = Z_{yz}u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{yz} + Z_{xz}v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{xz}$$

(21)

In our case, Eq. (16), the predominance of the SC yz orbital component is achieved via ${\mathrm{\Delta }}_{yz}^h \gg {\mathrm{\Delta }}_{xz}^h$, as guaranteed by the nematic-pairing condition $g_X \gg g_Y$. Instead in Eq. (21) this is mainly due to the rescaling of the orbital occupation factors by the corresponding spectral weights Z_yz/xz. By assuming $Z_{yz} \gg Z_{xz}$^22,31 one finds ${\mathrm{\Delta }}_{{\mathrm{\Gamma }},{\mathbf{k}}} \approx Z_{yz}u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2{\mathrm{\Delta }}_h^{yz}$, consistently with the measured gap anisotropy. However, the rescaling of the yz orbital occupation to $Z_{yz}u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2$ is operative not only on the SC gap function, but also on the band structure above T_c. This restores the yz character of the Γ pocket,⁴⁵ in contrast with ARPES measurements which clearly indicate^26,27 its predominant xz character.

To reconcile ARPES with RPA-based calculations of the spin-mediated pairing interactions, the authors of ref. ²⁶ use the alternative approach to remove intentionally the contribution of the Y pocket from the RPA-mediated pairing interaction. This is equivalent to put g_Y = 0 in Eqs. (10)–(13), so that ${\mathrm{\Delta }}_h^{xz} = 0$ and the modulation of the gap at Γ follows again the yz orbital weight, even if it is largely subdominant. With respect to these approaches, the main advantage of our model is to provide, via the orbital selectivity of the OSSF, a mechanism able to achieve the g_X > g_Y nematic pairing without affecting strongly the quasiparticle spectral weights, while the main disadvantage is the lack of a theoretical justification for the missing Y pocket. However, we cannot help noticing that this point is also controversial from the experimental point of view, due to different reports claiming to observe^19,28 or not^22,26 the Y pocket.

In summary, our work provides a paradigm for the emergence of superconductivity in FeSe from an orbital-selective nematic SF mechanism. By combining the orbital ordering induced by the nematic shrinking of the FS pockets below the nematic transition with the anisotropic pairing interaction mediated by nematic SF, we explain the gap hierarchy reported experimentally on hole and electron pockets, and its variation with k_z. Our findings also offer a fresh perspective on previous attempts to explain the SC properties of FeSe, highlighting from one side the crucial role of spin-mediated pairing, and from the other side clarifying the importance of spin–spin interactions beyond RPA level. This result then represents a serious challenge for a full microscopic approach, that must account self-consistently for the emergence of Ising-nematic SF below the nematic transition temperature.

Methods

Pairing by orbital-selective spin fluctuations

The mean-field equations for the pairing Hamiltonian, Eqs. (8) and (9), can be easily derived by defining the orbital-dependent SC order parameters for the hole $\left( {{\mathrm{\Delta }}_h^{yz},{\mathrm{\Delta }}_h^{xz}} \right)$ and electron $\left( {{\mathrm{\Delta }}_e^{yz},{\mathrm{\Delta }}_e^{xz}} \right)$ pockets as:

$${\mathrm{\Delta }}_e^{yz} = - g_X\langle u_{{\mathrm{\Gamma }},{\mathbf{k}}}^2h_{\mathbf{k}}h_{ - {\mathbf{k}}}\rangle ,$$

(22)

$${\mathrm{\Delta }}_e^{xz} = - g_Y\langle v_{{\mathrm{\Gamma }},{\mathbf{k}}}^2h_{\mathbf{k}}h_{ - {\mathbf{k}}}\rangle ,$$

(23)

$${\mathrm{\Delta }}_h^{yz} = - g_X\langle u_{X,{\mathbf{k}}}^2e_{X,{\mathbf{k}}}e_{X, - {\mathbf{k}}}\rangle ,$$

(24)

$${\mathrm{\Delta }}_h^{xz} = - g_Y\langle u_{Y,{\mathbf{k}}}^2e_{Y,{\mathbf{k}}}e_{Y, - {\mathbf{k}}}\rangle ,$$

(25)

$${\mathrm{\Delta }}_X^{xy} = - g_{xy}\langle v_{Y,{\mathbf{k}}}^2e_{Y,{\mathbf{k}}}e_{Y, - {\mathbf{k}}}\rangle ,$$

(26)

$${\mathrm{\Delta }}_Y^{xy} = - g_{xy}\langle v_{x,{\mathbf{k}}}^2e_{X,{\mathbf{k}}}e_{X, - {\mathbf{k}}}\rangle ,$$

(27)

The corresponding self-consistent BCS equations at T = 0 are the ones reported in the text, Eqs. (10)–(15). To solve them, we introduce polar coordinates and we approximate the orbital factors and the density of states with their values at the Fermi level for each pocket. This implies that the various integrals can be computed as for example:

$$\begin{array}{*{20}{l}} {\mathop {\sum}\limits_{\mathbf{k}} u_{X,{\mathbf{k}}}^2\frac{{{\mathrm{\Delta }}_{X,{\mathbf{k}}}}}{{E_{X,{\mathbf{k}}}}}} \hfill & = \hfill & {{\int} \frac{{kdkd\theta }}{{(2\pi )^2}}u_X^2(\theta )\frac{{{\mathrm{\Delta }}_X(\theta )}}{{\sqrt {\varepsilon _{X,{\mathbf{k}}}^2 + {\mathrm{\Delta }}_X^2(\theta )} }}} \hfill \\ {} \hfill & = \hfill & {{\int} \frac{{d\varepsilon d\theta }}{{(2\pi )}}N_X(\varepsilon _F,\theta )u_X^2(\theta )\frac{{{\mathrm{\Delta }}_X(\theta )}}{{\sqrt {\varepsilon ^2 + {\mathrm{\Delta }}_X^2(\theta )} }}} \hfill \\ {} \hfill & = \hfill & {{\int} \frac{{d\varepsilon d\theta }}{{(2\pi )}}N_X(\varepsilon _F,\theta )u_X^2(\theta ){\mathrm{\Delta }}_X(\theta ){\mathrm{log}}\frac{{\omega _D}}{{{\mathrm{\Delta }}_X(\theta )}}} \hfill \end{array}$$

(28)

where we defined $u_X^4(\theta ) \equiv u_X^4(k_F(\theta ))$ and ${\mathrm{\Delta }}_X(\theta ) \equiv {\mathrm{\Delta }}_e^{yz}u_X^2(\theta )+{\mathrm{\Delta }}_X^{xy}v_X^2(\theta )$. The cut-off ω_D represents the range of the spin-mediated pairing interaction, and it has been taken of order of 0.1 eV. The angular-dependent density of state is defined as usual as $N_X(\varepsilon _F,\theta ) = {\int} (kdk)/(2\pi )\delta (\varepsilon _F - \varepsilon _{X,{\mathbf{k}}}) = k_F(\theta )/2\pi |v_F(\theta )|$, where k_F(θ) and v_F(θ) are the wavevector and velocity at the Fermi level, respectively. For a parabolic band dispersion, N_X(ε_F, θ) reduces to an angular-independent constant. Even though in Eq. (28) the angular integration involves both the orbital factor and the density of states, we checked that the results do not change considerably if the angular-averaged density of states is taken outside the integral. For this reason, accounting separately for the angular averages of the orbital factors alone, as shown in Fig. 3, allows one to have a rough estimate of the numerical results, as discussed in the text. The results of the full numerical self-consistent calculations of Eqs. (10)–(13) are displayed in Figs. 4 and 5 for g_X/g_Y = 21 and |g_xy|/g_X = 0.076. The numerical values of the band parameters can be found in Supplementary Note 3.