Critical charge fluctuations and emergent coherence in a strongly correlated excitonic insulator

Abstract

Excitonic insulator is a coherent electronic phase that results from the formation of a macroscopic population of bound particle-hole pairs—excitons. With only a few candidate materials known, the collective excitonic behavior is challenging to observe, being obscured by crystalline lattice effects. Here we use polarization-resolved Raman spectroscopy to reveal the quadrupolar excitonic mode in the candidate zero-gap semiconductor Ta₂NiSe₅ disentangling it from the lattice phonons. The excitonic mode pronouncedly softens close to the phase transition, showing its electronic character, while its coupling to noncritical lattice modes is shown to enhance the transition temperature. On cooling, we observe the gradual emergence of coherent superpositions of band states at the correlated insulator gap edge, with strong departures from mean-field theory predictions. Our results demonstrate the realization of a strongly correlated excitonic state in an equilibrium bulk material.

Introduction

Attractive interactions between fermions are known to lead to a proliferation of bound pairs of particles at low temperatures causing a transition into superconducting or superfluid phases. In a semiconductor or a compensated semimetal, the Coulomb attraction between electrons and holes may induce a similar transition where electron-hole pairs, the excitons, develop macroscopic coherence^1,2,3,4. The resulting state, characterized by an interaction-induced gap, has been dubbed as the excitonic insulator⁵. However, so far only a few materials have been identified as excitonic insulator candidates^{6,7,8,9,10,11}—possibly because its formation requires strong attraction or matching energy dispersions of the electron- and hole-like carriers¹. These restrictions can be overcome by creating a non-equilibrium exciton population and cooling below their degeneracy temperature, in which case the coherent state may be observable only as a transient due to the finite lifetime of the excited state^12,13,14. An equilibrium excitonic phase in a bulk material, on the other hand, would allow for a far wider range of questions to be asked and answered regarding the excitonic states of matter and their formation.

As an example, controlling the bare band gap of an excitonic insulator allows one to explore a range of correlated regimes: from a weakly correlated electon-hole condensate analogous to the Bardeen–Cooper–Schrieffer (BCS) condensate in fermionic superfluids in the negative-gap (semimetallic) regime to a weakly interacting gas of tightly bound excitons in the opposite limit of a gapped (semiconductor) band structure^3,5,15. In the former case, weakly bound excitons are characterized by size ξ_ex larger than the interparticle distance l_eh and the exciton wavefunctions overlap strongly, while in the latter one, a dilute gas of tightly bound excitons with ξ_ex ≪ l_eh exists also above the transition temperature, with their chemical potential going to zero at the transition, in analogy with the Bose-Einstein condensation (BEC). The “BCS” and “BEC” regimes are also characterized by a different dynamics of the excitons. In particular, for a semimetallic normal state (as in Fig. 1d), the exciton, regardless of its energy, may decay into unbound particle-hole pairs (Landau damping), leading to overdamped dynamics. On the other hand, for a semiconductor-like normal state, the exciton energy is within the direct gap at T > T_c, such that energy conservation ensures the undamped dynamics of the excitons. Most interesting is the strongly correlated crossover regime^16,17, where neither of the weakly coupled bosonic or fermionic descriptions applies and the signatures of excitons above T_c may coexist with the strongly coupled electron-hole plasma.

Fig. 1: Schematics of the excitonic insulator physics for Ta₂NiSe₅.

a Crystal structure in the high-temperature orthorhombic phase. Three unit cells in a direction are shown. b Possible mechanisms of the phase transition: below T_c, the symmetry of the lattice can be reduced due to structural distortion or coherent excitonic condensation. c Illustration of the polarization-resolved Raman process: the aa scattering geometry (top) probes excitations with the full symmetry of the lattice (A_g), while the ac scattering geometry (bottom) couples to the symmetry-breaking excitations with the symmetry of the ac-like quadrupolar order parameter (B_2g). d and e Excitonic transition in a semimetal: above T_c. d conduction (blue) and valence (red) bands cross at the Fermi level; pre-formed excitons are coupled to interband transitions forming an overdamped collective mode in the B_2g channel. e At low temperatures excitonic order hybridizes the bands, opening a spectral gap. At the gap edge, the eigenstates are equal-weight superpositions of the two bands, shown on the right. As a result, distinct interference effects occur for ac (black arrows) and aa (red/blue arrows) geometries. f Expected Raman spectra for the ac scattering geometry near T_c. Left: for a structural transition one of the phonon modes softens to zero energy. Right: for an excitonic insulator, the excitonic mode is overdamped and softens to zero energy at T_c. g Expected Raman spectra at low temperature: due to the coherence factors (e), the response at the gap edge is suppressed to zero in the ac scattering geometry, but not in aa (see Methods for details). On heating the features move to lower frequencies (dashed lines).

However, proving the excitonic origin of the phase transition is challenging as it is expected to be obscured by an accompanying structural transition of the crystalline lattice. Formation of a macroscopic excitonic population in a real material may break some of the lattice symmetries: translational one if the excitons are indirect (i.e., have a nonzero center-of-mass momentum)⁹ or point-group ones otherwise³. In the latter case, the nontrivial effect on the lattice comes about due to the transformation properties of the particle and hole wavefunctions involved in forming an exciton: if those are distinct, e.g., an s-like electron is paired with a d-like hole, the exciton wavefunction would have symmetry lower than the lattice one, even if the relative motion of the particle and hole is in a fully symmetric s-like state. The discrete nature of the point-group symmetry broken in the excitonic insulator at the transition temperature T_c bears important consequences for its properties: in particular, its excitations are expected to have a finite energy gap, in contrast to the non-equilibrium excitonic condensates, where a continuous U(1) symmetry associated with approximate exciton number conservation is broken resulting in superfluidity and a gapless Bogoliubov-Goldstone mode at T < T_c^12,13,14.

The above mentioned difficulties are pertinent to the case of Ta₂NiSe₅, a material showing a semiconducting behavior at low temperatures with a phase transition from high-temperature orthorhombic phase to a monoclinic one at T_c = 328 K¹⁸, breaking two of the mirror symmetries (Fig. 1, a and b). Symmetry-wise, this transition corresponds to the reduction of point-group symmetry from D_2h to C_2h with the order parameter transforming as the B_2g irreducible representation of D_2h (xz-like quadrupole). The electronic structure of Ta₂NiSe₅ has been predicted to have a small or negative (as in Fig. 1d) direct gap at the Brillouin zone center^19,20, in agreement with experiments above T_c^15,21. The two bands closest to Fermi energy have quasi-one-dimensional (1D) character and are derived from superpositions of Ta 5d and Ni 3d orbitals at the multiple sites of the unit cell. Hybridization between them in the k_x = 0 plane, forbidden by x → −x symmetry above T_c, can serve as the order parameter that is microscopically induced by a condensation of the resulting interband excitons^22,23. This points to the quadrupolar character of these excitons.

Experimentally, below T_c formation of a gap has been observed in transport¹⁵ and optical²¹ measurements, and an anomalous dispersion of the hole-like band²⁴ has been taken as an indication for the excitonic character of the transition. Above T_c, a gap-like spectral weight suppression has been observed in ARPES studies²⁵, suggesting a BEC-like picture for the excitons, while the low values of transport gap suggest otherwise¹⁵. On the other hand, the changes in spectral and transport properties could also be due to the change of the lattice structure below T_c. While Ta₂NiSe₅ has been actively investigated since then^{15,21,23,25,26,27}, a structural origin of the transition has not been excluded^28,29,30. Additionally, the origin of the low-temperature spectral gap has not been directly probed: while the valence band dispersion at low temperatures is consistent with a symmetry-breaking hybridization forming below T_c²⁴, a direct proof of hybridization (regardless of the driving force behind the transition) requires showing the states of two bands being mixed into coherent superpositions of the states at the gap edge as in Fig. 1e.

The questions above can be addressed directly by Raman spectroscopy that probes the excitations of the system by an inelastic two-photon process. Polarization analysis of the incoming and outgoing photons further enables one to select excitations with a specific symmetry^31,32. Applied to Ta₂NiSe₅ above T_c, ac polarization geometry (Fig. 1c, bottom) probes excitations with the symmetry of ac-type quadrupole (i.e., B_2g, see Methods), the same as that of the order parameter, allowing direct observation of the soft mode expected at a second-order phase transition (Fig. 1d). Being even in parity, these excitations are invisible in a conventional light absorption experiment due to the dipole selection rules. The character of the soft mode reveals the origin of the transition. In an excitonic transition in a semimetal, critical fluctuations would have a broad relaxational lineshape due to the Landau damping and are enhanced at low frequencies close to T_c (Fig. 1f, right). On the other hand, in a structural transition driven by an optical phonon, a sharp spectral peak would soften to zero energy at T_c (Fig. 1f, left). In addition, the structural transition can also be driven by an instability of the acoustic modes (ferroelastic instability³³), which would lead to absence of signatures in Raman scattering, as coupling to light vanishes at q = 0 for acoustic modes³⁴.

Hybridization between bands can be further revealed by studying the contribution of electron-hole excitations to Raman scattering. Above T_c, the ac scattering geometry probes the interband transitions (Fig. 1d) between the valence (red) and conduction (blue) bands that have an ac-type quadrupole (B_2g) character. The aa scattering geometry, on the other hand, probes fully symmetric excitations (Fig. 1c, top), and is limited to intraband transitions only. Below T_c, hybridization mixes the states of two bands into coherent superpositions (Fig. 1e). This results in interference effects at the gap edge, in analogy with the effect of coherence factors in a supercondutor³⁵. In particular, for ac geometry a destructive interference occurs between transitions from “red” to “blue” states and vice versa resulting in an exact cancellation (Fig. 1e, on the right). In contrast, for aa geometry, the destructive interference is between two types of interband transitions, which do not cancel exactly, as “red” and “blue” bands couple to light differently (Fig. 1e). This results in the intensity close to the gap edge being strongly suppressed in ac geometry with respect to aa. In Fig. 1g we present the spectra expected in two polarization channels based on a mean-field model of an excitonic insulator (see Methods and Supplementary Note 2 for details). In contrast to the above description, if hybridization is absent and the low-temperature gap is between the conduction and valence bands, the gap edge corresponds to a purely interband transition. The intensity in ac geometry is then expected to be dominant, clearly distinct from the hybridization gap case.

Here, we employ polarization-resolved Raman spectroscopy to prove the excitonic origin of the transition in Ta₂NiSe₅, as opposed to structural one, and the hybridization nature of the low-temperature gap. We further determine that the resulting excitonic insulator is in the strongly correlated BCS-BEC crossover regime. In particular, close to T_c we observe critical softening of overdamped quadrupolar excitations Fig. 2b, f) that are consistent with excitonic fluctuations in a semimetal. In contrast, we find no softening of the optical phonon modes (Figs. 3 and 4b). At low temperatures, by comparing the intensities in aa and ac polarization geometries (Fig. 2e) we find direct evidence for hybridization-induced gap and coherent mixing of the two semimetallic bands driven by the excitonic order (as in Fig. 1d), i.e., spontaneously formed symmetry-breaking hybridization between the bands. With heating, the gap observed in fully symmetric channel fills in, rather than closes, characteristic of strong correlations beyond the mean-field regime. By estimating the exciton coherence length, we find that for Ta₂NiSe₅ the excitonic condensate lies within the strongly correlated BCS-BEC crossover regime, and argue that the whole body of experimental data for Ta₂NiSe₅ is consistent with this identification.

Fig. 2: Temperature dependence of Raman spectra.

a and b Raman response ${\chi }_{{\rm{aa}}}^{^{\prime\prime} }$ and ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }$ in the aa and ac scattering geometries, respectively. The horizontal frequency (Raman shift) scales are logarithmic. To emphasize the region around T_c = 328 K the vertical axes are rescaled by a nonlinear transformation arctanh [T/T₀], where T₀ = 390 K. χ^″ values are displayed on a logarithmic false color scale, with the strongest phononic resonances in saturation. In b, a critical enhancement at low frequencies is observed close to T_c. Owing to broken mirror symmetries below T_c, “leakages” of sharp phonon features appear in both (a and b). Additionally, a strong peak at 380 meV emerges in (a) with a weaker feature in (b). c and d Details of Raman response ${\chi }_{{\rm{aa}}}^{^{\prime\prime} }$ and ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }$, for T = 330 K (above T_c) and T = 35 K. Shading highlights the electronic contribution (see also Fig. 3). e and f${\chi }_{{\rm{aa}}}^{^{\prime\prime} }$ and ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }$ in the range enclosed by the dashed white boxes in (a and b) on a linear frequency scale with the resolution shown in top left corner. In e, ${\chi }_{{\rm{aa}}}^{^{\prime\prime} }(\omega ,T)$ all cross at the 280 meV isosbestic point, showing intensity transfer on heating from the 380 meV peak to the low-temperature gap region. ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }(\omega ,T)$ is shown for intensity comparison. In f, above T_c (red curve) three phonon modes are observed, with the first two showing a pronounced asymmetric Fano lineshape, pointing to an interaction with an electronic continuum. Below T_c the asymmetry becomes less pronounced and additional modes appear due to mutual aa-ac “leakage”.

Fig. 3: Temperature dependence of the low-energy Raman response in the ac scattering geometry.

a–h show ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }(\omega )$ data (black dots) for temperatures from 380 K to 35 K on a semilog scale. Thick red lines show the fits to the data with a generalized Fano model (see Methods) of three phonons coupled to a continuum with a relaxational response ${\chi }_{{\rm{cont}}}^{^{\prime\prime} }(\omega )$, Eq. (1). The deduced bare phonon and continuum responses are shown by green line and blue line with shading, respectively. Full Fano model response (red) does not equal sum of the two due to the presence of interference terms (see Supplementary Note 1.3). e, f Below T_c, an additional enhancement at low frequencies is modeled by a low-energy mode, originating from recoil scattering of acoustic phonons on the structural domains (see Methods), interacting with the continuum. Purple line shows its combined response with the continuum. g, h At lower temperatures ${\chi }_{{\rm{cont}}}^{^{\prime\prime} }(\omega )$ is suppressed due to gap opening, reducing the asymmetry of the phonon modes. Spectral resolution is shown in top right corner of the panels.

Fig. 4: Temperature dependence of the phonon parameters.

a The ratio between the integrated intensity of the lowest-energy ${A}_{{\rm{g}}}^{(1)}$ phonon mode in the ac and aa scattering geometries. Its appearance below T_c implies the onset of symmetry breaking that mixes the responses in aa and ac geometries. For comparison, the temperature derivative of the transport activation gap ${E}_{tr}(T)={k}_{{\rm{B}}}T{\mathrm{log}}\,\left[\frac{R(T)}{R(360\ K)}\right]$ is shown in black, displaying a discontinuity at T_c Error bars are the standard deviation. b The temperature dependence of the frequency of the two lowest-energy optical phonon modes ω_ph1,2 in ac geometry, that exhibit strong coupling with the excitonic continuum. Error bars are the 95% confidence intervals of the Fano model fit. The phonon frequencies soften on heating consistent with anharmonic decay model (solid lines) below T_c and above 350 K (see Supplementary Note 1.3.3 for details). The more pronounced change between this regimes is attributed to a nonlinear exciton-phonon coupling, not included in the Fano model (see text).

Results

Overview

In Fig. 2a–d we present an overview of the Raman spectra. For ac geometry, probing excitations with the symmetry of ac-type quadrupole above T_c, same as that of the order parameter (Fig. 1c, bottom), a pronounced enhancement of the low-energy response is observed around T_c. This is characteristic of a soft mode development near a second-order phase transition. For aa geometry, the most prominent feature is the redistribution of intensity towards higher energies below T_c with a pronounced gap-like suppression below 380 meV; at low energies conventional phonon peaks are observed (Fig. 2c and Supplementary Fig. 8).

The presence of a symmetry-breaking phase transition at T_c is evident from the appearance of new sharp optical phonon modes in both geometries. Their appearance is related to the change in selection rules below T_c (see Methods); in what follows, we will call this intensity admixture from an another polarization as “leakage”.

This reflects that below T_c the two polarization geometries are no longer orthogonal, thus all excitations may appear in both geometries. In Fig. 4a we further quantify this effect by showing the temperature dependence of the integrated intensity of the lowest-energy fully symmetric (above T_c) phonon mode in ac scattering geometry, which grows substantially below 328 K. The parameters of the fully symmetric phonon modes are also consistent with a recent study of Raman scattering in aa geometry³⁶.

Identification of the excitonic soft mode

We now focus on the low-energy lineshapes in the ac geometry spectra close to the transition temperature, detailed in Figs. 2f and 3. Three distinct peaks are observed, which correspond to three optical phonons of B_2g symmetry expected from the space group of Ta₂NiSe₅ (Cmcm), ruling the presence of other optical modes out. The striking feature of the raw data is the notably asymmetric shape of the two lowest-energy modes, that cannot be described as conventional Lorentzian oscillators. Instead, the data above T_c is well described by a generalized Fano model (see Methods and Supplementary Note 1.3 for details), including three phononic oscillators interacting with a continuum of overdamped excitonic excitations described by a purely relaxational response

$${\chi }_{{\rm{cont}}}(\omega )\propto \frac{1}{-i\omega +{{{\Omega }}}_{{\rm{e}}}(T)},$$

(1)

where its imaginary part χ″(ω) exhibits a maximum at Ω_e(T). This continuum is clearly distinct from the structural phonons and suggests the presence of an overdamped bosonic mode emerging from the electronic system consistent with a Landau-damped exciton in a semimetal (Fig. 1d). The parameter Ω_e(T) can then be represented by an analogy with an overdamped oscillator as ${{{\Omega }}}_{{\rm{e}}}(T)\equiv {\omega }_{0}^{2}/{{\Gamma }}$, ω₀ being the exciton frequency (which is a collective mode of the semimetal, similar to the collective Cooper pair mode above T_c in a superconductor³⁷) and Γ ≫ ω₀ —the damping rate. This form can be alternatively derived using the time-dependent Ginzburg-Landau equation (see Supplementary Note 1.3.1). Upon cooling towards T_c, Ω_e(T) linearly decreases as ${{{\Omega }}}_{{\rm{e}}}(T)\propto T-{T}_{{\rm{c}}}^{el}$ (see Supplementary Fig. 5), where ${T}_{{\rm{c}}}^{el}=137(16)$ K, consistent with a critical softening of this excitonic collective mode. Just below T_c, the ac Raman response is additionally enhanced at the lowest frequencies (Fig. 3e, f). We associate this enhancement with coupling to acoustic B_2g modes with finite momenta, that is mediated by a quasiperiodic pattern of structural domains (which form below T_c) that takes the quasimomentum recoil (see Methods for the details).

Most importantly, we do not observe phonon softening: instead, the two bare lowest B_2g mode frequencies harden on cooling, Fig. 4b. Consequently, an instability of a zone-center optical phonon²⁹ can be ruled out by our results. Away from T_c, the hardening on cooling is consistent with the expectations due to lowest-order phonon anharmonicity (lines in Fig. 4b); however, a more pronounced increase is observed close to T_c. We attribute this increase to a nonlinear coupling between the excitonic fluctuations and the phonons. In particular, introducing the electronic order parameter φ, the phonon frequency can be expressed as ω_ph(T, φ) ≈ ω_ph(T, 0) + α_phφ². For α_ph > 0 (which points to a competition between the electronic and lattice orders), below T_c an additional increase of the phonon frequency is expected. In our results, the increase starts above T_c, where 〈φ〉 = 0, pointing to the influence of the fluctuations beyond the mean-field; in particular, the increase starts at 350 K, suggesting a fluctuation region of around 20 K for the phonon properties.

To study the effects of the exciton-phonon coupling on the transition, we analyze the static order parameter susceptibilities (Fig. 5). From the Fano model, one can deduce the individual susceptibilities of the excitonic continuum, optical phonons as well as the combined one, that includes the effects of the coupling between them. The purely electronic contribution to the inverse susceptibility (Fig. 5b) ${\chi }_{{\rm{cont}}}^{-1}(0,T)={{{\Omega }}}_{{\rm{e}}}(T)/{t}_{{\rm{e}}}^{2}$ (from Eq. (1)) follows the Curie-Weiss form $\propto (T-{T}_{{\rm{c}}}^{el})$ above T_c, indicating the softening of Ω_e(T). Extrapolating this trend to T < T_c suggests that a purely electronic transition would have taken place at ${T}_{{\rm{c}}}^{el}\,<\,{T}_{{\rm{c}}}$ marked by the divergence of ${\chi }_{{\rm{cont}}}(0,{T}_{{\rm{c}}}^{{\rm{el}}})$. On the other hand, the bare optical phonon susceptibility remains almost constant and is even reduced around T_c.

Fig. 5: Static B_2g susceptibility derived from the Fano fits to the Raman data shown in Fig. 3.

a Temperature dependence of the ac quadrupole static susceptibilities χ_ac(0, T) derived from the decomposition of ${\chi }_{{\rm{ac}}}^{^{\prime\prime} }({\omega},\;{T})$ (Fig. 3): excitonic (blue), phononic (green) and combined (red). Above T_c (purple background), while the phononic susceptibility mildly decreases on cooling, the electronic and combined ones grow: lines represent an extrapolation using the Fano model parameters above 350 K. The extrapolated electronic susceptibility diverges at ${T}_{{\rm{c}}}^{el}=137(16)$ K (blue background below). Black dashed line illustrates the total susceptibility including the exciton-strain coupling (see text and Supplementary Note 1.3.4), that diverges at the actual transition temperature T_c. Below T_c (pink background), the electronic and combined susceptibilities are suppressed, indicating the formation of the order parameter. b Same for inverse static susceptibilities 1/χ_ac(0, T). Extrapolated electronic inverse susceptibility shows a linear (Curie-Weiss) behavior vanishing at ${T}_{{\rm{c}}}^{{\mathrm{el}}}$, while the combined inverse susceptibility is nonlinear and vanishes at a higher temperature, showing that the coupling to otherwise stable (green points) optical phonons can strongly enhance transition temperature. Same is true for coupling to acoustic strain, that can further enhance it to the actually observed T_c. Error bars in all panels are the 95% confidence intervals of the Fano model fit.

Nevertheless, the coupling between the normal modes of the lattice and the excitonic continuum can increase the transition temperature. Extrapolating the susceptibility of the full Fano model, which includes the exciton-phonon interaction, from above T_c (red line in Fig. 5) we obtained a transition temperature ${T}_{{\rm{c}}}^{{\mathrm{comb}}}=238(18)$ K, larger than ${T}_{{\rm{c}}}^{{\mathrm{el}}}$ by about 100 K. However, ${T}_{{\rm{c}}}^{{\mathrm{comb}}}$ is still smaller then the actual T_c, because apart from the optical phonons that we have observed, the coupling to the acoustic B_2g strain modes has also to be considered. Such a coupling has been demonstrated to increase the temperature of an electronic nematic ordering to the actually observed one in iron-based superconductors^38,39. These effects can be understood within the Landau theory, where the electronic order parameter φ couples linearly to (optical) phononic and strain order parameters (denoted as η_i and ε_ac, respectively) of the same symmetry resulting in the free energy expansion

$$F[\varphi ,{\eta }_{i},{\varepsilon }_{{\rm{ac}}},T]={\chi }_{{\rm{cont}}}^{-1}(0,T)\frac{{\varphi }^{2}}{2}+\mathop{\sum }\limits_{i=1}^{3}{\chi }_{{\rm{opt}},i}^{-1}(0,T)\frac{{\eta }_{i}^{2}}{2}+{\lambda }_{{\rm{opt}},i}\varphi {\eta }_{i}+\kappa \frac{{\varepsilon }_{{\rm{ac}}}^{2}}{2}+{\lambda }_{{\rm{ac}}}\varphi {\varepsilon }_{{\rm{ac}}}+O({\varphi }^{4}),$$

(2)

where κ is the B_2g shear modulus. Minimizing the quadratic term one obtains the condition for the transition temperature ${\chi }_{{\rm{cont}}}^{-1}(0,{T}_{{\rm{c}}})-{\lambda }^{2}/\kappa -\mathop{\sum }\nolimits_{i = 1}^{3}{\lambda }_{{\rm{opt}},i}^{2}{\chi }_{{\rm{opt}},i}(0,{T}_{{\rm{c}}})=0$, as opposed to ${\chi }_{{\rm{cont}}}^{-1}(0,{T}_{{\rm{c}}}^{el})=0$ in the purely electronic case, leading to an enhanced transition temperature T_c. In Fig. 5 we illustrate this effect (dashed black line) by choosing the value of the interaction with strain such that the total susceptibility diverges at T_c. We find the value of the exciton-strain coupling obtained this way to be consistent with a recent study of the sound velocity renormalization above T_c⁴⁰ (see Supplementary Note 1.3.5), and the Raman response from finite-momentum acoustic phonons below T_c in Fig. 3e, f.

The analysis above leads us to the following conclusions: (a) the excitonic mode exhibits a strong tendency to soften, suggesting a purely electronic transition temperature of ${T}_{{\rm{c}}}^{el}=137(16)$ K; and (b) the transition temperature is boosted to the observed T_c = 328 K due to the coupling to noncritical optical phonon modes and acoustic strain.

Coherent factors in the excitonic insulator

Having established the excitonic origin of the transition in Ta₂NiSe₅ we now demonstrate that an interband hybridization emerges in the excitonic insulator phase. At low temperatures, an intense peak at about 380 meV in the aa geometry emerges with a much subtler feature in the ac geometry (Fig. 2e). This stark contrast cannot be attributed to the difference of the Raman coupling in two geometries: while the intensity in the ac geometry is indeed weaker than in the aa one at all temperatures, the evolution of the intensity in aa geometry at T < T_c is much more pronounced, allowing to relate this effect to the symmetry breaking (see also Supplementary Note 1.5). Moreover, a substantial part of the signal observed in ac geometry at low temperatures can be attributed to the change in selection rules below T_c, making the ac intensity actually related to the interband transition even smaller (see Supplementary Fig. 11). Both band structure calculations²² and experiments^41,42,43 suggest that only two bands are mostly affected by this symmetry breaking, allowing us to limit ourselves to the simplest excitonic insulator model of Fig. 1. The strong temperature dependence of this feature below T_c (also discussed below) also precludes its attribution to transitions away from k_x = 0 plane, where hybridization can be present already above T_c. For a hybridization-induced gap in a semimetal, a suppression of interband transitions at the gap edge in the ac geometry is indeed expected (Fig. 1e). Within a simplified two-band model of the excitonic insulator (Fig. 1g), a divergence of the intensity at the gap edge occurs in aa geometry, but not in the ac one, where interference effects lead to an exact cancellation of the intensity at the gap edge. These predictions are in agreement with the pronounced peak being observed in the aa geometry only (which is further exacerbated if the leakage from aa to ac geometry is accounted for, see Supplementary Fig. 11). On the other hand, in the case of a semiconductor with a large direct gap, such that hybridization effects can be ignored, the opposite is true: the interband transition would have predominantly ac quadrupolar character allowing its observation only in ac geometry (see Methods and Supplementary Note 2 for details), in stark contrast to the data in Fig. 2e. Thus, the observation of an intense peak emerging at low temperatures in aa geometry clearly favors a semimetallic band structure with a hybridization-induced gap.

Now we discuss the temperature dependence of the high-energy Raman response in aa geometry (Fig. 2e). Within the mean-field theory, the energy of the peak is related to the amplitude of the order parameter, which is expected to diminish upon heating towards T_c (Fig. 1g), and zero intensity is expected within the hybridization gap. We have observed a pronounced redistribution of intensity in a broad energy range, with intensity appearing at the lowest energies at temperatures significantly smaller than T_c. Such behavior appears inconsistent with the mean-field expectation, pointing to a role of correlations beyond the mean-field picture. This is further emphasized by all the curves crossing at 280 meV—the so-called “isosbestic point”^32,44,45, characteristic of strongly correlated systems (see also the discussion of BCS-BEC crossover effects and pseudogap below). It was previously observed in materials where spectral gap has a many-body origin, such as SmB₆⁴⁶ or cuprates⁴⁷. While band gap renormalization in semiconductors can be also related to electron-phonon interactions⁴⁸, the broadening of the gap edge features is expected to be rather small in that case; on the contrary, the spectral features we have observed quickly become broader than their energy on heating (Fig. 2e). Furthermore, the renormalization effects are expected to diminish with phonon energy⁴⁹ and in our case all phonon energies are below 50 meV, suggesting the scale of the electron-phonon effects to remain below 100 meV.

Discussion

Our results unambiguously point to the excitonic physics playing a crucial role in Ta₂NiSe₅; we further found evidence for strong correlation effects beyond the mean-field expectations. The important role of correlations in the formation of excitonic order in Ta₂NiSe₅ is further corroborated by the large ratio $\frac{2{{\Delta }}}{{k}_{{\rm{B}}}{T}_{{\rm{c}}}}\approx 13$, k_B being the Boltzmann constant (2Δ ≈ 380 meV taking the energy of the spectral peak in Fig. 2e as an estimate for the order parameter value), well beyond the BCS prediction, but consistent with the suppression of T_c occurring in BCS-BEC crossover¹⁶. Moreover, we can estimate the coherence length of the excitonic order using the bare electronic transition temperature with the BCS expression ${\xi }_{{\rm{ex}}}=\frac{\hslash {v}_{{\rm{F}}}}{1.76\pi {k}_{{\rm{B}}}{T}_{{\rm{c}}}^{el}}$. As the system is highly anisotropic, we use the value of v_F along the most dispersive a direction, where the exciton size is expected to be largest. This yields ξ_ex = 32 Å, while the distance between particles can be estimated in a quasi-1D system from ${l}_{{\rm{eh}}}=\frac{\pi }{2{k}_{{\rm{F}}}}\approx 16$ Å (k_F ~ 0.1 Å⁻¹, m^* ≈ 0.37m₀⁴¹). The estimate places the excitons in Ta₂NiSe₅ in the correlated BCS-BEC crossover regime l_eh ~ ξ_ex.

Identification of Ta₂NiSe₅ as a correlated excitonic insulator finally allows to clarify the character of the electronic structure above T_c. Semimetallic character is suggested by the absence of a discernible gap in spectroscopy (Fig. 2e and refs. ^21,41) and the overdamped lineshape of the excitonic fluctuations (Fig. 3). A direct semiconducting gap at T > T_c would further contradict the absence of a noticeable peak in ac Raman intensity (Fig. 2e) at low temperatures that requires strong mixing of the bands as in Fig. 1e. On the other hand, an apparent gap feature has been observed in ARPES studies above T_c²⁵. It is known however, that strong fluctuations of a particle-hole order, such as a charge density wave, result in a pseudogap^50,51,52 in a metallic system—a suppression of the density of states already above T_c, which has been experimentally observed close to charge- or spin-density wave transitions^53,54. In all of these cases, the pseudogap opens above the ordering transition due to the thermal order parameter fluctuations, regardless of the microscopic mechanism of the transition. Strong excitonic fluctuations in the BCS-BEC crossover regime may be expected to lead to similar effects, which explains the spectral weight suppression observed in ARPES²⁵. Moreover, as the density of states in the pseudogap is finite⁵⁰, it can be reconciled with both the zero-gap behavior of the high-temperature transport¹⁵, as well as the overdamped character of the excitonic mode (Fig. 3).

In conclusion, by using polarization-resolved Raman spectroscopy, we have directly revealed the excitonic fluctuations driving a phase transition in Ta₂NiSe₅, shown the coherence of the low-temperature insulator by revealing the coherent band superpositions, and identified the fingerprints of strong electronic correlations, similar to the ones occuring in the BCS-BEC crossover. The strongly correlated excitonic insulator nature of Ta₂NiSe₅ accommodates for its anomalous features and provides a unified view of its electronic structure, consistent with the previous experiments^21,24,25. Furthermore, as the band structure of this material can be tuned by chemical substitution or pressure¹⁵, this opens perspectives for the exploration of different correlation regimes in excitonic insulators.

Methods

Sample preparation

Single crystals of Ta₂NiSe₅ were grown using the chemical vapor transport (CVT) method. Elemental powders of tantalum, nickel and selenium were mixed with a stoichiometric ratio and then sealed in an evacuated quartz ampoule with a small amount of iodine as the transport agent. The mixture was placed in the hot end of the ampoule (~950 ^∘C) under a temperature gradient of about 10 ^∘C cm⁻¹. After about a week mm-sized needle-like single crystals were found at the cold end of the ampoule. These crystals are shiny and cleave easily. We have used X-ray diffraction and Electron Dispersive X-ray Spectroscopy (EDS) to verify the exact composition of the crystals and their uniformity.

Resistance measurement

The resistance along the a-axis (direction along the Ta/Ni chains) of a very thin needle-like single crystal was measured in a four-probe configuration using a Quantum Design DynaCool PPMS system (see Supplementary Note 1.1 for details).

Polarization-resolved Raman measurements

The samples used for Raman measurement were cleaved to expose (010) crystallographic plane (ac plane). The cleaved surface was then examined under a Nomarski microscope to find a strain-free area.

Raman measurements were performed in a quasi-back-scattering geometry from the samples placed in a continuous helium-gas-flow cryostat. The 647 nm line from a Kr⁺ ion laser was used for excitation. Incident light was focused into a 50 × 100 μm² spot of freshly cleaved crystal surface. For data taken below 310 K, laser power of 8 mW was used. To reach temperature above 310 K, we kept the environmental temperature at 295 K and increased laser power to reach higher sample temperature in the excitation spot. The reported temperatures were corrected for laser heating, which was determined to be 1.29 ± 0.17 K mW⁻¹ from the Stokes/anti-Stokes intensity measurement consistent with detailed balance (see Supplementary Note 1.2 for details). We employed a custom triple-grating spectrometer with a liquid-nitrogen-cooled charge-coupled device detector for analysis and acquisition of the Raman signal. All the data were corrected for the spectral response of the spectrometer and the CCD detector. For low-frequency Raman shift data below 50 meV, the spectral resolution was set to 0.06 meV for data taken below 100 K; to 0.10 meV for data taken from 100 to 295 K; and to 0.19 meV for data taken above 295 K. For high-frequency spectral features above 50 meV about 2.5 meV resolution was used.

Two incident/scattered light polarization configurations were employed to probe excitations in different symmetry channels. The relationship between the scattering geometries and the symmetry channels is given in Table 1.

Table 1 The Raman selection rules in the high-temperature orthorhombic (point-group D_2h) and low-temperature monoclinic (point-group C_2h) phases.

The Raman response function χ″(ω, T) was calculated from the measured Raman intensity I(ω, T) by the relation I(ω, T) = [1 + n(ω, T)]χ″(ω, T), where n(ω, T) is the Bose factor, ω is Raman shift energy, and T is temperature.

Fitting model for the Raman susceptibility

We adapt an extended Fano model⁵⁵ comprising three phonon modes interacting with an electronic continuum to fit the ac-polarization Raman susceptibility data above T_c, see Figs. 2f and 3 of the main text. It was found that the electronic continuum is best described by a simple relaxational response ${\chi }_{{\rm{cont}}}^{^{\prime\prime} }(\omega )=\frac{{t}_{{\rm{e}}}^{2}\omega }{{\omega }^{2}+{{{\Omega }}}_{{\rm{e}}}^{2}}$, corresponding to the overdamped limit of the Drude-Lorentz model (see Eq. (1) in the main text). Here t_e is the Raman coupling vertex to electronic response and ${{{\Omega }}}_{{\rm{e}}}\equiv \frac{{\omega }_{0}^{2}}{{{\Gamma }}}$ is a single fit parameter representing ratio of the square of electronic excitation frequency to its relaxation rate.

The resulting purely phononic and electronic parts of the Raman susceptibility are shown in Fig. 3 in green and blue lines correspondingly. The temperature dependencies of the frequencies of the lowest phonons are shown in Fig. 4b. The electronic part of the inverse susceptibility (Fig. 5b, red points) is, on the other hand, proportional to the electronic oscillator parameter Ω_e(T) (see Eq. (1)). The static susceptibilities shown in Fig. 5 are obtained by taking the Fano model susceptibility at ω = 0. As the model susceptibility is an analytic function and matches the raw data well, the result is equivalent to that of a Kramers-Kronig transformation.

By analyzing the data just below T_c (Fig. 3 e, f), we have found an additional enhancement at low frequencies can not be described with the same model as the characteristic continuum energy Ω_e(T) only grows below T_c (see Supplementary Fig. 5) and the resulting continuum intensity at low energies only weakens (Fig. 3d–g). This suggests the presence of an additional source of low-frequency response. The appearance of a low-energy mode can be appreciated from the formation of domains observed in transmission electron microscopy (see Supplementary Note 1.3.2 and ref. ¹⁸). In particular, the domains have an elongated needle-like shape along the a axis, with their size along the c axis on the order of $\bar{d}=200$ Å. Acoustic phonon, scattering on this quasiperiodic structure, can take a recoil momentum of the order $2\pi /\bar{d}$; this way a finite ${q}_{{\rm{d}}}=\pm \!2\pi /\bar{d}$ phonon can contribute to the Raman intensity (where the momentum transfer is approximately zero). At finite q, introducing the acoustic phonon coordinate ${x}_{q}=\sqrt{q}{u}_{{\rm{q}}}$, where u_q - is the Fourier transform of the atomic displacement amplitude, one obtains that the Raman vertex ${t}_{{\rm{s}}}={\tau }_{{\rm{s}}}\sqrt{q}$ and the coupling to the excitonic mode ${v}_{{\rm{s}}}={\beta }_{{\rm{s}}}\sqrt{q}$ of the acoustic phonon are both proportional to $\sqrt{q}$. Both vanish at q = 0 and thus the acoustic modes can not be observed in the absence of domains. The energy of the relevant B_2g acoustic mode is ω_s(q) = c_sq, where c_s is the sound velocity, that can be deduced from the X-ray measurements⁴⁰ to be c_s ≈ 30 meV Å away from T_c. Note that the coupling to the soft excitonic mode results in the renormalization of the observed c_s close to T_c; in Supplementary Note 1.3.5 it is shown that the softening of c_s close to T_c can be fully attributed to this effect. To fit the data we further fix the ratio ${\beta }_{{\rm{s}}}^{2}/{c}_{{\rm{s}}}$ such that the total static susceptibility (Eq. 1) diverges at T_c and assume that the domain size i in units of the crystal unit cell constant c follows the Poisson distribution. In addition to that, we allow for an intrinsic linewidth of the phonons at finite q: ω_s → ω_s − ir_sq. Thus, the parameters τ_s and r_s remain the only fit parameters. Additional details can be found in Supplementary Note 1.

Effect of coherence factors on low-temperature Raman scattering

To describe an excitonic insulator at low temperature we use a two-band mean-field model (see Supplementary Note 2 for details): ${\hat{H}}_{{\rm{MF}}}={\sum }_{{\bf{p}}}{\varepsilon }_{{\rm{c}}}({\bf{p}}){\hat{c}}_{{\rm{c}}}^{\dagger }({\bf{p}}){\hat{c}}_{{\rm{c}}}({\bf{p}})+{\varepsilon }_{{\rm{v}}}({\bf{p}}){\hat{c}}_{{\rm{v}}}^{\dagger }({\bf{p}}){\hat{c}}_{{\rm{v}}}({\bf{p}})+W({\hat{c}}_{{\rm{c}}}^{\dagger }({\bf{p}}){\hat{c}}_{{\rm{v}}}({\bf{p}})+{\hat{c}}_{{\rm{v}}}^{\dagger }({\bf{p}}){\hat{c}}_{{\rm{c}}}({\bf{p}}))$, where ε_c(v)(p) is the dispersion of the conduction (valence) band and W—is the excitonic order parameter (spin index is omitted for brevity). The order parameter mixes two bands such that the eigenstates are superpositions of band states with coherent factors, analogous to the BCS ones. To calculate the Raman intensity, we use the following form of the electron-light vertex, consistent with the system’s symmetry above T_c: ${\hat{R}}_{{\rm{aa}}}={\sum }_{p}{g}_{{\rm{c}}}{\hat{c}}_{p}^{\dagger }{\hat{c}}_{p}+{g}_{{\rm{v}}}{\hat{v}}_{p}^{\dagger }{\hat{v}}_{p};\ {\hat{R}}_{{\rm{ac}}}={g}_{{\rm{ac}}}{\sum }_{p}{\hat{c}}_{p}^{\dagger }{\hat{v}}_{p}+{\hat{v}}_{p}^{\dagger }{\hat{c}}_{p},$ where ${\hat{R}}_{{\rm{aa}}}$ is the vertex for aa polarization geometry and ${\hat{R}}_{{\rm{ac}}}$— for ac geometry, and g_c/v, g_ac are coupling constants.

For the semimetallic case one can linearize the dispersion ε_c,v(p) around the Fermi surface. The resulting Raman intensity obtained using Fermi’s Golden Rule is:

$${I}_{{R}_{{\rm{ac}}}} \sim \frac{4\pi {g}_{{\rm{ac}}}^{2}{\nu }_{0}\sqrt{{\omega }^{2}/4-{W}^{2}}}{\omega };\ {I}_{{R}_{{\rm{aa}}}} \sim \frac{\pi {({g}_{{\rm{c}}}-{g}_{{\rm{v}}})}^{2}{\nu }_{0}{W}^{2}}{\omega \sqrt{{\omega }^{2}/4-{W}^{2}}},$$

(3)

for ω > 2W and zero otherwise, where ν₀ is the density of states. One can see that at the gap edge, the ac intensity vanishes due to the vanishing matrix element of the transition, as is shown in Fig. 1e, while the aa diverges as a square-root, similar to the case of a BCS superconductor³⁵, as is shown in Fig. 1g.

For the semiconductor-like 1D band structure with a direct gap 2E₀, on the other hand, we find that the interference effects are weakened such that a singularity at the gap edge is present in aa and ac scattering (See Supplementary Note 2 for details).