Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Antiferromagnetic excitonic insulator state in Sr3Ir2O7

## Abstract

Excitonic insulators are usually considered to form via the condensation of a soft charge mode of bound electron-hole pairs. This, however, presumes that the soft exciton is of spin-singlet character. Early theoretical considerations have also predicted a very distinct scenario, in which the condensation of magnetic excitons results in an antiferromagnetic excitonic insulator state. Here we report resonant inelastic x-ray scattering (RIXS) measurements of Sr3Ir2O7. By isolating the longitudinal component of the spectra, we identify a magnetic mode that is well-defined at the magnetic and structural Brillouin zone centers, but which merges with the electronic continuum in between these high symmetry points and which decays upon heating concurrent with a decrease in the material’s resistivity. We show that a bilayer Hubbard model, in which electron-hole pairs are bound by exchange interactions, consistently explains all the electronic and magnetic properties of Sr3Ir2O7 indicating that this material is a realization of the long-predicted antiferromagnetic excitonic insulator phase.

## Introduction

Detailed theoretical considerations of narrow-gap insulators date back to the 1960s, when it was realized that if the energy required to form an electron-hole pair becomes negative, a phase transition into an excitonic insulator state can occur1,2,3,4. Unscreened electron-hole Coulomb attraction is perhaps the most obvious driving force behind this phase transition, and excitonic charge insulator states are indeed thought to occur in materials such as TmSe0.45Te0.55, 1T-TiSe2, and Ta2NiSe55,6,7,8,9. Although less intuitive, effective electron-hole attraction can also arise from on-site electron-electron Coulomb repulsion U via magnetic exchange interactions between the electron and hole10. In this case, the soft exciton is expected to be a spin-triplet, which passes through a quantum critical point (QCP) with increasing effective U. The condensation of the relevant triplet exciton at the QCP gives rise to an antiferromagnetic ground state hosting a well-defined excitonic longitudinal mode4, which coexists with transverse modes that are a generic feature of ordered antiferromagnets. This longitudinal mode features excitonic character, in the sense that it modifies the local spin amplitude by creating electron-hole pairs4. In this work, we identify and study a longitudinal mode in Sr3Ir2O7, the presence of which is the key experimental signature of an antiferromagnetic excitonic insulator.

## Results

The formation of an antiferromagnetic excitonic insulator requires a very specific set of conditions. We need (i) a charge gap of similar magnitude to its magnetic energy scale and (ii) strong easy-axis anisotropy. Property (i) is a sign that the material is close to the excitonic QCP (see Fig. 1). Property (ii) is not a strict condition, but it facilitates the identification of an antiferromagnetic excitonic insulator, because the opening of a spin gap Δs protects the longitudinal mode from decay. This is because longitudinal fluctuations are often kinetically predisposed to decay into transverse modes generating a longitudinal continuum with no well-defined modes. This decay can be avoided when the energy of the longitudinal mode is lower than twice the spin gap. Iridates host strong spin-orbit coupling (SOC), which can help realize a large spin gap and bilayer Sr3Ir2O7 shown in Fig. 2a is known to have a narrow charge gap of order Δc ~ 150 meV11. The essential magnetic unit, with c-axis ordered moments, is shown in Fig. 2b12. In view of the antiferromagnetic order in Sr3Ir2O7, the material would be predicted to lie in the magnetically ordered region to the right of the QCP where the excitonic longitudinal mode is expected to appear. Because the exciton is predicted to have odd parity under exchange of the two Ir layers, we expect the excitonic longitudinal mode to be present at c-axis wavevectors corresponding to antisymmetric bilayer contributions and absent at the symmetric condition. We label these wavevectors qc = 0.5 and qc = 0, respectively. In contrast, transverse magnetic modes are expected to be present at all c-axis wavevectors, allowing the transverse and longitudinal modes to be readily distinguished.

The excitation spectrum of Sr3Ir2O7 was studied with RIXS. Figure 2c–e displays energy-loss spectra at T = 20 K, well below the Néel temperature TN = 285 K and qc = 0, 0.25 and 0.5, corresponding to L = 25.65, 26.95 and 28.25 in reciprocal lattice units (r.l.u.). These irrational L values arise because the bilayer separation d is not a rational fraction of the unit cell height c (see Methods section for details). The spectrum at qc = 0 is composed of a phonon-decorated quasi-elastic feature, a pronounced magnetic excitation at ~ 100 meV, which we later identified as the transverse mode, and a high-energy continuum. As explained above, changing qc is expected to isolate the anticipated excitonic mode. A longitudinal mode is indeed observed, reaching maximum intensity at qc = 0.5, and is highlighted by red shading in Fig. 2d, e.

In isolation, the presence of a longitudinal magnetic mode in this symmetry channel is a necessary but insufficient condition to establish an antiferromagnetic excitonic insulator, so we leverage the specific symmetry, decay, and temperature dependence of the longitudinal and transverse magnetic modes to establish the presence of the novel state. The only other candidate magnetic model that hosts a longitudinal mode of this type is a specific configuration of the bilayer Heisenberg Hamiltonian, in which the charge degrees of freedom are projected out. In particular, a model with a c-axis magnetic exchange Jc that is larger than, but not dramatically larger than, the in-plane exchange Jab is needed to produce a longitudinal mode and large easy-axis magnetic anisotropy is required to reproduce the spin gap. If JcJab, the spectrum would show only a spin-wave-like in-plane dispersion contrary to the observed qc dependence in Fig. 2c–e, and in the JcJab limit the system would become a quantum paramagnet. For Jc/Jab of order two, the bilayer Heisenberg Hamiltonian supports a longitudinal mode and for the current case of large easy-axis anisotropy, the transverse and longitudinal modes appear as well-defined modes throughout the Brillouin zone13,14,15,16. In fact, earlier reports have proposed this spin dimer model to explain RIXS measurements of the longitudinal ~ 170 meV feature in Sr3Ir2O714, 17. Although prior and subsequent non-dimerized models have also been proposed to describe Sr3Ir2O7 as rival candidates12, 18,19,20,21. These models, however, do not support a longitudinal mode (a detailed comparison between the different models is given in Supplementary Information (SI) Section 1). We, therefore, map the in-plane dispersion relations at qc = 0 and 0.5 and show them in Fig. 3a, b. At qc = 0, where the longitudinal mode is suppressed by symmetry, we observe an excitation dispersing from ~ 90 to 170 meV and a continuum at higher energies. Simultaneously analyzing qc = 0.5 and qc = 0 for each in-plane reciprocal-lattice wavevector, while leveraging the distinct symmetry properties of the longitudinal and transverse modes, allows us to isolate the longitudinal mode (see Methods section). We plot the position and peak width of the longitudinal mode in green in Fig. 3b. The transverse mode, on the other hand, is symmetry-allowed at qc = 0.5 and qc = 0 and is shown in black on Fig. 3a, b. We find that the longitudinal mode is well-defined around (0, 0) (Figs. 2c–d, 3i), but decays into the high-energy continuum as it disperses away, becoming undetectable at (1/4, 1/4) (Fig. 3h). The longitudinal mode is also detectable as a shoulder feature on the transverse mode at (1/2, 1/2) before dispersing upwards and broadening at neighboring momenta (Fig. 3e, g). The decay and merging of the longitudinal mode into the electron-hole continuum was not detected previously and suggests the realization of an antiferromagnetic excitonic insulator state because the longitudinal mode in this model has a bound electron-hole pair character and therefore will necessarily decay when it overlaps with the electron-hole continuum. This longitudinal mode decay is incompatible with a longitudinal mode arising from spin dimer excitations in a strongly isotropic bilayer Heisenberg model, which predicts well-defined modes throughout the Brillouin zone and projects out the high-energy particle-hole continuum13,14,15,16.

Since optical conductivity, tunneling spectroscopy, and photo-emission studies all report charge gaps Δc on the same energy scale of magnetic excitations (100–200 meV)11, 22,23,24,25, we model the microscopic interactions within a Hubbard Hamiltonian that retains the charge degree of freedom. In particular, the crucial difference with the Heisenberg description is that the Hubbard model retains the electron-hole continuum, whose lower edge at ω = Δc is below the onset of the two-magnon continuum: Δc < 2Δs. We considered a half-filled bilayer, which includes a single “Jeff = 1/2” effective orbital for each of the two Ir sites in the unit cell, following methods developed in parallel with this experimental study26. The model contains an effective Coulomb repulsion U, and three electron hopping parameters: nearest and next-nearest in-plane hopping terms tν (ν = 1, 2) within each Ir layer, and the spin dependent hopping strength tz(α) between Ir layers (Fig. 2b). tz(α) is composed of an amplitude tz and a phase α arising from the appreciable SOC in the material (further details are given in the Methods section)27. The model was solved using the random phase approximation (RPA) in the thermodynamic limit (SI Section 2), which is valid for intermediately correlated materials even at finite temperature28. We constrain tν and tz to values compatible with density functional theory and photo-emission measurements and consider the effective U, which is strongly influenced by screening, as the primary tuning parameter29. Figure 3c, d show the results of calculations with t1 = 0.115 eV, t2 = 0.012 eV, tz = 0.084 eV, α = 1.41, and U = 0.325 eV. The small U is due to the extended Ir orbitals and because this effective parameterization reflects the difference between on-site and longer-range interactions in the real material. The model identifies the quasiparticle dispersion at qc = 0 as the transverse mode with a persistent well-defined nature even at high energies. Above the transverse mode, the spin response is fundamentally influenced by the finite charge gap. A broad continuum involving electron-hole spin transitions across the charge gap is present for all qc values covering a broad energy-momentum range. A new mode emerges around (0, 0) and (0.5, 0.5) for qc = 0.5, which we identify as the excitonic longitudinal mode.

To understand the excitonic longitudinal mode discussed, we first note that the tight-binding band structure analysis of Sr3Ir2O7 suggests that it would be a narrow-gap band insulator or semi-metal even when Coulomb repulsion is neglected29. This occurs due to bonding-antibonding band splitting arising from the bilayer hopping alongside SOC, generating a minimum of the conduction band dispersion near the Brillouin zone center and a maximum in the valence band dispersion near the antiferromagnetic zone center. A finite value of U in a quasi-two-dimensional bilayer structure such as Sr3Ir2O7 produces an attractive particle-hole interaction in the triplet channel because of the well-known direct-exchange mechanism. In turn, particle-hole pairs at wavevectors favored by the band structure form bound states, i.e., excitons, in the magnetic channel appearing at qc = 0.5, because of the odd parity of the exciton under exchange of the two layers. The spin anisotropy arising from SOC splits the exciton triplet into a low energy state with c-axis spin quantum number Sz = 0 and higher energy Sz = ± 1 states. Strictly speaking, SOC means that total spin is not a good quantum number, but we retain the singlet-triplet labels for clarity. As shown in the schematic representation in Fig. 1, the Sz = 0 exciton condenses to form magnetic order at a wavevector of (0.5, 0.5) (qc = 0.5). The corresponding QCP, which exists at U = Uc = 0.27 eV (for t1 = 0.115 eV), then signals the onset of the antiferromagnetic excitonic insulator state in Sr3Ir2O7. Within the ordered state, what was a gapless Sz = 0 exciton mode at U = Uc becomes a gapped excitonic longitudinal mode for U > Uc. The existence and relatively low energy of this mode implies that U in Sr3Ir2O7 is only slightly above Uc. This property, together with the sufficiently large transverse mode gap Δs, protects the excitonic longitudinal mode from decay into pairs of transverse modes. The longitudinal mode’s bound electron-hole pair nature is especially vividly illustrated by its smooth merging with the particle-hole continuum away from (0.5, 0.5) and (0, 0). We plot the layer-resolved charge structure of the exciton in SI Section 3.

When heating an antiferromagnetic excitonic insulator, thermal fluctuations modify the magnetic properties via two different processes. The first one corresponds to the destruction of Néel order via softening of the longitudinal mode. This softening signals the exciton condensation below T = TN. The second process, that takes place at a higher temperature T*, corresponds to thermal breaking of the excitons (unbinding of particle-hole pairs). A RIXS temperature series designed to test this idea at different high symmetry locations is plotted in Fig. 4a–d (linecuts at selected temperatures are shown in Supplementary Fig. S2). As expected, heating up from base temperature towards TN enhances the decay of the modes into the electron-hole continuum broadening the spectra and making it difficult to isolate the two modes in a single spectrum. We can, however, leverage the symmetry properties of the modes at different reciprocal space points to clarify the soft mode phenomenology. Since the transverse mode occurs at the same energy independent of qc, and the longitudinal mode is present at qc = 0.5 and absent at qc = 0, the transverse mode temperature dependence can be studied in isolation at qc = 0 (Fig. 4a, b). We observe that this mode has only minimal detectable softening, which is expected in view of the Ising nature of magnetism. In contrast, a substantial softening is seen at (0.5, 0.5) in Fig. 4d. Although both modes are present at qc = 0.5, we know from qc = 0 measurements that the transverse mode displays only minimal softening. Thus the longitudinal mode must play a major role in the softening to form the antiferromagnetic state. Our observed phenomenology is only captured with the intermediate coupling regime (U/t1 = 2.83) that we conclude is relevant for Sr3Ir2O7. The strong coupling limit (U/t1 1) would require a charge gap much larger than the observed values of 100–200 meV and, to our knowledge, it has not been able to predict any aspects of the temperature-dependent phenomenology of Sr3Ir2O7. The excitonic insulator model is also supported by our temperature-dependent calculations, which are shown as dashed lines in Fig. 4c, d. Full calculations are shown in Supplementary Fig. S5 and explained in SI Section 2. Theory shows that exciton formation takes place at T* ≈ 2TN, controlled by the exciton binding energy, which is of order the charge gap minus the longitudinal mode energy at the ordering wavevector. The mean-field transition temperature prediction is TN = 424 K, which is not too far above the measured TN = 285 K and which is expected since fluctuations are expected to reduce TN below the mean-field prediction. The predictions in Fig. 4c, d are shown with temperatures re-normalized to the experimental TN.

The involvement of the longitudinal mode in magnetic long-range order is also evident from the temperature dependent quasi-elastic intensity. While most spectra feature the expected gradual enhancement in the quasi-elastic channel upon increasing temperature (Fig. 4e for (0, 0) and S2 for other reciprocal-lattice positions), the (0.5, 0.5) spectrum at qc = 0.5 displays a pronounced rise of intensity around TN (Fig. 4f). Note that neither qc = 0 nor qc = 0.5 correspond to the magnetic Bragg peak location, because the bilayer separation is incommensurate with respect to the c-axis lattice constant. Since in our setup qc = 0 is closer to a magnetic Bragg peak than qc = 0.5, we can exclude critical scattering from the long-range antiferromagnetic order as a significant contributor to this intensity as it would predict the opposite intensity behavior to what we observe (a more extensive demonstration of this is in SI Section 5). Thus the observed quasi-elastic anomaly at TN is indicative of substantial longitudinal mode condensation. The excitonic insulator character of the ground state is further supported by a large increase in resistivity below TN (see Fig. 4f)30, as the condensation of the excitonic mode leads to a reduction in the electronic carriers participating in electrical transport. This property is distinct from what is expected for a strongly-coupled Mott insulator (i.e., the large U limit of Fig. 1) where all charge-related processes are frozen out. The resistivity increase below TN could, in principle, also arise from Slater-type interactions, which can open a charge gap upon magnetic ordering. Sr3Ir2O7, however, lacks strong Fermi surface nesting23,24,25, 29 and is in the intermediately correlated (t1 ~ U) rather than the weakly correlated (t1U) regime, so the Slater mechanism is expected to have minimal relevance.

## Discussion

In summary, we have isolated and characterized a longitudinal magnetic mode in Sr3Ir2O7, which merges with the electron-hole continuum at certain points in the Brillouin zone, and which softens upon heating concurrent with a decrease in the material’s resistivity. These properties are consistent with those of an antiferromagnetic excitonic insulator state4. We substantiate this via calculations of a bilayer Hubbard model, in which electron-hole pairs are bound by magnetic exchange interactions between the electron and hole. This consistently explains all the electronic and magnetic properties of Sr3Ir2O7 based on only one free parameter U, since all other parameters are strongly constrained by the electronic band structure of the material. The totality of these results identifies Sr3Ir2O7 as a compelling candidate for the long-sought-after antiferromagnetic excitonic insulator.

Looking to the future, the intrinsically coupled spin and charge degrees of freedom in this state could have the potential for realizing new functionalities31, and suitably tuned material and/or laser-based approaches could realize methods to photo-excited these modes32. Further research on the topic may also include efforts to identify materials closer to the QCP, which in our study occurs at U/t1 = 2.35. This could extend the reciprocal space regions where the excitonic longitudinal mode exists. Another interesting direction would involve identifying excitonic easy-plane, rather than easy-axis, bilayer systems. These would host a different kind of soft excitonic longitudinal mode, often called “Higgs” mode, and could be used to study Higgs decay and renormalization effects in the presence of strong charge fluctuations. Careful selection of materials with multiple active orbitals could realize orbitally-ordered excitonic insulator states. Experimental realizations using chemical substitutions, strained thin films, high pressure, or different bilayer materials, including ruthenates, osmates, and other iridates, may help to answer some of these intriguing questions.

## Methods

### Samples

Sr3Ir2O7 single crystals were synthesized using the flux method33. Starting materials of IrO2, SrCO3, and SrCl2 6H2O were mixed with a molar ratio of 1:2:20, and heated at 1200 C for 10 h in a platinum crucible. The melt was then cooled to 800°C at a rate of 3 C/h, before quenching to room temperature. We index reciprocal space using a pseudo-tetragonal unit cell with a = b = 3.896 Å and c = 20.88 Å at room temperature.

### Resonant inelastic X-ray scattering (RIXS) setup

RIXS spectra were measured at the 27-ID-B station of the Advanced Photon Source at Argonne National Laboratory. The incident x-ray beam was tuned to the Ir L3-edge at 11.215 keV and monochromated using a Si (884) channel-cut monochromator. The exact x-ray energy was refined via resonant energy of a standard IrO2 and the Sr3Ir2O7 sample and was set 3 eV below the resonant edge. Scattered photons were analyzed using a spherically bent diced silicon (844) analyzer with a curvature radius of 2 m. The energy and Q-resolution were 32.0(2) meV and 0.105 Å−1 full-width at half-maximum (FWHM), respectively. A small background contribution arising from air scattering was removed by subtracting a constant value from the measured intensity. The value was determined by fitting the intensity on the energy-gain side of the spectra.

The L values in Fig. 2c–e were chosen such that they correspond to specific reciprocal-lattice positions with respect to the Ir-Ir interlayer spacing (see also Fig. 2a), i.e., G + qc = Ld/c, where G is an integer, qc the reduced c-axis reciprocal lattice position in terms of the Ir-Ir spacing, d = 4.07 Å the shortest Ir-Ir interlayer spacing and c = 20.88 Å the out-of-plane lattice constant. qc equals 0, 0.25 and 0.5 for L = 25.65, 26.95 and 28.25, respectively.

The magnetic dispersions in Fig. 3a, b were measured along (H1, K1, 25.65) and (H2, K2, 28.25) with H1 and K1 ranging between 0.5 and 1 and H2 and K2 between 0 and 0.5. The particular Brillouin zones were chosen to ensure a scattering geometry close to 90, minimizing Thompson scattering. For (0, 0, 25.65), (1, 1, 25.65), (0, 0, 26.92) and (0, 0, 28.25), 2θ = 85.5, 90.2, 90.9 and 96.8, respectively. The sample was aligned in the horizontal (H, H, L) scattering plane, such that both dispersions could be probed through a sample rotation of Δχ ≤ 4.1 relative to the surface normal.

### Analysis of the RIXS data

The spectra were analyzed by decomposing them into four components: (1) A quasi-elastic contribution (possibly containing contributions from phonons) which was modeled using a pseudo-Voigt energy resolution function, along with an additional low energy feature, which was modeled using the resolution functions at ± 32 meV, whose relative weights were constrained to follow the Bose factor. (2) The transverse magnetic mode was accounted for by a pseudo-Voigt function multiplied with an error function to capture the high-energy tail arising from the interactions with continuum. The interactions are enhanced when the modes and the continuum are less separated in energy, which leads to a reduced quasiparticle lifetime. In this case, we used a damped harmonic oscillator (with Bose factor) that was convoluted with the resolution function, which was further multiplied by an error function. (3) The longitudinal mode was described by either a pseudo-Voigt function or a damped harmonic oscillator, depending on whether or not it was resolution limited. (4) The magnetic continuum was reproduced using a broad damped harmonic oscillator multiplied by an error function to mimic its onset.

The excitonic longitudinal mode is strongly qc dependent, whereas the transverse magnetic mode and the magnetic continuum vary very weakly with qc. Thus, we analyzed the spectra measured at qc = 0 and qc = 0.5 simultaneously to disentangle the excitonic contribution from the other components. The positions and lineshapes of the transverse magnetic mode and the electron-hole magnetic continuum were constrained to be independent of qc, i.e., only the amplitudes were varied. The extra peaks at qc = 0.5 give information about the excitonic longitudinal mode. During the procedure, the elastic energy was allowed to vary to correct for small fluctuations of the incident energy.

### Theoretical model

Sr3Ir2O7 hosts Ir4+ ions, which have 5 electrons in the active Ir 5d5 valance band. The dominant splitting of this band comes from the close-to-cubic crystal field leaving empty eg states and 5 electrons in the t2g states. SOC further splits the t2g manifold into a full Jeff = 3/2 orbital a half-filled Jeff = 1/2 orbital at the Fermi level34. Our model involves projecting the band structure onto this Jeff = 1/2 doublet. The basic structural unit, shown in Fig. 2b, contains two Ir atoms, so the experimental data were interpreted using a half-filled bilayer Hubbard model H = − HK + HI with HI = Urnrnr and

$${H}_{{{{{{{{\rm{K}}}}}}}}}=\mathop{\sum}\limits_{{{{{{{{\boldsymbol{r}}}}}}}},{{{{{{{{\boldsymbol{\delta }}}}}}}}}_{\nu }}{t}_{\nu }{c}_{{{{{{{{\boldsymbol{r}}}}}}}}}^{{{{\dagger}}} }{c}_{{{{{{{{\boldsymbol{r}}}}}}}}+{{{{{{{{\boldsymbol{\delta }}}}}}}}}_{\nu }}+\mathop{\sum}\limits_{{{{{{{{{\boldsymbol{r}}}}}}}}}_{\perp }}{c}_{({{{{{{{{\boldsymbol{r}}}}}}}}}_{\perp },1)}^{{{{\dagger}}} }{t}_{z}(\alpha ){c}_{({{{{{{{{\boldsymbol{r}}}}}}}}}_{\perp },2)}+{{{{{{{\rm{H}}}}}}}}.{{{{{{{\rm{c}}}}}}}}.,$$
(1)

where tν (ν = 1, 2) are the nearest- and next-nearest-neighbor hopping amplitudes within the square lattice of each Ir-layer, and $${t}_{z}(\alpha )=| {t}_{z}| {e}^{i\frac{\alpha }{2}{\varepsilon }_{{{{{{{{\boldsymbol{r}}}}}}}}}{\sigma }_{z}}$$, with σz the Pauli matrix describes the Jeff spin dependent hopping strength between layers. The overall phase was chosen to gauge away the phase for tν. The operator $${c}_{{{{{{{{\boldsymbol{r}}}}}}}}}^{{{{\dagger}}} }$$ = [$${c}_{\uparrow ,{{{{{{{\boldsymbol{r}}}}}}}}}^{{{{\dagger}}} }$$, $${c}_{\downarrow ,{{{{{{{\boldsymbol{r}}}}}}}}}^{{{{\dagger}}} }$$] creates the Nambu spinor of the electron field at r = (r, l) with l = 1, 2 denoting the layer index and r = r1a1 + r2a2. Here, the primitive in-plane lattice vectors are denoted by a1 and a2, and the directed neighboring bonds are represented by δ1 = a1, a2 and δ2 = a1 ± a2. In the interaction term HI, U is the effective Coulomb interaction, and nrσ is the density operator for electrons of spin σ at r. In the spin dependent hopping term, the sign εr takes the values ± 1 depending on which sublattice of the bipartite bilayer system r points to. The phase α arises from hopping matrix elements between dxz and dyz orbitals, which are allowed through the staggered octahedral rotations in the unit cell along side SOC27, 35. In the model, SOC enters via the phase of the c-axis hopping, which is smaller than the in-plane bandwidth, justifying the approximate use of singlet and triplet for labels of the different excitons. The model was studied at half-filling in the sense that it contains two bands (bonding and antibonding) in the model, which host two electrons as is appropriate for Sr3Ir2O723,24,25, 27. We solved the model using the RPA in the thermodynamic limit (detailed information is given in SI Section 2), which is valid for intermediately correlated materials even at finite temperatures28. The theoretically determined Néel temperature, in this case, is $${T}_{\,{{\mbox{N}}}}^{{{\mbox{cal}}}\,}=424$$ K which is slightly larger than the experimental value TN = 285 K. This is expected within the RPA we use here, as this ignores fluctuations than act to reduce the transition temperature. The dynamical spin structure factors in Fig. 3c, d are shown after convolution with the experimental resolution.

A more complex model could include all t2g or all d orbitals, rather than just effective Jeff = 1/2 doublets. The success of our Jeff = 1/2 only model suggests that orbital degrees of freedom are entirely frozen out of the problem or manifest themselves in very subtle ways beyond current detection limits. Due to this, the Sr3Ir2O7 excitonic insulator state has no orbital component (other than in the trivial sense that the Jeff = 1/2 states in themselves are a coupled modulation of spin and orbital angular momentum). A possible SOC-induced orbital order is discussed in SI Section 6.

## Data availability

The RIXS data generated in this study have been deposited in the Zenodo database under accession code 581298936.

## Code availability

The code used in this study is available from the authors upon reasonable request.

## References

1. Mott, N. F. The transition to the metallic state. Philos. Mag. 6, 287–309 (1961).

2. Keldysh, L. & Kopaev, Y. V. Possible instability of semimetallic state toward Coulomb interaction. Sov. Phys. Solid State, USSR 6, 2219–2224 (1965).

3. Jérome, D., Rice, T. M. & Kohn, W. Excitonic insulator. Phys. Rev. 158, 462–475 (1967).

4. Halperin, B. I. & Rice, T. M. Possible anomalies at a semimetal-semiconductor transistion. Rev. Mod. Phys. 40, 755–766 (1968).

5. Bucher, B., Steiner, P. & Wachter, P. Excitonic insulator phase in TmSe0.45Te0.55. Phys. Rev. Lett. 67, 2717–2720 (1991).

6. Wakisaka, Y. et al. Excitonic insulator state in Ta2NiSe5 probed by photoemission spectroscopy. Phys. Rev. Lett. 103, 026402 (2009).

7. Eisenstein, J. Exciton condensation in bilayer quantum hall systems. Annu. Rev. Condens. Matter Phys. 5, 159–181 (2014).

8. Kogar, A. et al. Signatures of exciton condensation in a transition metal dichalcogenide. Science. 358, 1314–1317 (2017).

9. Lu, Y. et al. Zero-gap semiconductor to excitonic insulator transition in Ta2NiSe5. Nat. Commun. 8, 1–7 (2017).

10. Slater, J. C. Magnetic effects and the Hartree-Fock equation. Phys. Rev. 82, 538–541 (1951).

11. Moon, S. J. et al. Dimensionality-controlled insulator-metal transition and correlated metallic state in 5d transition metal oxides Srn+1IrnO3n+1: (n = 1, 2, and ). Phys. Rev. Lett. 101, 226402 (2008).

12. Kim, J. et al. Large spin-wave energy gap in the bilayer iridate Sr3Ir2O7: Evidence for enhanced dipolar interactions near the Mott metal-insulator transition. Phys. Rev. Lett. 109, 157402 (2012).

13. Lohöfer, M. et al. Dynamical structure factors and excitation modes of the bilayer Heisenberg model. Phys. Rev. B 92, 245137 (2015).

14. Moretti Sala, M. et al. Evidence of quantum dimer excitations in Sr3Ir2O7. Phys. Rev. B. 92, 024405 (2015).

15. Zhou, C., Yan, Z., Wu, H.-Q., Sun, K., Starykh, O. A. & Meng, Z. Y. Amplitude mode in quantum magnets via dimensional crossover. Phys. Rev. Lett. 126, 227201 (2021).

16. Su, Y. et al. Stable Higgs mode in anisotropic quantum magnets. Phys. Rev. B. 102, 125102 (2020).

17. Hogan, T. et al. Disordered dimer state in electron-doped Sr3Ir2O7. Phys. Rev. B. 94, 100401 (2016).

18. Gretarsson, H. et al. Two-magnon raman scattering and pseudospin-lattice interactions in Sr2IrO4 and Sr3Ir2O7. Phys. Rev. Lett. 116, 136401 (2016).

19. Lu, X. et al. Doping evolution of magnetic order and magnetic excitations in $${({{{{{{{{\rm{Sr}}}}}}}}}_{1-x}{{{{{{{{\rm{La}}}}}}}}}_{x})}_{3}{{{{{{{{\rm{Ir}}}}}}}}}_{2}{{{{{{{{\rm{O}}}}}}}}}_{7}$$. Phys. Rev. Lett. 118, 027202 (2017).

20. Li, S. et al. Symmetry-resolved two-magnon excitations in a strong spin-orbit-coupled bilayer antiferromagnet. Phys. Rev. Lett. 125, 087202 (2020).

21. Mohapatra, S., van den Brink, J. & Singh, A. Magnetic excitations in a three-orbital model for the strongly spin-orbit coupled iridates: Effect of mixing between the $$j=\frac{1}{2}$$ and $$j=\frac{1}{2}$$ sectors. Phys. Rev. B. 95, 094435 (2017).

22. Okada, Y. et al. Imaging the evolution of metallic states in a correlated iridate. Nat. Mater. 12, 707–713 (2013).

23. Wang, Q. et al. Dimensionality-controlled mott transition and correlation effects in single-layer and bilayer perovskite iridates. Phys. Rev. B 87, 245109 (2013).

24. King, P. D. C. et al. Spectroscopic indications of polaronic behavior of the strong spin-orbit insulator Sr3Ir2O7. Phys. Rev. B. 87, 241106 (2013).

25. de la Torre, A. et al. Coherent quasiparticles with a small fermi surface in lightly doped Sr3Ir2O7. Phys. Rev. Lett. 113, 256402 (2014).

26. Suwa, H., Zhang, S.-S. & Batista, C. D. Exciton condensation in bilayer spin-orbit insulator. Phys. Rev. Res. 3, 013224 (2021).

27. Carter, J.-M., Shankar V., V. & Kee, H.-Y. Theory of metal-insulator transition in the family of perovskite iridium oxides. Phys. Rev. B 88, 035111 (2013).

28. Hirschmeier, D., Hafermann, H., Gull, E., Lichtenstein, A. I. & Antipov, A. E. Mechanisms of finite-temperature magnetism in the three-dimensional Hubbard model. Phys. Rev. B. 92, 144409 (2015).

29. Carter, J.-M. & Kee, H.-Y. Microscopic theory of magnetism in Sr3Ir2O7. Phys. Rev. B. 87, 014433 (2013).

30. Cao, G. et al. Anomalous magnetic and transport behavior in the magnetic insulator Sr3Ir2O7. Phys. Rev. B. 66, 214412 (2002).

31. Tokura, Y., Kawasaki, M. & Nagaosa, N. Emergent functions of quantum materials. Nat. Phys. 13, 1056–1068 (2017).

32. Kang, S. et al. Coherent many-body exciton in van der Waals antiferromagnet NiPS3. Nature. 583, 785–789 (2020).

33. Li, L. et al. Tuning the $${J}_{{{{{{{{\rm{eff}}}}}}}}}=\frac{1}{2}$$ insulating state via electron doping and pressure in the double-layered iridate Sr3Ir2O7. Phys. Rev. B. 87, 235127 (2013).

34. Kim, B. J. et al. Novel Jeff = 1/2 Mott state induced by relativistic spin-orbit coupling in Sr2IrO4. Phys. Rev. Lett. 101, 076402 (2008).

35. Cao, G. & Schlottmann, P. The challenge of spin–orbit-tuned ground states in iridates: a key issues review. Rep. Prog. Phys. 81, 042502 (2018).

36. Mazzone, D. G. et al. Data repository for: Antiferromagnetic excitonic insulator state in Sr3Ir2O7 https://doi.org/10.5281/zenodo.5812988 (2022).

## Acknowledgements

We thank D.F. McMorrow, B. Normand, Ch. Rüegg, and J.P. Hill for fruitful discussions. Work performed at Brookhaven National Laboratory was supported by the US Department of Energy, Division of Materials Science, under Contract No. DE-SC0012704. This research used resources of the Advanced Photon Source, a U.S. Department of Energy (DOE) Office of Science User Facility operated for the DOE Office of Science by Argonne National Laboratory under Contract No. DE-AC02-06CH11357. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. D.G.M. acknowledges support from the Swiss National Science Foundation, Fellowship No. P2EZP2_175092. H.S. acknowledges support from JSPS KAKENHI Grant No. JP19K14650. X.L. acknowledges the support from the National Natural Science Foundation of China under grant No. 11934017. K.J. and Y.G.S. acknowledge the support from the National Natural Science Foundation of China (Grants No. U2032204), and the K.C. Wong Education Foundation (GJTD-2018-01). Y.G.S. acknowledges the Chinese National Key Research and Development Program (No. 2017YFA0302901) and the Strategic Priority Research Program (B) of the Chinese Academy of Sciences (Grant No. XDB33000000). J.L. acknowledges support from the National Science Foundation under Grant No. DMR-1848269. J.Y. acknowledges funding from the State of Tennessee and Tennessee Higher Education Commission (THEC) through their support of the Center for Materials Processing. H.M. was sponsored by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U.S. Department of Energy.

## Author information

Authors

### Contributions

The project was conceived and initiated by M.P.M.D, C.D.B., J.L. and X.L. D.G.M., Y.S., G.F., J.Y., H.M., M.H.U. and D.M.C. prepared and performed the experiments. K.J. and Y.Shi synthesized the samples. D.G.M. and Y.S. analyzed the experimental data, H.S. and S.S.Z. provided the theoretical calculations. The results were interpreted by D.G.M., Y.S., H.S., J.S., J.L. C.D.B, and M.P.M.D. The paper was written by D.G.M., Y.S., H.S., X.L., C.D.B. and M.P.M.D. with the input from all co-authors.

### Corresponding authors

Correspondence to D. G. Mazzone, X. Liu or M. P. M. Dean.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Peer review

### Peer review information

Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Mazzone, D.G., Shen, Y., Suwa, H. et al. Antiferromagnetic excitonic insulator state in Sr3Ir2O7. Nat Commun 13, 913 (2022). https://doi.org/10.1038/s41467-022-28207-w

• Accepted:

• Published:

• DOI: https://doi.org/10.1038/s41467-022-28207-w