Introduction

The Dicke model in quantum optics describes the cooperative, coherent coupling of an ensemble of two-level atoms with a single-mode light field1. Despite its simplicity, the model hosts a rich variety of phenomena that are significant in diverse contexts, such as cavity quantum electrodynamics2, condensed matter physics3, and quantum information science4,5. A prominent feature of the model is a second-order quantum phase transition (QPT), known as the superradiant phase transition (SRPT), which occurs when the light–matter coupling strength, g, exceeds a threshold6,7. When the system enters the superradiant phase, atomic and photonic polarizations spontaneously emerge, producing a unique many-body ground state that enables studies of unusual light–matter entanglement8, two-mode squeezed states9,10,11, and quantum chaos12.

Although the atomic ensemble in the original Dicke model was assumed to be noninteracting, it has been known from the early days that atom–atom interactions are important for explaining, for example, the dephasing and intensity correlation functions of fluorescent spectra13,14. Hence, there has long been interest in extending the Dicke model to include an atom–atom interaction (represented by strength J); see Fig. 1. Such an extended Dicke model, or the g–J model, should display an interplay of two types of interatomic interactions – i.e., the photonic-field-mediated long-range interaction, and the direct short-range interaction. Intuitively, one can expect the ground state of the system to crucially depend on the ratio g/J, with a superradiant phase (an atomically ordered phase) favored for large (small) g/J. However, no analytical solutions can be obtained for the g–J model, motivating one to simulate it using a well-characterized many-body quantum system.

Fig. 1: The extended Dicke model, or the g–J model, where an ensemble of interacting two-level atoms collectively couples with a bosonic field.
figure 1

The cooperative boson–atom interaction, with strength g, mediates long-range atom–atom interactions, whereas the direct atom–atom interaction, with strength J, is short-ranged.

Full size image

Computational studies of the g–J model under various approximations have revealed an array of new phenomena, such as a first-order QPT15,16,17,18,19, a shift of the SRPT boundary20,21, amplification of the integrablity-to-chaos transition22, modifications of matter–matter entanglement20,23, and alteration of the nature of an excited-state QPT18,24. To examine these phenomena, several experimental platforms, including atomic Bose–Einstein condensates25,26, superconducting qubits27,28, and quantum dots15, have been proposed as quantum simulators, but successful simulations have not been achieved.

Here, we present a novel protocol of using a crystal of erbium orthoferrite (ErFeO3), an antiferromagnetic (AFM) insulator, as a solid-state quantum simulator of the g–J model. The magnetic properties of ErFeO3 are governed by the moments carried by the Er3+ and Fe3+ spin subsystems and their interplay29. A previous study has revealed Dicke cooperativity in the Er3+–Fe3+ interaction30, demonstrating the resemblance of the magnetic Hamiltonian of ErFeO3 to the Dicke Hamiltonian. Namely, the paramagnetic Er3+ ions (the magnons of ordered Fe3+ spins) play the role of the atomic ensemble (light field), and the spin–magnon interaction is formally similar to the g-term in the Dicke model. What further strengthens this analogy is a magnetic phase transition of the crystal that exhibits all traits that would be expected for a Dicke SRPT. When the temperature (T) becomes lower than 4 K, the Er3+ lattice develops C-type AFM order31 (with the ferromagnetic chains along z), and a zone-boundary Fe3+ magnon mode condenses, displacing the staggered moments away from the x–z plane32,33; this corresponds to the emergence of atomic and photon polarizations in the standard SRPT. In Bertaut’s notation, the magnetic transition is of the Γ2 → Γ12 type (Fig. 2a). Mean-field calculations using a realistic spin model captures the simultaneous order parameter (OP) onset of both the Er3+ and Fe3+ spin components, \(\langle {\Sigma }_{z}^{-}\rangle\) and 〈Sy〉, respectively (Fig. 2b), indicating that the Γ2 → Γ12 transition is a magnonic SRPT34, with the Γ2 and Γ12 phases corresponding to the normal (N) and superradiant (S) phases, respectively.

Fig. 2: The Γ2 → Γ12 transition in ErFeO3 as a magnonic analog of the SRPT.
figure 2

a Lattice structure and spin configurations within the Γ12 and Γ2 phases. Brown polyhedra represent octahedrally coordinated FeO6 cages. b Temperature dependence of the Er3+ and Fe3+ spin components (normalized) across the phase transition at 0 T. \({\hat{\Sigma }}_{p}\), where p ∈ {x, y, z}, is the collective Er3+ spin operator, with its superscript “+” (“ − ”) denoting the sum (difference) of the two sublattices. \({\hat{S}}_{p}\) are the components of Fe3+ spins. c Temperature dependence of THz absorption spectra taken at zero magnetic field. The bright line, corresponding to the qAFM magnon mode of Fe3+ spins, shows a kink at 4 K, which is the superradiant-normal phase boundary at zero magnetic field.

Full size image

Results

One way to observe the OP onset is to monitor the quasi-antiferromagnetic (qAFM) magnon mode of Fe3+ spins through terahertz (THz) time-domain spectroscopy, which has been utilized to reveal the configuration of Fe3+ ions in rare-earth orthoferrites35. By performing THz transmission measurements on a z-cut ErFeO3 crystal in the Faraday geometry, we obtained absorption coefficient (α) spectra, derived from the imaginary part of the refractive index (see Methods), as a function of T, as shown in Fig. 2c. The observed bright absorption line is the qAFM mode, which has been thoroughly studied in previous studies35. It is the evolution of this mode in distinct phases of the g–J model that is of interest throughout this study. A continuous OP-like onset, or a kink, is observed at the N → S transition boundary (<4 K, blue dashed line). The frequency shift of the qAFM magnon mode in the S phase from that in the N phase is thus a sensitive reporter of the qAFM magnon condensate density, namely, the Fe3+ OP of the S phase.

The J-term is inherently built into the magnetic Hamiltonian of ErFeO3 since the Er3+–Er3+ exchange interaction, albeit being weak, is known to be present36. Spectroscopic measurements have also revealed a fine frequency splitting within the Er3+ electron paramagnetic resonance lines30, which is attributable to the Er3+–Er3+ exchange interaction. The presence of both the g- and J-terms sets the stage for ErFeO3 to simulate the g–J model. In the limit of strong interatomic interaction J ≫ g, one expects to find an atomic (A) phase characterized by an interatomic OP, without the superradiant order. Nonetheless, although the g-term-driven S phase can find correspondence to the Γ12 phase in ErFeO3, the g/J ratio set for the crystal stipulates that a pure atomic (A) phase, which is driven exclusively by the J-term, would not appear in equilibrium. For ErFeO3, the A phase would be an Er3+ ordered phase without involving any OP onset in the Fe3+ subsystem. Therefore, to achieve quantum simulation of the g–J model, we must search for a way to invoke an explicit A phase through an S → A transition.

Our theoretical consideration suggests that subjecting ErFeO3 to a static magnetic field (H) along the z axis can potentially induce an S → A transition. This can be understood by writing the simplified magnetic Hamiltonian (see Methods) in the second-quantized form as

$$\begin{array}{r}\hat{{{{{{{{\mathcal{H}}}}}}}}}/\hslash ={\omega }_{\pi }{\hat{a}}_{\pi }^{{{{\dagger}}} }{\hat{a}}_{\pi }+{\omega }_{{{{{{{{\rm{Er}}}}}}}}}{\hat{\Sigma }}_{x}^{+}+{\omega }_{z}{\hat{\Sigma }}_{z}^{+}+g\sqrt{\frac{2}{{N}_{0}}}i({\hat{a}}_{\pi }^{{{{\dagger}}} }-{\hat{a}}_{\pi }){\hat{\Sigma }}_{z}^{-}\\ +J\frac{6}{{N}_{0}\hslash }\left[{\big({\hat{\Sigma }}_{x}^{+}\big)}^{2}+{\big({\hat{\Sigma }}_{z}^{+}\big)}^{2}-{\big({\hat{\Sigma }}_{x}^{-}\big)}^{2}-{\big({\hat{\Sigma }}_{z}^{-}\big)}^{2}\right],\end{array}$$
(1)

where a two-sublattice approximation is adopted for both Er3+ and Fe3+ for a total of N0 unit cells. Here, ωπ, \({\hat{a}}_{\pi }^{{{{\dagger}}} }\), and \({\hat{a}}_{\pi }\) are the energy, creation, and annihilation operators for the Fe3+ qAFM magnon mode, respectively; ωEr is the frequency of Er3+ spins as two-level systems at H = 0; \({\omega }_{z}=| {{\mathfrak{g}}}_{z}{\mu }_{{{{{{{{\rm{B}}}}}}}}}{\mu }_{0}H| /\hslash\), where \({{\mathfrak{g}}}_{z}\) is the Landè g factor, μB is the Bohr magneton, and μ0 is the vacuum permeability, is the H-induced Zeeman frequency of Er3+; and g and J are the Er3+–magnon and Er3+–Er3+ coupling strengths, leading to the g- and J-terms of the g–J Hamiltonian, respectively. \({\hat{\Sigma }}_{p}=\mathop{\sum }\nolimits_{i = 1}^{2{N}_{0}}{\hat{\sigma }}_{i,p}/2\), where \({\hat{\sigma }}_{p}\) are Pauli matrices and p ∈ {x, y, z}, is the collective Er3+ spin operator, with its superscript “+” (" − ”) denoting the sum (difference) of the two sublattices. The way these operators appear in Eq. (1) is crucial for interpreting the ground-state energetics. Specifically, the g-term features a product of the Fe3+ magnon field operator \(i({\hat{a}}_{\pi }^{{{{\dagger}}} }-{\hat{a}}_{\pi })\) and the \({\hat{\Sigma }}_{z}^{-}\) component of Er3+ spins, thereby favoring antiparallel alignment of Er3+ sublattices and Fe3+ magnon condensation in the S phase (the onsets of \(\langle {\hat{\Sigma }}_{z}^{-}\rangle\) and 〈Sy〉 in Fig. 2b), whereas the J-term couples Er3+ antiferromagnetically; larger \(\langle {\hat{\Sigma }}_{x}^{-}\rangle\) and \(\langle {\hat{\Sigma }}_{z}^{-}\rangle\), where 〈. . . 〉 denotes expectation values, are energetically more favorable.

It is important to note that supplying the Zeeman term \({\hat{{{{{{{{\mathcal{H}}}}}}}}}}_{{{{{{{{\rm{Zeeman}}}}}}}}}/\hslash ={\omega }_{z}{\hat{\Sigma }}_{z}^{+}\) provides quantum controllability. The term promotes \(| \langle {\hat{\Sigma }}_{z}^{+}\rangle |\), the net moment of Er3+ sublattices, through Zeeman coupling to H∥z. Due to the commutation relation

$$\left[{\hat{\Sigma }}_{z}^{+},{\hat{\Sigma }}_{z}^{-}\right]=0\, \ne \left[{\hat{\Sigma }}_{z}^{+},{\hat{\Sigma }}_{x}^{-}\right],$$
(2)

modification to \(\langle {\hat{\Sigma }}_{z}^{+}\rangle\) would impact \(\langle {\hat{\Sigma }}_{x}^{-}\rangle\) much more than \(\langle {\hat{\Sigma }}_{z}^{-}\rangle\). This would tip the balance between the g-term and the J-term, since \({\hat{\Sigma }}_{x}^{-}\) appears only in the J-term but not in the g-term.

As shown in Fig. 3a, an S → A transition is indeed recovered in the calculated mean-field phase diagram of the spin Hamiltonian (see Methods) within the T–H parameter space, for T < 2.8 K, with a critical field ranging from 0.35 T to 0.5 T, depending on T. Increasing the field to above 1 T and elevating T to above 4 K would both push the system across the thermodynamic phase boundary into the N phase. A triple point (at 2.8 K and 0.5 T, decorated by a yellow star) marks the location where the S, A, and N phases converge. Figure 3b shows the calculated normalized spin components as the OPs of the magnetic phases, for a line cut along the H axis at T = 0 K, traversing sequentially the S → A and the A → N boundaries. We identify that the Fe3+ OP, represented by 〈Sy〉, is finite in the S phase but near-zero in the A phase. The Er3+ OP, on the other hand, is finite in both the S and A phases, but undergoes a switch from the \(\langle {\hat{\Sigma }}_{z}^{-}\rangle \, \ne \, 0\), \(\langle {\hat{\Sigma }}_{x}^{-}\rangle \approx \, 0\) type (S phase) to the \(\langle {\hat{\Sigma }}_{z}^{-}\rangle \approx \, 0\), \(\langle {\hat{\Sigma }}_{x}^{-}\rangle \, \ne \, 0\) type (A phase). Further, the OP evolution indicates that the S → A boundary is an abrupt-type, first-order phase transition, while the A → N boundary is a continuous-type, second-order phase transition.

Fig. 3: Mean-field solution for the spin Hamiltonian of ErFeO3 in H∥z.
figure 3

a Theoretical T–H phase diagram mapped by Er3+ spin components. b H-dependent evolution of the Er3+ and Fe3+ spin components (normalized) at T = 0 K. c Schematic diagrams of the spin configuration in each phase.

Full size image

Summarizing the mean-field calculation results, Fig. 3c pictorially shows the predicted Fe3+ and Er3+ spin order in each phase. Starting from the N phase, the two sublattices of Fe3+ are antiparallel along z with zero y-component, while Er3+ spins remain paramagnetic (no order). The A phase is characterized by Fe3+ order that is identical to that of the N phase, but the Er3+ subsystem develops canted AFM order where the sublattice moments are antiparallel along x (\(\langle {\hat{\Sigma }}_{x}^{-}\rangle \, \ne \, 0\)), with canting along z (\(\langle {\hat{\Sigma }}_{z}^{+}\rangle \, \ne \, 0\)). In the S phase, the Er3+ order takes the \(\langle {\hat{\Sigma }}_{x}^{+}\rangle \, \ne \, 0\), \(\langle {\hat{\Sigma }}_{z}^{-}\rangle \, \ne \, 0\) configuration, and the staggered moment of the Fe3+ sublattices undergoes a rotation about the x axis, bringing its y-component to nonzero.

The S → A transition can be considered as a spin-flop transition in terms of Er3+ ions. One conventional way to characterize the transition is to monitor the magnetic susceptibility through which the existence of the AFM ordering of Er3+ spins in the A phase has been previously observed37, although the configuration of Fe3+ spins was left ambiguous. Our magnetization measurements showed clear S → A and S → N phase boundaries (squares in Fig. 4a and Supplementary Fig. 1). However, a strong and non-uniform demagnetizing effect that broadens the phase boundary38 likely prevented us from clearly identifying the A → N phase boundary. This is because the shape of our sample for magnetization measurements was a thin irregularly shaped disk cut from the sample used for THz measurements, rather than a sphere, which would have produced a uniform demagnetizing field inside the sample37. Nonetheless, a disk-shaped sample with a large lateral size was necessary for performing THz transmission measurements.

Fig. 4: Mapping out the T–H phase diagram of ErFeO3 in H∥z.
figure 4

a Phase boundaries determined by THz measurements (solid circles) and magnetization measurements (squares) overlaid on the dT/d(μ0H) color map determined from MCE measurements. Red (blue) dashed line denotes a first- (second-) order phase boundary. b–d THz absorption spectra mapped vs μ0H for select T values. e–g THz absorption spectra mapped vs T for select μ0H values. Red and blue dashed lines mark the same boundaries as those in (a). All features except for those labeled “qFM” are qAFM magnon modes of Fe3+ spins.

Full size image

To demonstrate the A → N phase boundary, i.e., the breakdown of the AFM order of Er3+ spins, we performed magnetocaloric effect (MCE) experiments which are sensitive to the magnetic entropy landscape of a material. Namely, the Grüneisen ratio39

$${\Gamma }_{H}=-\frac{{(\partial S/\partial H)}_{T}}{{C}_{H}}={\left.\frac{1}{T}\frac{\partial T}{\partial H}\right| }_{S},$$
(3)

measures the slope of isentropes in the T−H plane40. Since the heat capacity CH is always a positive quantity, the sign of \({(\frac{\partial T}{\partial H})}_{S}\) is always opposite to \({(\frac{\partial S}{\partial H})}_{T}\). Furthermore, sharp changes in entropy S due to phase transitions will appear as step functions in \({(\frac{\partial T}{\partial H})}_{S}\)40, or peaks if \((\frac{\partial T}{\partial H})\) is measured in a quasi-adiabatic environment41. Thus, by measuring the differential change in sample temperature with respect to the magnetic field, \({(\frac{\partial T}{\partial H})}_{S}\), the T−H magnetic phase diagram can be measured. We note that the demagnetization factor can have a small effect on the MCE measurements42, namely that temperature and field shifts can occur, but the qualitative features should be present.

The T–H phase diagram of ErFeO3, and the obtained results are summarized in Fig. 4a. We configured a MCE measurement in a Physical Property Measurement System in the quasi-adiabatic condition43, and took raw data traces of sample temperature variation versus magnetic field at a ramping rate of 5 × 10−3 T/s with dH > 0 (Supplementary Fig. 2a); the sensitivity of temperature variation of our instrument reached 5 × 10−4 K. To identify H-induced phase transitions, the first-order derivative \({(\frac{\partial T}{\partial H})}_{S}\) was approximated as dT/d(μ0H) (Supplementary Fig. 2b), whose local extremes correspond to the transition boundaries41. The traces clearly exhibit two maxima for T < 2.8 K, corresponding to the S → A (red dashed line in Fig. 4a) and A → N (blue dashed line) boundaries, and one maximum for 2.8 K < T < 4 K, corresponding to the S → N (blue dashed line) boundary. These results are qualitatively consistent with the T–H phase diagram reported previously37, where quantitative shifts likely come from demagnetization effects.

Once we experimentally investigated the evolution of the atomic ensemble (or the Er3+ spins) in the extended Dicke Hamiltonian, we turned to elucidate the photonic counterpart (or the Fe3+ spins). The ambiguity of the configuration of Fe3+ spins and the nature of the transition boundaries require us to monitor the qAFM magnon mode of Fe3+ spins in THz magnetospectroscopy experiments. Unlike the static measurements, responses from different domains in the A phase can be distinguished in the frequency domain, illuminating the nature of the phase transition. The measurements were performed within the same T–H parameter space as that of the MCE experiments. Figure 4b–d and e–g shows the H-dependence of α spectra at select T values and the T-dependence of α spectra at select μ0H values, respectively. We found that the bright absorption lines can be assigned to either the quasi-ferromagnetic (qFM) mode or the qAFM mode35, the latter of which can be an OP for the Fe3+ spins.

In the H-dependent color map at 1.4 K (Fig. 4b), three lines are observed. The lowest frequency line, which does not pick up intensity until 0.8 T, is the qFM mode, while the other two are both qAFM magnons, albeit belonging to distinct phases. The middle (upper) line, which is located at 0.8 THz at 0 T (1 THz at 0.5 T), is the qAFM mode of the S (A & N) phase. The S → A transition can be identified to occur at 0.5 T (red dashed line), where the upper line emerges. The qAFM magnons belonging to the S and A phases coexist within 0.5 T < μ0H < 1 T, consistent with the prediction that the S → A transition is of first order and is thus inhomogeneous, until the middle line vanishes at > 1 T (blue dashed line) owing to entrance into the N phase. The 3.2 K map (Fig. 4c) shows a different behavior; the qAFM magnon (0.88 THz at 0 T) of the S phase continuously shifts to connect with that of the N phase in frequency, forming an OP-like onset for μ0H < 0.7 T (blue dashed line), signaling a second-order N → S transition boundary. Such a frequency shift is absent in the 4.4 K map (Fig. 4d) since the N phase persists throughout the whole H range.

T-dependent color maps at constant H further corroborate our assignments of the phase transitions. Again from the 0 T map (Fig. 4e), a continuous OP-like onset of the qAFM mode shift is observed (<4 K, blue dashed line). This echoes Fig. 4c in showing the continuous nature of the N → S transition, and establishes that the frequency shift of the qAFM magnon in the S phase from that in the N phase can be the Fe3+ OP of the S phase. Intriguingly, this OP is demonstrated to be zero in the A phase. We read this fact from the 0.75 T map (Fig. 4f), for which an N → A transition is expected upon lowering T. Although a residual mode pertaining to the S phase exists (as mentioned earlier when discussing Fig. 4b), the qAFM mode (unlabeled line) frequency does not undergo any noticeable OP-like anomaly across the N → A transition; it is as featureless as the qAFM mode within the 1.25 T map (Fig. 4g), for which the N phase persists throughout the whole T range. This unambiguously demonstrates that the spin order in the A phase only involves Er3+ ordering but not any Fe3+ OP, consistent with our expectation depicted in Fig. 3. Finally, phase boundaries determined by the THz experiments are overlaid (as solid circles) on top of the MCE phase diagram in Fig. 4a, showing overall agreement.

Discussion

A potential impact of this analogy is the possibility of being applied to other members of the rare-earth orthoferrite family or orthochromite compounds. For example, spin-reorientation phase transitions44,45 (Γ4 → Γ2) in RFeO3 (R=Yb, Er, and Tb) would mimic the SRPT. In YbFeO3, where the Yb3+–Yb3+ interaction (J) is negligible, it would be a potential playground for studying the standard Dicke model (g model). At the boundaries of the phase transition of YbFeO3, the qFM mode of Fe3+ shows a kink, and a transition inside the ground doublet of Yb3+ ions shows a softening46. This simultaneous kink and softening is one of the hallmarks of a magnonic SRPT47. It was also suggested that TbFeO3 can be regarded as the magnetic phase transition of the Jahn-Teller type48,49 that would resemble a magnonic SRPT. In ErFeO3, where a crystal field transition (~1.5 THz) is responsible for the spin-reorientation transition (T = 87 K), the crystal field levels would play the role of an ensemble of two-level atoms in the Dicke model. To prove that Dicke physics is at work, however, one must show Dicke cooperativity, i.e., the coupling strength g must exhibit cooperative enhancement \(g\propto \sqrt{N}\), where N is the number of two-level atoms. In addition, mapping their spin Hamiltonians into the Dicke models is required to establish this analogy. No attempts have been made to develop an analogy between the spin-reorientation transition and the Dicke superradiant phase transition.

The advantages of using the low-temperature phase transition, as opposed to the ~80 K spin-reorientation phase transition, of ErFeO3 in simulating the extended Dicke model can be summarized as follows. First, the low-temperature phase transition allows us to simulate the first-order phase transition into the A phase, which is the main point of this work and does not exist in the spin-reorientation transition at 87 K. Second, since we deal with the lowest two energy levels (Kramers doublet) of Er3+ ions in the low-temperature phase transition, theoretical analysis is directly relevant to the Dicke model, compared to the multiple crystal-field energy levels involved in the 87 K phase transition. Third, and most importantly, at high temperatures, thermally populated magnons are not negligible. Such thermal magnons will prevent studies of the vacuum magnons responsible for the Dicke superradiant phase transition, which occurs in thermal equilibrium without any external driving. For example, one consequence of the superradiant phase transition induced by vacuum bosonic fields is a two-mode perfect squeezed vacuum at the critical point34. A finite number of thermally excited magnons will mask such interesting quantum phenomena.

Conclusion

In summary, through THz magnetospectroscopy and magnetocaloric effect experiments, we studied a crystal of ErFeO3 to simulate the g–J model, which is an extended Dicke model that includes not only the bosonic-field-mediated long-range interatomic interactions but also direct short-range interactomic interactions. In addition to the superradiant and normal phases expected from the standard Dicke model, we identified a new phase, an atomic phase, which is driven by the short-range J-term in the Hamiltonian. Further, we elucidated the nature of the various phase boundaries, distinguishing between first-order and second-order transitions. These results demonstrated the potential of ErFeO3 as a simulator of quantum optics Hamiltonians. More specifically, in the context of Dicke physics, this condensed matter platform may lead to the possibilities of assisting quantum chaos22 and modifying matter–matter entanglement20,23 with tunability given through an external magnetic field. Bridging the gap between quantum optics and many-body correlated physics, our results will find broader application in the design of hybrid quantum systems with superior controllability, such as the Dicke-Ising machine27,50 and the Dicke-Lipkin-Meshkov-Glick model18,23. Furthermore, the ability to transition between the superradiant and atomic phases via a nonthermal knob provides opportunities to study unconventional quantum criticality51 and chaos-assisted thermalization52.

Methods

Spin Hamiltonian

Following the prior work34, the spin Hamiltonian taking into account all the spin subsystems and their mutual interactions is first introduced. We analyzed the ErFeO3 spin system from a microscopic model originally derived in ref. 34. The total microscopic Hamiltonian is

$${{{{{{{\mathcal{H}}}}}}}}={{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}}+{{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}}+{{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}-{{{{{{{\rm{Er}}}}}}}}}.$$
(4)

\({{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}}\), \({{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}}\), and \({{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}-{{{{{{{\rm{Er}}}}}}}}}\) are Fe3+, Er3+, and Fe3+–Er3+ interaction Hamiltonian, respectively. As in our previous studies30,53 and Herrmann’s model54, we take the two-sublattice model both for Er and Fe spins.

The Fe3+ Hamiltonian is

$${{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}} = \mathop{\sum}\limits_{s={{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{B}}}}}}}}}\mathop{\sum }\limits_{i=1}^{{N}_{0}}{\mu }_{{{{{{{{\rm{B}}}}}}}}}{\mu }_{0}{\hat{S}}_{i}^{s}\cdot {{\mathfrak{g}}}^{{{{{{{{\rm{Fe}}}}}}}}}H+{J}_{{{{{{{{\rm{Fe}}}}}}}}}\mathop{\sum}\limits_{{{{{{{{\rm{n.n.}}}}}}}}}{\hat{S}}_{i}^{{{{{{{{\rm{A}}}}}}}}}\cdot {\hat{S}}_{{i}^{{\prime} }}^{{{{{{{{\rm{B}}}}}}}}}\\ -{D}_{y}^{{{{{{{{\rm{Fe}}}}}}}}}\mathop{\sum}\limits_{{{{{{{{\rm{n.n.}}}}}}}}}\left({\hat{S}}_{i,z}^{{{{{{{{\rm{A}}}}}}}}}{\hat{S}}_{{i}^{{\prime} },x}^{{{{{{{{\rm{B}}}}}}}}}-{\hat{S}}_{i,z}^{{{{{{{{\rm{B}}}}}}}}}{\hat{S}}_{{i}^{{\prime} },x}^{{{{{{{{\rm{A}}}}}}}}}\right)\\ -\mathop{\sum }\limits_{i=1}^{{N}_{0}}\left({A}_{x}{\hat{S}}_{i,x}^{{{{{{{{\rm{A}}}}}}}}\,2}+{A}_{z}{\hat{S}}_{i,z}^{{{{{{{{\rm{A}}}}}}}}\,2}+{A}_{xz}{\hat{S}}_{i,x}^{{{{{{{{\rm{A}}}}}}}}\quad }{\hat{S}}_{i,z}^{{{{{{{{\rm{A}}}}}}}}}\right)\\ -\mathop{\sum }\limits_{i=1}^{{N}_{0}}\left({A}_{x}{\hat{S}}_{i,x}^{{{{{{{{\rm{B}}}}}}}}\,2}+{A}_{z}{\hat{S}}_{i,z}^{{{{{{{{\rm{B}}}}}}}}\,2}-{A}_{xz}{\hat{S}}_{i,x}^{{{{{{{{\rm{B}}}}}}}}\quad }{\hat{S}}_{i,z}^{{{{{{{{\rm{B}}}}}}}}}\right).$$

\({\hat{S}}_{i}^{s}\) is the Fe3+ spin operator with S = 5/2 at the i-th site and s sublattice. JFe, \({D}_{y}^{{{{{{{{\rm{Fe}}}}}}}}}\), Ax(z, xz) are a Fe3+ isotropic exchange constant, Dzyaloshinskii-Moriya interaction for the y(b)-axis, and the single ion anisotropy for the x(z, xz) spin components. ∑n.n. is a summation for the nearest neighbors and the number of nearest neighbors is zFe = 6. N0 is the number of unit cells in ErFeO3. The g-factor of Fe3+ is

$${{\mathfrak{g}}}^{{{{{{{{\rm{Fe}}}}}}}}}=\left(\begin{array}{ccc}{{\mathfrak{g}}}_{x}^{{{{{{{{\rm{Fe}}}}}}}}}&0&0\\ 0&{{\mathfrak{g}}}_{y}^{{{{{{{{\rm{Fe}}}}}}}}}&0\\ 0&0&{{\mathfrak{g}}}_{z}^{{{{{{{{\rm{Fe}}}}}}}}}\\ \end{array}\right).$$

μB is the Bohr magneton and H is an external magnetic field.

The Er3+ Hamiltonian is

$${{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}}=\mathop{\sum}\limits_{s={{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{B}}}}}}}}}\mathop{\sum }\limits_{i=1}^{{N}_{0}}\frac{{\mu }_{0}}{2}{\hat{\sigma }}_{i}^{{{{{{{{\rm{s}}}}}}}}}\cdot {{\mathfrak{g}}}^{{{{{{{{\rm{Er}}}}}}}}}\cdot H+{J}_{{{{{{{{\rm{Er}}}}}}}}}\mathop{\sum}\limits_{{{{{{{{\rm{n.n.}}}}}}}}}{\hat{\sigma }}_{i}^{{{{{{{{\rm{A}}}}}}}}}\cdot {\hat{\sigma }}_{{i}^{{\prime} }}^{{{{{{{{\rm{B}}}}}}}}}.$$

\({\hat{\sigma }}_{i}^{{{{{{{{\rm{s}}}}}}}}}\) is the Er3+ spin operator at the i-th site and s sublattice. JEr represents Er3+–Er3+ isotorpic exchange constant. The g-factor of Er3+ is

$${{\mathfrak{g}}}^{{{{{{{{\rm{Er}}}}}}}}}=\left(\begin{array}{ccc}{{\mathfrak{g}}}_{x}&0&0\\ 0&{{\mathfrak{g}}}_{y}&0\\ 0&0&{{\mathfrak{g}}}_{z}\\ \end{array}\right).$$

Finally, the Fe3+–Er3+ interaction Hamiltonian is

$${{{{{{{{\mathcal{H}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}-{{{{{{{\rm{Fe}}}}}}}}}=\mathop{\sum }\limits_{i=1}^{{N}_{0}}\mathop{\sum}\limits_{s,{s}^{{\prime} }={{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{B}}}}}}}}}\left[J{\hat{\sigma }}_{i}^{{{{{{{{\rm{s}}}}}}}}}\cdot {\hat{S}}^{{s}^{{\prime} }}+{D}^{s,{s}^{{\prime} }}\cdot \left({\hat{\sigma }}_{i}^{{{{{{{{\rm{s}}}}}}}}}\times {\hat{S}}^{{s}^{{\prime} }}\right)\right].$$

In our Fe3+–Er3+ interaction Hamiltonian, Fe3+ and Er3+ interact within the same unit cell. J is a Fe3+–Er3+ isotropic exchange constant. \({D}^{s,{s}^{{\prime} }}\) is the Dzyaloshinskii-Moriya interaction and

$$ {D}^{{{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{A}}}}}}}}} = {\left({D}_{x},{D}_{y},0\right)}^{\!\!t},\\ {D}^{{{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{B}}}}}}}}} = {\left(-{D}_{x},-{D}_{y},0\right)}^{\!\!t},\\ {D}^{{{{{{{{\rm{B}}}}}}}},{{{{{{{\rm{A}}}}}}}}} = {\left(-{D}_{x},{D}_{y},0\right)}^{\!\!t},\\ {D}^{{{{{{{{\rm{B}}}}}}}},{{{{{{{\rm{B}}}}}}}}} = {\left({D}_{x},-{D}_{y},0\right)}^{\!\!t}.$$

The state of spins are determined by the equations of motion in which we assume each individual spin experiences a uniform mean field supplied by its surrounding magnetic ions. Assuming the dynamics within all unit cells are identical, we can replace spins dependent on unit cells, \({S}_{i}^{s}\) and \({\sigma }_{i}^{s}\), with uniform spins, Ss and σs keeping the sublattice dependence. From the Heisenberg equation from the Hamiltonian (Eq. (4)), the equations of motion are

$$\hslash \frac{\partial }{\partial t}{\sigma }^{s}=-{\sigma }^{s}\times {\mu }_{0}{\mathfrak{g}}{\mu }_{B}{H}_{{{{{{{{\rm{Er}}}}}}}}}^{s}(\{{\sigma }^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\},\{{S}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\}),$$
(5)
$$\hslash \frac{\partial }{\partial t}{S}^{s}=-{S}^{s}\times {\mu }_{0}{\mathfrak{g}}{\mu }_{B}{H}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}(\{{\sigma }^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\},\{{S}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\}).$$
(6)

\({\mathfrak{g}}\) is the free electron g-factor. \({H}_{{{{{{{{\rm{Er}}}}}}}}}^{s}\) and \({H}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}\) are the mean-fields for Er3+ and Fe3+ spins, respectively. The mean-fields are defined as derivatives of the total Hamiltonian (Eq. (4)) with respect to the corresponding spin variables, for example \(g{\mu }_{0}{\mu }_{{{{{{{{\rm{B}}}}}}}}}{H}_{{{{{{{{\rm{Er}}}}}}}}}^{s}=2\partial {{{{{{{\mathcal{H}}}}}}}}/\partial {\sigma }^{s}\). The phase diagrams shown in Figs. 2 and 3 are calculated by the following procedure.

From the equations of motion, the equilibrium spins (\({\bar{\sigma }}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\) and \({\bar{S}}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\)) are parallel to the mean-fields \({\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}={H}_{{{{{{{{\rm{Er}}}}}}}}}^{s}(\{{\bar{\sigma }}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\},\{{\bar{S}}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\})\) and \({\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}={H}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}(\{{\bar{\sigma }}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\},\{{\bar{S}}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\})\). They are connected as

$${\bar{\sigma }}^{s}=\langle {\hat{\sigma }}^{s}\rangle =\langle {\hat{\sigma }}_{\parallel }^{s}\rangle {u}_{{{{{{{{\rm{Er}}}}}}}}}^{s},\quad {\hat{\sigma }}_{\parallel }^{s}={\hat{\sigma }}^{s}\cdot {u}_{{{{{{{{\rm{Er}}}}}}}}}^{s},\quad {u}_{{{{{{{{\rm{Er}}}}}}}}}^{s}={\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}/| {\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}| ,$$
(7)
$${\bar{S}}^{s}=\langle {\hat{S}}^{s}\rangle =\langle {\hat{S}}_{\parallel }^{s}\rangle {u}_{{{{{{{{\rm{Fe}}}}}}}}}^{s},\quad {\hat{S}}_{\parallel }^{s}={\hat{S}}^{s}\cdot {u}_{{{{{{{{\rm{Fe}}}}}}}}}^{s},\quad {u}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}={\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}/| {\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}| .$$
(8)

We determine the equilibrium spins (\({\bar{\sigma }}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\) and \({\bar{S}}^{{{{{{{{\rm{A}}}}}}}}/{{{{{{{\rm{B}}}}}}}}}\)) in the following self-consistent equations

$$\langle {\hat{\sigma }}_{\parallel }^{s}\rangle =-\frac{\partial }{\partial {y}_{s}}\ln {Z}_{{{{{{{{\rm{Er}}}}}}}}}^{s}=-\tanh ({y}_{s}),$$
(9)
$$\langle {\hat{S}}_{\parallel }^{s}\rangle =-\frac{\partial }{\partial {x}_{s}}\ln {Z}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}=-S{B}_{S}(S{x}_{s}),$$
(10)

BJ(z) is the Brillouin function

$${B}_{J}(z)=\frac{2J+1}{2J}\coth \left(\frac{2J+1}{2J}z\right)-\frac{1}{2J}\coth \left(\frac{z}{2J}\right),$$

and the partition functions are

$${Z}_{{{{{{{{\rm{Er}}}}}}}}}^{s}\equiv {{{{{{{\rm{Tr}}}}}}}}\left[{{{{{{{{\rm{e}}}}}}}}}^{-{\hat{{{{{{{{\mathcal{H}}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}/{k}_{B}T}\right]=\mathop{\sum}\limits_{m=\pm 1}{{{{{{{{\rm{e}}}}}}}}}^{-m{y}_{\!s}}=2\cosh ({y}_{s}),$$
(11)
$${Z}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}\equiv {{{{{{{\rm{Tr}}}}}}}}\left[{{{{{{{{\rm{e}}}}}}}}}^{-{\hat{{{{{{{{\mathcal{H}}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}/{k}_{B}T}\right]=\mathop{\sum }\limits_{m=-S}^{S}{{{{{{{{\rm{e}}}}}}}}}^{-m{x}_{s}}=\frac{\sinh [(S+1/2){x}_{s}]}{\sinh ({x}_{s}/2)},$$
(12)
$${y}_{s}=\frac{{\mathfrak{g}}{\mu }_{0}{\mu }_{B}| {\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}| }{2{k}_{B}T},\quad {x}_{s}=\frac{{\mathfrak{g}}{\mu }_{0}{\mu }_{B}| {\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}| }{{k}_{B}T}.$$

kB is Boltzmann constant and T is temperature. Finally, the effective Hamiltonians of given mean-fields \({\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}\) and \({\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}\) are

$$ {\hat{{{{{{{{\mathcal{H}}}}}}}}}}_{{{{{{{{\rm{Er}}}}}}}}}^{s} = \frac{1}{2}{\mathfrak{g}}{\mu }_{0}{\mu }_{B}{\hat{\sigma }}^{s}\cdot {\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}=\frac{1}{2}{\mathfrak{g}}{\mu }_{0}{\mu }_{B}{\hat{\sigma }}_{\parallel }^{s}| {\bar{H}}_{{{{{{{{\rm{Er}}}}}}}}}^{s}| ,\\ {\hat{{{{{{{{\mathcal{H}}}}}}}}}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s} = {\mathfrak{g}}{\mu }_{0}{\mu }_{B}{\hat{S}}^{s}\cdot {\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}={\mathfrak{g}}{\mu }_{0}{\mu }_{B}{\hat{S}}_{\parallel }^{s}| {\bar{H}}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}| .$$

Here, \({\hat{\sigma }}^{s}\) and \({\hat{S}}^{s}\) are vectors of the Pauli operators and angular momentum with the magnitude S.

To determine the ground state, we calculate free energy from the partition functions, Eqs. (11) and (12), and pick the configuration with the lowest energy. The free energies are defined as

$${F}_{{{{{{{{\rm{Er}}}}}}}}}^{s} = -{k}_{B}T\ln {Z}_{{{{{{{{\rm{Er}}}}}}}}}^{s},\\ {F}_{{{{{{{{\rm{Fe}}}}}}}}}^{s} = -{k}_{B}T\ln {Z}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}.$$

The total free energy of Er3+ and Fe3+ spins are

$$F=\mathop{\sum}\limits_{s={{{{{{{\rm{A}}}}}}}},{{{{{{{\rm{B}}}}}}}}}\left({F}_{{{{{{{{\rm{Er}}}}}}}}}^{s}+{F}_{{{{{{{{\rm{Fe}}}}}}}}}^{s}\right)/2.$$
(13)

Extended Dicke model

Second quantization of the spin Hamiltonian Eq. (4) has been carried out by ref. 34. We omit the derivation here since details can be found in the reference. For the sake of consistency, notation will follow those in ref. 34, but will be substituted by those used in Eq. (1) eventually. Ref. 34 expressed the total Hamiltonian as

$$\hat{{{{{{{{\mathcal{H}}}}}}}}} \approx \mathop{\sum}\limits_{K=0,\pi }\hslash {\omega }_{K}{\hat{a}}_{K}^{{{{\dagger}}} }{\hat{a}}_{K}+{\omega }_{{{{{{{{\rm{Er}}}}}}}}}{\hat{\Sigma }}_{x}^{+}+\mathop{\sum}\limits_{\xi =x,y,z}{{\mathfrak{g}}}_{\xi }^{{{{{{{{\rm{Er}}}}}}}}}{\mu }_{{{{{{{{\rm{B}}}}}}}}}{B}_{\xi }^{{{{{{{{\rm{DC}}}}}}}}}{\hat{\Sigma }}_{\xi }^{+}\\ +\frac{8{z}_{{{{{{{{\rm{Er}}}}}}}}}{J}_{{{{{{{{\rm{Er}}}}}}}}}}{N}{\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{A}}}}}}}}}\cdot {\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{B}}}}}}}}}+\frac{2\hslash {g}_{x}}{\sqrt{N}}({\hat{a}}_{\pi }^{{{{\dagger}}} }+{\hat{a}}_{\pi }){\hat{\Sigma }}_{x}^{+}\\ +\frac{{{{{{{{\rm{i}}}}}}}}2\hslash {g}_{y}}{\sqrt{N}}({\hat{a}}_{0}^{{{{\dagger}}} }-{\hat{a}}_{0}){\hat{\Sigma }}_{y}^{+}+\frac{2\hslash {g}_{y}^{{\prime} }}{\sqrt{N}}({\hat{a}}_{\pi }^{{{{\dagger}}} }+{\hat{a}}_{\pi }){\hat{\Sigma }}_{y}^{-}\\ +\frac{{{{{{{{\rm{i}}}}}}}}2\hslash {g}_{z}}{\sqrt{N}}({\hat{a}}_{\pi }^{{{{\dagger}}} }-{\hat{a}}_{\pi }){\hat{\Sigma }}_{z}^{-}+\frac{2\hslash {g}_{z}^{{\prime} }}{\sqrt{N}}({\hat{a}}_{0}^{{{{\dagger}}} }+{\hat{a}}_{0}){\hat{\Sigma }}_{z}^{+}.$$
(14)

Here, K = 0 and π corresponds to the qFM and qAFM magnon modes. \({\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{A/B}}}}}}}}}\) is a spin-\(\frac{N}{4}\) operator representing the rare-earth spins in the A/B sublattice, satisfying

$${\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{A/B}}}}}}}}}\equiv \frac{1}{2}\mathop{\sum }\limits_{i=1}^{{N}_{0}}{\hat{{{{{{{{\boldsymbol{R}}}}}}}}}}_{i}^{{{{{{{{\rm{A/B}}}}}}}}},$$
(15)

where \((1/2){\hat{{{{{{{{\boldsymbol{R}}}}}}}}}}_{i}^{{{{{{{{\rm{A/B}}}}}}}}}\) is a spin-\(\frac{1}{2}\) operator for an Er3+ ion. We also define the sum and difference of the two sublattice spins as

$${\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{\pm }\equiv {\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{A}}}}}}}}}\pm {\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{B}}}}}}}}}.$$
(16)

The total number of spin-\(\frac{1}{2}\) spins (Er3+ spins) in the two sublattices is

$$N\equiv 2{N}_{0}.$$
(17)

The five Er3+–magnon coupling terms were rewritten in terms of the annihilation (creation) operators \({\hat{a}}_{K}\) (\({\hat{a}}_{K}^{{{{\dagger}}} }\)) of a magnon, with their respective coupling strengths defined as

$$\hslash {g}_{x} = \sqrt{2S}(J\cos {\beta }_{0}-{D}_{y}\sin {\beta }_{0}){\left(\frac{b+a}{d-c}\right)}^{\!1/4}\\ = h\times 0.051\ {{{{{{{\rm{THz}}}}}}}},$$
(18a)
$$\hslash {g}_{y} = \sqrt{2S}J{\left(\frac{d+c}{b-a}\right)}^{1/4}\\ = h\times 0.041\ {{{{{{{\rm{THz}}}}}}}},$$
(18b)
$$\hslash {g}_{y}^{{\prime} } = \sqrt{2S}({D}_{x}\sin {\beta }_{0}){\left(\frac{b+a}{d-c}\right)}^{1/4}\\ = h\times 3.1\times 1{0}^{-5}\ {{{{{{{\rm{THz}}}}}}}},$$
(18c)
$$\hslash {g}_{z} = \sqrt{2S}{D}_{x}{\left(\frac{d-c}{b+a}\right)}^{1/4}\\ = h\times 0.116\ {{{{{{{\rm{THz}}}}}}}},$$
(18d)
$$\hslash {g}_{z}^{{\prime} } = \sqrt{2S}(-J\sin {\beta }_{0}-{D}_{y}\cos {\beta }_{0}){\left(\frac{b-a}{d+c}\right)}^{1/4}\\ = h\times (-0.040\ {{{{{{{\rm{THz}}}}}}}}).$$
(18e)

The numerical values of these coupling strengths are evaluated by the set of parameters, a, b, c, d, J, β0, Dx, Dy, which are defined and quantitatively given in ref. 34. We found that the ℏgz is the dominant term, for which we retain in Eq. (1) and all other Fe3+–Er3+ coupling terms are dropped as an approximation.

Regarding the Er3+–Er3+ interaction term, while the Er3+ spin ensemble is described by six operators, \({\hat{\Sigma }}_{x,y,z}^{+}\) and \({\hat{\Sigma }}_{x,y,z}^{-}\), in the extended Dicke Hamiltonian, only \({\hat{\Sigma }}_{x}^{+}\) and \({\hat{\Sigma }}_{z}^{-}\) are relevant to the low-temperature phase transition. \({\hat{\Sigma }}_{x}^{+}\) corresponds to the paramagnetic alignment by the Fe3+ magnetization along the a axis, and \({\hat{\Sigma }}_{z}^{-}\) corresponds to the antiferromagnetic ordering along the c axis. Then, for analyzing the thermal-equilibrium values of the spins, we need to consider only the following terms in the Er3+–Er3+ exchange interactions:

$$ \frac{8{z}_{{{{{{{{\rm{Er}}}}}}}}}{J}_{{{{{{{{\rm{Er}}}}}}}}}}{N}{\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{A}}}}}}}}}\cdot {\hat{{{{{{{{\boldsymbol{\Sigma }}}}}}}}}}^{{{{{{{{\rm{B}}}}}}}}} = \frac{2{z}_{{{{{{{{\rm{Er}}}}}}}}}{J}_{{{{{{{{\rm{Er}}}}}}}}}}{N}\mathop{\sum}\limits_{\xi =x,y,z}\left[{\left({\hat{\Sigma }}_{\xi }^{+}\right)}^{2}-{\left({\hat{\Sigma }}_{\xi }^{-}\right)}^{2}\right]\\ \to \frac{2{z}_{{{{{{{{\rm{Er}}}}}}}}}{J}_{{{{{{{{\rm{Er}}}}}}}}}}{N}\left[{\left({\hat{\Sigma }}_{x}^{+}\right)}^{2}+{\left({\hat{\Sigma }}_{z}^{+}\right)}^{2}-{\left({\hat{\Sigma }}_{x}^{-}\right)}^{2}-{\left({\hat{\Sigma }}_{z}^{-}\right)}^{2}\right].$$
(19)

After this substitution, a notation substitution of gz → g and JEr → J, and incorporating the aforementioned simplification about the coupling terms, Eq. (14) becomes Eq. (1). Numerical values of the material parameters therein are:

$$J=0.037\,{{{{{{{\rm{meV}}}}}}}}$$
(20)
$$g=0.48\,{{{{{{{\rm{meV}}}}}}}}$$
(21)
$${\omega }_{\pi }=2\pi \times 0.896\,{{{{{{{\rm{THz}}}}}}}}$$
(22)
$${\omega }_{{{{{{{{\rm{Er}}}}}}}}}=2\pi \times 0.023\,{{{{{{{\rm{THz}}}}}}}}.$$
(23)

Sample preparation

Polycrystalline ErFeO3 was first synthesized by a conventional solid state reaction method using Er2O3 (99.9%) and Fe2O3 (99.98%) powders. According to the stoichiometric ratio, the original reagents were weighted carefully and pulverized with moderate anhydrous ethanol in an agate mortar. Mixtures were sintered at 1300 ∘C for 1000 min and then cooled down to room temperature. The sintered powders were thoroughly reground and pressed into a rod that is 70 mm in length and 5–6 mm in diameter by a Hydrostatic Press System (Riken Seiki CO. Ltd, model HP-M-SD-200) at 70 MPa, and then sintered again at 1300 ∘C for sufficient reaction. Single crystal samples were then grown by an optical floating zone furnace (FZT-10000-H-VI-P-SH, Crystal Systems Corp; heat source: four 1 kW halogen lamps). The polycrystalline samples were melted in an airflow. Conditions like the melting power and the rate of sample rotation were stabilized and controlled in the molten zone.

Magnetization measurements

We measured iso-field (Supplementary Fig. 1a, left axis, blue circles) and isothermal (Supplementary Fig. 1b, left axis, blue circles) magnetization M to determine the phase transitions in the T–H plane of ErFeO3 for magnetic field H∥c. The transition from the N → S states is marked by the maximum in the derivative d(MT)/dT55 (Supplementary Fig. 1a right axis, red squares). The transition from the S → A states in the isothermal M measured at T = 1.5 K (Supplementary Fig. 1b, left axis, blue circles) is clearly marked by the maximum in dM/dH (Supplementary Fig. 1b right axis, red triangles). The transition from the A → N states is much less obvious, marked by a small kink in dM/dH (Supplementary Fig. 1b right axis, red triangles) marked by the gray dashed lane. This may be due to demagnetization effects due to the irregularity of the sample shape we measured. Nonetheless, we are able to use MCE measurements that clearly exhibit signatures of the A → N transition. It is noted that transitions shown here from magnetization measurements are consistent within error bars with the T–H phase diagram presented in Fig. 4a.

THz magnetospectroscopy

We performed time-domain THz transmission magnetospectroscopy measurements in the Faraday geometry. The sample is placed in a liquid-helium-cooled magneto-optical cryostat (Oxford Instruments Spectromag-10T) with variable temperatures T between 1.4 and 300 K and static magnetic fields μ0H up to 10 T. We generate THz pulses via optical rectification using a Ti:sapphire regenerative amplifier (775 nm, 0.7 mJ, 150 fs, 1 kHz, Clark-MXR, Inc., CPA2001) as a laser source that pumps a (110) zinc telluride (ZnTe) crystal, while detection is accomplished through electro-optical sampling in another ZnTe crystal.

Index of refraction and absorption coefficient

In this section we derive the standard equations used in the extraction of the complex index of refraction of a sample using THz-TDS. Let \({\tilde{E}}_{0}(\omega )\) be the Fourier transform of an incoming THz pulse E0(t) incident on two linear media surrounding a homogeneous dielectric slab of thickness d (the sample). We assume that trailing pulses due to multiple reflections within the sample (the Fabry-Pérot effect) are well separated in time from the main transmitted THz pulse and that the incidence is normal to the sample surfaces (assumed parallel and flat). Experimentally, two separate measurements are consecutively carried out. First, the THz electric field transmitting without a sample in place is measured and the reference electric field \({\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(\omega )\) is obtained. Second, both the sample and its surroundings is measured and \({\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(\omega )\) is extracted. Under these assumptions, each transmitted electric field can be written as56,57:

$${\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(\omega )={\tilde{t}}_{13}(\omega ){\tilde{P}}_{{{{{{{{\rm{vac}}}}}}}}}(\omega ,d){\tilde{E}}_{0}(\omega )$$
(24)
$${\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(\omega )={\tilde{t}}_{12}(\omega ){\tilde{P}}_{{{{{{{{\rm{s}}}}}}}}}(\omega ,d){\tilde{t}}_{23}{\tilde{E}}_{0}(\omega )$$
(25)

where \({\tilde{t}}_{jk}=\frac{2{\tilde{n}}_{j}}{{\tilde{n}}_{j}+{\tilde{n}}_{k}}\) is the complex Fresnel transmission coefficient between mediums j and k, \({\tilde{P}}_{j}(\omega ,{d}_{j})={e}^{i{k}_{0}{d}_{j}{\tilde{n}}_{j}}={e}^{i(\omega {d}_{j}/c){\tilde{n}}_{j}}\) is the propagator through medium j, and the subscripts vac, r, and s refer to vacuum, reference, and sample, respectively. The ratio between \({\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(\omega )\) and \({\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(\omega )\) is the transfer function \(\tilde{H}(\omega )\), and it follows from Eqs. (24) and (25) that:

$$\tilde{H}(\omega )=\frac{{\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(\omega )}{{\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(\omega )}=\frac{{\tilde{t}}_{12}{\tilde{t}}_{23}}{{\tilde{t}}_{13}}\frac{{\tilde{P}}_{{{{{{{{\rm{s}}}}}}}}}(\omega ,d)}{{\tilde{P}}_{{{{{{{{\rm{vac}}}}}}}}}(\omega ,d)}=\frac{2{\tilde{n}}_{2}({\tilde{n}}_{1}+{\tilde{n}}_{3})}{({\tilde{n}}_{1}+{\tilde{n}}_{2})({\tilde{n}}_{2}+{\tilde{n}}_{3})}{e}^{i(\omega d/c)({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}-1)}$$
(26)

The bulk samples characterized in this work are single crystals grown without a substrate, and therefore, the surrounding mediums can be taken as vacuum by setting \({\tilde{n}}_{1}={\tilde{n}}_{3}=1\) in Eq. (26). With this simplification, the coefficient in front of the exponential becomes \(\frac{4{\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}}{{({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}+1)}^{2}}\), where we have change the subscript \({\tilde{n}}_{2}\) to \({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}\) for convenience. Furthermore, we can set \({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}={n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )\) for \({\tilde{t}}_{jk}\) and solve Eq. (26) for \({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}={n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )+i{\kappa }_{{{{{{{{\rm{s}}}}}}}}}(\omega )\) in the exponential. Here, ns(ω) is the index of refraction of the sample, and κs(ω) its extinction coefficient. This approximation is justified by the fact that the sample absorption is negligible in the Fresnel transmission coefficient compared to the exponential term and is thus used in the data analysis here described. We obtain:

$$\tilde{H}(\omega )=\frac{4{n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )}{{({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )+1)}^{2}}{e}^{i(\omega d/c)({\tilde{n}}_{{{{{{{{\rm{s}}}}}}}}}-1)}=\frac{4{n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )}{{({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )+1)}^{2}}{e}^{i(\omega d/c)({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )-1)}{e}^{-(\omega d/c){\kappa }_{{{{{{{{\rm{s}}}}}}}}}(\omega )}$$
(27)

Taking the modulus and phase of Eq. (27) leads to:

$$\Phi [\tilde{H}(\omega )]=\left(\frac{\omega d}{c}\right)({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )-1)\to {n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )=1+\frac{c}{\omega d}\Phi [\tilde{H}(\omega )]$$
(28)
$$| \tilde{H}(\omega )| =\frac{4{n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )}{{({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )+1)}^{2}}{e}^{-(\omega d/c){\kappa }_{{{{{{{{\rm{s}}}}}}}}}(\omega )}\to {\kappa }_{{{{{{{{\rm{s}}}}}}}}}(\omega )=-\frac{c}{\omega d}\ln \left[\frac{{({n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )+1)}^{2}}{4{n}_{{{{{{{{\rm{s}}}}}}}}}(\omega )}| \tilde{H}(\omega )| \right]$$
(29)

We can also re-write this result in terms of the absorption coefficient α(ω) of the sample as:

$$n(\omega )=1+\frac{c}{\omega d}\Phi [\tilde{H}(\omega )]$$
(30)
$$\alpha (\omega )=\frac{2\omega }{c}\kappa (\omega )=-\frac{2}{d}\ln \left[\frac{{(n(\omega )+1)}^{2}}{4n(\omega )}| \tilde{H}(\omega )| \right]$$
(31)

where we have dropped the subscript s for convenience. In conclusion, by Fourier transforming \({\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(t)\) and \({\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(t)\), which are obtained experimentally, the transfer function \(\tilde{H}(\omega )\) can be calculated as \({\tilde{E}}_{{{{{{{{\rm{s}}}}}}}}}(\omega )/{\tilde{E}}_{{{{{{{{\rm{r}}}}}}}}}(\omega )\), and n(ω) and α(ω) follow from Eqs. (30) and (31), respectively.