Controlling the magnetic state of the proximate quantum spin liquid α -RuCl 3 with an optical cavity

,


INTRODUCTION
2][3] Due to a competition of strong anisotropy, fluctuations, and spin-orbit effects, vdW materials are prime candidates to host exotic phenomena such as topological phase transitions, magnetic skyrmions and quantum spin liquids. 4,5In addition, the electronic, magnetic and optical properties of these materials are sensitive to a wide range of material engineering techniques such as strain, 6,7 nanostructuring, 8 electric fields 9,10 and moiré twisting, 11,12 allowing their state to be tuned with high precision.
4][15][16] However, driving a system with lasers is associated with excessive heating when the frequency becomes multi-photon resonant with electronic transitions. 17,180][21][22][23][24][25] Due to the strong interaction between light and charged excitations polaritonic control of material and chemical properties has, with a few exceptions, [26][27][28][29] mainly been considered for electronic states. 30,31While currently efforts are made to extend the cavity framework to a broader class of materials, and to construct a unified first principles description of cavity quantum fluctuations and quantum matter, 19,21,32 experiments demonstrating polaritonic control of materials are scarce. 33,34Therefore, to transform this promising approach into a powerful experimental tool, it is of key importance to identify candidate materials where cavity engineering techniques can be explored.
Here, we extend the concept of cavity quantum electrodynamics (c-QED) engineering into the magnetic regime and identify such a candidate system, by demonstrating how an optical cavity can be used to control the magnetic ground state of the proximate quantum spin liquid α-RuCl 3 via shaping the quantum fluctuations of the cavity. 19,21,31,35Depending on the cavity frequency, photon occupation, and the strength of the effective lightmatter coupling, we find that it is possible to transform the equilibrium zigzag antiferromagnetic order into any of the magnetic phases supported by the extended Kitaev model (see Eq. 1 and Fig. 2).As a key result we find that for frequencies of a few THz and for moderate light-matter couplings, the interaction between the magnetic system and the vacuum fluctuations of the cavity is sufficient to transform α-RuCl 3 from a zigzag antiferromagnet to a ferromagnet.In contrast to the meta-stable states obtained by driving the system with classical light, the magnetic state resulting from the interaction with the quantum fluctuations of the cavity is a true equilibrium state denoted the photo ground state (PGS). 22Pumping the cavity in the few photon regime, it is further possible to push the system into the Kitaev quantum spin liquid state and to retrieve the non-equilibrium phase diagram of the semi-classical limit. 36Our results pave the way for utilizing c-QED to induce and control long-lived exotic states in quantum materials.Experimentally α-RuCl 3 is found to exhibit a zigzag antiferromagnetic order below the Néel temperature T N ≈ 7 K, as indicated in Fig. 1.This zigzag state is energetically very close to a ferromagnetic state [23, 24], and might only be stabilized by spin quantum fluctuations [25].In addition, signatures of a Kitaev quantum spin liquid (QSL) state have been found upon applying an external magnetic field along the out-of-plane direction [6, 26, 27].Together these results indicate that although α-RuCl 3 orders at low temperatures, its ground state is adjacent to several competing magnetic states and the magnetic phase diagram is determined by a delicate competition of magnetic interactions.This makes α-RuCl 3 an excellent candidate to explore material engineering techniques based on c-QED.
The system will in the following be assumed to have C 3 symmetry, which is satified to a very good degree in α-RuCl 3 .All parameters of the local and kinetic Hamiltonians H U and H t have been calculated from first principles, as discussed in the Methods section.

Light-matter coupling in a cavity
Within the low-energy model described above the main effect of a cavity field is to modify the hopping amplitudes of the Ru holes.Inside the cavity the total Hamiltonian is given by H = H U + H t + λ ℏΩ λ nλ , where ℏΩ λ is the energy of photon mode λ and nλ is the corresponding number operator.The photon field modifies the Hamiltonian H t by introducing the replacements ĉ † iασ ĉiβσ → e iϕij ĉ † iασ ĉiβσ , where the Peierls phases are ϕ ij = (ea/ℏ)d ij • Â.Here d ij is the vector between atomic sites i and j, measured in units of the Ru-Ru distance a, and the quantum vector potential is Â = λ (g λ e λ â † λ + g * λ e * λ âλ ).For a given mode λ the light-matter coupling is defined as where e is the elementary charge, ℏ the reduced Planck constant, ϵ the relative permittivity and V the cavity mode volume.For a two-dimensional Fabry-Pérot cavity the photon mode with the lowest energy has a frequency Ω 0 = πc/L z .Therefore the light-matter coupling of this mode is independent of the photon frequency, and assuming L x = L y = N a with N the number of unit cells along a linear dimension of the system, g 0 = e/ √ 2πϵℏcN 2 ≈ 0.12/N .

Effective coupled spin-photon model
The total Hamiltonian H is down-folded to the spin sector by eliminating H t to fourth order in t/U using quasi-degenerate perturbation theory [28].The structure of the perturbation expansion allows the down-folding to be performed separately within each given photon sector, resulting in a coupled spin-photon Hamiltonian of the form Here H s,nm is the spin Hamiltonian in the sector connecting photon number states |n⟩ and |m⟩, where m = {m 1 , m 2 , . . ., m L } for L modes.The Hamiltonian in each photon sector is where the magnetic parameters J, K and Γ all depend on the light-matter coupling g = {g 1 , g 2 , . . ., g L } and the photon numbers n and m.Each bond ⟨ij⟩ is labeled by

Low-energy model and equilibrium magnetic order
The vdW material α-RuCl 3 consists of layers of Ru atoms arranged in an hexagonal lattice and surrounded by edge-sharing octahedra of Cl ions (Fig. 1a).Due to the crystal field the Ru d-orbitals are split by an energy ∆ cf into a lower e g and a higher t 2g manifold, and the strong spin-orbit coupling further splits the t 2g states by an energy ∆ soc into a lower j eff = 3/2 quartet, fully occupied in the ground state, and a j eff = 1/2 doublet with a single hole.The Cl ions are assumed to be completely filled in the ground state, with their p-orbitals separated from the Ru d-orbitals by a charge-transfer energy ∆ pd .The Ru energy level structure is schematically shown in Fig. 1c.The local interactions of the t 2g manifold are described by a Hubbard-Kanamori Hamiltonian H U , 36,37 which takes into account the intra-orbital Hubbard interaction U , the inter-orbital interaction U , the Hund's coupling J H and the spin-orbit coupling λ (see Methods for a discussion of the model).The Ru holes can either hop directly between Ru atoms or move via the Cl ligands, as described by the kinetic Hamiltonian H t .In the strong coupling limit t/U 1 virtual hopping processes give rise to effective magnetic interactions via the superexchange mechanism.Due to the orbital alignment a Kitaev interaction arises from ligand-mediated hopping over 90 • bond angles, 38 while sub-dominant exchange and anisotropy terms arise from direct Ru-Ru interactions.
Experimentally α-RuCl 3 is found to exhibit a zigzag antiferromagnetic order below the Néel temperature T N ≈ 7 K, as indicated in Fig. 1.At zero temperature, first principles calculations show that this zigzag state is approximately degenerate with a ferromagnetic state, 39 and might only be stabilized by spin quantum fluctuations. 40In addition, signatures of a Kitaev quantum spin liquid (QSL) state have been found upon applying an external magnetic field along the out-of-plane direction. 41,42Together these results indicate that although α-RuCl 3 orders at low temperatures, its ground state is proximate to several competing magnetic orders and the magnetic phase diagram is determined by a delicate competition of different magnetic interactions.This makes α-RuCl 3 an excellent candidate material to explore the competition between cavity and spin quantum fluctuations.
In the following the material will be assumed to have C 3 symmetry, which is satisfied to a very good degree in α-RuCl 3 .All parameters of the local and kinetic Hamiltonians H U and H t were calculated from first principles as discussed in the Methods section, and give a zigzag antiferromagnetic ground state in line with observations.

Light-matter coupling in a cavity
Within the low-energy description discussed above, the main effect of the cavity is to modify the hopping amplitudes of the Ru holes.Inside the cavity the total Hamiltonian is given by H = H U + H t + λ Ω λ nλ , where Ω λ is the energy of photon mode λ and nλ is the corresponding number operator.The photon field modifies the kinetic Hamiltonian H t by introducing the replacements ĉ † iασ ĉiβσ → e iφij ĉ † iασ ĉiβσ , where the Peierls phases φ ij are proportional to the quantum vector potential is Â = λ (g λ e λ â † λ + g * λ e * λ âλ ).For a given mode λ the bare light-matter coupling is defined as , where e is the elementary charge, the reduced Planck constant, the relative permittivity and V the cavity mode volume.For a two-dimensional Fabry-Pérot cavity the lowest energy photon mode has a frequency Ω 0 = πc/L z , and assuming a cavity with linear dimensions L x = L y = La where L is width of the cavity in units of the Ru-Ru distance a, the bare light-matter coupling of this mode is g 0 = e/ √ 2π cL 2 ≈ 0.12/L.
However, below we consider the effective single mode approximation obtained by integrating over photon modes with momenta q ≈ 0, where the effective light-matter coupling g is determined by an effective mode volume V eff . 21,22,43,44Although the value of V eff can in principle be fixed by comparisons to experiment, we here take it as a free parameter.

Effective coupled spin-photon model
The total Hamiltonian is down-folded to the spin sector by eliminating H t to fourth order in virtual ligand mediated processes using quasi-degenerate perturbation theory.The structure of the perturbation expansion allows for the down-folding to be performed separately within each given photon sector, resulting in a coupled spinphoton Hamiltonian of the form (1) Here H s,nm is the spin Hamiltonian in the sector connecting cavity number states |n and |m , where m = {m 1 , m 2 , . . ., m L } for L modes.The Hamiltonian in each photon sector is where the magnetic interactions J, K, Γ and Γ all depend on the light-matter coupling g = {g 1 , g 2 , . . ., g L } and the photon numbers n and m.Each bond ij is labeled by the indexes αβ(γ) ∈ {xy(z), yz(x), zx(y)} and is denoted a γ-bond in accordance with Fig. 1d.The induced magnetic field B points along the [111] direction of the local spin axes, as defined by the unit vector êB .
The form of the spin-photon Hamiltonian in Eq. 2 is valid as long as the C 3 symmetry is preserved.Since a single linearly polarized mode breaks rotational symmetry, such a mode induces additional terms in the spin Hamiltonian.The rotational symmetry breaking is also reflected in a strong dependence of the cavity-modified spin parameters on the polarization direction, as shown in Supplemental Figure S1.However, in a Fabry-Pérot cavity the lowest mode is doubly degenerate with two possible in-plane polarizations, and including both these modes in the description restores the rotational symmetry of H.The C 3 symmetry can also be maintained by considering a single circularly polarized mode in a chiral cavity, 21 as will be done in the following.

Magnetic phases of the cavity photo ground state
The PGS and magnetic phase diagram of the cavityembedded spin system was obtained by exact diagonal-ization of Eq. 1 on the 24-site spin cluster shown in Fig. 2, including a single effective circularly polarized cavity mode.This spin cluster is the minimal one known to respect all sub-lattice symmetries of the magnetic system. 45To perform the calculations we assume a perfect cavity, while a more quantitative description should account for dissipation.However, the results discussed below are expected to be robust against such effects.
To find how the magnetic ground state flows through the phase diagram of the extended Kitaev model as a function of light-matter coupling and cavity frequency (see Fig. 2b), the magnetic interactions are parameterized by J = sin θ cos φ, K = sin θ sin φ and Γ = cos θ.
Here J, K and Γ are the cavity-renormalized spin parameters divided by the energy The spin parameters for g = 0 were obtained via first principles calculations as discussed in the Methods section, and give φ = −89.43• and θ = 69.44 • placing the equilibrium system in a zigzag antiferromagnetic state adjacent to the ferromagnetic Kitaev QSL.This is in good agreement with experimental findings. 41,42To identify the magnetic state we calculate the spin-spin correlation function S z i S z j and the expectation value of the plaquette flux operator W p = 2 −6 S x i S y j S z k S x l S y m S z n .In W p the site indexes traverses a given hexagon p in the positive direction, and the spin component at a given site is determined by the bond pointing out from the hexagon.Magnetic phase transitions are identified by the second order derivatives of the total energy.Fig. 2b shows the paths traced out by the magnetic ground state as g is increased from the initial value g i = 0 to the final value g f = 0.5, for a number of photon energies in the range Ω ∈ [0, 0.2] eV.For all parameters the system flows away from the ferromagnetic Kitaev point, and depending on Ω either remain in the zigzag state or enter the ferromagnetic domain.In particular, for photon energies below Ω ≈ 10 meV, the PGS evolves from a zigzag antiferromagnetic state into a ferromagnetic state as the light-matter coupling is increased (see Figs. 2d and 2e).7][48] The increase of g is associated with an increase in photon occupation, which diverges in the limit Ω → 0 (see Fig. 2f).The reason for this divergence is a class of fourth order virtual electronic processes where a single charge excitation moves while simultaneously emitting/absorbing a virtual cavity photon (see Fig. 2g).Since the magnetic interaction parameters are inversely proportional to the energy difference between connected virtual states, this leads to a J ∼ 1/Ω enhancement in the low frequency limit.This behavior has been validated in small electronphoton clusters (see Supplementary Figure S2), which show good qualitative agreement with the larger-scale spin-photon model simulations.
The microscopic origin of the magnetic phase transition is a simultaneous suppression of the Kitaev interaction and increase of the isotropic exchange, as seen from 4 tor displayed in Supplementary Figure S3.In particular, the Kitaev interaction changes sign as a function of lightmatter coupling at a point coinciding with the magnetic phase transition.This change in parameters is consistent with a simultaneous suppression of the effective ligand mediated hopping and increase of direct Ru−Ru hopping.To see how this comes about we note that while the direct hopping process depicted in Fig. 2g implies an enhancement of J ∼ 1/Ω at small frequencies, the ligand mediated process shown in Fig. 2h is proportional to the reciprocal of the charge-transfer energy.Therefore K ∼ 1/∆ pd is largely unaffected by the low frequency limit, leading to a relative enhancement of J/K.

Magnetic phase diagram of a pumped cavity
The results of Fig. 2 show that the quantum fluctuations of the cavity vacuum are sufficient to bring about a change in the magnetic ground state of α-RuCl 3 in the THz regime.In contrast, for photon energies large compared to the magnetic energy scale (K/ Ω 1), the fluctuations of real photons are strongly suppressed.This restricts the system to a single photon sector, and the Hamiltonian is well approximated by keeping only the diagonal terms H s,nm = H s,nn δ nm .In this regime the spin and photon fluctuations therefore decouples, and the magnetic ground state can be obtained from a pure spin model with effective cavity-renormalized parameters.
The magnetic phase diagram of α-RuCl 3 , interacting with a single circularly polarized cavity mode with average photon occupation n av = 1, is shown in Fig. 3.The results are displayed as a function of the effective lightmatter coupling ḡ = 2g √ n av , and again the cavity is seen to make the magnetic ground state flow towards the antiferromagnetic Kitaev point (Fig. 3a).The cavity has a drastic effect on the spin parameters as the photon energy is tuned through the charge resonances of the underlying electronic system, as illustrated by the sharp variations of the spin-spin correlation function and plaquette flux around Ω = 2 eV (see Figs. 3c and 3d as well as Supplementary Figure S4).For a given light-matter coupling this leads to a series of magnetic phase transitions as a function of frequency, stabilizing quasi-stationary states the normalized spin parameters of the zero photon sector displayed in Supplementary Figure S3.In particular, the Kitaev interaction changes sign as a function of lightmatter coupling at a point coinciding with the magnetic phase transition.This change in parameters is consistent with a simultaneous suppression of the effective ligand mediated hopping and increase of direct Ru−Ru hopping.To see how this comes about we note that while the direct hopping process depicted in Fig. 2g implies an enhancement of J ∼ 1/Ω at small frequencies, the ligand mediated process shown in Fig. 2h is proportional to the reciprocal of the charge-transfer energy.Therefore K ∼ 1/∆ pd is largely unaffected by the low frequency limit, leading to a relative enhancement of J/K.

Magnetic phase diagram of a pumped cavity
The results of Fig. 2 show that the quantum fluctuations of the cavity vacuum are sufficient to bring about a change in the magnetic ground state of α-RuCl 3 in the THz regime.In contrast, for photon energies large compared to the magnetic energy scale (K/ Ω 1), the fluctuations of real photons are strongly suppressed.This restricts the system to a single photon sector, and the Hamiltonian is well approximated by keeping only the diagonal terms H s,nm = H s,nn δ nm .In this regime the spin and photon fluctuations therefore decouples, and the magnetic ground state can be obtained from a pure spin model with effective cavity-renormalized parameters.
The magnetic phase diagram of α-RuCl 3 , interacting with a single effective circularly polarized cavity mode with average photon occupation n av = 1, is shown in Fig. 3.The results are displayed as a function of the effective light-matter coupling ḡ = 2g √ n av , and again the cavity is seen to make the magnetic ground state flow towards the antiferromagnetic Kitaev point (Fig. 3a).The cavity has a drastic effect on the spin parameters as the photon energy is tuned through the charge resonances of the underlying electronic system, as illustrated by the sharp variations of the spin-spin correlation function and plaquette flux around Ω = 2 eV (see Figs. 3c  and 3d as well as Supplementary Figure S4).For a given light-matter coupling this leads to a series of magnetic phase transitions as a function of frequency, stabilizing quasi-stationary states with either ferromagnetic, antiferromagnetic or spiral order depending on the parameters.In addition, there is a large region of light-matter cou-4 in Fig. 2, interacting with a single circularly polarized cavity mode (see Methods).This spin cluster is the minimal one known to respect all sub-lattice symmetries of the magnetic system [44].In contrast to the meta-stable states obtained in a pumped cavity or with classical driving fields, the coupled spin-photon ground state is a true equilibrium state henceforth denoted the photo ground state (PGS).
To study how the magnetic ground state flows through the phase diagram of the extended Kitaev model as a function of light-matter coupling and cavity frequency (see Fig. 2b), the magnetic interactions are parameterized by J = sin θ cos ϕ, K = sin θ sin ϕ and Γ = cos θ where J, K and Γ are the spin parameters divided by the energy scale E = √ J 2 + K 2 + Γ 2 .The spin parameters in the decoupled limit (i.e.g = 0) were obtained from first principles calculations as discussed in the Methods section, and gives ϕ = −89.43• and θ = 69.44 • placing the equilibrium system in a zigzag antiferromagnetic state adjacent to the ferromagnetic Kitaev QSL.This is in good agreement with experimental findings.To identify the magnetic state we use the spin-spin correlation function ⟨S z i S z j ⟩ as well as the expectation value of the plaquette flux operator W p = 2 −6 S x i S y j S z k S x l S y m S z n .Here the site indexes traverse a given hexagon in the positive direction, and the spin component at a given site is determined by the bond pointing out from the hexagon.Magnetic phase transitions are identified by the second order derivatives of the total energy, ∂ 2 g E and ∂ 2 Ω E.
Fig. 2b shows the paths traced out by the magnetic ground state as g is increased from the initial value g i = 0 to the final value g f = 0.5 for a number of photon energies in the range ℏΩ ∈ [0, 0.2] eV.In all cases the system flows away from the ferromagnetic Kitaev point, and depending on Ω either remain in the zigzag state or enter the ferromagnetic region.In particular, for photon energies below ℏΩ ≈ 10 meV, the PGS evolves from a zigzag antiferromagnetic state into a ferromagnetic states as the light-matter coupling is increased (see Figs. 2d  and 2e).This evolution is associated with an increase in photon occupation, which diverges in the limit Ω → 0 (see Figs. 2f).The reason for this divergence is a class of fourth order virtual electronic processes where a single holon (or doublon) excitation moves while simultaneously emitting/absorbing a photon (see Fig. 2g).Since the magnetic interaction parameters are inversely proportional to the energy difference between participating virtual states, this leads to a behavior J ∼ 1/Ω in the low frequency limit.This behavior has been validated in mall electron-photon clusters (see Supplementary Fig- plings ḡ = 0.2 − 0.6 and photon energies Ω = 2.8 − 3.2 eV where the system enters the antiferromagnetic Kitaev QSL state.The presence of a Kitaev QSL has been verified by calculations of the spin-spin correlation function as well as the plaquette flux operator W p , which takes on a quantized value W p = ±1 in the QSL phase. 5Although the equilibrium system displays a ferromagnetic Kitaev interaction, the QSL state appears in a region of antiferromagnetic Kitaev interaction implying that the cavity induces a sign reversal of K.The QSL is found to be stabilized by the effective magnetic field induced by the cavity (Fig. 3e), as well as a relative increase of the Γ interaction.
As seen from Fig. 3, α-RuCl 3 also enters the QSL phase in a narrow region of cavity mode frequencies around Ω = 2.1 eV.This coincides with a region where the photo-modified exchange interaction J changes sign, and therefore is very small.More specifically, since the photomodified magnetic exchange is given close to a resonance by J ∼ 1/(Ω − E res ), and two resonances appear close together at about Ω = 2.0 eV and Ω = 2.2 eV, the different sign of J above the lower and below the upper resonance implies a sign reversal between the resonances.This gives a relative amplification of the Kitaev interaction K inducing the QSL state.

DISCUSSION
The results of Fig. 2 and 3 show that the magnetic ground state of α-RuCl 3 is sensitive to the interaction with the quantum fluctuations of the cavity field, and depending on the photon frequency, average cavity occupation and strength of the light-matter coupling, the equilibrium magnetic order can be transformed into any of the magnetic states supported by the extended Kitaev model of Eq. 1.In particular, our results show that the cavity vacuum fluctuations alone are indeed sufficient to change the magnetic order of the photo ground state when the light-matter coupling is sufficiently strong (but within the range of present experimental light-matter couplings).However, as the value of the light-matter coupling is strongly dependent on the cavity size, and scales as g ∼ 1/L with the linear size of the system, it seems as though the effect of the cavity vanishes quickly in the thermodynamic limit.This naive scaling argument might fail for several reasons, the most important being the neglect of additional cavity modes with momenta q ≈ 0. In fact, the number of modes with momenta smaller than some given cut-off q max scales as N (q max ) ∼ L, 22,34,49 and can potentially cancel the volume scaling of g.For the effective single mode approximation employed here, the mode volume appearing in the light-matter coupling should therefore be interpreted as an effective mode volume to be fixed either by additional calculations or by comparison to experiment. 21,22,43 additional subtlety of the thermodynamic limit is the potential coupling of the cavity modes to collective fluctuations of the magnetic system.In particular, close to a magnetic phase boundary, collective fluctuations with a correlation length ξ ∼ L are expected to appear.Such collective modes can be expected to play a crucial role in stabilizing the magnetic order, and to couple more strongly to the cavity field. 50However, an explicit treatment of such macroscopic effects, while simultaneously retaining an exact description of spin and photon quantum fluctuations, requires a more advanced methodology going beyond the current work.
For practical purposes, we note that for a given lightmatter coupling g the effective coupling ḡ can be enhanced by increasing the photon occupation of the cavity.In particular, since ḡ = 2g √ n av , a substantial effective coupling (and corresponding modification of the magnetic interactions in the spin-photon Hamiltonian of Eq. 1) can be obtained with relatively few photons.Although the present work neglects any cavity dissipation as might be relevant in a non-equilibrium setting or for a dark cavity with lossy mirrors, the qualitative conclusions are expected to be robust against such effects.This includes in particular the identification of a zigzag antiferromagnetic to ferromagnetic transition induced by the cavity vacuum fluctuations, and the presence of a large Kitaev QSL domain in the magnetic phase diagram in the few photon regime.Together these results demonstrate the feasibility of c-QED engineering of the magnetic state of the proximate spin liquid α-RuCl 3 in the few photon limit.Looking ahead, we note that the cavity photon momentum constitutes a tuning parameter in addition to the light-matter coupling, and we therefore envisage that c-QED can be used to also modify phases with finiteq ordering vectors (such as charge density waves).Our work thereby paves the way for utilizing c-QED to control exotic magnetic states in real cavity-embedded quantum materials. 19,21

Effective low-energy lattice electron Hamiltonian
The crystal structure of a single layer α-RuCl 3 consists of a hexagonal lattice of Ru atoms, surrounded by edge-sharing octahedra of Cl atoms.Due to the strong octahedral crystal field the Ru 3+ d-orbitals are split into a lower e g and a higher t 2g manifold, the latter consisting of the orbitals d xy , d xz and d yz .Due to the strong spinorbit coupling the t 2g states are further split into a lower j eff = 3/2 quartet, fully occupied in the ground state, and a j eff = 1/2 doublet with a single hole.The Cl − ions are assumed to be completely filled in the ground state, with their p x , p y and p z orbitals separated from the d-orbitals by a charge-transfer energy ∆ pd .
The local part of the Hamiltonian is given by 36,37 Here U is the intraorbital Hubbard interaction, U the interorbital interaction and J H the Hund's coupling between the Ru d-orbitals α, β ∈ {yz, xz, xy}.Further, ∆ pd is the single-particle charge transfer energy to add a hole to the Cl ligands, λ is the strength of the spinorbit coupling (SOC), and the vector of operators is ĉ † In this basis, the inner product of orbital and spin angular momentum may be written To account for hopping processes between the Ru dorbitals and Cl p-orbitals, we assume the system has C 3 symmetry.The hopping processes along a Z-bond are described by Here, t i with i ∈ {1, 2, 3, 4} parameterize direct d − d hopping processes, while t pd determines the strength of ligand mediated hopping via the Cl p z -orbitals.

Light-matter coupling
In presence of the cavity the total Hamiltonian in the dipole approximation is given by H = H U + H t + λ Ω λ nλ .Here, Ω λ is the photon energy of a mode described by the label λ and nλ is the corresponding number operator.Inside the cavity the Hamiltonian H t is modified by the replacements ĉ † iασ ĉiβσ → e iφij ĉ † iασ ĉiβσ , where the Peierls phases are φ ij = (ea/ )d ij • Â, d ij = r j − r i is the vector between atomic sites i and j measured in units of the Ru-Ru distance a, and the quantum vector potential is For a given mode λ this defines the light-matter coupling g λ = (ea/ )A λ = ea/ √ 2 0 Ω λ V .Thus, the Peierls phases can be written as and the modified hopping Hamiltonian is Hamiltonian in the photon number basis The Hamiltonian is expanded in the photon number basis |n = |n 1 , n 2 , . . ., n N according to 26,51,52 Here 1 e is the identity operator in the electronic Hilbert space, and the Hamiltonian H nm is given by The matrix elements n|e idij • Â|m are obtained by noting that since [â † λ , âλ ] = 0, the Peierls phases factorize over different modes and where Âλ = g λ e λ â † λ + g * λ e * λ âλ .
The single-mode expressions are calculated by introducing the variables η ijλ = g λ (d ij • e λ ).Using the Baker-Hausdorff formula to expand the exponential, the matrix elements are n λ |e idij • Âλ |m λ = i |n λ −m λ | j ij n λ ,m λ where Using the notation g = {g 1 , g 2 , . . ., g N } to denote the full set of light-matter couplings, the light-matter coupling is described by the function With this function, the hopping Hamiltonian in presence of the cavity can be written as + J j i nm (g)p † j σ ĉiyzσ + J jj nm (g)ĉ † jxzσ pj σ + H.c. .

Single-mode light-matter coupling and polarization dependence
For a single photon mode the index λ can be dropped, and the function J ij nm (g) = J ij nm (g) = j ij n,m .Writing the intersite vector on a given bond as d ij = d(cos θ ij x + sin θ ij ŷ), the parameter η ij is given for a circularly polarized field by η ij = g(d ij • e) = gde ±iθij , where ± denotes a left-handed/right-handed polarization.Noting that |η ij | = |η| = gd so that the spatial dependence of the function j ij n,m factorizes, we have j ij n,m = j n,m e ±i(n−m)θij with for n ≥ m and similarly but with n ↔ m for m ≥ n.
For linear polarization the polarization vector can be written as e = cos φx + sin φŷ, and we have η ij = gd cos(θ ij − φ).

Connection to Floquet Hamiltonian
As a consistency check, the Hamiltonian H t,nm for a single circularly polarized photon mode is shown to reduce to the Floquet Hamiltonian of Ref. 36 in the semiclassical limit.For a single photon mode, we can drop the mode index, and the function J ij nm (g) = J ij nm (g) = j ij n,m .Taking the polarization vector of the photon mode to be e = (x ± iŷ)/ √ 2, and writing To establish the connection to the Floquet Hamiltonian, what remains is now to show that the function j n,m (g) reduces to the Bessel function J n−m (A) in the semi-classical limit.To do this, we write n = m + l and take the limit m → ∞ and g → 0 while keeping A = 2gd √ m constant.Noting that Since l = n − m the Hamiltonian H t,nm reproduces the correct Floquet Hamiltonian in the semi-classical limit.

Cavity-modified spin Hamiltonian
The total Hamiltonian H is down-folded to the spin sector by using quasi-degenerate perturbation theory to eliminate H t order by order. 53This perturbation expansion is performed numerically up to fourth order in t/U using exact diagonalization of the Ru 2 Cl 2 cluster shown in Fig. 3e.This allows to extract the nearest-neighbor magnetic interactions and their dependence on the lightmatter coupling, cavity frequency and polarization.The structure of the perturbation expansion allows the downfolding to be performed separately within each photon sector, giving a coupled spin-photon Hamiltonian of the form Here H s,nm is the spin Hamiltonian in the photon sector connecting photon numbers n and m, and is given by where each bond (ij) is labeled by the indexes αβ(γ) ∈ {xy(z), yz(x), zx(y)}.In this Hamiltonian, the magnetic parameters J, K and Γ all depend on the light-matter coupling g and the photon numbers n and m.

Exact diagonalization
The ground state of the coupled spin-photon system was obtained by exact diagonalization of a 24 site spin cluster interacting with a single photon mode.To perform the calculations we used to open-source Python package QuSpin. 54,55

Model parameters from first principles
To determine the parameters of the electronic Hamiltonians H U and H t , we performed first principles simulations of monolayer α-RuCl 3 with the Octopus electronic structure code. 56,57The single particle parameters where obtained from a Wannierization of the Ru 4d and Cl 3p orbitals in the paramagnetic state using Wannier90, 58,59 while the interaction parameters where determined using the hybrid DFT+U functional ACBN0 in the zigzag state. 56We have checked that the single particle parameters differ by less than one percent between the paramagnetic and zigzag states.The resulting electronic parameters are given in Tab.I, the equilibrium spin parameters in Tab.II, and the electronic band structure is shown in Fig. 4.
The calculations were performed in a 1 × √ 3 supercell to account for the zigzag magnetic structure, using the experimental lattice parameters a = 5.98 Å and b = 10.35

FIG. 1 .
FIG. 1. Crystal and magnetic structure of monolayer α-RuCl3.a, Crystal structure of monolayer α-RuCl3, with the magnetic Ru ions (orange) arranged in a hexagonal lattice and exhibiting a zigzag antiferromagnetic order in equilibrium.The surrounding octahedra of Cl ions give rise to crystal field splitting of the Ru d-orbitals (see panel c.) b, The magnetic system interacts with a strength g with a cavity electric field of frequency Ω. c, Energy level structure of magnetic Ru ions, giving rise to an effective j eff = 1/2 magnetic moment in the Ru t2g manifold.d, Magnetic interactions of Ru moments, colored according to the Kitaev bonds x, y and z.

FIG. 1 .
FIG. 1. Crystal and magnetic structure of monolayer α-RuCl3.a, Crystal structure of monolayer α-RuCl3, with the magnetic Ru ions (orange) arranged in a hexagonal lattice and exhibiting a zigzag antiferromagnetic order.The surrounding octahedra of Cl ions give rise to a crystal field splitting of the Ru d-orbitals (see panel c.) b, The magnetic system interacts with a strength g with a cavity electric field of frequency Ω. c, Energy level structure of the magnetic Ru ions, leading to an effective j eff = 1/2 magnetic moment in the Ru t2g manifold.d, Magnetic interactions of Ru moments, colored according to the Kitaev bonds x, y and z.

FIG. 2 .
FIG. 2. Magnetic phases of the photo ground state.a, Spin cluster employed for the exact diagonalization studies.b, Paths through the magnetic phase diagram traced out by the system as the light-matter coupling g increases from gi = 0 to g f = 0.5.The paths are colored according to the cavity photon energy Ω, and the small arrows show the direction of the paths.c, Parametrization angles θ and φ as a function of g for Ω = 10 meV.d, Nearest neighbor spin-spin correlation functions S z i S z j .e, Expectation value of the plaquette flux operator Wp. f, Average photon number nav = n of the photo ground state.g, Schematic of a fourth order direct process showing a 1/Ω enhancement at low photon energies.h, Schematic of a fourth order ligand mediated process unaffected by the low-frequency limit.

FIG. 2 .
FIG. 2. Magnetic phases of the photo ground state.a, Spin cluster employed for the exact diagonalization studies.b, Paths through the magnetic phase diagram traced out by the system as the light-matter coupling g increases from gi = 0 to g f = 0.5.The paths are colored according to the cavity photon energy Ω, and the small arrows show the direction of the paths.c, Parametrization angles θ and φ as a function of g for Ω = 10 meV.d, Nearest neighbor spin-spin correlation functions S z i S z j .e, Expectation value of the plaquette flux operator Wp. f, Average photon number nav = n of the photo ground state.g, Schematic of a fourth order direct process showing a 1/Ω enhancement at low photon energies.h, Schematic of a fourth order ligand mediated process unaffected by the low-frequency limit.

BFIG. 3 .
FIG. 3. Magnetic phases in a seeded cavity.a, Paths through the magnetic phase diagram traced out by the spin system as the light-matter coupling g increases from g i = 0 to g f = 0.5.The paths are colored according to the cavity photon energy ℏΩ, and the small arrows show the direction of the paths.b, Parametrization angles θ and ϕ as a function of g for ℏΩ = 3 eV.c, Schematic of third order processes where hopping around isosceles triangles leads to an induced magnetic field.d, Spin-spin correlation function ⟨S z i S z j ⟩ as a function of photon energy and light-matter coupling.e, Average of the plaquette flux operator W p as a function of photon energy and light-matter coupling.In all panels the average photon number is n av = 1.

FIG. 3 .
FIG. 3. Magnetic phases in a seeded cavity.a, Paths through the magnetic phase diagram traced out by the spin system as the light-matter coupling g increases from gi = 0 to g f = 0.5.The paths are colored according to the cavity photon energy Ω, and the small arrows show the direction of the paths.b, Parametrization angles θ and φ as a function of g for Ω = 3 eV.c, Spin-spin correlation function S z i S z j as a function of photon energy and light-matter coupling.d, Average of the plaquette flux operator Wp as a function of photon energy and light-matter coupling.e, Schematic of third order processes where hopping around isosceles triangles leads to an induced magnetic field.In all panels the average photon number is nav = 1.

Parameter U JH λ ∆ pd t1 t2 t3 t4 t pd eV 3 .= ea 2 0
FIG. S1.Polarization dependence of the Kitaev interaction for a linearly polarized cavity.Effective Kitaev interaction K as a function of light-matter coupling g/(2√ nav) and frequency Ω for a cavity with nav = 1.The different panels correspond to polarization vectors making and angle ϕ with the x-axis.Due to the C3 symmetry of the magnetic system, the polarization dependence shows a period of ϕ0 = 2π/3.

12 ϕ
FIG. S1.Polarization dependence of the Kitaev interaction for a linearly polarized cavity.Effective Kitaev interaction K as a function of light-matter coupling g/(2√ nav) and frequency Ω for a cavity with nav = 1.The different panels correspond to polarization vectors making and angle ϕ with the x-axis.Due to the C3 symmetry of the magnetic system, the polarization dependence shows a period of ϕ0 = 2π/3.