Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Identification of magnetic interactions and high-field quantum spin liquid in α-RuCl3

## Abstract

The frustrated magnet α-RuCl3 constitutes a fascinating quantum material platform that harbors the intriguing Kitaev physics. However, a consensus on its intricate spin interactions and field-induced quantum phases has not been reached yet. Here we exploit multiple state-of-the-art many-body methods and determine the microscopic spin model that quantitatively explains major observations in α-RuCl3, including the zigzag order, double-peak specific heat, magnetic anisotropy, and the characteristic M-star dynamical spin structure, etc. According to our model simulations, the in-plane field drives the system into the polarized phase at about 7 T and a thermal fractionalization occurs at finite temperature, reconciling observations in different experiments. Under out-of-plane fields, the zigzag order is suppressed at 35 T, above which, and below a polarization field of 100 T level, there emerges a field-induced quantum spin liquid. The fractional entropy and algebraic low-temperature specific heat unveil the nature of a gapless spin liquid, which can be explored in high-field measurements on α-RuCl3.

## Introduction

The spin-orbit magnet α-RuCl3, with edge-sharing RuCl6 octahedra and a nearly perfect honeycomb plane, has been widely believed to be a correlated insulator with the Kitaev interaction1,2,3,4,5,6. The compound α-RuCl3, and the Kitaev materials in general, have recently raised great research interest in exploring the inherent Kitaev physics7,8,9,10,11, which can realize non-Abelian anyon with potential applications in topological quantum computations12,13. Due to additional non-Kitaev interactions in the material, α-RuCl3 exhibits a zigzag antiferromagnetic (AF) order at sufficiently low temperature (Tc 7 K)2,14,15, which can be suppressed by an external in-plane field of 7-8 T16,17,18. Surprisingly, the thermodynamics and the unusual excitation continuum observed in the inelastic neutron scattering (INS) measurements suggest the presence of fractional excitations and the proximity of α-RuCl3 to a quantum spin liquid (QSL) phase14,15,19. Furthermore, experimental probes including the nuclear magnetic resonance (NMR)17,20,21, Raman scattering22, electron spin resonance (ESR)23, THz spectroscopy24,25, and magnetic torque26,27, etc, have been employed to address the possible Kitaev physics in α-RuCl3 from all conceivable angles. In particular, the unusual (even half-integer quantized) thermal Hall signal was observed in a certain temperature and field window28,29,30,31, suggesting the emergent Majorana fractional excitations. However, significant open questions remain to be addressed: whether the in-plane field in α-RuCl3 induces a QSL ground state that supports the spin-liquid signals in experiment, and furthermore, is there a QSL phase induced by fields along other direction?

To accommodate the QSL states in quantum materials like α-RuCl3, realization of magnetic interactions of Kitaev type plays a central role. Therefore, the very first step toward the precise answer to above questions is to pin down an effective low-energy spin model of α-RuCl3. As a matter of fact, people have proposed a number of spin models with various couplings19,32,33,34,35,36,37,38,39,40,41,42, yet even the signs of the couplings are not easy to determine and currently no single model can simultaneously cover the major experimental observations43, leaving a gap between theoretical understanding and experimental observations. In this work, we exploit multiple accurate many-body approaches to tackle this problem, including the exponential tensor renormalization group (XTRG)44,45 for thermal states, the density matrix renormalization group (DMRG) and variational Monte Carlo (VMC) for the ground state, and the exact diagonalization (ED) for the spectral properties. Through large-scale calculations, we determine an effective Kitaev-Heisenberg-Gamma-Gamma$$^{\prime}$$ (K-J-Γ-$${{\Gamma }}^{\prime}$$) model [cf. Eq. (1) below] that can perfectly reproduce the major experimental features in the equilibrium and dynamic measurements.

Specifically, in our K-J-Γ-$${{\Gamma }}^{\prime}$$ model the Kitaev interaction K is much greater than other non-Kitaev terms and found to play the predominant role in the intermediate temperature regime, showing that α-RuCl3 is indeed in close proximity to a QSL. As the compound, our model also possesses a low-T zigzag order, which is melted at about 7 K. At intermediate energy scale, a characteristic M star in the dynamical spin structure is unambiguously reproduced. Moreover, we find that in-plane magnetic field suppresses the zigzag order at around 7 T, and drives the system into a trivial polarized phase. Nevertheless, even above the partially polarized states, our finite-temperature calculations suggest that α-RuCl3 could have a fractional liquid regime with exotic Kitaev paramagnetism, reconciling previous experimental debates. We put forward proposals to explore the fractional liquid in α-RuCl3 via thermodynamic and spin-polarized INS measurements. Remarkably, when the magnetic field is applied perpendicular to the honeycomb plane, we disclose a QSL phase driven by high fields, which sheds new light on the search of QSL in Kitaev materials. Furthermore, we propose experimental probes through magnetization and calorimetry46 measurements under 100-T class pulsed magnetic fields47,48.

## Results

### Effective spin model and quantum many-body methods

We study the K-J-Γ-$${{\Gamma }}^{\prime}$$ honeycomb model with the interactions constrained within the nearest-neighbor sites, i.e.,

$$\begin{array}{ll}H=&\mathop{\sum}\limits_{{\langle i,j\rangle }_{\gamma }}\left[K{S}_{i}^{\gamma }{S}_{j}^{\gamma }+J\ {{\bf{S}}}_{i}\cdot {{\bf{S}}}_{j}+{{\Gamma }}\big({S}_{i}^{\alpha }{S}_{j}^{\beta }+{S}_{i}^{\beta }{S}_{j}^{\alpha }\big)\right.\\ &\kern-1.5pc\left.+\,{{\Gamma }}^{\prime} \big({S}_{i}^{\gamma }{S}_{j}^{\alpha }+{S}_{i}^{\gamma }{S}_{j}^{\beta }+{S}_{i}^{\alpha }{S}_{j}^{\gamma }+{S}_{i}^{\beta }{S}_{j}^{\gamma }\big)\right],\end{array}$$
(1)

where $${{\bf{S}}}_{i}=\{{S}_{i}^{x},{S}_{i}^{y},{S}_{i}^{z}\}$$ are the pseudo spin-1/2 operators at site i, and 〈i, jγ denotes the nearest-neighbor pair on the γ bond, with {α, β, γ} being {x, y, z} under a cyclic permutation. K is the Kitaev coupling, Γ and $${{\Gamma }}^{\prime}$$ the off-diagonal couplings, and J is the Heisenberg term. The symmetry of the model, besides the lattice translation symmetries, is described by the finite magnetic point group $${D}_{{\rm{3d}}}\times {Z}_{2}^{T}$$, where $${Z}_{2}^{T}=\{E,T\}$$ is the time-reversal symmetry group and each element in D3d stands for a combination of lattice rotation and spin rotation due to the spin-orbit coupling. The symmetry group restricts the physical properties of the system. For instance, the Landé g tensor and the magnetic susceptibility tensor, should be uni-axial.

We recall that the $${{\Gamma }}^{\prime}$$ term is important for stabilizing the zigzag magnetic order at low temperature in the extended ferromagnetic (FM) Kitaev model with K < 036,49,50. While the zigzag order can also be induced by the third-neighbor Heisenberg coupling J332,33,51, we constrain ourselves within a minimal K-J-Γ-$${{\Gamma }}^{\prime}$$ model in the present study and leave the discussion on the J3 coupling in the Supplementary Note 1. In the simulations of α-RuCl3 under magnetic fields, we mainly consider the in-plane field along the $$[11\bar{2}]$$ direction, $${H}_{[11\bar{2}]}\parallel {\bf{a}}$$, and the out-of-plane field along the [111] direction, H[111]c*, with the corresponding Landé factors $${g}_{{\rm{ab}}}(={g}_{[11\bar{2}]})$$ and $${g}_{{\rm{{c}}^{* }}}(={g}_{[111]})$$, respectively. The index [l, m, n] represents the field direction in the spin space depicted in Fig. 1b. Therefore, the Zeeman coupling between field H[l, m, n] to local moments can be written as $${H}_{{\rm{Zeeman}}}={g}_{[l,m,n]}{\mu }_{{\rm{B}}}{\mu }_{0}{H}_{[l,m,n]}{S}_{i}^{[l,m,n]}$$, where S[l, m, n] ≡ Sdl,m,n with S = (Sx, Sy, Sz) and $${{\bf{d}}}_{l,m,n}={(l,m,n)}^{T}/\sqrt{{l}^{2}+{m}^{2}+{n}^{2}}$$. The site index i = 1,  , N, with N ≡ W × L × 2 the total site number.

In the simulations, various quantum many-body calculation methods have been employed (see Methods). The thermodynamic properties under zero and finite magnetic fields are computed by XTRG on finite-size systems (see, e.g, YC4 systems shown in Fig. 1a). The model parameters are pinpointed by fitting the XTRG results to the thermodynamic measurements, and then confirmed by the ground-state magnetization calculations by DMRG with the same geometry and VMC on an 8 × 8 × 2 torus. Moreover, the ED calculations of the dynamical properties are performed on a 24-site torus, which are in remarkable agreement to experiments and further strengthen the validity and accuracy of our spin model. Therefore, by combining these cutting-edge many-body approaches, we explain the experimental observations from the determined effective spin Hamiltonian, and explore the field-induced QSL in α-RuCl3 under magnetic fields.

### Model parameters

As shown in Fig. 2a-b, through simulating the experimental measurements, including the magnetic specific heat and both in- and out-of-plane susceptibility data3,14,15,52,53,54,55, we accurately determine the parameters in the Hamiltonian Eq. (1), which read K = − 25 meV, Γ = 0.3K, $${{\Gamma }}^{\prime} =-0.02\,| K|$$, and J = − 0.1K. The in- and out-of-plane Landé factors are found to be gab = 2.5 and $${g}_{{\rm{{c}}^{* }}}=2.3$$, respectively. We find that both the magnetic specific heat Cm and the two susceptibilities (in-plane χab and out-of-plane $${\chi }_{{\rm{{c}}^{* }}}$$) are quite sensitive to the Γ term, and the inclusion of $${{\Gamma }}^{\prime}$$(J) term can significantly change the low-Cm(χab) data. Based on these observations, we accurately pinpoint the various couplings. The details of parameter determination, with comparisons to the previously proposed candidate models can be found in Supplementary Notes 1, 2. To check the robustness and uniqueness of the parameter fittings, we have also performed an automatic Hamiltonian searching56 with the Bayesian optimization combined large-scale thermodynamics solver XTRG, and find that the above effective parameter set indeed locates within the optimal regime of the optimization (Supplementary Note 1). In addition, the validity of our α-RuCl3 model is firmly supported by directly comparing the model calculations to the measured magnetization curves in Fig. 2c and INS measurements in Fig. 2d–f.

In our K-J-Γ-$${{\Gamma }}^{\prime}$$ model of α-RuCl3, we see a dominating FM Kitaev interaction and a sub-leading positive Γ term (Γ > 0), which fulfill the interaction signs proposed from recent experiments6,34,39 and agree with some ab initio studies9,33,36,37,43,57. The strong Kitaev interaction seems to play a predominant role at intermediate temperature, which leads to the fractional liquid regime and therefore naturally explains the observed proximate spin liquid behaviors14,15,19.

### Magnetic specific heat and two-temperature scales

We now show our simulations of the K-J-Γ-$${{\Gamma }}^{\prime}$$ model and compare the results to the thermodynamic measurements. In Fig. 2a, the XTRG results accurately capture the prominent double-peak feature of the magnetic specific heat Cm, i.e., a round high-T peak at Th 100 K and a low-T one at Tl 7 K. As Th 100 K is a relatively high-temperature scale where the phonon background needs to be carefully deal with52, and there exists quantitative difference among the various Cm measurements in the high-T regime3,15,52. Nevertheless, the high-T scale Th itself is relatively stable, and in Fig. 2a our XTRG result indeed exhibits a high-T peak centered at around 100 K, in good agreement with various experiments. Note that the high-temperature crossover at Th corresponds to the establishment of short-range spin correlations, which can be ascribed to the emergence of itinerant Majorana fermions15,58 in the fractional liquid picture that we will discuss. Such a crossover can also be observed in the susceptibilities, which deviate the high-T Curie-Weiss law and exhibit an intermediate-T Curie-Weiss scaling below Th59, as shown in Fig. 2b for χab (the same for $${\chi }_{{\rm{{c}}^{* }}}$$).

At the temperature Tl 7 K, the experimental Cm curves of α-RuCl3 exhibit a very sharp peak, corresponding to the establishment of a zigzag magnetic order2,3,14,15,52. Such a low-T scale can be accurately reproduced by our model calculations, as shown in Fig. 2a. As our calculations are performed on the cylinders of a finite width, the height of the Tl peak is less prominent than experiments, as the transition in the compound α-RuCl3 may be enhanced by the inter-layer couplings. Importantly, the location of Tl fits excellently to the experimental results. Below Tl our model indeed shows significantly enhanced zigzag spin correlation, which is evidenced by the low-energy dynamical spin structure in Fig. 2f and the low-T static structure in the inset of Fig. 3a.

### Anisotropic susceptibility and magnetization curves

It has been noticed from early experimental studies of α-RuCl3 that there exists a very strong magnetic anisotropy in the compound2,3,6,14,53,54,55, which was firstly ascribed to anisotropic Landé g factor3,55, and recently to the existence of the off-diagonal Γ interaction,6,53,60. We compute the magnetic susceptibilities along two prominent field directions, i.e., $${H}_{[11\bar{2}]}$$ and H[111], and compare them to experiments in Fig. 2b14,53,54. The discussions on different in-plane and tilted fields are left in the Supplementary Note 3.

In Fig. 2b, we show that both the in- and out-of-plane magnetic susceptibilities χab and $${\chi }_{{\rm{{c}}^{* }}}$$ can be well fitted using our K-J-Γ-$${{\Gamma }}^{\prime}$$ model, with dominant Kitaev K, considerable off-diagonal Γ, as well as similar in-plane (gab) and out-of-plane ($${g}_{{\rm{{c}}^{* }}}$$) Landé factors. Therefore, our many-body simulation results indicate that the anisotropic susceptibilities mainly originate from the off-diagonal Γ coupling (cf. Supplementary Fig. 2), in consistent with the resonant elastic X-ray scattering6 and susceptibility measurements53. Moreover, with the parameter set of K, Γ, $${{\Gamma }}^{\prime}$$, J, gab, and $${g}_{{\rm{{c}}^{* }}}$$ determined from our thermodynamics simulations, we compute the magnetization curves $$M({H}_{[l,m,n]})=1/N\mathop{\sum }\nolimits_{i = 1}^{N}{g}_{[l,m,n]}{\mu }_{{\rm{B}}}\langle {S}_{i}^{[l,m,n]}\rangle$$ along the $$[11\bar{2}]$$ and [111] directions using DMRG, as shown in Fig. 2c. The two simulated curves, showing clear magnetic anisotropy, are in quantitative agreement with the experimental measurements at very low temperature3,55.

### Dynamical spin structure and the M star

The INS measurements on α-RuCl3 revealed iconic dynamical structure features at low and intermediate energies14,15. With the determined α-RuCl3 model, we compute the dynamical spin structure factors using ED, and compare the results to experiments. First, we show in Fig. 2d the constant k-cut at the k = Γ and M points (as indicated in Fig. 2e), where a quantitative agreement between theory and experiment can be observed. In particular, the positions of the intensity peak ωΓ = 2.69 ± 0.11 meV and ωM = 2.2 ± 0.2 meV from the INS measurements14, are accurately reproduced with our determined model. For the Γ-point intensity, the double-peak structure, which was observed in experimental measurements18, can also be well captured.

We then integrate the INS intensity $${\mathcal{I}}({\bf{k}},\omega )$$ over the low- and intermediate-energy regime with the atomic form factor taken into account, and check their k-dependence in Fig. 2e-f. In experiment, a structure factor with bright Γ and M points was observed at low energy, and, on the other hand, a renowned six-pointed star shape (dubbed M star43) was reported at intermediate energies14,15. In Fig. 2e-f, these two characteristic dynamical spin structures are reproduced, in exactly the same energy interval as experiments. Specifically, the zigzag order at low temperature is reflected in the bright M points in the Brillouin zone (BZ) when integrated over [2, 3] meV, and the Γ point in the BZ is also turned on. As the energy interval increases to [4.5, 7.5] meV, the M star emerges as the zigzag correlation is weakened while the continuous dispersion near the Γ point remains prominent. The round Γ peak, which also appears in the pure Kitaev model, is consistent with the strong Kitaev term in our α-RuCl3 model.

### Suppressing the zigzag order by in-plane fields

In experiments, the low-T zigzag magnetic order has been observed to be suppressed by the in-plane magnetic fields above 7-8 T3,17,18,55. We hereby investigate this field-induced effect by computing the spin structure factors under finite fields. The M-point peak of the structure factor S(M) in the T-$${H}_{[11\bar{2}]}$$ plane characterizes the zigzag magnetic order as shown in Fig. 3a. The derivatives $$\frac{dS({\rm{M}})}{dT}{| }_{H = 0}$$ and $$\frac{dS({\rm{M}})}{dH}{| }_{T = 1.9{\rm{K}}}$$ are calculated in Fig. 3b-c, which can only show a round peak at the transition as limited by our finite-size simulation. For H = 0, the turning temperature is at about 7 K, below which the zigzag order builds up; on the other hand, the isothermal M-H curves in Fig. 3c suggest a transition point at $${h}_{[11\bar{2}]}^{c}={\mu }_{0}{H}_{[11\bar{2}]}\simeq 7$$ T, beyond which the zigzag order is suppressed. Correspondingly, in Fig. 4c [and also in Fig. 4d], the low-temperature scale Tl decreases as the fields increase, initially very slow for small fields and then quickly approaches zero only in the field regime near the critical point, again in very good consistency with experimental measurements3,17,20.

Besides, from the contour plots of Cm/T and the isentropes in Fig. 4a, c, one can also recognize the critical temperature and field consistent with the above estimations. Moreover, when the field direction is tilted about 55 away from the a axis in the a-c* plane (i.e., H[110] along the $$c^{\prime}$$ axis), as shown in Fig. 4b, d, our model calculations suggest a critical field $${h}_{[110]}^{c}={\mu }_{0}{H}_{[110]}\simeq 10$$ T with suppressed zigzag order, in accordance with recent NMR probe20. Overall, the excellent agreements of the finite-field simulations with different experiments further confirm our K-J-Γ-$${{\Gamma }}^{\prime}$$ model as an accurate description of the Kitaev material α-RuCl3.

### Finite-T phase diagram under in-plane fields

Despite intensive experimental and theoretical studies, the phase diagram of α-RuCl3 under in-plane fields remains an interesting open question. The thermal Hall29, Raman scattering22, and thermal expansion61 measurements suggest the existence of an intermediate QSL phase between the zigzag and polarized phases. On the other hand, the magnetization3,55, INS18, NMR17,20,21, ESR23, Grüneisen parameter62, and magnetic torque measurements27 support a single-transition scenario (leaving aside the transition between two zigzag phases due to different inter-layer stackings63). Nevertheless, most experiments found signatures of fractional excitations at finite temperature, although an alternative multi-magnon interpretation has also been proposed23.

Now with the accurate α-RuCl3 model and multiple many-body computation approaches, we aim to determine the phase diagram and nature of the field-driven phase(s). Our main results are summarized in Fig. 1c, e, where a single quantum phase transition (QPT) is observed as the in-plane fields $${H}_{[11\bar{2}]}$$ increases. Both VMC and DMRG calculations find a trivial polarized phase in the large-field side ($${\mu }_{0}{H}_{[11\bar{2}]}\,> \,{h}_{[11\bar{2}]}^{c}$$), as evidenced by the magnetization curve in Fig. 2c as well as the results in Supplementary Note 3.

Despite the QSL phase is absent under in-plane fields, we nevertheless find a Kitaev fractional liquid at finite temperature, whose properties are determined by the fractional excitations of the system. For the pure Kitaev model, it has been established that the itinerant Majorana fermions and Z2 fluxes each releases half of the entropy at two crossover temperature scales15,58. Such an intriguing regime is also found robust in the extended Kitaev model with additional non-Kitaev couplings59. Now for the realistic α-RuCl3 model in Eq. (1), we find again the presence of fractional liquid at intermediate T. As shown in Fig. 2b (zero field) and Fig. 3d (finite in-plane fields), the intermediate-T Curie-Weiss susceptibility can be clearly observed, with the fitted Curie constant $$C^{\prime}$$ distinct from the high-T paramagnetic constant C. This indicates the emergence of a novel correlated paramagnetism—Kitaev paramagnetism—in the material α-RuCl3. The fractional liquid constitutes an exotic finite-temperature quantum state with disordered fluxes and itinerant Majorana fermions, driven by the strong Kitaev interaction that dominates the intermediate-T regime59.

In Fig. 4c-d of the isentropes, we find that the Kitaev fractional liquid regime is rather broad under either in-plane ($${H}_{[11\bar{2}]}$$) or tilted (H[110]) fields. When the field is beyond the critical value, the fractional liquid regime gradually gets narrowed, from high-T scale Th down to a new lower temperature scale $${T}_{{\rm{l}}}^{\prime}$$, below which the field-induced uniform magnetization builds up (see Supplementary Note 3). From the specific heat and isentropes in Fig. 4, we find in the polarized phase $${T}_{{\rm{l}}}^{\prime}$$ increases linearly as field increases, suggesting that such a low-T scale can be ascribed to the Zeeman energy. At the intermediate temperature, the thermal entropy is around $$({\mathrm{ln}}\,2)/2$$ [see Fig. 4c-d], indicating that “one-half” of the spin degree of freedom, mainly associated with the itinerant Majorana fermions, has been gradually frozen below Th.

Besides, we also compute the spin structure factors $${S}^{\gamma \gamma }({\bf{k}})=1/{N}_{{\rm{Bulk}}}\ {\sum }_{i,j\in {\rm{Bulk}}}{e}^{i{\bf{k}}({{\bf{r}}}_{j}-{{\bf{r}}}_{i})}\langle {S}_{i}^{\gamma }{S}_{j}^{\gamma }\rangle$$ under an in-plane field of $${\mu }_{0}{H}_{[11\bar{2}]}=4.2$$ T in the fractional liquid regime, where NBulk is the number of bulk sites (with left and right outmost columns skipped), i, j run over the bulk sites, and γ = x, y, z. Except for the bright spots at Γ and M points, there appears stripy background in Fig. 3e-g very similar to that observed in the pure Kitaev model59, which reflects the extremely short-range and bond-directional spin correlations there. The stripe rotates as the spin component γ switches, because the γ-type spin correlations $${\langle {S}_{i}^{\gamma }{S}_{j}^{\gamma }\rangle }_{\gamma }$$ are nonzero only on the nearest-neighbor γ-type bond. As indicated in the realistic model calculations, we propose such distinct features in Sγγ(k) can be observed in the material α-RuCl3 via the polarized neutron diffusive scatterings.

### Signature of Majorana fermions and the Kitaev fractional liquid

It has been highly debated that whether there exists a QSL phase under intermediate in-plane fields. Although more recent experiments favor the single-transition scenario27,62, there is indeed signature of fractional Majorana fermions and spin liquid observed in the intermediate-field regime18,21,22,27,29. Based on the model simulations, here we show that our finite-T phase diagram in Fig. 1 provides a consistent scenario that reconciles these different in-plane field experiments.

For example, large28,64 or even half-quantized thermal Hall conductivity was observed at intermediate fields and between 4 and 6 K30,31,65. However, it has also been reported that the thermal Hall conductivity vanishes rapidly when the field further varies or the temperature lowers below ~2 K66. Therefore, one possible explanation, according to our model calculations, is that the ground state under in-plane fields above 7 T is a trivial polarized phase [see Fig. 1c], while the large thermal Hall conductivity at intermediate fields may originate from the Majorana fermion excitations in the finite-T fractional liquid58.

In the intermediate-T fractional liquid regime, the Kitaev interaction is predominating and the system resembles a pure Kitaev model under external fields and at a finite temperature. This effect is particularly prominent as the field approaches the intermediate regime, i.e., near the quantum critical point, where the fractional liquid can persist to much lower temperature. Matter of fact, given a fixed low temperature, when the field is too small or too large, the system leaves the fractional liquid regime (cf. Fig. 4) and the signatures of the fractional excitation become blurred, as if there were a finite-field window of “intermediate spin liquid phase”. Such fractional liquid constitutes a Majorana metal state with a Fermi surface58,59, accounting possibly for the observed quantum oscillation in longitudinal thermal transport66. Besides thermal transport, the fractional liquid dominated by fractional excitations can lead to rich experimental ramifications, e.g., the emergent Curie-Weiss susceptibilities in susceptibility measurements [see Fig. 3d] and the stripy spin structure background in the spin-resolved neutron or resonating X-ray scatterings [Fig. 3e-g], which can be employed to probe the finite-T fractional liquid in the compound α-RuCl3.

### Quantum spin liquid induced by out-of-plane fields

Now we apply the H[111]c* field out of the plane and investigate the field-induced quantum phases in α-RuCl3. As shown in the phase diagram in Fig. 1d, f, under the H[111] fields a field-induced QSL phase emerges at intermediate fields between the zigzag and the polarized phases, confirmed in both the thermal and ground-state calculations. The existence of two QPTs and an intermediate phase can also be seen from the color maps of Cm, Z2 fluxes, and thermal entropies shown in Figs. 5a-b and 6a.

To accurately nail down the two QPTs, we plot the c* DMRG magnetization curve M(H[111]) and the derivative dM/dH[111] in Fig. 5c, together with the ED energy spectra in Fig. 5d, from which the ground-state phase diagram can be determined (cf. Fig. 1f). In particular, the lower transition field, $${h}_{[111]}^{c1}\simeq 35$$ T, estimated from both XTRG and DMRG, is in excellent agreement with recent experiment through measuring the magnetotropic coefficient27. The existence of the upper critical field $${h}_{[111]}^{c2}$$ at 100 T level can also be probed with current pulsed high field techniques48.

Correspondingly, we find in Fig. 5b that the Z2 flux $${W}_{{\rm{p}}}={2}^{6}\langle {S}_{i}^{x}{S}_{j}^{y}{S}_{k}^{z}{S}_{l}^{x}{S}_{m}^{y}{S}_{n}^{z}\rangle$$ (where i, j, k, l, m, n denote the six vertices of a hexagon p) changes its sign from negative to positive at $${h}_{[111]}^{c1}$$, then to virtually zero at $${h}_{[111]}^{c2}$$, and finally converge to very small (positive) values in the polarized phase. These observations of flux signs in different phases are consistent with recent DMRG and tensor network studies on a K-Γ-$${{\Gamma }}^{\prime}$$ model49,50, and it is noteworthy that the flux is no longer a strictly conserved quantity as in the pure Kitaev model, the low-T expectation values of Wp thus would be very close to 1 only deep in the Kitaev spin liquid phase49,59,67.

From the Cm color map in Fig. 5a and Cm curves in Fig. 6c, we find double-peaked specific heat curves in the QSL phase, which clearly indicate the two temperature scales, e.g., Th 105 K and $${T}_{{\rm{l}}}^{^{\prime\prime} }\simeq 10$$ K for μ0H[111] = 45.5 T. They correspond to the establishment of spin correlations at Th and the alignment of Z2 fluxes at $${T}_{{\rm{l}}}^{^{\prime\prime} }$$, respectively, as shown in Fig. 5f. As a result, in Fig. 6a-b the system releases $$({\mathrm{ln}}\,2)/2$$ thermal entropy around Th, and the rest half is released at around $${T}_{{\rm{l}}}^{^{\prime\prime} }$$. The magnetic susceptibility curves in Fig. 5e fall into an intermediate-T Curie-Weiss behavior below Th when the spin correlations are established, and deviate such emergent universal behavior when approaching $${T}_{{\rm{l}}}^{^{\prime\prime} }$$ as the gauge degree of freedom (flux) gradually freezes.

Furthermore, we study the properties of the QSL phase. We find very peculiar spin correlations as evidenced by the (modified) structure factor

$${\tilde{S}}^{_{zz}}({\bf{k}})=\frac{1}{{N}_{{\rm{Bulk}}}}\mathop{\sum}\limits_{i,j\in {\rm{Bulk}}}{e}^{i{\bf{k}}({{\bf{r}}}_{j}-{{\bf{r}}}_{i})}(\langle {S}_{i}^{z}{S}_{j}^{z}\rangle -\langle {S}_{i}^{z}\rangle \langle {S}_{j}^{z}\rangle ).$$
(2)

As shown in the inset of Fig. 5b, there appears no prominent peak in $${\tilde{S}}^{_{zz}}({\bf{k}})$$ at both finite and zero temperatures, for a typical field of 45.5 T in the QSL phase. As shown in Fig. 5f, we find the dominating nearest-neighboring correlations at T > 0 are bond-directional and the longer-range correlations are rather weak, same for the QSL ground state at T = 0. This is in sharp contrast to the spin correlations in the zigzag phase. More spin structure results computed with both XTRG and DMRG can be found in Supplementary Note 4. Below the temperature $${T}_{{\rm{l}}}^{^{\prime\prime} }$$, we observe an algebraic specific heat behavior as shown in Fig. 6c-d, which strongly suggests a gapless QSL. We remark that the gapless spin liquid in the pure Kitaev model also has bond-directional and extremely short-range spin correlations13. The similar features of spin correlations in this QSL state may be owing to the predominant Kitaev interaction in our model.

Overall, under the intermediate fields between $${h}_{[111]}^{c1}$$ and $${h}_{[111]}^{c2}$$, and below the low-temperature scale $${T}_{{\rm{l}}}^{^{\prime\prime} }$$, our model calculations predict the presence of the long-sought QSL phase in the compound α-RuCl3.

## Discussion

First, we discuss the nature of the QSL driven by the out-of-plane fields. In Fig. 5d, the ED calculation suggests a gapless spectrum in the intermediate phase, and the DMRG simulations on long cylinders find the logarithmic correction in the entanglement entropy scaling (see Supplementary Note 4), which further supports a gapless QSL phase. On the other hand, the VMC calculations identify the intermediate phase as an Abelian chiral spin liquid, which is topologically nontrivial with a quantized Chern number ν = 2. Overall, various approaches consistently find the same scenario of two QPTs and an intermediate-field QSL phase under high magnetic fields [cf. Fig. 1d, f]. It is worth noticing that a similar scenario of a gapless QSL phase induced by out-of-plane fields has also been revealed in a Kitaev-Heisenberg model with K > 0 and J < 068, where the intermediate QSL was found to be smoothly connected to the field-induced spin liquid in a pure AF Kitaev model67,69,70. We note the extended AF Kitaev model in Ref. 68 can be transformed into a K-J-Γ-$${{\Gamma }}^{\prime}$$ model with FM Kitaev term (and other non-Kitaev terms still distinct from our model) through a global spin rotation71. Therefore, despite the rather different spin structure factors of the field-induced QSL in the AF Kitaev model from ours, it is interesting to explore the possible connections between the two in the future.

Based on our α-RuCl3 model and precise many-body calculations, we offer concrete experimental proposals for detecting the intermediate QSL phase via the magnetothermodynamic measurements under high magnetic fields. The two QPTs are within the scope of contemporary technique of pulsed high fields, and can be confirmed by measuring the magnetization curves47,48. The specific heat measurements can also be employed to confirm the two-transition scenario and the high-field gapless QSL states. As field increases, the lower temperature scale Tl first decreases to zero as the zigzag order is suppressed, and then rises up again (i.e., $${T}_{{\rm{l}}}^{^{\prime\prime} }$$) in the QSL phase [cf. Fig. 5a]. As the specific heat exhibits a double-peak structure in the high-field QSL regime, the thermal entropy correspondingly undergoes a two-step release in the QSL phase and exhibits a quasi-plateau near the fractional entropy $$({\mathrm{ln}}\,2)/2$$ [cf. 45.5 T and 91 T lines in Fig. 6b]. This, together with the low-T (below $${T}_{{\rm{l}}}^{^{\prime\prime} }$$) algebraic specific heat behavior reflecting the gapless excitations, can be probed through high-field calorimetry46.

Lastly, it is interesting to note that the emergent high-field QSL under out-of-plane fields may be closely related to the off-diagonal Γ term (see, e.g., Refs. 49,50) in the compound α-RuCl3. The Γ term has relatively small influences in ab plane, while introduces strong effects along the c* axis — from which the magnetic anisotropy in α-RuCl3 mainly originates. The zigzag order can be suppressed by relatively small in-plane fields and the system enters the polarized phase, as the Γ term does not provide a strong “protection” of both zigzag and QSL phases under in-plane fields (recall the QSL phase in pure FM Kitaev model is fragile under external fields59,67,72). The situation is very different for out-of-plane fields, where the Kitaev (and also Γ) interactions survive the QSL after the zigzag order is suppressed by high fields. Intuitively, the emergence of this QSL phase can therefore be ascribed to the strong competition between the Γ interaction and magnetic field along the hard axis c* of α-RuCl3. With such insight, we expect a smaller critical field for compounds with a less significant Γ interaction. In the fast-moving Kitaev materials studies, such compounds with relatively weaker magnetic anisotropy, e.g., the recent Na2Co2TeO6 and Na2Co2SbO673,74,75,76, have been found, which may also host QSL induced by out-of-plane fields at lower field strengths.

## Methods

### Exponential tensor renormalization group

The thermodynamic quantities including the specific heat, magnetic susceptibility, Z2 flux, and the spin correlations can be computed with the exponential tensor renormalization group (XTRG) method44,45 on the Y-type cylinders with width W = 4 and length up to L = 6 (i.e., YC4 × 6 × 2). We retain up to D = 800 states in the XTRG calculations, with truncation errors ϵ 2 × 10−5, which guarantees a high accuracy of computed thermal data down to the lowest temperature T 1.9 K. Note the truncation errors in XTRG, different from that in DMRG, directly reflects the relative errors in the free energy and other thermodynamics quantities. The low-T data are shown to approach the T = 0 DMRG results (see Fig. 5f). In the thermodynamics simulations of the α-RuCl3 model, one needs to cover a rather wide range of temperatures as the high- and low-T scales are different by more than one order of magnitude (100 K vs. 7 K in α-RuCl3 under zero field). In the XTRG cooling procedure, we represent the initial density matrix ρ0(τ) at a very high temperature T ≡ 1/τ (with τ 1) as a matrix product operator (MPO), and the series of lower temperature density matrices ρn(2nτ) (n ≥ 1) are obtained by successively multiplying and compressing ρn = ρn−1ρn−1 via the tensor-network techniques. Thus XTRG is very suitable to deal with such Kitaev model problems, as it cools down the system exponentially fast in temperature59.

### Density matrix renormalization group

The ground state properties are computed by the density matrix renormalization group (DMRG) method, which can be considered as a variational algorithm based on the matrix product state (MPS) ansatz. We keep up to D = 2048 states to reduce the truncation errors ϵ 1 × 10−8 with a very good convergence. The simulations are based on the high-performance MPS algorithm library GraceQ/MPS277.

### Variational Monte Carlo

The ground state of α-RuCl3 model are evaluated by the variational Monte Carlo (VMC) method based on the fermionic spinon representation. The spin operators are written in quadratic forms of fermionic spinons $${S}_{i}^{m}=\frac{1}{2}{C}_{i}^{\dagger }{\sigma }^{m}{C}_{i},m=x,y,z$$ under the local constraint $${\hat{N}}_{i}={c}_{i\uparrow }^{\dagger }{c}_{i\uparrow }+{c}_{i\downarrow }^{\dagger }{c}_{i\downarrow }=1$$, where $${C}_{i}^{\dagger }=({c}_{i\uparrow }^{\dagger },{c}_{i\downarrow }^{\dagger })$$ and σm are Pauli matrices. Through this mapping, the spin interactions are expressed in terms of fermionic operators and are further decoupled into a non-interacting mean-field Hamiltonian Hmf(R), where R denotes a set of parameters (see Supplementary Note 5). Then we perform Gutzwiller projection to the mean-field ground state $$\left|{{{\Phi }}}_{{\rm{mf}}}({\boldsymbol{R}})\right\rangle$$ to enforce the particle number constraint ($${\hat{N}}_{i}=1$$). The projected states $$\left|{{\Psi }}({\boldsymbol{R}})\right\rangle ={P}_{G}\left|{{{\Phi }}}_{{\rm{mf}}}({\boldsymbol{R}})\right\rangle ={\sum }_{\alpha }f(\alpha )\left|\alpha \right\rangle$$ (here α stands for the Ising bases in the many-body Hilbert space, same for β and γ below) provide a series of trial wave functions, depending on the specific choice of the mean-field Hamiltonian Hmf(R). Owing to the huge size of the many-body Hilbert space, the energy of the trial state $$E({\boldsymbol{R}})=\left\langle {{\Psi }}({\boldsymbol{R}})\right|H\left|{{\Psi }}({\boldsymbol{R}})\right\rangle /\langle {{\Psi }}({\boldsymbol{R}})| {{\Psi }}({\boldsymbol{R}})\rangle ={\sum }_{\alpha }\frac{| f(\alpha ){| }^{2}}{{\sum }_{\gamma }| f(\gamma ){| }^{2}}\left(\right.{\sum }_{\beta }\langle \beta | H| \alpha \rangle \frac{f{(\beta )}^{* }}{f{(\alpha )}^{* }}\left)\right.$$ is computed using Monte Carlo sampling. The optimal parameters R are determined by minimizing the energy E(R). While the VMC calculations are performed on a relatively small size (up to 128 sites), once the optimal parameters are determined we can plot the spinon dispersion of a QSL state by diagonalizing the mean-field Hamiltonian on a larger lattice size, e.g., 120 × 120 unit cells in practice.

### Exact diagonalization

The 24-site exact diagonalization (ED) is employed to compute the zero-temperature dynamical correlations and energy spectra. The clusters with periodic boundary conditions are depicted in the inset of Fig. 2d and the Supplementary Information, and the α-RuCl3 model under in-plane fields ($${H}_{[11\bar{2}]}\parallel {\bf{a}}$$ and $${H}_{[1\bar{1}0]}\parallel {\bf{b}}$$) as well as out-of-plane fields H[111]c* have been calculated, as shown in Fig. 5 and Supplementary Note 3. Regarding the dynamical results—the neutron scattering intensity—is defined as

$$\begin{array}{ll}{\mathcal{I}}({\bf{k}},\omega )\propto &{f}^{2}({\bf{k}})\int dt\mathop{\sum}\limits_{\mu ,\nu }\big({\delta }_{\mu ,\nu }-{k}_{\mu }{k}_{\nu }/{k}^{2}\big)\\ &\times \mathop{\sum}\limits_{i,j}\langle {S}_{i}^{\mu }(t){S}_{j}^{\nu }(0)\rangle {e}^{i{\bf{k}}\cdot ({{\bf{r}}}_{j}-{{\bf{r}}}_{i})}{e}^{-i\omega t},\\ \end{array}$$
(3)

where f(k) is the atomic form factor of Ru3+, which can be fitted by an analytical function as reported in Ref. 78. $${S}^{\mu \nu }({\bf{k}},\omega )={\sum }_{i,j}\langle {S}_{i}^{\mu }(t){S}_{j}^{\nu }(0)\rangle {e}^{i{\bf{k}}\cdot ({{\bf{r}}}_{j}-{{\bf{r}}}_{i})}{e}^{-i\omega t}$$ is the dynamical spin structure factor, which can be expressed by the continued fraction expansion in the tridiagonal basis of the Hamiltonian using Lanczos iterative method. For the diagonal part,

$$\begin{array}{ll}{S}^{\mu \mu }({\bf{k}},\omega )&=-\frac{1}{\pi }{\rm{Im}}\big[\left\langle {\psi }_{0}\right|{\hat{S}}_{-{\bf{k}}}^{\mu }\frac{1}{z-\hat{H}}{\hat{S}}_{{\bf{k}}}^{\mu }\left|{\psi }_{0}\right\rangle \big]\\ &\!\!\!\!\!\!\!\!=-\frac{1}{\pi }{\rm{Im}}\Big[\frac{\left\langle {\psi }_{0}\right|{\hat{S}}_{-{\bf{k}}}^{\mu }{\hat{S}}_{{\bf{k}}}^{\mu }\left|{\psi }_{0}\right\rangle }{z-{a}_{0}-\frac{{b}_{1}^{2}}{z-{a}_{1}-\frac{{b}_{2}^{2}}{z-{a}_{2}-\cdot \cdot \cdot }}}\Big],\end{array}$$
(4)

where z = ω + E0 + iη, E0 is the ground state energy, $$\left|{\psi }_{0}\right\rangle$$ is the ground state wave function, η is the Lorentzian broadening factor (here we take η = 0.5 meV in the calculations, i.e., 0.02 times the Kitaev interaction strength K = 25 meV), and ai (bi+1) is the diagonal (sub-diagonal) matrix element of the tridiagonal Hamiltonian. On the other hand, for the off-diagonal part, we define a Hermitian operator $${\hat{S}}_{{\bf{k}}}^{\mu }+{\hat{S}}_{{\bf{k}}}^{\nu }$$ to do the continued fraction expansion,

$$\begin{array}{ll}{S}^{(\mu +\nu )(\mu +\nu )}({\bf{k}},\omega )\\ &=-\frac{1}{\pi }{\rm{Im}}\left[\left\langle {\psi }_{0}\right|\big({\hat{S}}_{-{\bf{k}}}^{\mu }+{\hat{S}}_{-{\bf{k}}}^{\nu }\big)\frac{1}{z-\hat{H}}\big({\hat{S}}_{{\bf{k}}}^{\mu }+{\hat{S}}_{{\bf{k}}}^{\nu }\big)\left|{\psi }_{0}\right\rangle \right]\\ &\kern-4.3pc=-\frac{1}{\pi }{\rm{Im}}\left[\frac{\left\langle {\psi }_{0}\right|({\hat{S}}_{-{\bf{k}}}^{\mu }+{\hat{S}}_{-{\bf{k}}}^{\nu })({\hat{S}}_{{\bf{k}}}^{\mu }+{\hat{S}}_{{\bf{k}}}^{\nu })\left|{\psi }_{0}\right\rangle }{z-{a}_{0}-\frac{{b}_{1}^{2}}{z-{a}_{1}-\frac{{b}_{2}^{2}}{z-{a}_{2}-\cdot \cdot \cdot }}}\right].\end{array}$$
(5)

Then the off-diagonal Sμν(k, ω) can be computed by Sμν(k, ω) + Sνμ(k, ω) = S(μ+ν) (μ+ν)(k, ω) − Sμμ(k, ω) − Sνν(k, ω). Following the INS experiments14, the shown scattering intensities in Fig. 2 are integrated over perpendicular momenta kz [ − 5π, 5π], assuming perfect two-dimensionality of α-RuCl3 in the ED calculations.

## Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

## Code availability

All numerical codes in this paper are available upon request to the authors.

## References

1. 1.

Plumb, K. W. et al. α-RuCl3: A spin-orbit assisted Mott insulator on a honeycomb lattice. Phys. Rev. B 90, 041112(R) (2014).

2. 2.

Sears, J. A. et al. Magnetic order in α-RuCl3: A honeycomb-lattice quantum magnet with strong spin-orbit coupling. Phys. Rev. B 91, 144420 (2015).

3. 3.

Kubota, Y., Tanaka, H., Ono, T., Narumi, Y. & Kindo, K. Successive magnetic phase transitions in α-RuCl3: XY-like frustrated magnet on the honeycomb lattice. Phys. Rev. B 91, 094422 (2015).

4. 4.

Sandilands, L. J. et al. Spin-orbit excitations and electronic structure of the putative Kitaev magnet α-RuCl3. Phys. Rev. B 93, 075144 (2016).

5. 5.

Zhou, X. et al. Angle-resolved photoemission study of the Kitaev candidate α-RuCl3. Phys. Rev. B 94, 161106 (2016).

6. 6.

Sears, J. A. et al. Ferromagnetic Kitaev interaction and the origin of large magnetic anisotropy in α-RuCl3. Nat. Phys. 16, 837–840 (2020).

7. 7.

Jackeli, G. & Khaliullin, G. Mott insulators in the strong spin-orbit coupling limit: From Heisenberg to a quantum compass and Kitaev models. Phys. Rev. Lett. 102, 017205 (2009).

8. 8.

Trebst, S. "Kitaev materials,” http://arxiv.org/abs/1701.07056 (2017).

9. 9.

Winter, S. M. et al. Models and materials for generalized Kitaev magnetism. J. Phys.: Condens. Matter 29, 493002 (2017).

10. 10.

Takagi, H., Takayama, T., Jackeli, G., Khaliullin, G. & Nagler, S. E. Concept and realization of Kitaev quantum spin liquids. Nat. Rev. Phys. 1, 264–280 (2019).

11. 11.

Janssen, L. & Vojta, M. Heisenberg–Kitaev physics in magnetic fields. J. Phys.: Condens. Matter 31, 423002 (2019).

12. 12.

Kitaev, A. Fault-tolerant quantum computation by anyons. Ann. Phys. 303, 2–30 (2003).

13. 13.

Kitaev, A. Anyons in an exactly solved model and beyond. Ann. Phys. 321, 2–111 (2006).

14. 14.

Banerjee, A. et al. Neutron scattering in the proximate quantum spin liquid α-RuCl3. Science 356, 1055–1059 (2017).

15. 15.

Do, S.-H. et al. Majorana fermions in the Kitaev quantum spin system α-RuCl3. Nat. Phys. 13, 1079 (2017).

16. 16.

Sears, J. A., Zhao, Y., Xu, Z., Lynn, J. W. & Kim, Y.-J. Phase diagram of α-RuCl3 in an in-plane magnetic field. Phys. Rev. B 95, 180411 (2017).

17. 17.

Zheng, J. et al. Gapless spin excitations in the field-induced quantum spin liquid phase of α-RuCl3. Phys. Rev. Lett. 119, 227208 (2017).

18. 18.

Banerjee, A. et al. Excitations in the field-induced quantum spin liquid state of α-RuCl3. npj Quantum Mater. 3, 8 (2018).

19. 19.

Banerjee, A. et al. Proximate Kitaev quantum spin liquid behaviour in a honeycomb magnet. Nat. Mater. 15, 733–740 (2016).

20. 20.

Baek, S.-H. et al. Evidence for a field-induced quantum spin liquid in α-RuCl3. Phys. Rev. Lett. 119, 037201 (2017).

21. 21.

Janša, N. et al. Observation of two types of fractional excitation in the Kitaev honeycomb magnet. Nat. Phys. 14, 786–790 (2018).

22. 22.

Wulferding, D. et al. Magnon bound states versus anyonic Majorana excitations in the Kitaev honeycomb magnet α-RuCl3. Nat. Commun. 11, 1603 (2020).

23. 23.

Ponomaryov, A. N. et al. Nature of magnetic excitations in the high-field phase of α-RuCl3. Phys. Rev. Lett. 125, 037202 (2020).

24. 24.

Little, A. et al. Antiferromagnetic resonance and terahertz continuum in α-RuCl3. Phys. Rev. Lett. 119, 227201 (2017).

25. 25.

Wang, Z. et al. Magnetic excitations and continuum of a possibly field-induced quantum spin liquid in α-RuCl3. Phys. Rev. Lett. 119, 227202 (2017).

26. 26.

Leahy, I. A. et al. Anomalous thermal conductivity and magnetic torque response in the honeycomb magnet α-RuCl3. Phys. Rev. Lett. 118, 187203 (2017).

27. 27.

Modic, K. A. et al. Scale-invariant magnetic anisotropy in RuCl3 at high magnetic fields. Nat. Phys. 17, 240–244 (2021).

28. 28.

Kasahara, Y. et al. Unusual thermal Hall effect in a Kitaev spin liquid candidate α-RuCl3. Phys. Rev. Lett. 120, 217205 (2018).

29. 29.

Kasahara, Y. et al. Majorana quantization and half-integer thermal quantum Hall effect in a Kitaev spin liquid. Nature 559, 227–231 (2018).

30. 30.

Yokoi, T. et al. “Half-integer quantized anomalous thermal Hall effect in the Kitaev material α-RuCl3,” http://arxiv.org/abs/2001.01899 (2020).

31. 31.

Yamashita, M., Gouchi, J., Uwatoko, Y., Kurita, N. & Tanaka, H. Sample dependence of half-integer quantized thermal Hall effect in the Kitaev spin-liquid candidate α-RuCl3. Phys. Rev. B 102, 220404 (2020).

32. 32.

Winter, S. M., Li, Y., Jeschke, H. O. & Valentí, R. Challenges in design of Kitaev materials: Magnetic interactions from competing energy scales. Phys. Rev. B 93, 214431 (2016).

33. 33.

Winter, S. M. et al. Breakdown of magnons in a strongly spin-orbital coupled magnet. Nat. Commun. 8, 1152 (2017).

34. 34.

Wu, L. et al. Field evolution of magnons in α-RuCl3 by high-resolution polarized terahertz spectroscopy. Phys. Rev. B 98, 094425 (2018).

35. 35.

Cookmeyer, J. & Moore, J. E. Spin-wave analysis of the low-temperature thermal Hall effect in the candidate Kitaev spin liquid α-RuCl3. Phys. Rev. B 98, 060412 (2018).

36. 36.

Kim, H.-S. & Kee, H.-Y. Crystal structure and magnetism in α-RuCl3: An ab initio study. Phys. Rev. B 93, 155143 (2016).

37. 37.

Suzuki, T. & Suga, S.-i Effective model with strong Kitaev interactions for α-RuCl3. Phys. Rev. B 97, 134424 (2018).

38. 38.

Suzuki, T. & Suga, S.-i Erratum: Effective model with strong Kitaev interactions for α-RuCl3 [Phys. Rev. B 97, 134424 (2018)]. Phys. Rev. B 99, 249902 (2019).

39. 39.

Ran, K. et al. Spin-wave excitations evidencing the Kitaev interaction in single crystalline α-RuCl3. Phys. Rev. Lett. 118, 107203 (2017).

40. 40.

Wang, W., Dong, Z.-Y., Yu, S.-L. & Li, J.-X. Theoretical investigation of magnetic dynamics in α-RuCl3. Phys. Rev. B 96, 115103 (2017).

41. 41.

Ozel, I. O. et al. Magnetic field-dependent low-energy magnon dynamics in α-RuCl3. Phys. Rev. B 100, 085108 (2019).

42. 42.

Kim, H.-S., Shankar, V. V., Catuneanu, A. & Kee, H.-Y. Kitaev magnetism in honeycomb RuCl3 with intermediate spin-orbit coupling. Phys. Rev. B 91, 241110 (2015).

43. 43.

Laurell, P. & Okamoto, S. Dynamical and thermal magnetic properties of the Kitaev spin liquid candidate α-RuCl3. npj Quantum Mater. 5, 2 (2020).

44. 44.

Chen, B.-B., Chen, L., Chen, Z., Li, W. & Weichselbaum, A. Exponential thermal tensor network approach for quantum lattice models. Phys. Rev. X 8, 031082 (2018).

45. 45.

Li, H. et al. Thermal tensor renormalization group simulations of square-lattice quantum spin models. Phys. Rev. B 100, 045110 (2019).

46. 46.

Imajo, S., Dong, C., Matsuo, A., Kindo, K. and Kohama, Y. "High-resolution calorimetry in pulsed magnetic fields,” http://arxiv.org/abs/2012.02411 (2020).

47. 47.

Zhou, X.-G. et al. "Particle-hole symmetry breaking in a spin-dimer system TlCuCl3 observed at 100 T”, http://arxiv.org/abs/2009.11028 (2020).

48. 48.

Zhou, X.-G. & Matsuda, Y. H. priviate communication.

49. 49.

Gordon, J. S., Catuneanu, A., Sørensen, E. S. & Kee, H.-Y. Theory of the field-revealed Kitaev spin liquid. Nat. Commun. 10, 2470 (2019).

50. 50.

Lee, H.-Y. et al. Magnetic field induced quantum phases in a tensor network study of Kitaev magnets. Nat. Commun. 11, 1639 (2020).

51. 51.

Maksimov, P. A., & Chernyshev, A. L. Rethinking α-RuCl3. Phys. Rev. Research 2, 033011 (2020).

52. 52.

Widmann, S. et al. Thermodynamic evidence of fractionalized excitations in α-RuCl3. Phys. Rev. B 99, 094415 (2019).

53. 53.

Lampen-Kelley, P. et al. Anisotropic susceptibilities in the honeycomb Kitaev system α-RuCl3. Phys. Rev. B 98, 100403 (2018).

54. 54.

Weber, D. et al. Magnetic properties of restacked 2D Spin 1/2 honeycomb RuCl3 nanosheets. Nano Lett. 16, 3578–3584 (2016).

55. 55.

Johnson, R. D. et al. Monoclinic crystal structure of α-RuCl3 and the zigzag antiferromagnetic ground state. Phys. Rev. B 92, 235119 (2015).

56. 56.

Yu, S., Gao, Y., Chen, B.-B. and Li, W. "Learning the effective spin Hamiltonian of a quantum magnet”, http://arxiv.org/abs/2011.12282 (2020).

57. 57.

Eichstaedt, C. et al. Deriving models for the Kitaev spin-liquid candidate material α-RuCl3 from first principles. Phys. Rev. B 100, 075110 (2019).

58. 58.

Nasu, J., Udagawa, M. & Motome, Y. Thermal fractionalization of quantum spins in a Kitaev model: Temperature-linear specific heat and coherent transport of majorana fermions. Phys. Rev. B 92, 115122 (2015).

59. 59.

Li, H. et al. Universal thermodynamics in the Kitaev fractional liquid. Phys. Rev. Res. 2, 043015 (2020).

60. 60.

Janssen, L., Andrade, E. C. & Vojta, M. Magnetization processes of zigzag states on the honeycomb lattice: Identifying spin models for α-RuCl3 and Na2IrO3. Phys. Rev. B 96, 064430 (2017).

61. 61.

Gass, S. et al. Field-induced transitions in the Kitaev material α-RuCl3 probed by thermal expansion and magnetostriction. Phys. Rev. B 101, 245158 (2020).

62. 62.

Bachus, S. et al. Thermodynamic perspective on field-induced behavior of α-RuCl3. Phys. Rev. Lett. 125, 097203 (2020).

63. 63.

Balz, C. et al. Finite field regime for a quantum spin liquid in α-RuCl3. Phys. Rev. B 100, 060405 (2019).

64. 64.

Hentrich, R. et al. Large thermal Hall effect in α-RuCl3: evidence for heat transport by Kitaev-Heisenberg paramagnons. Phys. Rev. B 99, 085136 (2019).

65. 65.

Kasahara, Y. et al. Majorana quantization and half-integer thermal quantum Hall effect in a Kitaev spin liquid. Nature 559, 227–231 (2018).

66. 66.

Czajka, P. et al. "Oscillations of the thermal conductivity observed in the spin-liquid state of α-RuCl3,” https://doi.org/10.1038/s41567-021-01243-x Nat. Phys. (2021).

67. 67.

Hickey, C. & Trebst, S. Emergence of a field-driven U(1) spin liquid in the Kitaev honeycomb model. Nat. Commun. 10, 530 (2019).

68. 68.

Jiang, Y.-F., Devereaux, T. P. & Jiang, H.-C. Field-induced quantum spin liquid in the Kitaev-Heisenberg model and its relation to α-RuCl3. Phys. Rev. B 100, 165123 (2019).

69. 69.

Zhu, Z., Kimchi, I., Sheng, D. N. & Fu, L. Robust non-Abelian spin liquid and a possible intermediate phase in the antiferromagnetic Kitaev model with magnetic field. Phys. Rev. B 97, 241110 (2018).

70. 70.

Patel, N. D. & Trivedi, N. Magnetic field-induced intermediate quantum spin liquid with a spinon Fermi surface. PNAS 116, 12199–12203 (2019).

71. 71.

Chaloupka, J. & Khaliullin, G. Hidden symmetries of the extended Kitaev-Heisenberg model: Implications for the honeycomb-lattice iridates A2IrO3. Phys. Rev. B 92, 024413 (2015).

72. 72.

Jiang, H.-C., Gu, Z.-C., Qi, X.-L. & Trebst, S. Possible proximity of the Mott insulating iridate Na2IrO3 to a topological phase: Phase diagram of the Heisenberg-Kitaev model in a magnetic field. Phys. Rev. B 83, 245104 (2011).

73. 73.

Yao, W. & Li, Y. Ferrimagnetism and anisotropic phase tunability by magnetic fields in Na2Co2TeO6. Phys. Rev. B 101, 085120 (2020).

74. 74.

Hong, X. et al. “Unusual heat transport of the Kitaev material Na2Co2TeO6: putative quantum spin liquid and low-energy spin excitations,” http://arxiv.org/abs/2101.12199 (2021).

75. 75.

Lin, G. et al. “Field-induced quantum spin disordered state in spin-1/2 honeycomb magnet Na2Co2TeO6 with small Kitaev interaction,” http://arxiv.org/abs/2012.00940 (2020).

76. 76.

Songvilay, M. et al. Kitaev interactions in the Co honeycomb antiferromagnets Na3Co2SbO6 and Na2Co2TeO6. Phys. Rev. B 102, 224429 (2020).

77. 77.

Sun, R. Y. and Peng, C. “GraceQ/MPS2: A high-performance matrix product state (MPS) algorithms library based on GraceQ/tensor,” https://github.com/gracequantum/MPS2.

78. 78.

Cromer, D. T. & Waber, J. T. Scattering factors computed from relativistic Dirac–Slater wave functions. Acta Cryst. 18, 104–109 (1965).

## Acknowledgements

We are indebted to Yang Qi, Xu-Guang Zhou, Yasuhiro Matsuda, Wentao Jin, Weiqiang Yu, Jinsheng Wen, Kejing Ran, Yanyan Shangguan and Hong Yao for helpful discussions. This work was supported by the National Natural Science Foundation of China (Grant Nos. 11834014, 11974036, 11974421, 11804401), Ministry of Science and Technology of China (Grant No. 2016YFA0300504), and the Fundamental Research Funds for the Central Universities (BeihangU-ZG216S2113, SYSU-2021qntd27). We thank the High-Performance Computing Cluster of Institute of Theoretical Physics-CAS for the technical support and generous allocation of CPU time.

## Author information

Authors

### Contributions

W.L., S.S.G. and Z.X.L initiated this work. H.L. and D.W.Q. performed the thermal tensor network calculations, H.L. obtained the Hamiltonian parameters by fitting experiments, Y.G. confirmed the model parameters with Bayesian searching, H.K.Z. (DMRG) and J.C.W. (VMC) performed the ground state simulations, and H.Q.W. computed the dynamical correlations by ED. All authors contributed to the analysis of the results and the preparation of the draft. S.S.G., Z.X.L., and W.L. supervised the project.

### Corresponding authors

Correspondence to Zheng-Xin Liu, Shou-Shu Gong or Wei Li.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Peer review informationNature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Li, H., Zhang, HK., Wang, J. et al. Identification of magnetic interactions and high-field quantum spin liquid in α-RuCl3. Nat Commun 12, 4007 (2021). https://doi.org/10.1038/s41467-021-24257-8

• Accepted:

• Published: