Abstract
Frustrated magnets hold the promise of material realizations of exotic phases of quantum matter, but direct comparisons of unbiased model calculations with experimental measurements remain very challenging. Here we design and implement a protocol of employing manybody computation methodologies for accurate model calculations—of both equilibrium and dynamical properties—for a frustrated rareearth magnet TmMgGaO_{4} (TMGO), which explains the corresponding experimental findings. Our results confirm TMGO is an ideal realization of triangularlattice Ising model with an intrinsic transverse field. The magnetic order of TMGO is predicted to melt through two successive Kosterlitz–Thouless (KT) phase transitions, with a floating KT phase in between. The dynamical spectra calculated suggest remnant images of a vanishing magnetic stripe order that represent vortex–antivortex pairs, resembling rotons in a superfluid helium film. TMGO therefore constitutes a rare quantum magnet for realizing KT physics, and we further propose experimental detection of its intriguing properties.
Introduction
Kosterlitz–Thouless (KT) physics bestows interesting mechanism of phase transition upon twodimension (2D) interacting system with a continuous symmetry. Although such symmetry is not allowed to break spontaneously at any finite temperature^{1}, phase transition can still take place from the hightemperature disordered phase to a KT phase with quasilongrange order, which has a topological root in the binding of the vortex and antivortex pair^{2,3}. Experimentally, the KT transition has been observed in thin helium films^{4} and ultracold 2D Bose gases^{5,6}, etc. Two distinct types of elementary excitations, i.e., phonons and rotons, play essential roles in the related superfluid phenomena^{7,8}, and they are important for understanding liquid helium thermodynamics^{9}. Besides interacting bosons in liquid and gas, there are also theoretical proposals of KT transitions in solidstate magnetic systems such as the 2D classical XY^{2,3} and the frustrated quantum Ising models^{10}. However, to date, the material realization of the KT transition in 2D magnets has rarely been reported.
In the mean time, the search of exotic quantum magnetic states in the triangularlattice spin systems—the motif of frustrated magnets—has attracted great attention over the decades. Experimentally, the triangularlattice quantum magnets have been synthesized only very recently, including compounds Ba_{3}CoSb_{2}O_{9}^{11,12,13,14}, Ba_{8}CoNb_{6}O_{24}^{15,16}, and a rareearth oxide YbMgGaO_{4}—which has been suggested as a quantum spin liquid candidate^{17,18,19,20}—whereas an alternative scenario of glassy and disorderinduced state has also been proposed recently^{21,22,23}. On the other hand, an Isingtype triangular antiferromagnet TmMgGaO_{4} (TMGO, with Yb^{3+} replaced by another rareearth ion Tm^{3+})^{24,25,26}, as shown in Fig. 1a and explained in details in this study, is the successful material realization of a quantum magnet with strong Ising anisotropy.
In this work, we construct the microscopic model of TMGO and employ two stateoftheart quantum manybody simulation approaches: the exponential tensor renormalization group (XTRG)^{27} and quantum Monte Carlo (QMC) equipped with stochastic analytic continuation (QMCSAC)^{28,29,30,31,32}, to calculate both the thermodynamic and dynamic properties. By scanning various parameters and fit our simulation results to the existing experimental data^{24,25,26}, we find TMGO realizes a triangularlattice transversefield Ising model and determine accurately its model parameters. Based on this, we conclude that TMGO should host the celebrated KT phase and further predict several prominent features to be observed in TMGO, inspired by the experimental measurements for detecting KT physics in a superfluid thin film^{4}. It is worthwhile to point out that our calculation of quantum fluctuations goes beyond the linear spinwave approximation in ref. ^{26} and puts the system in the clockordered (later melted through KT transitions), rather than disordered, regime. Therefore, our methods and results do not only explain the experimental findings but, more importantly, establish a protocol for acquiring equilibrium and dynamic experiments of strongly correlated quantum materials, such as TMGO, in an unbiased manner.
Results
Microscopic spin model
Due to strong spin–orbit coupling and crystal electric field splitting, TMGO can be described as an effective spin1/2 model with strong easyaxis anisotropy, i.e., a triangularlattice Ising model (TLI), as shown in Fig. 1a. Firstprinciple calculations of the TMGO material are performed based on the density functional theory (DFT)^{33,34,35,36} where we see a large easyaxis anisotropy of roomtemperature energy scale. By comparing the DFT energies of antiferromagnetic spin configuration with a ferromagnetic one, one finds the former has a lower energy and the coupling strength can be estimated as a few tenths of meV. Density distribution of 4f electrons in TMGO can also be obtained, where it is observed that the 4f electrons of Tm^{3+} are coupled via superexchange mediated by 2p electrons of O^{2−} within the triangularlattice plane (Supplementary Note 1).
In ref. ^{25}, the authors took the lowest two levels \(\left{\Phi }_{\pm }\right\rangle\) of Tm^{3+} as nonKramers doublet and construct a classical TLI with both nearestneighbor (NN) and nextnearestneighbor (NNN) interactions to account for the absence of zeropoint entropy observed in experiments. Substantial randomness was also introduced to explain the smooth magnetization curves even at a very low temperature. Later on, inelastic neutron scattering (INS) results of TMGO reveal a clear magnon band^{26}, suggesting the influence of coupling randomness should be modest in TMGO and an adequate modeling of the material shall include in the Hamiltonian noncommuting terms with quantum fluctuations. As the Kramers theorem is absent in Tm^{3+} system with total angular momentum J = 6, a small level splitting Δ between the quasidoublet \(\left{\Phi }_{\!\pm }\!\right\rangle\) is involved, as shown in Fig. 1a. Therefore, a quantum TLI model was proposed^{26}, with spin1/2 Hamiltonian
where 〈,〉\(\left(\langle \langle ,\rangle \rangle \right)\) stands for NN (NNN) couplings J_{1} (J_{2}), Δ the energy splitting between \(\left{\Phi }_{\!\pm }\!\right\rangle\) (i.e., the intrinsic transverse field), and h is the external magnetic field. g_{∥} = 2Jg_{J} constitutes the effective spin1/2 g factor, with g_{J} the Landé factor.
The phase diagram of quantum TLI has been studied intensively with analytic and numeric methods in the past^{10,37,38,39}, and is schematically shown in Fig. 1b. We indicate the TMGO model parameter with the vertical arrow (the determination of parameters is given below). From high to low temperatures, the system first goes through an upper KT transition at T_{h} from the paramagnetic phase to a KT phase with powerlaw (algebraic) spin correlations. At a lower temperature T_{l}, the system enters the clock phase with a true longrange order depicted in Fig. 1a. This threesublattice clock order breaks the discrete lattice point group and the Z_{2} spin symmetries, giving rise to a low but finite transition temperature T_{l}.
Increasing the nextnearestneighbor coupling J_{2}, say, at J_{2}/J_{1} = 0.2, we find the static magnetic structure factor develops a stripe order^{40} with structure factor peak at M point (see, e.g., Fig. 4a) of the Brillouin zone (BZ) (see Supplementary Notes 2 and 3). This magnetic stripe order, as shown in Fig. 1c, has been observed previously in TLI material AgNiO_{2}, where J_{2} coupling is relatively strong (~0.15J_{1}, along with other interactions) and the exotic KT physics is absent there^{41}. In TMGO, however, the clock order wins over the stripe order as J_{2} ∕J_{1} ≃ 0.05 is relatively small in this material. Nevertheless, as will be shown below, a ghost of the stripe order—the M rotonlike modes—remains in the spin spectrum^{26}, which turns out to be related to a vortex–antivortex pair excitation in the topological language (see Discussion section).
Thermodynamics and parameter fittings
The model parameters in Eq. (1) can be accurately determined through fitting the available experimental data of TMGO^{24,25,26}, from which we find J_{1} = 0.99 meV, J_{2} = 0.05J_{1}, Δ = 0.54J_{1} and g_{∥} = 13.212. We present in Fig. 2 the calculated thermodynamic quantities and their experimental counterparts^{25,26}, where excellent agreements are seen. In Fig. 2a, at T > 30 K, the magnetic entropy S_{m} approaches \(R\mathrm{ln}\,2\), corresponding to the hightemperature paramagnetic phase with effective spin1/2. As temperature decreases, S_{m} gradually releases throughout the intermediatetemperature regime and approaches zero below \({T}_{{\rm{l}}}^{* }\simeq 1\) K, as the longrange clock order develops. In Fig. 2b, the very good agreement in magneticspecific heat C_{m} extends from high T (~30 K) all the way down to low temperature T ~ 1 K.
In the fittings of the magnetic entropy and specific heat curves in Fig. 2a, b, we rescale the Taxes (in the unit of J_{1}) to lay the model calculations on top of the experimental measurements, and in this way we find the optimized J_{1} = 0.99 meV. In both plots, the yaxes scaling ratios are associated with the ideal gas constant R = 8.313 J mol^{−1} K^{−1} and thus fixed. In Fig. 2c, we fit the magnetic susceptibility \(\chi (h)=\frac{M(h)}{h}\), with M(h) the uniform magnetization (per Tm^{3+}), under external fields of a small value h = 1 kOe and a larger one 10 kOe. As shown in Fig. 2c, by setting the effective Landé factor g_{∥} = 13.212, we fit both susceptibility curves very well. This completes the model parameters in the Hamiltonian Eq. (1) (see more fitting details in Supplementary Note 4) and note this parameter set also leads to accurate entropy results at 5 kOe when put in a direct comparison with the experimental data in Fig. 2a.
With the parameters J_{1}, J_{2}, Δ, and g_{∥} determined from the above fittings, we can compute the magnetization curve M(h) and compare it directly with several independently measured experimental curves in Fig. 2d. It is noteworthy that there exists a turning point at about 1/3 magnetization, under a magnetic field around 20–25 kOe. It becomes clearer as T decreases further down to 40 mK (see the inset of Fig. 2d), suggesting the existence of fieldinduced quantumphase transition in TMGO. This sharp change of behaviors can also be witnessed in the specific heat curves under various magnetic fields (see Supplementary Note 6), which can also be understood very well within the set of parameters obtained above.
Lastly, we briefly discuss the scalings in the uniform susceptibility χ in Figs. 2c and 3, where susceptibility curves under more external fields are computed theoretically in the latter plot as a complementary. These χ data reflect the twostep establishment of magnetic order as T lowers. As pointed out in refs. ^{42,43}, a universal scaling χ(h) = h^{−α} appears for small fields h with \(\alpha =\frac{4 \,  \, 18\eta (T)}{4\,  \, 9\eta (T)}\) in the KT regime, where \(\eta (T)\in [\frac{1}{9},\frac{2}{9}]\) is the anomalous dimension exponent of the emergent XYorder parameter varying with temperature. For T = T_{l}, \(\eta =\frac{1}{9}\) and χ(h) ~ h^{−2/3}, which diverges as h approaches zero, whereas above some higher temperature \(T^* _h\), \(\eta =\frac{2}{9}\) and χ(h) remains a constant vs. h. Therefore, at small external field, the increase of χ at intermediate T reflects the decrease of η(T) vs. T and such enhancement becomes less prominent for a relatively larger field, say, h = 10 kOe. This salient difference indeed can be noticed in the experimental and our numerical curves in Fig. 2c. Moreover, in Fig. 3 we plot in the inset susceptibility curves under various magnetic fields h between 6.5 kOe and 9 kOe, where the powerlaw scaling is shown explicitly and the anomalous exponent η can be extracted therein.
Spin spectra and magnetic structure factors
Spin frustration can lead to strong renormalization effects, which in turn gives rise to interesting spectrum in dynamics. Here we employ the QMCSAC approach^{28,29,30,31,32} to compute the spin spectra S(q, ω) from S^{z} spin correlations, at various temperatures (see Methods). The obtained spectra, with model parameters determined from equilibrium data fittings, are plotted in Fig. 4 and are compared directly with the INS results^{26}. Figure 4a depicts the spin spectrum inside the clock phase, at a low temperature T = 0.5 K. As the clock phase is of discrete symmetry breaking, the S(q, ω = 0) at K point signals the Bragg peak of the clock order and there exists a small gap ~0.1 meV between the ω = 0 and finite ω spectra, consistent with the INS result. The rotonlike modes are also clearly present in the QMCSAC results, with an energy gap about 0.4 meV, in quantitative agreement with that in ref. ^{26}.
Figure 4b is the QMCSAC spectrum calculated according to the parameters (Δ/J_{1} ≃ 1.36, J_{2}/J_{1} ≃ 0.046) given in ref. ^{26}. As mentioned in the introduction, we find, via spin structure factor calculations, that such set of parameters actually put the model in the disordered paramagnetic phase with Δ > Δ_{c} ~ 0.8J_{1}. It is possible that the fitting scheme adopted in ref. ^{26} is based on meanfield treatment and cannot capture the quantum fluctuations inherent to the quantum TLI model and the material TMGO. This is a clear sign that the unbiased quantum manybody calculation scheme in our work is the adequate approach to explain the experimental results.
We continue with the correct parameter set and rise the temperature to T = 1.45 K in Fig. 4c. It is interesting to see that the dispersion still resembles that in the clock phase of Fig. 4a but with a vanishing gap at the K point and softened M roton modes. To show it more clearly, we plot the intensity at M in Fig. 4d, where the roton gap gets reduced as T increases, with substantially broadened linewidths. As M rotonlike excitations can be related to vortexpair excitation (see Discussions), this softening of M roton is consistent with the scenario of vortex proliferations near the upper KT transition. Such remarkable spectra constitute a nontrivial prediction to be confirmed in future INS experiments.
Besides, the static magnetic structure factor \(S({\bf{q}})=\sum _{i,j}\) e\(^{i{\bf{q}}\cdot ({{\bf{r}}}_{i}{{\bf{r}}}_{j})}\langle {S}_{i}^{z}\ {S}_{j}^{z}\rangle\) are also simulated, where r_{i} and r_{j} run throughout the lattice. Figure 5a shows the temperature dependence of S_{K} and S_{M}, where one observes an enhancement of S_{M} at intermediate temperature, signifying its closeness towards the stripe order. At \(T \, < \, {T}_{{\rm{l}}}^{* }\), the enhancement of S_{M} vanishes and instead the S_{K} intensity becomes fully dominant. Figure 5b–d show the S(q) results at low (T = 0.57 K), intermediate (T = 2.2 K), and high temperaure (T = 4.5 K). In the clock phase, S(q) evidently peaks at the K point, the ordering wavevector of the threesublattice clock phase, whereas in the intermediate temperature regime, notably there exists a “ghost” peak at the M point, manifesting the existence of shortrange stripe order selected by thermal fluctuations. These interesting features are gone at higher T = 4.5 K, where strong fluctuations considerably weaken the structure factor peaks.
Discussion
As postulated by Landau, the phonon–roton spectrum play an essential role in understanding lowT thermodynamics and critical velocities of superfluidity, etc, in liquid helium, where the rotons are believed to be related to local vorticity of the fluid^{7}. Roton constitutes a minima in the spectrum at finite momentum and energy, and has been regarded as a quantum analog of hydrodynamic vortex ring, as coined “the ghost of vanished vortex ring” by Onsager^{44}. On the other hand, as derived from a trial wavefunction introduced by Feynman^{8}, the roton excitation has energy ϵ_{q} ~ ℏq^{2}/[2m S(q)], with m is the helium atom mass and S(q) the structure factor peak, and it is therefore also associated with an incipient crystalization Bragg peak competing with superfluidity^{45}. Rotonlike excitations are also found in thin helium films^{46} and frustrated triangular Heisenberg (TLH) magnets. In the latter case, M point rotonlike modes were predicted theoretically^{47,48,49,50} and confirmed in recent experiments^{13,14}, whose nature is under ongoing investigations^{51,52}. Notably, it has been proposed that the M rotons in TLH can be softened by further enhancing spin frustration (and thus quantum fluctuations)^{52} or thermal fluctuations^{53}, which melts the longrange or incipient semiclassical 120° order, driving the system into liquidlike spin states.
In the frustrated magnet TMGO, as mentioned earlier in Fig. 4, there exists rotonlike modes with large density of states, which becomes softened even at low temperatures, melting the clock order, and strongly influences thermodynamics of the system. Similar to that in liquid helium, roton in TMGO has also a topological origin: we demonstrate below that the rotonlike modes represent bound states of topological vortex excitations, via a pseudospin mapping of the spin stripe order. As shown in Fig. 1c, the stripe order constitutes a proximate competing order to the clock state. Although it eventually gets perished in TMGO at low T, the stripe order leaves a “ghost image” in the excitation spectrum, i.e., rotonlike dip along the ΓMΓ path in Fig. 4a. Correspondingly, there exists an incipient S_{M} peak in the static structure factor at intermediate T (Fig. 5c), i.e., ϵ_{q} ~ 1/S(q), similar to rotons in the superfluid helium discussed above^{8,45}.
On top of the spin stripe order, the M rotonlike excitation can be related to a locally bounded vortex pair, some form of “rotational motion” happening in TMGO. We perform a pseudospin mapping
where ψ = ∣ψ∣e^{iθ} is the complex order parameter^{37}, i.e., the pseudospin. As shown in Fig. 1c, ψ is located in the center of each triangle, with emergent XY degree of freedom θ. In Eq. (2), \({m}_{\gamma }^{z}=\pm \! 1/2\) represents the spinup(down) of corresponding spin S^{z} components at γsublattice (γ = A, B, C). This mapping helps establishing a Landau–Ginzburg theory of TLI^{10,37} and the clock order shown in Fig. 1a corresponds to a ferromagnetic order of pseudo spins.
As shown in Fig. 1c, we create a vortex pair by applying S^{x} operator on two adjacent sites to simultaneously flip their spin orientations. We note that any closed loop enclosing only the vortex defect (red dot in Fig. 1c) leads to a winding number 1 (modulo 2π), whereas those around the antivortex (black dot) to −1. Zero winding number can be counted when a pair of defects (or no defects at all) are enclosed by the loop. Moreover, one can further move the vortex on the triangular lattice, such as in a “tightbinding” model, by flipping spins on further neighboring sites, which naturally leads to a quadratictype lowenergy dispersion near M point along ΓMΓ (see Supplementary Note 7). As the ghost peak in Fig. 5c only suggests a shortrange stripe correlation, the vortex pair can thus move only within a small cluster with incipient stripe order, i.e., they are bounded. The vortex pair only unbinds at the upper KT transition T_{h} where vortices are proliferated^{2,3}, as seen by the “softening” of M rotonlike mode in dynamical spectrum in Fig. 4.
To conclude, in this work, we have established a protocol of understanding and explaining experiments of frustrated magnets in an unbiased manner, with XTRG and QMCSAC machinery. The thermodynamic and dynamic results of TMGO are captured to great accuracy, thus allowing comprehensive studies of KT physics therein. At intermediate temperature, the KT phase of TMGO realizes a magnetic analog of 2D superfluid phase with several intriguing properties: (i) there emerges a spin XY symmetry and correspondingly complex order parameter ψ, which bears quasilongrange correlation and phase coherence; (ii) the finiteT spin spectrum contains the longwavelength magnon and competing gapped rotonlike modes near the BZ boundary with energy signifying the binding of vortex–antivortex pair, which plays a key role in determining finiteT phase diagram of the system; (iii) the quasilongrange XY order melts and the TMGO becomes paramagnetic as T is above T_{h}, driven by the proliferation of vortex excitations, in analogy to the superfluid transition in a helium thin film^{4,54}. We note that, different from the liquid helium, TMGO has twotemperature scales that outline the intermediatetemperature KT phase. Similar separation of scales has been seen in other quantum magnetic materials, by temperature^{53,55} or spatial dimensions^{56}, but the KT phase is the first time to be seen. Nevertheless, the \({T}_{{\rm{l}}}^{* },{T}_{{\rm{h}}}^{* }\) in the present work constitute tentative estimated of two KT transition temperatures roughly from thermodynamics, which still needs to be precisely determined both numerically and experimentally in the future.
The extraction of anomalous exponents η(T) constitutes another interesting future study. The exponent η(T) of the KT phase appears in the algebraic correlations in \(\langle {S}_{i}^{z}{S}_{j}^{z}\rangle \sim  {r}_{i}{r}_{j}{ }^{\eta (T)}\) and seems rather indirect to measure in solidstate experiements. Nevertheless, as discussed above in Fig. 3, η(T) can be determined from the uniform susceptibility χ, a routine magnetic quantity in experiments. Moreover, it would be interesting to check several distinct predictions of KT physics in this 2D magnetic material. One renowned phenomenon is the universal jump in superfluid density at the KT transition^{57}, as observed experimentally in helium film^{4}. Through calculations of the qclock model (q = 5, 6), people have revealed universal jumps in the spin stiffness at both upper and lower KT transitions^{58}, which certainly is an interesting prediction to check in TMGO. Dynamically, nuclear magnetic resonance measurements of relaxation time 1/T_{1} can be conducted, which probe signals of lowenergy magnetic dissipations at the KT transition where vortices proliferate. Besides, nonequilibrium thermodynamics such as thermal transport, would also be very worthwhile to explore in the TMGO magnet.
Methods
Quantum manybody computations
In this work, we combined two manybody numerical approaches: QMC^{37} and XTRG, the latter method is recently introduced based on matrix product operators (MPOs) and logarithmic temperature scales^{27}. XTRG is employed to simulate the TLI down to temperatures T < 0.5 K on YC W × L geometries up to width W = 9 with various lengths up to L = 12. Both dynamical and equilibrium properties are simulated, with the purpose of fitting the experimental data and obtaining the right parameters, as well as to make predictions for experiments. The QMC is performed in the spacetime lattice of L × L × L_{τ}, where L = 36 and L_{τ} = β/Δτ with Δτ = 0.05 and β ≡ 1/T. The spacetime configuration is written in the \({S}_{i,\tau }^{z}\) basis with both local and Wolffcluster updates to overcome the long autocorrelation time. As the QMC method is standard, we will only introduce the SAC scheme below and leave the QMC itself to the Supplementary Note 8.
Exponential thermal tensor network method
For the calculations of equilibrium properties, we start from a highT density matrix \(\hat{\rho }(\Delta \tau )={e}^{\Delta \tau H}\), whose MPO representation can be obtained conveniently and accurately (up to machine precision), at a small Δτ ~ 10^{−3~−4}. One way to obtain such an accurate MPO representation is to exploit the seriesexpansion thermal tensor network technique^{59} via the expansion
Given the \(\hat{\rho }(\Delta \tau )\) representation, traditionally one evolves \(\hat{\rho }(\beta )\) linear in β to reach various lower temperatures, i.e., β = L_{τ}Δτ increases by a small value Δτ after each step by multiplying \(\hat{\rho }(\Delta \tau )\) to the density matrix^{60}. However, this linear scheme is not optimal in certain aspects, and encounters challenges in generalization to 2D. Instead, recent study shows that the block entanglement entropy of MPO is bound by \({S}_{E}\le a\ \mathrm{ln}\,\beta +{\rm{const.}}\) at a conformal critical point, with a an universal coefficient proportional to the central charge^{27}. This suggests an exponential procedure of performing cooling procedure. Based on this idea, we have developed the XTRG method, which turns out to be highly efficient in simulating both onedimensional (1D) critical quantum chains and various 2D lattice systems^{27,53,61}.
In XTRG, we cool down the system by multiplying the thermal state by itself, i.e., ρ_{0} ≡ ρ(Δτ), ρ_{1} ≡ ρ_{0} ⋅ ρ_{0} = ρ(2Δτ), thus ρ_{n} ≡ ρ_{n−1} ⋅ ρ_{n−1} = ρ(2^{n}Δτ), and reach the lowT thermal states exponentially fast. Efficient compression of MPO bonds is then required to maintain the cooling procedure, where a truncation scheme optimizing the free energy and in the mean time maintaining the thermal entanglement, is involved. One advantage of XTRG is the convenience and high efficiency to deal with longrange interactions after the quasi1D mapping. For the TLI model with NN (J_{1}) and NNN (J_{2}) interactions considered in this work, we map the 2D lattice into a quasi1D geometry following a snakelike path. The Hamiltonian thus contains “longrange” interactions and has an efficient MPO representation with geometric bond dimension D_{H} = 2W + 2, with W the width of the lattice. In XTRG calculations, the computational costs scale with power O(D^{4}), with D the retained bond dimension in MPO, which is chosen as large as 500–600 in the present study, assuring accurate thermodynamical results down to subKelvin regime.
QMCSAC approach
We exploit the path integral QMC^{37}, equipped with SAC approach, to compute the dynamical properties. The time displaced correlated function, defined as G(τ) = 〈S^{z}(τ)S^{z}(0)〉, for a set of imaginary times τ_{i}(i = 0, 1, ⋯ , L_{τ}) with statistical errors can be obtained from QMC simulations. By SAC method^{28,29,31,32}, the corresponding realfrequency spectral function S(ω) can be obtained via \(S(\tau )={\int }_{\infty }^{\infty }d\omega S(\omega )K(\tau ,\omega )\), where the kernel K(τ, ω) depends on the type of the spectral function, i.e., fermionic or bosonic, finite or zero temperature. The spectra at positive and negative frequencies obey the relation of S(−ω) = e^{−βω}S(ω) and we are restricted at the positive frequencies and the kernel can therefore be written as \(K(\tau ,\omega )=\frac{1}{\pi }({e}^{\tau \omega }+{e}^{(\beta \tau )\omega })\). To work with a spectral function that is itself normalized to unity, we further modify the kernel and the spectral function, and arrive at the transformation between the imaginary time Green’s function G(q, τ) and realfrequency spectral function B(q, ω)
where B(q, ω) = S(q, ω)(1 + e^{−βω}).
In the practical calculation, we parametrize the B(q, ω) with a large number of equalamplitude δfunctions sampled at locations in a frequency continuum as \(B(\omega )=\sum _{i=0}^{{N}_{\omega }1}{a}_{i}\delta (\omega {\omega }_{i})\). Then the relationship between Green’s function obtained from Eq. (4) and from QMC can be described by the goodness of fit χ^{2}, i.e., \({\chi }^{2}=\sum _{i=1}^{{N}_{\tau }}\sum _{j=1}^{{N}_{\tau }}({G}_{i}{\overline{G}}_{i}){C}_{ij}^{1}({G}_{j}{\overline{G}}_{j})\), where \({\overline{G}}_{i}\) is the average of QMC measurement and C_{ij} is covariance matrix \({C}_{ij}=\frac{1}{{N}_{B}({N}_{B} \, \, 1)}\sum _{b=1}^{{N}_{B}}({G}_{i}^{b}{\overline{G}}_{i})({G}_{j}^{b}{\overline{G}}_{j})\), with N_{B} the number of bins. Then we update the series of δfunctions in a Metropolis process, from (a_{i}, ω_{i}) to \(({a}_{i}^{\prime},{\omega }_{i}^{\prime})\), to get a more probable configuration of B(q, ω). The weight for a given spectrum follows the Boltzmann distribution \(P(B)\propto \exp ({\chi }^{2}/2\Theta )\), with Θ a fictitious temperature chosen in an optimal way so as to give a statistically sound mean χ^{2}value, while still staying in the regime of significant fluctuations of the sampled spectra so that a smooth averaged spectral function is obtained. The resulting spectra will be collected as an ensemble average of the Metropolis process within the configurational space of {a_{i}, ω_{i}}, as detailed in refs. ^{28,29,31,32}.
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Code availability
All numerical codes in this paper are available upon request to the authors.
References
 1.
Mermin, N. D. & Wagner, H. Absence of ferromagnetism or antiferromagnetism in one or twodimensional isotropic Heisenberg models. Phys. Rev. Lett. 17, 1133–1136 (1966).
 2.
Kosterlitz, J. M. & Thouless, D. J. Ordering, metastability and phase transitions in twodimensional systems. J. Phys. C: Solid State Phys. 6, 1181–1203 (1973).
 3.
Kosterlitz, J. M. The critical properties of the twodimensional xy model. J. Phys. C: Solid State Phys. 7, 1046–1060 (1974).
 4.
Bishop, D. J. & Reppy, J. D. Study of the superfluid transition in twodimensional ^{4}He films. Phys. Rev. Lett. 40, 1727–1730 (1978).
 5.
Hadzibabic, Z., Krüger, P., Cheneau, M., Battelier, B. & Dalibard, J. BerezinskiiKosterlitzThouless crossover in a trapped atomic gas. Nature 441, 1118–1121 (2006).
 6.
Desbuquois, R. et al. Superfluid behaviour of a twodimensional Bose gas. Nature Phys. 8, 645–648 (2012).
 7.
Landau, L. On the theory of superfluidity. Phys. Rev. 75, 884–885 (1949).
 8.
Feynman, R. P. Progress in Low Temperature Physics: Chapter II, Application of Quantum Mechanics to Liquid Helium (ed Gorter, C. G.) Vol. 1 (North Holland, 1955).
 9.
Bendt, P. J., Cowan, R. D. & Yarnell, J. L. Excitations in liquid helium: thermodynamic calculations. Phys. Rev. 113, 1386–1395 (1959).
 10.
Isakov, S. V. & Moessner, R. Interplay of quantum and thermal fluctuations in a frustrated magnet. Phys. Rev. B 68, 104409 (2003).
 11.
Zhou, H. D. et al. Successive phase transitions and extended spinexcitation continuum in the \(S\, {\boldsymbol{=}}\,\frac{1}{2}\) triangularlattice antiferromagnet Ba_{3}CoSb_{2}O_{9}. Phys. Rev. Lett. 109, 267206 (2012).
 12.
Susuki, T. et al. Magnetization process and collective excitations in the S = 1/2 triangularlattice Heisenberg antiferromagnet Ba_{3}CoSb_{2}O_{9}. Phys. Rev. Lett. 110, 267201 (2013).
 13.
Ma, J. et al. Static and dynamical properties of the spin1/2 equilateral triangularlattice antiferromagnet Ba_{3}CoSb_{2}O_{9}. Phys. Rev. Lett. 116, 087201 (2016).
 14.
Ito, S. et al. Structure of the magnetic excitations in the spin1/2 triangularlattice Heisenberg antiferromagnet Ba_{3}CoSb_{2}O_{9}. Nat. Commun. 8, 235 (2017).
 15.
Rawl, R. et al. Ba_{8}CoNb_{6}O_{24}: a spin\(\frac{1}{2}\) triangularlattice Heisenberg antiferromagnet in the twodimensional limit. Phys. Rev. B 95, 060412(R) (2017).
 16.
Cui, Y. et al. MerminWagner physics, (H,T) phase diagram, and candidate quantum spinliquid phase in the spin\(\frac{1}{2}\) triangularlattice antiferromagnet Ba_{8}CoNb_{6}O_{24}. Phys. Rev. Mater. 2, 044403 (2018).
 17.
Li, Y. et al. Rareearth triangular lattice spin liquid: a singlecrystal study of YbMgGaO_{4}. Phys. Rev. Lett. 115, 167203 (2015).
 18.
Li, Y. et al. Muon spin relaxation evidence for the U(1) quantum spinliquid ground state in the triangular antiferromagnet YbMgGaO_{4}. Phys. Rev. Lett. 117, 097201 (2016).
 19.
Shen, Y. et al. Evidence for a spinon Fermi surface in a triangularlattice quantumspinliquid candidate. Nature 540, 559–562 (2016).
 20.
Li, Y. et al. Rearrangement of uncorrelated valence bonds evidenced by lowenergy spin excitations in YbMgGaO_{4}. Phys. Rev. Lett. 122, 137201 (2019).
 21.
Zhu, Z., Maksimov, P. A., White, S. R. & Chernyshev, A. L. Disorderinduced mimicry of a spin liquid in YbMgGaO_{4}. Phys. Rev. Lett. 119, 157201 (2017).
 22.
Kimchi, I., Nahum, A. & Senthil, T. Valence bonds in random quantum magnets: Theory and application to YbMgGaO_{4}. Phys. Rev. X 8, 031028 (2018).
 23.
Ma, Z. et al. Spinglass ground state in a triangularlattice compound YbZnGaO_{4}. Phys. Rev. Lett. 120, 087201 (2018).
 24.
Cevallos, F. A., Stolze, K., Kong, T. & Cava, R. J. Anisotropic magnetic properties of the triangular plane lattice material TmMgGaO_{4}. Mater. Res. Bull. 105, 154–158 (2018).
 25.
Li, Y. et al. Partial upupdown order with the continuously distributed order parameter in the triangular antiferromagnet TmMgGaO_{4}. Phys. Rev. X 10, 011007 (2020).
 26.
Shen, Y. et al. Intertwined dipolar and multipolar order in the triangularlattice magnet TmMgGaO_{4}. Nat. Commun. 10, 4530 (2019).
 27.
Chen, B.B., Chen, L., Chen, Z., Li, W. & Weichselbaum, A. Exponential thermal tensor network approach for quantum lattice models. Phys. Rev. X 8, 031082 (2018).
 28.
Sandvik, A. W. Constrained sampling method for analytic continuation. Phys. Rev. E 94, 063308 (2016).
 29.
Qin, Y. Q., Normand, B., Sandvik, A. W. & Meng, Z. Y. Amplitude mode in threedimensional dimerized antiferromagnets. Phys. Rev. Lett. 118, 147207 (2017).
 30.
Shao, H., Qin, Y. Q., Capponi, S., Chesi, S., Meng, Z. Y. & Sandvik, A. W. Nearly deconfined spinon excitations in the squarelattice spin1∕2 Heisenberg antiferromagnet. Phys. Rev. X 7, 041072 (2017).
 31.
Huang, C.J., Deng, Y., Wan, Y. & Meng, Z. Y. Dynamics of topological excitations in a model quantum spin ice. Phys. Rev. Lett. 120, 167202 (2018).
 32.
Sun, G.Y. et al. Dynamical signature of symmetry fractionalization in frustrated magnets. Phys. Rev. Lett. 121, 077201 (2018).
 33.
Kresse, G. & Furthmüller, J. Efficient iterative schemes for ab initio totalenergy calculations using a planewave basis set. Phys. Rev. B 54, 11169–11186 (1996).
 34.
Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmentedwave method. Phys. Rev. B 59, 1758–1775 (1999).
 35.
Giannozzi, P. et al. QUANTUM ESPRESSO: a modular and opensource software project for quantum simulations of materials. J. Phys. Condens. Matter 21, 395502 (2009).
 36.
Giannozzi, P. et al. Advanced capabilities for materials modelling with Quantum ESPRESSO. J. Phys. Condens. Matter 29, 465901 (2017).
 37.
Wang, Y.C., Qi, Y., Chen, S. & Meng, Z. Y. Caution on emergent continuous symmetry: a Monte Carlo investigation of the transversefield frustrated Ising model on the triangular and honeycomb lattices. Phys. Rev. B 96, 115160 (2017).
 38.
Powalski, M., Coester, K., Moessner, R. & Schmidt, K. P. Disorder by disorder and flat bands in the kagome transverse field Ising model. Phys. Rev. B 87, 054404 (2013).
 39.
Fey, S., Kapfer, S. C. & Schmidt, K. P. Quantum criticality of twodimensional quantum magnets with longrange interactions. Phys. Rev. Lett. 122, 017203 (2019).
 40.
Metcalf, B. D. Ground state spin orderings of the triangular ising model with the nearest and next nearest neighbor interaction. Phys. Lett. A 46, 325–326 (1974).
 41.
Wheeler, E. M. et al. Spin dynamics of the frustrated easyaxis triangular antiferromagnet 2HAgNiO_{2} explored by inelastic neutron scattering. Phys. Rev. B 79, 104421 (2009).
 42.
Damle, K. Melting of threesublattice order in easyaxis antiferromagnets on triangular and kagome lattices. Phys. Rev. Lett. 115, 127204 (2015).
 43.
Biswas, S. & Damle, K. Singular ferromagnetic susceptibility of the transversefield Ising antiferromagnet on the triangular lattice. Phys. Rev. B 97, 085114 (2018).
 44.
Donnelly, R. J. In Quantum Statistical Mechanics in the Natural Sciences: A Volume Dedicated to Lars Onsager on the Occasion of his Seventieth Birthday (eds. Mintz, S. L. & Widmayer, S. M.) p. 359–402 (Springer US, Boston, MA, 1974).
 45.
Nozières, P. Is the roton in superfluid He^{4} the ghost of a Bragg spot? J. Low Temp. Phys. 137, 45–67 (2004).
 46.
Carneiro, K., Ellenson, W. D., Passell, L., McTague, J. P. & Taub, H. Neutronscattering study of the structure of adsorbed helium monolayers and of the excitation spectrum of fewatomiclayer superfluid films. Phys. Rev. Lett. 37, 1695–1698 (1976).
 47.
Starykh, O. A., Chubukov, A. V. & Abanov, A. G. Flat spinwave dispersion in a triangular antiferromagnet. Phys. Rev. B 74, 180403(R) (2006).
 48.
Chernyshev, A. L. & Zhitomirsky, M. E. Magnon decay in noncollinear quantum antiferromagnets. Phys. Rev. Lett. 97, 207202 (2006).
 49.
Zheng, W., Fjærestad, J. O., Singh, R. R. P., McKenzie, R. H. & Coldea, R. Anomalous excitation spectra of frustrated quantum antiferromagnets. Phys. Rev. Lett. 96, 057201 (2006).
 50.
Zheng, W., Fjærestad, J. O., Singh, R. R. P., McKenzie, R. H. & Coldea, R. Excitation spectra of the spin\(\frac{1}{2}\) triangularlattice Heisenberg antiferromagnet. Phys. Rev. B 74, 224420 (2006).
 51.
Verresen, R., Moessner, R. & Pollmann, F. Avoided quasiparticle decay from strong quantum interactions. Nat. Phys. 15, 750–753 (2019).
 52.
Ferrari, F. & Becca, F. Dynamical structure factor of the J _{1} − J _{2} Heisenberg model on the triangular lattice: magnons, spinons, and gauge fields. Phys. Rev. X 9, 031026 (2019).
 53.
Chen, L. et al. Twotemperature scales in the triangular lattice Heisenberg antiferromagnet. Phys. Rev. B 99, 140404(R) (2019).
 54.
Minnhagen, P. The twodimensional Coulomb gas, vortex unbinding, and superfluidsuperconducting films. Rev. Mod. Phys. 59, 1001–1066 (1987).
 55.
Banerjee, A. et al. Proximate kitaev quantum spin liquid behaviour in a honeycomb magnet. Nat. Mater. 15, 733–740 (2016).
 56.
Lake, B., Tennant, D. A., Frost, C. D. & Nagler, S. E. Quantum criticality and universal scaling of a quantum antiferromagnet. Nat. Mater. 4, 329–334 (2005).
 57.
Nelson, D. R. & Kosterlitz, J. M. Universal jump in the superfluid density of twodimensional superfluids. Phys. Rev. Lett. 39, 1201–1205 (1977).
 58.
Chatelain, C. DMRG study of the Berezinskii–Kosterlitz–Thouless transitions of the 2D fivestate clock model. J. Stat. Mech. P11022 (2014).
 59.
Chen, B.B., Liu, Y.J., Chen, Z. & Li, W. Seriesexpansion thermal tensor network approach for quantum lattice models. Phys. Rev. B 95, 161104(R) (2017).
 60.
Li, W. et al. Linearized tensor renormalization group algorithm for the calculation of thermodynamic properties of quantum lattice models. Phys. Rev. Lett. 106, 127202 (2011).
 61.
Li, H. et al. Thermal tensor renormalization group simulations of squarelattice quantum spin models. Phys. Rev. B 100, 045110 (2019).
Acknowledgements
We are indebted to Ziyu Chen, Wentao Jin, Andreas Weichselbaum, Yao Shen, and Yuesheng Li for stimulating discussions. This work was supported by the Ministry of Science and Technology of China through the National Key Research and Development Program (Grant Number 2016YFA0300502), the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant Number XDB28000000), the National Science Foundation of China (Grant Numbers 11421092, 11574359, 11674370, 11974036, 11874115, and 11834014), and Research Grants Council of Hong Kong Special Administrative Region of China through 17303019 and the Aspen Center for Physics, which is supported by National Science Foundation grant PHY1607611. B.B.C. was supported by the German Research foundation, DFG WE4819/31. W.L. was supported by the Fundamental Research Funds for the Central Universities. We thank the Center for Quantum Simulation Sciences at Institute of Physics, Chinese Academy of Sciences, the Computational Initiative at the Faculty of Science at the University of Hong Kong, the Tianhe1A platform at the National Supercomputer Center in Tianjin, and Tianhe2 platform at the National Supercomputer Center in Guangzhou for their technical support and generous allocation of CPU time.
Author information
Affiliations
Contributions
H.L. and Y.D.L. contributed equally to this work. W.L., Y.Q., and Z.Y.M. initiated the work. H.L. and B.B.C. performed the calculations of thermodynamics. X.T.Z. and X.L.S. performed firstprinciple calculations of the band structure and Y.D.L. performed the dynamical property simulations. All authors contributed to the analysis of the results. Z.Y.M. and W.L. supervised the project.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Communications thanks Arnab Banerjee, Frédéric Mila and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, H., Liao, Y.D., Chen, BB. et al. KosterlitzThouless melting of magnetic order in the triangular quantum Ising material TmMgGaO_{4}. Nat Commun 11, 1111 (2020). https://doi.org/10.1038/s41467020149078
Received:
Accepted:
Published:
Further reading

Effective pointcharge analysis of crystal fields: Application to rareearth pyrochlores and tripod kagome magnets R3Mg2Sb3O14
Physical Review Research (2021)

Quantum manybody simulations of the twodimensional FermiHubbard model in ultracold optical lattices
Physical Review B (2021)

Neutron scattering investigation of proposed KosterlitzThouless transitions in the triangularlattice Ising antiferromagnet TmMgGaO4
Physical Review B (2021)

Phase diagram of the quantum Ising model on a triangular lattice under external field
Physical Review B (2021)

Fermi arcs and pseudogap in a lattice model of a doped orthogonal metal
Physical Review B (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.