## Abstract

Predicting magnetic ordering in kagome compounds offers the possibility of harnessing topological or flat-band physical properties through tuning of the magnetism. Here, we examine the magnetic interactions and phases of ErMn_{6}Sn_{6} which belongs to a family of *R*Mn_{6}Sn_{6}, *R* = Sc, Y, Gd–Lu, compounds with magnetic kagome Mn layers, triangular *R* layers, and signatures of topological properties. Using results from single-crystal neutron diffraction and mean-field analysis, we find that ErMn_{6}Sn_{6} sits close to the critical boundary separating the spiral-magnetic and ferrimagnetic ordered states typical for non-magnetic versus magnetic *R* layers, respectively. Finding interlayer magnetic interactions and easy-plane Mn magnetic anisotropy consistent with other members of the family, we predict the existence of a number of temperature and field dependent collinear, noncollinear, and noncoplanar magnetic phases. We show that thermal fluctuations of the Er magnetic moment, which act to weaken the Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy, dictate magnetic phase stability. Our results provide a starting point and outline a multitude of possibilities for studying the behavior of Dirac fermions in *R*Mn_{6}Sn_{6} compounds with control of the Mn spin orientation and real-space spin chirality.

### Similar content being viewed by others

## Introduction

The kagome lattice has a two-dimensional corner-sharing triangular arrangement which supports frustrated electronic interactions. Its electronic band structure generally contains both flat bands and linear (Dirac-like) band crossings, both of which can lead to important correlated electronic phenomena; flat bands can give rise to itinerant magnetic correlations and Stoner-type magnetism, whereas linear band crossings create Dirac cones and the associated topological (Dirac) fermions^{1,2,3,4,5}. These latter features can lead to topological phenomena that intimately link charge and spin, such as dissipationless spin-momentum-locked (chiral) charge transport and quantum-anomalous-Hall effects^{6}.

Kagome compounds with competing interlayer magnetic interactions are particularly important as they can offer the ability to tune the electronic states of the kagome layers by manipulating the magnetism. Hexagonal *R*Mn_{6}Sn_{6} (*R*166) metals with *R* = Sc, Y, Gd–Lu, fall into this category. These compounds have emerged as model layered systems for magnetic tuning due to the combination of largely defect-free magnetic Mn kagome nets hosting topological electronic bands and *R*-site magnetism that controls details of the magnetic order^{4,7,8,9,10,11,12,13,14,15,16}. Crucial to harnessing the interplay between the magnetic order and topological properties is understanding the microscopic magnetic interactions determining the magnetic order and how to manipulate or influence the magnetic interactions to create a desired magnetic state. Here, we address these challenges via a single-crystal neutron diffraction and mean-field analysis study of ErMn_{6}Sn_{6}.

As shown in Fig. 1a for Er166, *R*166 compounds feature Mn kagome bilayers separated by triangular *R* layers. Importantly, the magnetic layers individually exhibit ferromagnetic (FM) ordering of their magnetic moments (spins), and a majority of *R*166 compounds with magnetic *R* ions exhibit collinear ferrimagnetic (FIM) order driven by strong antiferromagnetic (AFM) coupling between the Mn and *R* layers^{17}. For non-magnetic *R*, such as Y, long range and competing Mn-Mn interlayer couplings lead to spiral magnetic order^{9,18,19}, and applied magnetic fields lead to distorted and noncoplanar spin configurations where the topological-Hall effect and planar-anisotropic magnetoresistance have been observed^{9,10,20,21}. Thus, the strength of the Mn-*R* coupling controls the stability between collinear and noncollinear magnetism in the *R*166 compounds and we show in this work that Er166 sits close to the critical boundary between the two types of ordering.

Another key aspect of *R*166 compounds is that *R* ions with nonzero orbital angular momentum adopt complex magnetic anisotropy through the crystalline-electric-field (CEF) splitting of their 4*f* orbitals. Easy-plane Mn anisotropy is present, but strong *R* anisotropy will dictate the Mn magnetization direction at low temperature and zero magnetic field. For the FM Mn layers, a uniaxial anisotropy is proposed to dramatically enhance the spin-orbit splitting of Dirac cones, forming a Chern insulator^{7}. Further, the competition of *R* anisotropy with the easy-plane Mn anisotropy can result in temperature and field-driven spin-reorientation transitions between easy-plane and uniaxial (*R* = Tb) or tilted (*R* = Dy, Ho) FIM states. These transitions can facilitate magnetization switching via small changes in temperature or magnetic field^{14,22,23} which can create or destroy a Chern gap^{7} or Weyl nodes^{24}. We find that the Er^{3+} ion has weakly uniaxial and strongly temperature-dependent anisotropy in Er166, resulting in facile magnetic switching. Small magnetic fields of *μ*_{0}*H* < 1 T can trigger spin-reorientation transitions where large anomalous-Hall and topological-Hall responses have been reported^{25}.

Here, we present single-crystal neutron diffraction and mean-field analysis results that illustrate how competition between interlayer magnetic interactions and magnetic anisotropy in Er166 leads to a number of nearly degenerate magnetic states. Using this information, we characterize and understand the temperature-driven first-order transition from planar-FIM to a distorted-triple-spiral order upon warming through *T*_{spiral} = 92(1) K, showing that spins in the Er layers remain magnetically ordered above *T*_{spiral} rather than their previously reported paramagnetic behavior^{25}. Using additional results from inelastic neutron scattering and magnetization measurements, we determine the key microscopic magnetic interactions and predict the emergence of various collinear, noncollinear, and noncoplanar magnetic phases as functions of temperature and magnetic field. Importantly, we find that the stability of the magnetic phases is controlled by thermal fluctuations of the Er spins which act to weaken the effective Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy.

## Results and discussion

### Experimental results

The magnetization *M* of Er166 for a weak magnetic field of *μ*_{0}*H* = 0.005 T applied perpendicular to the crystalline *c* axis is shown in Fig. 2a and the zero-field neutron diffraction pattern for (0, 0, *l*) reciprocal-lattice points is shown in Fig. 2b for increasing temperature. The main features in both figures agree with previously published data^{22,26}; with cooling, a peak in [*M*/*H*](*T*) in Fig. 2a at the Néel temperature of *T*_{N} = 348(1) K coincides with the emergence of satellite magnetic-Bragg peaks in Fig. 2b surrounding the (0, 0, 2) structural-Bragg peak; a large jump in [*M*/*H*](*T*) at *T*_{spiral} accompanies the disappearance of the satellite peaks. The primary satellites [those satellite peaks closest to (0, 0, 2)] correspond to a temperature-dependent AFM propagation vector of ** τ** = (0, 0,

*τ*) which indicates the presence of magnetic ordering with a temperature-dependent structure that is modulated along

**c**. Analysis of satellites for different values of the

*h*and

*l*reciprocal-lattice coordinates indicates that the spins primarily lie in the crystalline

**ab**plane. Second-harmonic (2

*τ*) and third-harmonic (3

*τ*) satellites are also present in the figure and are addressed below.

Figure 2 c shows that the integrated intensity of the \((1,0,\bar{\tau })\) primary satellite monotonically increases while cooling down to *T*_{spiral} and that a jump in the integrated intensity of the (1, 0, 0) Bragg peak accompanies the disappearance of the satellite at *T*_{spiral}. This jump signals the emergence of a magnetic-Bragg peak on top of the (1, 0, 0) structural-Bragg peak and is due to a transition to planar-FIM order with the spins continuing to lie in the **ab** plane (FIM-ab). The sharp jump and temperature hysteresis of the integrated intensity of the (1, 0, 0) peak around *T*_{spiral} are both consistent with a first-order transition. Figure 3d shows that the lattice parameters experience a small change in slope at *T*_{spiral} but there is no conclusive sign of an accompanying structural transition.

Previous reports attributed the transition at *T*_{spiral} to a loss of Er magnetic ordering for *T* > *T*_{spiral}^{22}, leaving a Mn only double-spiral structure similar to that found for Y166^{9,10,19}. The double-spiral order is described by a small angle *δ* between the spin directions of the Mn bilayers [e.g. layers (1) and (2) in Fig. 1b and c], which are strongly coupled via the exchange interaction \({{{{\mathcal{J}}}}}_{2}^{{{{\rm{MM}}}}}\) indicated in Fig. 1a. A larger angle Φ = 2*π**τ* describes a compound rotation of the spins between bilayers and encapsulates the periodicity of the spiral. For Er166, however, the temperature dependence of *τ* that we observe is much stronger than it is for Y166^{9,20} and is similar to observations for the triple-spiral order of TmMn_{6}Sn_{5.8}Ge_{0.2}^{27}. In addition, we find that if we assume double-spiral order, the integrated intensity of the \((1,0,\bar{\tau })\) satellite would give a Mn ordered magnetic moment of *μ*_{Mn} ≈ 8*μ*_{B} at *T* = 108 K. This value is too large for Mn-only magnetic ordering. Furthermore, as described below, the temperature dependence of the integrated intensity is in reasonable agreement with the mean-field analysis which predicts triple-spiral magnetic order.

The ideal triple-spiral magnetic order is similar to double-spiral order but with the Er spins participating in the ordering and pointing antiparallel to the bisector of (Φ − *δ*). This is shown in Fig. 1b and c. The spins of the FM Er layer rotate in phase with the Mn spiral order, with the total triple spiral having a period of *c*/*τ*. As we show below, the mean-field results in combination with the single-crystal neutron diffraction data strongly support the presence of triple-spiral ordering between *T*_{N} and *T*_{spiral}. We also have performed a Rietveld refinement using FULLPROF^{28} to powder neutron diffraction data taken at *T* = 200 K. Supplementary Section 1.2 shows that the refinement yields good agreement with the triple-spiral structure, with *τ* = 0.1876(6) and *δ* = 14.0(2)^{∘}, and reasonable values for the ordered Er and Mn magnetic moments of *μ*_{Er} = 3.9(3)*μ*_{B} and *μ*_{Mn} = 2.0(1)*μ*_{B}, respectively.

In addition to the temperature-dependent primary satellites, two other remarkable features appear in the neutron diffraction data. First, much weaker satellite peaks are evident in Fig. 2b that are 2** τ** and 3

**away from (0, 0, 2). These higher-harmonic satellite peaks are highlighted in the constant-temperature cuts in Fig. 3a. The second remarkable feature is the splitting of the lineshapes of individual satellites, as shown in Fig. 4. Both of these features suggest that the triple-spiral order is distorted in Er166 and we next address each feature in turn.**

*τ*Odd-harmonic magnetic satellites are usually associated with squaring up of the AFM order^{29} (such as “bunched-spiral” order) whereas it is much less common to observe even-harmonic magnetic satellites. Possible sources for even-harmonic satellites include distortions of the chemical lattice, magnetoelastic distortions of the ideal triple-spiral magnetic order, or the existence of fan-type magnetic order similar to that which occurs when applying an in-plane magnetic field to a magnetic spiral^{30,31,32}. Figure 3e–g show that we do not observe the 2*τ* peaks in single-crystal x-ray diffraction data, which suggests that the 2*τ* satellites arise from magnetic diffraction.

Regarding the possibility of fan-type magnetic order, we hypothesize that a net magnetization must exist within a magnetic domain in order for it to occur with no applied magnetic field. This could happen if the orientation of the Er spins is pinned by a strong CEF potential. However, both our neutron diffraction and magnetization measurements do not find evidence for a net magnetization existing between *T*_{spiral} and *T*_{N}, as we do not conclusively measure any magnetic contributions at integer (*h*, *k*, *l*) positions in the diffraction data and zero-field-cooled and field-cooled magnetization measurements made with *μ*_{0}*H* = 0.1 T do not reveal a domain population imbalance. Our mean-field modeling, described below, is also dismissive of zero-field fan-type order. Thus, we associate the 2*τ* peaks with an unknown distortion of the ideal triple-spiral magnetic order.

Next, the splitting of the lineshapes of the magnetic-Bragg peaks is also suggestive of a distortion of the ideal triple spiral. The cuts in Fig. 4a to c show that the lineshape of a primary-satellite peak is not described by a gaussian lineshape. Rather, depending on temperature, the lineshape is better fit by the sum of two (*T* ≤ 256 K) or three (*T* > 256 K) gaussian peaks and a background. This is exemplified in Fig. 4b for the \((1,0,\bar{\tau })\) primary satellite at 345 K and 98 K where the individual gaussian components are indicated by dashed lines.

A similar splitting of the primary-satellite lineshape has been reported for Y166^{9,10,18} and Ga-substituted Y166^{33,34}, with the splitting for Y166 being characterized by Δ*τ*_{1} = *τ*^{+} − *τ*^{−} ≈ 0.05 at room temperature. Here, *τ*^{+} and *τ*^{−} refer to the centers of the two gaussian components contributing to the lineshape. As shown in Fig. 4b and d, we find that three rather than two gaussian components contribute to the lineshapes of the (1, 0, ± *τ*) primary satellites of Er166 around room temperature, with a somewhat smaller splitting of Δ*τ*_{1} ≈ 0.03 between components. Similar to Y166, Fig. 4d shows that the splitting is reduced with decreasing temperature, becoming barely resolvable at intermediate temperatures. However, further cooling results in an increase of the splitting where the third gaussian component never reemerges.

Like the double-spiral ordering of Y166^{9}, the origin of the splitting of the primary-satellite lineshapes for the triple-spiral ordering of Er166 is not yet completely clear. We show in Supplementary Sections 1.2 and 1.3 that the lineshape splitting is observed for different single-crystal samples and even in the powder neutron diffraction data. This points to the splitting being an intrinsic property. The lineshape splitting could arise from different magnetic domains with slightly different spiral periods or it could result from beating of the spiral order within a single magnetic domain that results in long-period (∝1/Δ*τ*_{1}) modulations of the magnetic structure. The single-domain origin of the lineshape splitting gains some credence from the observed broadening of the 2*τ* and 3*τ* satellites, which is consistent with splitting of their lineshapes. However, Fig. 3b and c surprisingly show that the splittings of the (1, 0, *τ*) and (1, 0, 2*τ*) lineshapes are unequal, with Δ*τ*_{2} ≈ 1.5Δ*τ*_{1}. Though not completely understood, this observation provides strong evidence supporting the magnetic nature of the 2*τ* satellites and lends weight to the single-domain hypothesis for the lineshape splitting.

### Magnetic Hamiltonian

The competition between exchange coupling and magnetic anisotropy makes the development of a microscopic magnetic model for Er166 challenging, but necessary to understand the magnetic phase stability. We define a magnetic Hamiltonian comprised of isotropic-exchange interactions (\({{{{\mathcal{H}}}}}_{ex}\)) between Mn-Er and Mn-Mn spins, a Zeeman term (\({{{{\mathcal{H}}}}}_{{{{\rm{Z}}}}}\)) for coupling spins to an externally applied magnetic field, and single-ion terms for the Er (\({{{{\mathcal{H}}}}}_{{{{\rm{Er}}}}}\)) and Mn (\(6{{{{\mathcal{H}}}}}_{{{{\rm{Mn}}}}}\)) crystallographic sites that capture the magnetic anisotropy.

The exchange Hamiltonian is given by

where we label various intralayer and interlayer interactions between Mn spins **s**_{i} as \({{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}\) where *k* is a layer index. \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\, > \,0\) is the AFM coupling between neighboring Mn and Er spins **S**_{j}, and all of the pertinent interlayer interactions are diagrammed in Fig. 1a. The values of \({{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}\) are similar to those found for Tb166^{12}, and whereas the intralayer Mn-Mn interaction \({{{{\mathcal{J}}}}}_{0}^{{{{\rm{MM}}}}}\) is large and FM, it is known that the frustrated Mn-Mn interlayer (*k* > 0) interactions lead to competition between collinear and spiral phases^{17,19}. The Zeeman energy is given by

where *g* ≈ 2 for Mn, and *g*_{J} = 6/5, *J* = 15/2 for Er, giving *g*_{J}*J* = 9.

Complex behavior of the magnetic anisotropy arises due to the action of the CEF potential of neighboring ions on the 4*f* orbital states of Er. The Hamiltonian for the CEF acting on Er is

where \({B}_{l}^{m}\) are the CEF parameters for Er^{3+} with hexagonal point-group symmetry and \({{{{\mathcal{O}}}}}_{l}^{m}\) are Stevens operators. We find that our CEF parameters lead to uniaxial magnetic anisotropy for Er in the ground state, although barely so. The uniaxial Er anisotropy competes with the simple Mn easy-plane anisotropy given by

where *K*^{M} > 0. Easy-plane Mn anisotropy is consistent with the planar-FIM or helical ground states found in *R*166 compounds with magnetically isotropic (*R* = Gd) or non-magnetic (*R* = Sc, Y, Lu) ions, respectively^{18,22}.

A representative set of parameters for the magnetic Hamiltonian are shown in Table 1 and their estimation is described in Supplementary Section 1.4. Using these parameters, we next describe a mean-field analysis of the equilibrium states which provides semi-quantitative agreement with observations of the temperature-driven FIM to triple-spiral transition and field and temperature-driven spin-reorientation and spin-flop transitions. The strong intralayer Mn-Mn exchange justifies the assumption that each Mn and Er layer remains FM upon cooling or upon the application of a magnetic field, but both the direction and magnitude of the spins can vary from layer to layer. Thus, determination of the magnetic structure can require minimization of the free energy for spin configurations potentially spanning dozens of unit cells. To facilitate the calculations, we initially ignore planar anisotropy of the Er ion [i.e. the \({B}_{6}^{6}\) term in Eq. (3)] and consider only uniaxial applied fields.

Within a mean-field description, the magnetic structure is determined by minimizing the magnetic free energy described by four angles *θ*_{Er}, *θ*_{Mn}, Φ, and *δ*. *θ*_{Er} (*θ*_{Mn}) describes the polar angle of the Er (Mn) spins away from **c**, and Φ and *δ* are the spiral angles defined above. Minimization of the \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\) exchange energy requires that *μ*_{Er} points antiparallel to the bisector of (Φ − *δ*), as shown in Fig. 1c. With these constraints, the possible magnetic phases are planar-FIM (FIM-ab), uniaxial-FIM (FIM-c), vertical-plane-canted (VP-canted), planar-spiral, vertical-conical-spiral (VCS), and forced-FM (FF). Further details of the mean-field calculations are given Supplementary Sections 1.5 and 1.6.

### Mean-field results and discussion

Competition between the weaker FM \({{{{\mathcal{J}}}}}_{1}^{{{{\rm{MM}}}}}\) and AFM \({{{{\mathcal{J}}}}}_{3}^{{{{\rm{MM}}}}}\) interactions lead to the zero-field spiral phases in the hexagonal *R*166s with non-magnetic *R* = Sc, Y, or Lu^{19}. For *R*166s with magnetic *R* ions, the \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\) coupling destabilizes the spiral state in favor of collinear-FIM order, as shown in Fig. 5a. The spiral phase is the preferred *T* = 0 K state for \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\approx 0\) and as \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\) is increased, Φ → 0 and FIM-ab becomes the preferred ground state above a critical value of \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\approx 1.1\) meV. Thus, the experimentally determined value of \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}=1.35\) meV for Er166 correctly predicts the FIM-ab ground state.

We have performed mean-field calculations for *T* > 0 K to test the stability of the FIM-ab phase of Er166 at higher temperatures. We find that with rising temperature the accompanying increase in thermal fluctuations of the Er spins destabilizes the FIM-ab phase in favor of the ideal triple spiral at *T*_{spiral}. This is exemplified by the plots in Fig. 5b which show that the ideal triple spiral becomes the equilibrium state above *T* = 75 K, which is in good agreement with the experimentally observed value of *T*_{spiral} = 92(1) K. Figure 5b also shows that the calculated temperature dependence of Φ aligns almost perfectly with the experimentally determined values of Φ from Fig. 2b. These results give very strong confidence in the chosen interlayer-exchange parameters shown in Table 1. Finally, Supplementary Figure 8 shows that only a small free energy difference exists between the FIM-ab and ideal triple-spiral phase. This is consistent with observations that the FIM-ab phase can be stabilized at *T* > *T*_{spiral} by a very small planar magnetic field^{35}.

Previous reports have ascribed the transition at *T*_{spiral} to a decoupling of Mn and Er magnetic sublattices and the complete loss of Er ordering^{22}. Our mean-field and experimental results indicate that Er participates in the spiral, leading to the experimentally observed distorted-triple-spiral phase, although there is a substantial reduction of *μ*_{Er} with increasing temperature, as shown in the inset to Fig. 5b. Figure 5c shows the squares of the magnetic structure factors for the double-spiral and ideal triple-spiral orders calculated for the \((1,0,\bar{\tau })\) satellite peak using the mean-field determined parameters. The curve for the ideal triple spiral agrees well with the neutron diffraction data, showing concave-up behavior. The double-spiral curve, on the other hand, shows virtually no temperature dependence below *T* = 300 K. From these comparisons, we also understand that the continued shortening of the triple-spiral period with increasing temperature is caused by growing thermal fluctuations of the Er spins which progressively reduce *μ*_{Er} and the effective Mn-Er coupling.

For more insight into the microscopic details of the magnetism giving rise to a high-temperature triple-spiral phase, we next consider the six-fold planar anisotropy term \({B}_{6}^{6}\) for the Er and whether its proper treatment leads to fan-like phases or distortions of the ideal triple-spiral order that generate the 2*τ* and 3*τ* satellite neutron diffraction peaks. We begin by estimating the temperature dependence of the planar MAE by calculating the free-energy difference \(\Delta {{{{\mathcal{F}}}}}_{\varphi }\) between FIM-ab phases with spins pointing either along the easy (*φ*_{Er} = 30^{∘}, *φ*_{Mn} = 210^{∘}) or hard (*φ*_{Er} = 0^{∘}, *φ*_{Mn} = 180^{∘}) planar axis. Figure 6a shows that thermal fluctuations of the Er spins leads to a rapid decrease of \(\Delta {{{{\mathcal{F}}}}}_{\varphi }\) with increasing temperature, with the planar MAE being reduced by 95% at *T*_{spiral} (\(\Delta {{{{\mathcal{F}}}}}_{\varphi }\,\approx \,0.035\) meV) and becoming negligible above *T* = 150 K.

We next examine whether a certain size of the planar-MAE constant \({K}_{3}^{{\prime} }\), where \({K}_{3}^{{\prime} }={J}^{(6)}{B}_{6}^{6}=\Delta {{{{\mathcal{F}}}}}_{\varphi }/2\), can lead to distorted-spiral order using results from classical energy-minimization calculations for *T* = 0 K and a stack of 36 Mn-Er-Mn layers. Figure 6b shows that larger values of \({K}_{3}^{{\prime} }\) do not lead to fan-like phases, but for moderate values of \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\) larger values of \({K}_{3}^{{\prime} }\) lead to a lock-in of the spiral periodicity at Φ = 60^{∘} (*τ* = 1/6), where the six-fold planar MAE of the Er spins is fully minimized. We also find that nonzero values of \({K}_{3}^{{\prime} }\) lead to a first-order like jump in the spiral periodicity at *T*_{spiral}, in agreement with observations.

It is difficult to address whether distortions of the ideal triple-spiral order close to *T*_{spiral} can result from moderate values of \({K}_{3}^{{\prime} }\, <\, 0.02\) meV. Such distortions, for example, could lead to even and odd-harmonic satellite peaks caused by the pinning of the Er spin direction by the planar MAE. However, three factors suggest that the planar anisotropy plays no role in producing the higher-harmonic satellites: (1) the higher-harmonic satellites persist well above *T* = 200 K, where the Er-planar MAE is fully quenched; (2) there is no experimental evidence for a lock-in of the spiral periodicity to 60°; (3) anharmonicity of the triple spiral resulting from Er spin bunching is expected to produce only odd-harmonic satellites^{36,37,38}.

*M*(*H*) data for **H**∥**c** measured for various temperatures are shown in Fig. 7a. Similar to previous results^{26}, sharp steps to a plateau are visible for *T* < *T*_{spiral}. These steps correspond to a first-order magnetization process (FOMP) that coherently rotates the planar spins of the FIM-ab phase to lay along **c**, resulting in the FIM-c state. The critical FOMP field is small at low temperature (*μ*_{0}*H*_{FOMP} = 0.65 T) and increases with increasing temperature to a maximum of ≈ 4 T close to *T*_{spiral}.

We understand the magnetization data using our mean-field model by calculating the polar MAE which we define as the free-energy difference \(\Delta {{{{\mathcal{F}}}}}_{\theta }\) between FIM-ab (*θ* = 90°) and FIM-c phases (*θ* = 0°). Figure 7c shows that the polar MAE is small at low temperatures, indicating the near degeneracy of FIM-ab and FIM-c phases caused by competing Er uniaxial and Mn easy-plane anisotropies. This is consistent with the small value for *H*_{FOMP} at *T* = 2 K seen by experiment, and our mean-field calculations for *M*(*H*) shown in Fig. 7b are similar to the experimental data in Fig. 7a, correctly predicting a small *H*_{FOMP} for low temperatures. Upon increasing the temperature, *H*_{FOMP} shifts to higher fields, indicating that thermal fluctuations and quenching of the Er anisotropy increase the net easy-plane anisotropy. Above *T*_{spiral}, the steps in *M*(*H*) broaden with increasing temperature until the curves show linear behavior, rather than a step, before plateauing. This departure from the step-like jump signals an (*H*, *T*) region where a gradual canting of the spins occurs with increasing field. The canting creates a VCS magnetic order which terminates in the FIM-c phase at higher field.

The increase of *H*_{FOMP} and disappearance of the FOMP above *T*_{spiral} is again due to increasing thermal fluctuations of the Er spins with increasing temperature, which in this case results in a temperature-driven decrease of the contribution of the Er uniaxial anisotropy to the MAE. Surprisingly, the single-ion anisotropy of Er attains a planar character above *T* ≈ 100 K, resulting in the maximum in \(\Delta {{{{\mathcal{F}}}}}_{\theta }\) shown in Fig. 7c. At higher temperatures, Er contributions to the MAE become completely quenched by thermal fluctuations and the MAE approaches the classical value for the Mn ions of 6*K*^{M} = 1.02 meV. At all temperatures, the mean-field calculations of *M*(*H*) show qualitative agreement with the experimental curves which confirms the assignment of the FIM-ab, VCS, and FIM-c magnetic phases. This is despite a likely overestimation of the size of the Er spin thermal fluctuations by the calculations.

Given its success in describing the low-field phases, we employ the mean-field model to predict the magnetic phase diagram for larger values of **H**∥**c**. The results are shown in Fig. 8. A low-temperature metamagnetic transition from FIM-c to a VP-canted structure is predicted at *μ*_{0}*H* = 23 T, similar to experimental observations^{26}. This transition is driven by a flop of the Er spins into the plane perpendicular to **H** (Er-flop). Another metamagnetic transition occurs at 58 T into the FF phase through which the Er spins flip to be parallel to **H** (Er-flip). The first-order character of both transitions are dictated by the large \({B}_{4}^{0}\) Er CEF term which creates an energy barrier between uniaxial (*θ*_{Er} = 0°,180°) and planar (*θ*_{Er} = 90°) configurations of the Er spins.

Like the disappearance of the low-field FOMP above *T*_{spiral}, weakening of the Er anisotropy by thermal fluctuations leads to the disappearance of the VP-canted phase. The transition from FIM-c to FF also becomes continuous at higher temperature. Notably, the ordered Er moment is completely quenched at the FIM-c to FF crossover. Here, the exchange and Zeeman energies for the Er spins exactly cancel at a crossover field of \(\approx 12{{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}({g}_{J}-1)/{g}_{J}{\mu }_{B}=47\) T.

Our experimental results and mean-field analysis have revealed the temperature and magnetic field responses of the magnetic order of ErMn_{6}Sn_{6}. Crucial to the compound’s magnetic tunability is competition between the various interlayer interactions and single-ion magnetic-anisotropy energies. At zero-field, the increase in thermal spin fluctuations with increasing temperature decreases *μ*_{Er} and results in a weakening of \({{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\) and of the uniaxial Er anisotropy. This leads to the first-order transition from the FIM-ab to a distorted-triple-spiral phase at *T*_{spiral}. For a nonzero field, weakening of *μ*_{Er} by thermal fluctuations is responsible for the increase and eventual disappearance of the low-field FOMP above *T*_{spiral} as well as the elimination of the field-induced VP-canted phase. Remarkably, our calculations also reveal that at high temperatures *μ*_{Er} is effectively quenched at the crossover between FIM-c and FF order due to an exact cancellation of the exchange and Zeeman energies for the Er spins.

We have also investigated whether the observed splitting of the lineshapes of the magnetic-Bragg peaks in the triple-spiral phase is explained by the in-plane magnetic anisotropy. This splitting is also observed for Y166’s double-spiral order but with a different temperature dependence, begging the question of whether the Er anisotropy explains the differences we observe for Er166. Interestingly, recent work on GdV_{6}Sn_{6} has found evidence for incommensurate amplitude-modulated magnetic order which produces diffraction signatures reminiscent of those seen for the double spiral of Y166^{39}. The amplitude-modulated order is associated with an RKKY mechanism within the Gd triangular sublattice, however, which is quite different than the Mn-Mn and Mn-Er interlayer exchange we discuss for Er166.

We conclude that an unknown distortion of the triple-spiral ordering of Er166 exists that is intrinsic to the material and our mean-field analysis verifies that the single-ion anisotropy of the Er, induced by CEF splitting, cannot be responsible in all instances for the distortions. We suggest that a non-sinusoidal layer-to-layer variation of the main spiral and a long-period modulation (beating) exist that occur within each magnetic domain. Similar distortions are often caused by the competition between isotropic-exchange interactions that favor uniform spiral phases and magnetic anisotropy that favors collinear or bunched-spiral distortions^{36,37,38}. Neither spin bunching resulting from anisotropy effects nor amplitude-modulated order, however, is expected to produce even-harmonic satellite diffraction peaks^{38} and would not explain the 2*τ* satellites that we observe.

The general understanding of the interactions controlling the low-field spin-reorientation transitions, higher field spin-flop transitions, and the development of conical-spiral phases in *R*166 kagome metals that we have presented gives a roadmap towards studying the behavior of Dirac fermions with control of the Mn spin orientation and real-space spin chirality. In the FIM phases, the low critical field for the FOMP can be a viable source for controlling the Chern gap, which is maximized in the FIM-c phase. For example, magneto-optical transitions between valence and conduction bands of massive Dirac fermions can be switched using small applied fields^{3,40,41}. Triple-spiral phases in Er166 with tunable periodicity also are attractive for studying the role of vector spin chirality in transport and optical properties^{42,43,44}. Indeed, the Hamiltonian parameters listed in Table 1 are largely consistent and scalable across the *R*Mn_{6}Sn_{6} series, with the details of the *R* magnetic anisotropy and effective magnetic coupling between the *R* and Mn ions being responsible for determining the magnetic ground state^{27,45}. This latter point is born out by the results of our in-depth study and allows for further predictions of novel magnetic and topological phases across the entire *R*166 series.

## Methods

### Single-crystal magnetization and neutron diffraction

Single-crystals of ErMn_{6}Sn_{6} (Er166) were grown from excess Sn flux as described previously^{12}. The samples were determined to be single-phase by x-ray diffraction. Magnetization *M* measurements were made on a Quantum Design, Inc., Magnetic Property Measurement System down to a temperature of *T* = 1.8 K and in magnetic fields up to *μ*_{0}*H* = 7 T. For measurements of the **H**∥**c** orientation, the plate-like samples were glued to a plastic disc and held inside a plastic drinking straw. Prior to measuring with the sample, the bare disc was measured for background subtractions.

Single-crystal neutron diffraction measurements were made on the Fixed-Incident-Energy Triple-Axis Spectrometer at the High-Flux Iostope Reactor, Oak Ridge National Laboratory. Neutrons with a wavelength of *λ* = 2.377 Å were selected by a double-bounce pyrolitic-graphite (PG) monochromator system and a PG analyzer using the PG (0, 0, 2) Bragg reflection. Söller slit collimators with collimations of \(4{0}^{{\prime} }\)-\(4{0}^{{\prime} }\)-\(4{0}^{{\prime} }\)-\(8{0}^{{\prime} }\) were placed before the monochromator, between the sample and monochromator, between the sample and analyzer, and between the analyzer and detector, respectively. PG filters placed after each monochromator were used to reduce contamination by higher-order wavelengths. The sample was mounted on an Al sample holder and cooled in a He closed-cycle refrigerator while either immersed in He exchange gas (*T* < 300 K) or in vacuum. Two different samples were studied with masses of 109.0(1) mg and 282.6(1) mg over two successive experiments. Both samples were aligned with the (*h*, 0, *l*) reciprocal-lattice plane set in the scattering plane of the instrument. Diagrams of the chemical structure were made using VESTA^{46}.

### Powder neutron diffraction

Powder neutron diffraction measurements were made on the time-of-flight diffractometer POWGEN at the Spallation Neutron Source, Oak Ridge National Laboratory. 6.874 g of Er166 powder was loaded into a 6 mm diameter single-wall vanadium can. The powderized sample was obtained by grinding several single crystals. The sample was cooled using a He closed-cycle refrigerator and the automatic sample changer (PAC) was used. Data were collected using the high-resolution setting.

### Single-crystal X-ray diffraction

High-resolution single-crystal x-ray diffraction measurements were performed at Ames National Laboratory using a four-circle diffractometer with Cu *K*_{α1} radiation from a rotating-anode source and a Ge (1, 1, 1) monochromator. The sample was attached to a flat Cu mount which was thermally anchored to the cold head of a He closed-cycle refrigerator. Be domes were used as vacuum shrouds and heat shields. A small amount of He exchange gas facilitated thermal equilibrium.

## Data availability

The datasets used or analyzed during the current study are available from the corresponding authors on reasonable request.

## Code availability

The code used for this study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.

## References

Yin, J.-X., Lian, B. & Hasan, M. Z. Topological kagome magnets and superconductors.

*Nature***612**, 647–657 (2022).Yang, S., Gu, Z.-C., Sun, K. & Das Sarma, S. Topological flat band models with arbitrary Chern numbers.

*Phys. Rev. B***86**, 241112(R) (2012).Tokura, Y., Yasuda, K. & Tsukazaki, A. Magnetic topological insulators.

*Nat. Rev. Phys.***1**, 126–143 (2019).Ma, W. et al. Rare earth engineering in \(R{{{{\rm{Mn}}}}}_{6}{{{{\rm{Sn}}}}}_{6}\) (

*R*= Gd–Tm, Lu) topological kagome magnets.*Phys. Rev. Lett.***126**, 246602 (2021).Regnault, N. et al. Catalogue of flat-band stoichiometric materials.

*Nature***603**, 824–828 (2022).Bernevig, B. A., Felser, C. & Beidenkopf, H. Progress and prospects in magnetic topological materials.

*Nature***603**, 41–51 (2022).Yin, J.-X. et al. Quantum-limit Chern topological magnetism in TbMn

_{6}Sn_{6}.*Nature***583**, 533–536 (2020).Zhang, H. et al. Topological magnon bands in a room-temperature kagome magnet.

*Phys. Rev. B***101**, 100405(R) (2020).Ghimire, N. J. et al. Competing magnetic phases and fluctuation-driven scalar spin chirality in the kagome metal YMn

_{6}Sn_{6}.*Sci. Adv.***6**, eabe2680 (2020).Dally, R. L. et al. Chiral properties of the zero-field spiral state and field-induced magnetic phases of the itinerant kagome metal \({{{{\rm{YMn}}}}}_{6}{{{{\rm{Sn}}}}}_{6}\).

*Phys. Rev. B***103**, 094413 (2021).Li, M. et al. Dirac cone, flat band and saddle point in kagome magnet YMn

_{6}Sn_{6}.*Nat. Commun.***12**, 3129 (2021).Riberolles, S. X. M. et al. Low-temperature competing magnetic energy scales in the topological ferrimagnet TbMn

_{6}Sn_{6}.*Phys. Rev. X***12**, 021043 (2022).Lee, Y. et al. Interplay between magnetism and band topology in the kagome magnets \(R{{{{\rm{Mn}}}}}_{6}{{{{\rm{Sn}}}}}_{6}\).

*Phys. Rev. B***108**, 045132 (2023).Riberolles, S. X. M. et al. Orbital character of the spin-reorientation transition in TbMn

_{6}Sn_{6}.*Nat. Commun.***14**, 2658 (2023).Annaberdiyev, A., Mandal, S., Mitas, L., Krogel, J. T. & Ganesh, P. The role of electron correlations in the electronic structure of putative Chern magnet TbMn

_{6}Sn_{6}.*npj Quant. Mater.***8**, 50 (2023).Lee, S. et al. Nature of charge density wave in kagome metal ScV

_{6}Sn_{6}.*npj Quant. Mater.***9**, 15 (2024).Baranov, N. V., Gerasimov, E. G. & Mushnikov, N. V. Magnetism of compounds with a layered crystal structure.

*Phys. Met. Metallogr.***112**, 711–744 (2011).Venturini, G., Fruchart, D. & Malaman, B. Incommensurate magnetic structures of

*R*Mn_{6}Sn_{6}(*R*= Sc, Y, Lu) compounds from neutron diffraction study.*J. Alloys Compd***236**, 102–110 (1996).Rosenfeld, E. V. & Mushnikov, N. V. Double-flat-spiral magnetic structures: theory and application to the

*R*Mn_{6}*X*_{6}compounds.*Phys. B: Condens. Matter***403**, 1898–1906 (2008).Wang, B. et al. Magnetotransport properties of the kagome magnet \({{{{\rm{TmMn}}}}}_{6}{{{{\rm{Sn}}}}}_{6}\).

*Phys. Rev. B***106**, 125107 (2022).Liu, C. et al. Nontrivial spin textures induced remarkable topological Hall effect and extraordinary magnetoresistance in kagome magnet TmMn

_{6}Sn_{6}.*Surf. Interfaces***39**, 102866 (2023).Malaman, B. et al. Magnetic properties of

*R*Mn_{6}Sn_{6}(*R*= Gd–Er) compounds from neutron diffraction and Mössbauer measurements.*J. Magn. Magn. Mater.***202**, 519–534 (1999).Jones, D. C. et al. Origin of spin reorientation and intrinsic anomalous Hall effect in the kagome ferrimagnet TbMn

_{6}Sn_{6}.*arXiv*https://arxiv.org/abs/2203.17246 (2022).Ghimire, M. P. et al. Creating Weyl nodes and controlling their energy by magnetization rotation.

*Phys. Rev. Res.***1**, 032044 (2019).Dhakal, G. et al. Anisotropically large anomalous and topological hall effect in a kagome magnet.

*Phys. Rev. B***104**, L161115 (2021).Suga, K. et al. High-field-magnetization study of an ErMn

_{6}Sn_{6}single crystal.*J. Alloys Compd***408-412**, 158–160 (2006).Lefèvre, C., Venturini, G. & Malaman, B. Neutron diffraction study of HfFe

_{6}Ge_{6}-type TmMn_{6}Sn_{6−x}Ga_{x}compounds (0.0 ≤*x*≤ 2.5).*J. Alloys Compd***346**, 84–94 (2002).Rodríguez-Carvajal, J. Recent advances in magnetic structure determination by neutron powder diffraction.

*Phys. B: Condens. Matter***192**, 55 – 69 (1993).Jayasekara, W. T. et al. Complex magnetic ordering in CeGe

_{1.76}studied by neutron diffraction.*Phys. Rev. B***90**, 134423 (2014).Johnston, D. C. Magnetic structure and magnetization of helical antiferromagnets in high magnetic fields perpendicular to the helix axis at zero temperature.

*Phys. Rev. B***96**, 104405 (2017).Johnston, D. C. Erratum: Magnetic structure and magnetization of helical antiferromagnets in high magnetic fields perpendicular to the helix axis at zero temperature [Phys. Rev. B

**96**, 104405 (2017)].*Phys. Rev. B***98**, 099903(E) (2018).Johnston, D. C. Magnetic structure and magnetization of

*z*-axis helical Heisenberg antiferromagnets with XY anisotropy in high magnetic fields transverse to the helix axis at zero temperature.*Phys. Rev. B***99**, 214438 (2019).Xu, C. Q., Heitmann, T. W., Zhang, H., Xu, X. & Ke, X. Magnetic phase transition, magnetoresistance, and anomalous Hall effect in Ga-substituted \({{{\rm{Y}}}}{{{{\rm{Mn}}}}}_{6}{{{{\rm{Sn}}}}}_{6}\) with a ferromagnetic kagome lattice.

*Phys. Rev. B***104**, 024413 (2021).Bhandari, H. et al. Magnetism and fermiology of kagome magnet YMn

_{6}Sn_{4}Ge_{2}.*npj Quant. Mater.***9**, 6 (2024).Canepa, F., Napoletano, M., Masini, R., Lefèvre, C. & Venturini, G. A magnetization study of ErMn

_{6}Sn_{6−x}Ga_{x}single crystals (0.11 ≤*x*≤ 1.20).*J. Phys. Condens.***17**, 1961 (2005).Ramazanoglu, M. et al. Temperature-dependent properties of the magnetic order in single-crystal BiFeO

_{3}.*Phys. Rev. B***83**, 174434 (2011).Ratcliff, W., Lynn, J. W., Kiryukhin, V., Jain, P. & Fitzsimmons, M. R. Magnetic structures and dynamics of multiferroic systems obtained with neutron scattering.

*npj Quant. Mater.***1**, 16003 (2016).Chaix, L. et al. Helical bunching and symmetry lowering inducing multiferroicity in Fe langasites.

*Phys. Rev. B***93**, 214419 (2016).Porter, Z., Pokharel, G., Kim, J.-W., Ryan, P. J. & Wilson, S. D. Incommensurate magnetic order in the \({{\mathbb{Z}}}_{2}\) kagome metal \({{\mathbb{Z}}}_{2}\).

*Phys. Rev. B***108**, 035134 (2023).Phuphachong, T. et al. Dirac landau level spectroscopy in Pb

_{1−x}Sn_{x}Se and Pb_{1−x}Sn_{x}Te across the topological phase transition: a review.*Crystals***7**, 29 (2017).Cheskis, D. Magneto-optical tools to study effects in dirac and weyl semimetals.

*Symmetry***12**, 1412 (2020).Nagaosa, N. & Tokura, Y. Topological properties and dynamics of magnetic skyrmions.

*Nat. Nanotechnol.***8**, 899–911 (2013).Šmejkal, L., Mokrousov, Y., Yan, B. & MacDonald, A. H. Topological antiferromagnetic spintronics.

*Nat. Phys.***14**, 242–251 (2018).Kurumaji, T. et al. Skyrmion lattice with a giant topological Hall effect in a frustrated triangular-lattice magnet.

*Science***365**, 914–918 (2019).Clatterbuck, D. M. & Gschneidner, K. A. Magnetic properties of

*R*Mn_{6}Sn_{6}(*R*=Tb, Ho, Er, Tm, Lu) single crystals.*J. Magn. Magn. Mater.***207**, 78–94 (1999).Momma, K. & Izumi, F.

*VESTA3*for three-dimensional visualization of crystal, volumetric and morphology data.*J. Appl. Crystallogr.***44**, 1272–1276 (2011).Olenich, R. R., Akselrud, L. G. & Yarmolyuk, Y. R. Crystal structure of ternary germanides

*R*Fe_{6}Ge_{6}(*R*– Sc, Ti, Zr, Hf, Nd) and*R*Co_{6}Ge_{6}(*R*= Ti, Zr, Hf).*Dopov. Akad. Nauk Ukr. R. S. R., Ser. A***2**, 84 (1981).

## Acknowledgements

S.X.M.R., T.J.S., T.H., P.C.C., R.J.M., and B.G.U. work was supported by the Center for the Advancement of Topological Semimetals (CATS), an Energy Frontier Research Center funded by the U.S. Department of Energy (DOE) Office of Science (SC) and Office of Basic Energy Sciences (BES) through the Ames National Laboratory. A.S., J.M.W., and S.L.B. work at the Ames National Laboratory is supported by the U.S. DOE SC, BES, Division of Materials Sciences and Engineering. Ames National Laboratory is operated for the U.S. DOE by Iowa State University under Contract No. DE-AC02-07CH11358. A portion of this research used resources at the High-Flux Isotope Reactor and Spallation Neutron Source, which are U.S. DOE SC User Facilities operated by Oak Ridge National Laboratory. This research used resources at the Missouri University Research Reactor (MURR).

## Author information

### Authors and Affiliations

### Contributions

S.X.M.R., T.H., W.T., Q.Z., D.L.A., L.D.S., R.J.M., and B.G.U. performed the neutron scattering experiments and analyzed the results. T.J.S., S.L.B., and P.C.C. synthesized the samples and performed magnetization measurements. J.M.W. and A.S. performed single-crystal x-ray diffraction measurements. R.J.M. performed the mean-field analysis. S.X.M.R., R.J.M., and B.G.U. wrote the manuscript with input from all of the authors.

### Corresponding authors

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Additional information

**Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Supplementary information

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Riberolles, S.X.M., Han, T., Slade, T.J. *et al.* New insight into tuning magnetic phases of *R*Mn_{6}Sn_{6} kagome metals.
*npj Quantum Mater.* **9**, 42 (2024). https://doi.org/10.1038/s41535-024-00656-0

Received:

Accepted:

Published:

DOI: https://doi.org/10.1038/s41535-024-00656-0