New insight into tuning magnetic phases of R Mn 6 Sn 6 kagome metals

.


Introduction
The kagome lattice has a two-dimensional corner-sharing triangular arrangement which supports frustrated electronic interactions.Its electronic band structure generally contains both flat bands and linear (Dirac-like) band crossings, both of which can lead to important correlated electronic phenomena; flat bands can give rise to itinerant magnetic correlations and Stoner-type magnetism, whereas linear band crossings create Dirac cones and the associated topological (Dirac) fermions [1][2][3][4][5].These latter features can lead to topological phenomena that intimately link charge and spin, such as dissipationless spin-momentum-locked (chiral) charge transport and quantum-anomalous-Hall effects [6].
Kagome compounds with competing interlayer magnetic interactions are particularly important as they can offer the ability to tune the electronic states of the kagome layers by manipulating the magnetism.Hexagonal RMn 6 Sn 6 (R166) metals with R = Sc, Y, Gd-Lu, fall into this category.These compounds have emerged as model layered systems for magnetic tuning due to the combination of largely defect-free magnetic Mn kagome nets hosting topological electronic bands and R-site magnetism that controls details of the magnetic order [4,[7][8][9][10][11][12][13][14][15][16].Crucial to harnessing the interplay between the magnetic order and topological properties is understanding the microscopic magnetic interactions determining the magnetic order and how to manipulate or influence the magnetic interactions to create a desired magnetic state.Here, we address these challenges via a single-crystal neutron diffraction and mean-field analysis study of ErMn 6 Sn 6 .
As shown in Fig. 1a for Er166, R166 compounds feature Mn kagome bilayers separated by triangular R layers.Importantly, the magnetic layers individually exhibit ferromagnetic (FM) ordering of their magnetic moments (spins), and a majority of R166 compounds with magnetic R ions exhibit collinear ferrimagnetic (FIM) order driven by strong antiferromagnetic (AFM) coupling between the Mn and R layers [17].For nonmagnetic R, such as Y, long range and competing Mn-Mn interlayer couplings lead to spiral magnetic order [9,18,19], and applied magnetic fields lead to distorted and noncoplanar spin configurations where the topological-Hall effect and planar-anisotropic magnetoresistance have been observed [9,10,20,21].Thus, the strength of the Mn-R coupling controls the stability between collinear and noncollinear magnetism in the R166 compounds and we show in this work that Er166 sits close to the critical boundary between the two types of ordering.
Another key aspect of R166 compounds is that R ions with nonzero orbital angular momentum adopt complex magnetic anisotropy through the crystalline-electric-field (CEF) splitting of their 4f orbitals.Easy-plane Mn anisotropy is present, but strong R anisotropy will dictate the Mn magnetization direction at low temperature and zero magnetic field.For the FM Mn layers, a uniaxial anisotropy is proposed to dramatically enhance the spin-orbit splitting of Dirac cones, forming a Chern insulator [7].Further, the competition of R anisotropy with the easy-plane Mn anisotropy can result in temperature and field driven spin-reorientation transitions between easy-plane and uniaxial (R = Tb) or tilted (R = Dy, Ho) FIM states.These transitions can facilitate magnetization switching via small changes in temperature or magnetic field [14,22,23] which can create or destroy a Chern gap [7] or Weyl nodes [24].We find that the Er 3+ ion has weakly uniaxial and strongly temperature dependent anisotropy in Er166, resulting in facile magnetic switching.Small magnetic fields of µ 0 H < 1 T can trigger spin-reorientation transitions where large anomalous-Hall and topological-Hall responses have been reported [25].
Here, we present single-crystal neutron diffraction and mean-field analysis results that illustrate how competition between interlayer magnetic interactions and magnetic anisotropy in Er166 leads to a number of nearly degenerate magnetic states.Using this information, we characterize and understand the temperature driven first-order transition from planar-FIM to a distorted-triple-spiral order upon warming through T spiral = 92(1) K, showing that spins in the Er layers remain magnetically ordered above T spiral rather than their previously reported paramagnetic behavior [25].Using additional results from inelastic neutron scattering and magnetization measurements, we determine the key microscopic magnetic interactions and predict the emergence of various collinear, noncollinear, and noncoplanar magnetic phases as functions of temperature and magnetic field.Importantly, we find that the stability of the magnetic phases is controlled by thermal fluctuations of the Er spins which act to weaken the effective Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy.

Experimental Results
The magnetization M of Er166 for a weak magnetic field of µ 0 H = 0.005 T applied perpendicular to the crystalline c axis is shown in Fig. 2a and the zero-field neutron diffraction pattern for (0, 0, l) reciprocal-lattice points is shown in Fig. 2b for increasing temperature.The main features in both figures agree with previously published data [22,26]; with cooling, a peak in [M/H](T ) in Fig. 2a at the Néel temperature of T N = 348(1) K coincides with the emergence of satellite magnetic-Bragg peaks in Fig. 2b surrounding the (0, 0, 2) structural-Bragg peak; a large jump in [M/H](T ) at T spiral accompanies the disappearance of the satellite peaks.The primary satellites [those satellite peaks closest to (0,0,2)] correspond to a temperature dependent AFM propagation vector of τ = (0, 0, τ ) which indicates the presence of magnetic ordering with a temperature dependent structure that is modulated along c. Analysis of satellites for different values of the h and l reciprocal-lattice coordinates indicates that the spins primarily lie in the crystalline ab plane.Second-harmonic (2τ ) and third-harmonic (3τ ) satellites are also present in the figure and are addressed below.
Figure 2c shows that the integrated intensity of the (1, 0, τ ) primary satellite monotonically increases while cooling down to T spiral and that a jump in the integrated intensity of the (1, 0, 0) Bragg peak accompanies the disappearance of the satellite at T spiral .This jump signals the emergence of a magnetic-Bragg peak on top of the (1, 0, 0) structural-Bragg peak and is due to a transition to planar-FIM order with the spins continuing to lie in the ab plane (FIM-ab).The sharp jump and temperature hysteresis of the integrated intensity of the (1, 0, 0) peak around T spiral are both consistent with a first-order transition.Figure 3d shows that the lattice parameters experience a small change in slope at T spiral but there is no conclusive sign of an accompanying structural transition.
Previous reports attributed the transition at T spiral to a loss of Er magnetic ordering for T > T spiral [22], leaving a Mn only double-spiral structure similar to that found for Y166 [9,10,19].The double-spiral order is described by a small angle δ between the spin directions of the Mn bilayers [e.g.layers (1) and (2) in Figs.1b and 1c], which are strongly coupled via the exchange interaction J MM 2 indicated in Fig. 1a.A larger angle Φ = 2πτ describes a compound rotation of the spins between bilayers and encapsulates the periodicity of the spiral.For Er166, however, the temperature dependence of τ that we observe is much stronger than it is for Y166 [9,20] and is similar to observations for the triple-spiral order of TmMn 6 Sn 5.8 Ge 0.2 [27].In addition, we find that if we assume double-spiral order, the integrated intensity of the (1, 0, τ ) satellite would give a Mn ordered magnetic moment of µ Mn ≈ 8 µ B at T = 108 K.This value is too large for Mn-only magnetic ordering.Furthermore, as described below, the temperature dependence of the integrated intensity is in reasonable agreement with the mean-field analysis which predicts triple-spiral magnetic order.
The ideal triple-spiral magnetic order is similar to double-spiral order but with the Er spins participating in the ordering and pointing antiparallel to the bisector of (Φ − δ).This is shown in Figs.1b and 1c.The spins of the FM Er layer rotate in phase with the Mn spiral order, with the total triple spiral having a period of c/τ .As we show below, the mean-field results in combination with the single-crystal neutron diffraction data strongly support the presence of triple-spiral ordering between T N and T spiral .We also have performed a Rietveld refinement using fullprof [28] to powder neutron diffraction data taken at T = 200 K. Supplementary Section 1.2 shows that the refinement yields good agreement with the triple-spiral structure, with τ = 0.1876 (6) and δ = 14.0(2)°, and reasonable values for the ordered Er and Mn magnetic moments of µ Er = 3.9(3) µ B and µ Mn = 2.0(1) µ B , respectively.
In addition to the temperature dependent primary satellites, two other remarkable features appear in the neutron diffraction data.First, much weaker satellite peaks are evident in Fig. 2b that are 2τ and 3τ away from (0, 0, 2).These higher-harmonic satellite peaks are highlighted in the constant-temperature cuts in Fig. 3a.The second remarkable feature is the splitting of the lineshapes of individual satellites, as shown in Fig. 4.Both of these features suggest that the triple-spiral order is distorted in Er166 and we next address each feature in turn.
Odd-harmonic magnetic satellites are usually associated with squaring up of the AFM order [29] (such as "bunched-spiral" order) whereas it is much less common to observe even-harmonic magnetic satellites.Possible sources for even-harmonic satellites include distortions of the chemical lattice, magnetoelastic distortions of the ideal triple-spiral magnetic order, or the existence of fan-type magnetic order similar to that which occurs when applying an in-plane magnetic field to a magnetic spiral [30][31][32].Figures 3e to 3g show that we do not observe the 2τ peaks in single-crystal x-ray diffraction data, which suggests that the 2τ satellites arise from magnetic diffraction.
Regarding the possibility of fan-type magnetic order, we hypothesize that a net magnetization must exist within a magnetic domain in order for it to occur with no applied magnetic field.This could happen if the orientation of the Er spins is pinned by a strong CEF potential.However, both our neutron diffraction and magnetization measurements do not find evidence for a net magnetization existing between T spiral and T N , as we do not conclusively measure any magnetic contributions at integer (h, k, l) positions in the diffraction data and zero-field-cooled and field-cooled magnetization measurements made with µ 0 H = 0.1 T do not reveal a domain population imbalance.Our mean-field modeling, described below, is also dismissive of zero-field fan-type order.Thus, we associate the 2τ peaks with an unknown distortion of the ideal triplespiral magnetic order.
Next, the splitting of the lineshapes of the magnetic-Bragg peaks is also suggestive of a distortion of the ideal triple spiral.The cuts in Figs.4a to 4c show that the lineshape of a primary-satellite peak is not described by a gaussian lineshape.Rather, depending on temperature, the lineshape is better fit by the sum of two (T ≤ 256 K) or three (T > 256 K) gaussian peaks and a background.This is exemplified in Fig. 4b for the (1, 0, τ ) primary satellite at 345 K and 98 K where the individual gaussian components are indicated by dashed lines.
A similar splitting of the primary-satellite lineshape has been reported for Y166 [9,10,18] and Ga-substituted Y166 [33,34], with the splitting for Y166 being characterized by ∆τ 1 = τ + − τ − ≈ 0.05 at room temperature.Here, τ + and τ − refer to the centers of the two gaussian components contributing to the lineshape.As shown in Figs.4b and 4d, we find that three rather than two gaussian components contribute to the lineshapes of the (1, 0, ±τ ) primary satellites of Er166 around room temperature, with a somewhat smaller splitting of ∆τ 1 ≈ 0.03 between components.Similar to Y166, Fig. 4d shows that the splitting is reduced with decreasing temperature, becoming barely resolvable at intermediate temperatures.However, further cooling results in an increase of the splitting where the third gaussian component never reemerges.
Like the double-spiral ordering of Y166 [9], the origin of the splitting of the primarysatellite lineshapes for the triple-spiral ordering of Er166 is not yet completely clear.We show in Supplementary Sections 1.2 and 1.3 that the lineshape splitting is observed for different single-crystal samples and even in the powder neutron diffraction data.This points to the splitting being an intrinsic property.The lineshape splitting could arise from different magnetic domains with slightly different spiral periods or it could result from beating of the spiral order within a single magnetic domain that results in long-period (∝ 1/∆τ 1 ) modulations of the magnetic structure.The single-domain origin of the lineshape splitting gains some credence from the observed broadening of the 2τ and 3τ satellites, which is consistent with splitting of their lineshapes.However, Figs. 3b and 3c surprisingly show that the splittings of the (1, 0, τ ) and (1, 0, 2τ ) lineshapes are unequal, with ∆τ 2 ≈ 1.5∆τ 1 .Though not completely understood, this observation provides strong evidence supporting the magnetic nature of the 2τ satellites and lends weight to the single-domain hypothesis for the lineshape splitting.

Magnetic Hamiltonian
The competition between exchange coupling and magnetic anisotropy makes the development of a microscopic magnetic model for Er166 challenging, but necessary to understand the magnetic phase stability.We define a magnetic Hamiltonian comprised of isotropic-exchange interactions (H ex ) between Mn-Er and Mn-Mn spins, a Zeeman term (H Z ) for coupling spins to an externally applied magnetic field, and single-ion terms for the Er (H Er ) and Mn (6H Mn ) crystallographic sites that capture the magnetic anisotropy.
The exchange Hamiltonian is given by where we label various intralayer and interlayer interactions between Mn spins s i as where k is a layer index.J ME > 0 is the AFM coupling between neighboring Mn and Er spins S j , and all of the pertinent interlayer interactions are diagrammed in Fig. 1a.The values of J MM k are similar to those found for Tb166 [12], and whereas the intralayer Mn-Mn interaction J MM 0 is large and FM, it is known that the frustrated Mn-Mn interlayer (k > 0) interactions lead to competition between collinear and spiral phases [17,19].The Zeeman energy is given by where g ≈ 2 for Mn, and g J = 6/5, J = 15/2 for Er, giving g J J = 9.Complex behavior of the magnetic anisotropy arises due to the action of the CEF potential of neighboring ions on the 4f orbital states of Er.The Hamiltonian for the CEF acting on Er is where B m l are the CEF parameters for Er 3+ with hexagonal point-group symmetry and O m l are Stevens operators.We find that our CEF parameters lead to uniaxial magnetic anisotropy for Er in the ground state, although barely so.The uniaxial Er anisotropy competes with the simple Mn easy-plane anisotropy given by where K M > 0. Easy-plane Mn anisotropy is consistent with the planar-FIM or helical ground states found in R166 compounds with magnetically isotropic (R = Gd) or non-magnetic (R = Sc, Y, Lu) ions, respectively [18,22].
A representative set of parameters for the magnetic Hamiltonian are shown in Table 1 and their estimation is described in Supplementary Section 1.4.Using these parameters, we next describe a mean-field analysis of the equilibrium states which provides semi-quantitative agreement with observations of the temperature driven FIM to triple-spiral transition and field and temperature driven spin-reorientation and spin-flop transitions.The strong intralayer Mn-Mn exchange justifies the assumption that each Mn and Er layer remains FM upon cooling or upon the application of a magnetic field, but both the direction and magnitude of the spins can vary from layer to layer.Thus, determination of the magnetic structure can require minimization of the free energy for spin configurations potentially spanning dozens of unit cells.To facilitate the calculations, we initially ignore planar anisotropy of the Er ion [i.e. the B 6  6 term in Eqn. ( 3)] and consider only uniaxial applied fields.Within a mean-field description, the magnetic structure is determined by minimizing the magnetic free energy described by four angles θ Er , θ Mn , Φ, and δ. θ Er (θ Mn ) describes the polar angle of the Er (Mn) spins away from c, and Φ and δ are the spiral angles defined above.Minimization of the J ME exchange energy requires that µ Er points antiparallel to the bisector of (Φ − δ), as shown in Fig. 1c.With these constraints, the possible magnetic phases are planar-FIM (FIM-ab), uniaxial-FIM (FIM-c), vertical-plane-canted (VP-canted), planar-spiral, vertical-conical-spiral (VCS), and forced-FM (FF).Further details of the mean-field calculations are given Supplementary Sections 1.5 and 1.6.

Mean-Field Results and Discussion
Competition between the weaker FM J MM 1 and AFM J MM 3 interactions lead to the zero-field spiral phases in the hexagonal R166s with non-magnetic R = Sc, Y, or Lu [19].For R166s with magnetic R ions, the J ME coupling destabilizes the spiral state in favor of collinear-FIM order, as shown in Fig. 5a.The spiral phase is the preferred T = 0 K state for J ME ≈ 0 and as J ME is increased, Φ → 0 and FIM-ab becomes the preferred ground state above a critical value of J ME ≈ 1.1 meV.Thus, the experimentally determined value of J ME = 1.35 meV for Er166 correctly predicts the FIM-ab ground state.
We have performed mean-field calculations for T > 0 K to test the stability of the FIM-ab phase of Er166 at higher temperatures.We find that with rising temperature the accompanying increase in thermal fluctuations of the Er spins destabilizes the FIM-ab phase in favor of the ideal triple spiral at T spiral .This is exemplified by the plots in Fig. 5b which show that the ideal triple spiral becomes the equilibrium state above T = 75 K, which is in good agreement with the experimentally observed value of T spiral = 92(1) K. Figure 5b also shows that the calculated temperature dependence of Φ aligns almost perfectly with the experimentally determined values of Φ from Fig. 2b.These results give very strong confidence in the chosen interlayer-exchange parameters shown in Table 1.Finally, Supplementary Figure 8 shows that only a small free energy difference exists between the FIM-ab and ideal triple-spiral phase.This is consistent with observations that the FIM-ab phase can be stabilized at T > T spiral by a very small planar magnetic field [35].
Previous reports have ascribed the transition at T spiral to a decoupling of Mn and Er magnetic sublattices and the complete loss of Er ordering [22].Our meanfield and experimental results indicate that Er participates in the spiral, leading to the experimentally observed distorted-triple-spiral phase, although there is a substantial reduction of µ Er with increasing temperature, as shown in the inset to Fig. 5b. Figure 5c shows the squares of the magnetic structure factors for the double-spiral and ideal triple-spiral orders calculated for the (1, 0, τ ) satellite peak using the meanfield determined parameters.The curve for the ideal triple spiral agrees well with the neutron diffraction data, showing concave-up behavior.The double-spiral curve, on the other hand, shows virtually no temperature dependence below T = 300 K. From these comparisons, we also understand that the continued shortening of the triplespiral period with increasing temperature is caused by growing thermal fluctuations of the Er spins which progressively reduce µ Er and the effective Mn-Er coupling.
For more insight into the microscopic details of the magnetism giving rise to a high temperature triple-spiral phase, we next consider the six-fold planar anisotropy term B 6  6 for the Er and whether its proper treatment leads to fan-like phases or distortions of the ideal triple-spiral order that generate the 2τ and 3τ satellite neutron diffraction peaks.We begin by estimating the temperature dependence of the planar MAE by calculating the free-energy difference ∆F φ between FIM-ab phases with spins pointing either along the easy (φ Er = 30°, φ Mn = 210°) or hard (φ Er = 0°, φ Mn = 180°) planar axis.Figure 6a shows that thermal fluctuations of the Er spins leads to a rapid decrease of ∆F φ with increasing temperature, with the planar MAE being reduced by 95% at T spiral (∆F φ ≈ 0.035 meV) and becoming negligible above T = 150 K.
We next examine whether a certain size of the planar-MAE constant K ′ 3 , where , can lead to distorted-spiral order using results from classical energy-minimization calculations for T = 0 K and a stack of 36 Mn-Er-Mn layers.Figure 6b shows that larger values of K ′ 3 do not lead to fan-like phases, but for moderate values of J ME larger values of K ′ 3 lead to a lock-in of the spiral periodicity at Φ = 60°(τ = 1/6), where the six-fold planar MAE of the Er spins is fully minimized.We also find that nonzero values of K ′ 3 lead to a first-order like jump in the spiral periodicity at T spiral , in agreement with observations.
It is difficult to address whether distortions of the ideal triple-spiral order close to T spiral can result from moderate values of K ′ 3 < 0.02 meV.Such distortions, for example, could lead to even and odd-harmonic satellite peaks caused by the pinning of the Er spin direction by the planar MAE.However, three factors suggest that the planar anisotropy plays no role in producing the higher-harmonic satellites: (1) the higher-harmonic satellites persist well above T = 200 K, where the Er-planar MAE is fully quenched; (2) there is no experimental evidence for a lock-in of the spiral periodicity to 60°; (3) anharmonicity of the triple spiral resulting from Er spin bunching is expected to produce only odd-harmonic satellites [36][37][38].
M (H) data for H ∥ c measured for various temperatures are shown in Fig. 7a.Similar to previous results [26], sharp steps to a plateau are visible for T < T spiral .These steps correspond to a first-order magnetization process (FOMP) that coherently rotates the planar spins of the FIM-ab phase to lay along c, resulting in the FIM-c state.The critical FOMP field is small at low temperature (µ 0 H FOMP = 0.65 T) and increases with increasing temperature to a maximum of ≈ 4 T close to T spiral .
We understand the magnetization data using our mean-field model by calculating the polar MAE which we define as the free-energy difference ∆F θ between FIM-ab (θ = 90°) and FIM-c phases (θ = 0°).Figure 7c shows that the polar MAE is small at low temperatures, indicating the near degeneracy of FIM-ab and FIM-c phases caused by competing Er uniaxial and Mn easy-plane anisotropies.This is consistent with the small value for H FOMP at T = 2 K seen by experiment, and our mean-field calculations for M (H) shown in Fig. 7b are similar to the experimental data in Fig. 7a, correctly predicting a small H FOMP for low temperatures.Upon increasing the temperature, H FOMP shifts to higher fields, indicating that thermal fluctuations and quenching of the Er anisotropy increase the net easy-plane anisotropy.Above T spiral , the steps in M (H) broaden with increasing temperature until the curves show linear behavior, rather than a step, before plateauing.This departure from the step-like jump signals an (H, T ) region where a gradual canting of the spins occurs with increasing field.The canting creates a VCS magnetic order which terminates in the FIM-c phase at higher field.
The increase of H FOMP and disappearance of the FOMP above T spiral is again due to increasing thermal fluctuations of the Er spins with increasing temperature, which in this case results in a temperature-driven decrease of the contribution of the Er uniaxial anisotropy to the MAE.Surprisingly, the single-ion anisotropy of Er attains a planar character above T ≈ 100 K, resulting in the maximum in ∆F θ shown in Fig. 7c.At higher temperatures, Er contributions to the MAE become completely quenched by thermal fluctuations and the MAE approaches the classical value for the Mn ions of 6K M = 1.02 meV.At all temperatures, the mean-field calculations of M (H) show qualitative agreement with the experimental curves which confirms the assignment of the FIM-ab, VCS, and FIM-c magnetic phases.This is despite a likely overestimation of the size of the Er spin thermal fluctuations by the calculations.
Given its success in describing the low-field phases, we employ the mean-field model to predict the magnetic phase diagram for larger values of H ∥ c.The results are shown in Fig. 8.A low temperature metamagnetic transition from FIM-c to a VPcanted structure is predicted at µ 0 H = 23 T, similar to experimental observations [26].This transition is driven by a flop of the Er spins into the plane perpendicular to H (Er-flop).Another metamagnetic transition occurs at 58 T into the FF phase through which the Er spins flip to be parallel to H (Er-flip).The first-order character of both transitions are dictated by the large B 0 4 Er CEF term which creates an energy barrier between uniaxial (θ Er = 0°,180°) and planar (θ Er = 90°) configurations of the Er spins.
Like the disappearance of the low-field FOMP above T spiral , weakening of the Er anisotropy by thermal fluctuations leads to the disappearance of the VP-canted phase.The transition from FIM-c to FF also becomes continuous at higher temperature.Notably, the ordered Er moment is completely quenched at the FIM-c to FF crossover.Here, the exchange and Zeeman energies for the Er spins exactly cancel at a crossover field of ≈ 12J ME (g J − 1)/g J µ B = 47 T.
Our experimental results and mean-field analysis have revealed the temperature and magnetic field responses of the magnetic order of ErMn 6 Sn 6 .Crucial to the compound's magnetic tunability is competition between the various interlayer interactions and single-ion magnetic-anisotropy energies.At zero-field, the increase in thermal spin fluctuations with increasing temperature decreases µ Er and results in a weakening of J ME and of the uniaxial Er anisotropy.This leads to the first-order transition from the FIM-ab to a distorted-triple-spiral phase at T spiral .For a nonzero field, weakening of µ Er by thermal fluctuations is responsible for the increase and eventual disappearance of the low-field FOMP above T spiral as well as the elimination of the field-induced VP-canted phase.Remarkably, our calculations also reveal that at high temperatures µ Er is effectively quenched at the crossover between FIM-c and FF order due to an exact cancellation of the exchange and Zeeman energies for the Er spins.
We have also investigated whether the observed splitting of the lineshapes of the magnetic-Bragg peaks in the triple-spiral phase is explained by the in-plane magnetic anisotropy.This splitting is also observed for Y166's double-spiral order but with a different temperature dependence, begging the question of whether the Er anisotropy explains the differences we observe for Er166.Interestingly, recent work on GdV 6 Sn 6 has found evidence for incommensurate amplitude-modulated magnetic order which produces diffraction signatures reminiscent of those seen for the double spiral of Y166 [39].The amplitude-modulated order is associated with an RKKY mechanism within the Gd triangular sublattice, however, which is quite different than the Mn-Mn and Mn-Er interlayer exchange we discuss for Er166.
We conclude that an unknown distortion of the triple-spiral ordering of Er166 exists that is intrinsic to the material and our mean-field analysis verifies that the single-ion anisotropy of the Er, induced by CEF splitting, cannot be responsible in all instances for the distortions.We suggest that a non-sinusoidal layer-to-layer variation of the main spiral and a long-period modulation (beating) exist that occur within each magnetic domain.Similar distortions are often caused by the competition between isotropic-exchange interactions that favor uniform spiral phases and magnetic anisotropy that favors collinear or bunched-spiral distortions [36][37][38].Neither spin bunching resulting from anisotropy effects nor amplitude-modulated order, however, is expected to produce even-harmonic satellite diffraction peaks [38] and would not explain the 2τ satellites that we observe.
The general understanding of the interactions controlling the low-field spinreorientation transitions, higher field spin-flop transitions, and the development of conical-spiral phases in R166 kagome metals that we have presented gives a roadmap towards studying the behavior of Dirac fermions with control of the Mn spin orientation and real-space spin chirality.In the FIM phases, the low critical field for the FOMP can be a viable source for controlling the Chern gap, which is maximized in the FIM-c phase.For example, magneto-optical transitions between valence and conduction bands of massive Dirac fermions can be switched using small applied fields [3,40,41].Triple-spiral phases in Er166 with tunable periodicity also are attractive for studying the role of vector spin chirality in transport and optical properties [42][43][44].Indeed, the Hamiltonian parameters listed in Table 1 are largely consistent and scalable across the RMn 6 Sn 6 series, with the details of the R magnetic anisotropy and effective magnetic coupling between the R and Mn ions being responsible for determining the magnetic ground state [27,45].This latter point is born out by the results of our in-depth study and allows for further predictions of novel magnetic and topological phases across the entire R166 series.

Single-Crystal Neutron Diffraction
Single-crystals of ErMn 6 Sn 6 (Er166) were grown from excess Sn flux as described previously [12].The samples were determined to be single phase by x-ray diffraction.Magnetization M measurements were made on a Quantum Design, Inc., Magnetic Property Measurement System down to a temperature of T = 1.8K and in magnetic fields up to µ 0 H = 7 T.For measurements of the H ∥ c orientation, the plate-like samples were glued to a plastic disc and held inside a plastic drinking straw.Prior to measuring with the sample, the bare disc was measured for background subtractions.
Single-crystal neutron diffraction measurements were made on the Fixed-Incident-Energy Triple-Axis Spectrometer at the High-Flux Iostope Reactor, Oak Ridge National Laboratory.Neutrons with a wavelength of λ = 2.377 Å were selected by a double-bounce pyrolitic-graphite (PG) monochromator system and a PG analyzer using the PG (0, 0, 2) Bragg reflection.Söller slit collimators with collimations of 40 ′ -40 ′ -40 ′ -80 ′ were placed before the monochromator, between the sample and monochromator, between the sample and analyzer, and between the analyzer and detector, respectively.PG filters placed after each monochromator were used to reduce contamination by higher-order wavelengths.The sample was mounted on an Al sample holder and cooled in a He closed-cycle refrigerator while either immersed in He exchange gas (T < 300 K) or in vacuum.Two different samples were studied with masses of 109.0(1)mg and 282.6(1) mg over two successive experiments.Both samples were aligned with the (h, 0, l) reciprocal-lattice plane set in the scattering plane of the instrument.Diagrams of the chemical structure were made using vesta [46].

Powder Neutron Diffraction
Powder neutron diffraction measurements were made on the time-of-flight diffractometer POWGEN at the Spallation Neutron Source, Oak Ridge National Laboratory.6.874 g of Er166 powder was loaded into a 6 mm diameter single-wall vanadium can.The powderized sample was obtained by grinding several single crystals.The sample was cooled using a He closed-cycle refrigerator and the automatic sample changer (PAC) was used.Data were collected using the high-resolution setting.

Single-Crystal X-Ray Diffraction
High-resolution single-crystal x-ray diffraction measurements were performed at Ames National Laboratory using a four-circle diffractometer with Cu K α1 radiation from a rotating-anode source and a Ge (1, 1, 1) monochromator.The sample was attached to a flat Cu mount which was thermally anchored to the cold head of a He closed-cycle refrigerator.Be domes were used as vacuum shrouds and heat shields.A small amount of He exchange gas facilitated thermal equilibrium.

Data availability
The datasets used or analyzed during the current study are available from the corresponding authors on reasonable request.

Code availability
The code used for this study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.
Table 1 Heisenberg and crystal field parameters for ErMn 6 Sn 6 .The listed values are given in units of meV and their estimations are described in Supplementary Section 1.4.Mn ( 1) Mn ( 2) Mn ( 4) Mn ( 5) .00Å at room temperature [17,47].The individually ferromagnetic kagome-Mn (triangular-Er) planes are indicated in red (blue).Interlayer Mn-Sn bonds are displayed and interlayer-exchange interactions between Mn layers (J MM k ) and Mn and Er layers (J ME ) are shown as black double arrows.b Parallel and c top-down views of ideal-triple-spiral order.Numbers label the sequence of the Mn and Er planes along c and each arrow represents a ferromagnetic Er or Mn plane.Each plane has an ordered magnetic moment oriented perpendicular to c and the moment orientation varies between layers as shown.The angles Φ and δ characterize the magnetic order, with the Er moment direction pointing antiparallel to the bisector of (Φ − δ).Φ = δ = 0 corresponds to the low-temperature ferrimagnetic phase with the Er and Mn moments pointing opposite to each another.Fig. 7 Field-induced magnetic phases and the role of polar magnetic anisotropy.a Measured magnetization versus magnetic field data for several temperatures with the field applied parallel to c. b Complementary results from mean-field calculations using the parameters in Table 1.c Results from mean-field calculations for the temperature evolution of the polar magnetic-anisotropy energy, which is estimated as the free-energy difference between uniaxial (θ Mn = 0°, θ Er = 180°) and planar (θ Mn = θ Er = 90°) spin orientations for a ferrimagnetic phase.K M is the Mn anisotropy parameter.

Estimation of Hamiltonian parameters
Exchange parameters.Fits to the spin-wave dispersions from previous inelasticneutron-scattering (INS) data for TbMn 6 Sn 6 (Tb166) provide a starting point for estimating the various Mn-Mn and Mn-Er isotropic exchange constants defined in the main text in Eq. ( 1).The values for Tb166, in meV, are; 8, and J MT = 1.8 meV [12,14].Here, J MM 2 is the dominant ferromagnetic (FM) interlayer coupling.The stability of spiral or ferrimagnetic (FIM) states is then dependent on the balance of the competing J MM 1 and J MM 3 interactions, which have opposite signs.We discuss this competition in terms of the ratios x = J MM 1 /J MM 2 > 0 and y = J MM 3 /J MM 2 < 0. The Tb166 value of x = 0.23 is consistent with estimates based on neutron diffraction and magnetization data for Y166, so we retain this value.The Tb166 value of y = |−0.09| is smaller than the value of y = |−0.12|reported for Y166.The smaller of the two y values is very close to the stability range of the FIM phase and therefore can only stabilize long-period spiral order.This is not consistent with experimental data in the spiral phase for Er166, so we set y to the value reported for Y166, or J MM 3 = 2.3 meV.The Mn-R interaction, J MR , is known to be antiferromagnetic (AFM) for the heavy rare-earths and both INS data [12,14] and density-functional-theory (DFT) calculations [13] indicate that J MR decreases in magnitude from R = Gd → Tm.We estimate the value of J ME = 1.35 meV using INS data for Er166 and magnetization data, as described below.This value is lower than reported values for Gd166 (J MG = 2 meV) and Tb166 (J MT = 1.8 meV) which is consistent with the expected trend.
Mn easy-plane anisotropy.Mn has an easy-plane (planar) anisotropy with K M > 0. The magnitude of K M in R166 compounds has been estimated to be in a range from 0.2 to 0.5 meV from a variety of experimental approaches; (1) fitting the spin gap of Tb166 results in K M = 0.44 meV [14]; (2) reported from analysis of the magnetization data as 0.47 meV for Tb166 [23] (3) reported to be 0.23 meV from the H ∥ c saturation magnetization field of Gd166 [19]; (4) reported as 0.2 meV for Y166 [9].
For Er166, the best estimate of K M is obtained from the high-temperature magnetization data with H ∥ c, as shown in Fig. 7a of the main text.At temperatures where the Er MAE is quenched (T ∼ 200 K), the critical field for ferrimagnetic alignment of Mn and Er moments is approximately µ 0 H c = 5 T. If this critical field is determined solely by the MAE of the Mn ion, we estimate that K M ≈ µ 0 H c /12M = 0.17 meV, where M = 7 µ B is the onset of the magnetization plateau at 200 K.This value of K M is on the low end of the range of the other R166 compounds and so we fix K M = 0.17 meV.
Er crystalline-electric-field parameters.The crystalline-electric-field (CEF) parameters B m l for different rare-earth ions scale according to the formula where θ l is the Steven's factor and ⟨r l ⟩ is the average l th order radial moment, both of which depend on the R ion.The parameter A m l is intrinsic to the crystalline potential and is expected to vary slowly across the R166 series.
From INS studies of Tb166, we are able to determine approximate values of A 0 2 = (4.1 meV)a −2 0 , A 0 4 = −(7.0meV)a −4 0 , and A 0 6 ≈ 0 (where a 0 is the Bohr radius).Assuming the transferability of the CEF potential for the hexagonal R166s and using Eq. ( 5), we obtain good starting values for the CEF parameters for the Er ion; B 0 2 = 0.0075 meV, B 0 4 = −0.00041meV and B 0 6 = 0.These values are reasonably consistent with DFT studies of the RMn 6 Sn 6 series [13] and slight refinement of these values are described below.For Er166, magnetization measurements indicate an easy-axis along the (1, 1, 0) direction in the hexagonal unit cell (see Ref. [26]) implying that the in-plane anisotropy term B 6  6 is positive (B 6 6 > 0).Global search.Next we describe estimates of B m l and J ME for Er166 from comparison of magnetization and INS data.In this search, we fix the other parameters of the model (i.e.J MM i and K M ).Supplementary Figure 12 shows data from INS measurements of Er166 where we observe four CEF excitations out of the ground state at T = 5 K.These excitations are not visible at 200 K due to thermal depopulation of the ground state.The energies of the first four transitions were obtained from gaussian fits to the peaks and are listed in Table 3.
As shown in Fig. 7a of the main text, an applied magnetic field along the hard (0, 0, 1) direction will generate a first-order magnetization process (FOMP) where the ground-state FIM structure jumps from spins ordered with their orientations in the abplane (planar) to their orientations lying along the c-axis (uniaxial).The small value of the critical FOMP field [µ 0 H c = 0.7(2) T] indicates that the planar and uniaxial configurations are nearly degenerate with a free energy difference of ∆F = F c − F ab ≈ M µ 0 H c = 0.16 (5) meV, where M = 4 µ B /fu is the net magnetization per formula unit in the FIM ground state.
We can use a combination of INS and magnetization data to refine our estimates of these parameters.Supplementary Figure 13 shows a representative slice of a search in the 4-D space of B m l and J ME parameters.The indicated regions for the different observables correspond to the values in Table 3.In Supplementary Figure 13, the We use these terms to generate local mean-field Hamiltonians for Er and Mn and where B Mn = −2J ME ⟨S⟩ − i γ i J MM i ⟨s i ⟩ and B Er = −12J ME ⟨s⟩ are the selfconsistently determined molecular fields acting on Mn and Er, respectively.γ i is the coordination number for J MM i .The self-consistent solution to the mean-field Hamiltonian is obtained by minimizing the free energy where Z Er and Z Mn are the partition functions for Er and Mn obtained from Eqs. ( 8) and ( 9), respectively.
To simplify our analysis, we consider only uniaxial fields H ∥ c (i.e.H z ) and make an approximation where the planar Er magnetic-anisotropy energy (MAE) given by K ′ 3 sin 6 θ Er cos(6φ Er ) (where ) is fixed at its minimum value of −K ′ 3 sin 6 θ Er for any in-plane angle φ Er .This is equivalent to an easy-plane anisotropy with nonzero B 0 6 , as the θ dependence plays a critical role in controlling the phase stability.Furthermore, Fig. 6a in the main text shows that the true easy-plane limit (K ′ 3 ≈ 0) is valid at high temperatures (T > 100 K) due to rapid thermal softening of the planar MAE upon warming.This approximation effectively describes the low-temperature magnetic phases, such as FIM, where all moments lie in a vertical plane containing the planar easy-axis and c-axis field directions.
Zero-field case.For easy-plane configurations [K ′ 3 cos(6φ Er ) = const.]in zero field, only the relative angles between spins matter.In the specific case of the ideal-triplespiral structure, the periodicity is defined by the angle Φ between alike layers in adjacent cells (coupled by J MM 3 ) and the angle δ is between strongly coupled Mn bilayers (coupled by J MM 2 ).Supplementary Figure 15 (and Fig. 1 of the main text) shows the relative angles of spins in neighboring layers and their coupling constants.In Supplementary Figure 15, the Er moment is arbitrarily chosen to point along ( 1, 0, 0), bisecting the angle (Φ − δ) and the Mn moment directions.The Mn moments are specified by layer designations A, B, A+, A-, and B-and their orientations are given relative to the Er moment orientation.The spin vectors are where the thermally averaged spin magnitudes ⟨s⟩ and ⟨S⟩ are determined selfconsistently.
Vertical-field case.For a vertical magnetic field along the z (i.e.c) direction, we maintain the double spiral angles δ and Φ and now introduce the polar angles of the Er and Mn sublattices θ Er and θ Mn .Similar to the definitions above, we assume that the Er moment is pointed in the (1, 0, 1) plane and always negative in the x−direction.The spin vectors are S = S [− sin θ Er , 0, cos θ Er ] , The corresponding molecular fields are given by similar expressions as above and include an externally applied magnetic field along the z−axis.be evaluated independently for each layer, we rather minimize the total energy of the form + 3J MM Here, p is the number of unit cells containing an Mn-Er-Mn trilayer and j labels a single unit cell.Mn and Er moments adopt a planar angle ϕ i in each successive layer in the stack with i = 1 to 2p and i = 2p + 1 to 3p, respectively, with open boundary conditions.We find the minimum energy using the exchange parameters in Table 1 of the main text and for different values of the in-plane anisotropy parameter K ′ 3 .The mean periodicity is shown in Fig. 6b of the main text for a stack consisting of p = 36 unit cells (108 layers).

Fig. 1
Fig.1Ideal triple-spiral magnetic ordering.a Chemical structure of ErMn 6 Sn 6 with the unit cell indicated by thin lines.The hexagonal R166 compounds crystallize in the HfFe 6 Ge 6 -type structure (space group P 6/mmm, No. 191) with lattice parameters of a = 5.51 Å and c = 9.00 Å at room temperature[17,47].The individually ferromagnetic kagome-Mn (triangular-Er) planes are indicated in red (blue).Interlayer Mn-Sn bonds are displayed and interlayer-exchange interactions between Mn layers (J MM

Fig. 2
Fig.2Temperature-driven distorted-triple-spiral to ferrimagnetic ordering.a Magnetization divided by field versus temperature for a magnetic field of µ 0 H = 0.005 T applied perpendicular to the c crystalline axis.The inset shows a zoomed-in view of the high-temperature peak.b Diffraction pattern for (0, 0, l) reciprocal-lattice points and increasing temperature.Data for T < 105 K and T > 105 K are from two different experiments.A log scale is used for the intensity and white circles are the fitted centers of the (0, 0, 2 ± τ ) magnetic-Bragg peaks.Similar data for (1, 0, l) are given in Supplementary Section 1.1.c The temperature evolution of the integrated intensities of the (1, 0, 0) and (1, 0, τ ) Bragg peaks.Darker (lighter) symbols correspond to measurements made on cooling (warming).

Fig. 3 Fig. 4 Fig. 5 Fig. 6
Fig. 3 Higher harmonic magnetic-Bragg peaks.a Diffraction patterns along (0, 0, l) for the indicated temperatures demonstrating the presence of higher-harmonic satellite peaks.Vertical lines indicate the centers of the τ , 2τ , and 3τ peaks.b,c Diffraction patterns showing the splitting of the lineshapes of the (1, 0, τ ) b and (1, 0, 2τ ) c satellite peaks.Dashed green and orange lines show individual gaussian components of the lineshapes and vertical dotted lines indicate the center of each component.∆τ 1 and ∆τ 2 show the distances between the centers of the components.d The a and c lattice parameters plotted versus temperature as determined from single-crystal x-ray diffraction.eg Single-crystal x-ray diffraction data for T = 140 K taken across the (0, 0, 4 − 2τ ) e, (0, 0, 4) f, and (0, 0, 4 + 2τ ) g positions.The color bar scales are counts per second.The absence of Bragg peaks in e and g is evidence that the 2τ peaks are not structural in origin.Error bars indicate one standard deviation.