New insight into tuning magnetic phases of RMn₆Sn₆ kagome metals

Abstract

Predicting magnetic ordering in kagome compounds offers the possibility of harnessing topological or flat-band physical properties through tuning of the magnetism. Here, we examine the magnetic interactions and phases of ErMn₆Sn₆ which belongs to a family of RMn₆Sn₆, R = Sc, Y, Gd–Lu, compounds with magnetic kagome Mn layers, triangular R layers, and signatures of topological properties. Using results from single-crystal neutron diffraction and mean-field analysis, we find that ErMn₆Sn₆ sits close to the critical boundary separating the spiral-magnetic and ferrimagnetic ordered states typical for non-magnetic versus magnetic R layers, respectively. Finding interlayer magnetic interactions and easy-plane Mn magnetic anisotropy consistent with other members of the family, we predict the existence of a number of temperature and field dependent collinear, noncollinear, and noncoplanar magnetic phases. We show that thermal fluctuations of the Er magnetic moment, which act to weaken the Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy, dictate magnetic phase stability. Our results provide a starting point and outline a multitude of possibilities for studying the behavior of Dirac fermions in RMn₆Sn₆ compounds with control of the Mn spin orientation and real-space spin chirality.

Introduction

The kagome lattice has a two-dimensional corner-sharing triangular arrangement which supports frustrated electronic interactions. Its electronic band structure generally contains both flat bands and linear (Dirac-like) band crossings, both of which can lead to important correlated electronic phenomena; flat bands can give rise to itinerant magnetic correlations and Stoner-type magnetism, whereas linear band crossings create Dirac cones and the associated topological (Dirac) fermions^1,2,3,4,5. These latter features can lead to topological phenomena that intimately link charge and spin, such as dissipationless spin-momentum-locked (chiral) charge transport and quantum-anomalous-Hall effects⁶.

Kagome compounds with competing interlayer magnetic interactions are particularly important as they can offer the ability to tune the electronic states of the kagome layers by manipulating the magnetism. Hexagonal RMn₆Sn₆ (R166) metals with R = Sc, Y, Gd–Lu, fall into this category. These compounds have emerged as model layered systems for magnetic tuning due to the combination of largely defect-free magnetic Mn kagome nets hosting topological electronic bands and R-site magnetism that controls details of the magnetic order^{4,7,8,9,10,11,12,13,14,15,16}. Crucial to harnessing the interplay between the magnetic order and topological properties is understanding the microscopic magnetic interactions determining the magnetic order and how to manipulate or influence the magnetic interactions to create a desired magnetic state. Here, we address these challenges via a single-crystal neutron diffraction and mean-field analysis study of ErMn₆Sn₆.

As shown in Fig. 1a for Er166, R166 compounds feature Mn kagome bilayers separated by triangular R layers. Importantly, the magnetic layers individually exhibit ferromagnetic (FM) ordering of their magnetic moments (spins), and a majority of R166 compounds with magnetic R ions exhibit collinear ferrimagnetic (FIM) order driven by strong antiferromagnetic (AFM) coupling between the Mn and R layers¹⁷. For non-magnetic R, such as Y, long range and competing Mn-Mn interlayer couplings lead to spiral magnetic order^9,18,19, and applied magnetic fields lead to distorted and noncoplanar spin configurations where the topological-Hall effect and planar-anisotropic magnetoresistance have been observed^9,10,20,21. Thus, the strength of the Mn-R coupling controls the stability between collinear and noncollinear magnetism in the R166 compounds and we show in this work that Er166 sits close to the critical boundary between the two types of ordering.

Fig. 1: Ideal triple-spiral magnetic ordering.

a Chemical structure of ErMn₆Sn₆ with the unit cell indicated by thin lines. The hexagonal R166 compounds crystallize in the HfFe₆Ge₆-type structure (space group P6/mmm, No. 191) with lattice parameters of a = 5.51 Å and c = 9.00 Å at room temperature^17,47. The individually ferromagnetic kagome-Mn (triangular-Er) planes are indicated in red (blue). Interlayer Mn-Sn bonds are displayed and interlayer-exchange interactions between Mn layers (${{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}$) and Mn and Er layers (${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$) are shown as black double arrows. b Parallel and c top-down views of ideal-triple-spiral order. Numbers label the sequence of the Mn and Er planes along c and each arrow represents a ferromagnetic Er or Mn plane. Each plane has an ordered magnetic moment oriented perpendicular to the c axis and the moment orientation varies between layers as shown. The angles Φ and δ characterize the magnetic order, with the Er moment direction pointing antiparallel to the bisector of (Φ − δ). Φ = δ = 0 corresponds to the low-temperature ferrimagnetic phase with the Er and Mn moments pointing opposite to each another.

Another key aspect of R166 compounds is that R ions with nonzero orbital angular momentum adopt complex magnetic anisotropy through the crystalline-electric-field (CEF) splitting of their 4f orbitals. Easy-plane Mn anisotropy is present, but strong R anisotropy will dictate the Mn magnetization direction at low temperature and zero magnetic field. For the FM Mn layers, a uniaxial anisotropy is proposed to dramatically enhance the spin-orbit splitting of Dirac cones, forming a Chern insulator⁷. Further, the competition of R anisotropy with the easy-plane Mn anisotropy can result in temperature and field-driven spin-reorientation transitions between easy-plane and uniaxial (R = Tb) or tilted (R = Dy, Ho) FIM states. These transitions can facilitate magnetization switching via small changes in temperature or magnetic field^14,22,23 which can create or destroy a Chern gap⁷ or Weyl nodes²⁴. We find that the Er³⁺ ion has weakly uniaxial and strongly temperature-dependent anisotropy in Er166, resulting in facile magnetic switching. Small magnetic fields of μ₀H < 1 T can trigger spin-reorientation transitions where large anomalous-Hall and topological-Hall responses have been reported²⁵.

Here, we present single-crystal neutron diffraction and mean-field analysis results that illustrate how competition between interlayer magnetic interactions and magnetic anisotropy in Er166 leads to a number of nearly degenerate magnetic states. Using this information, we characterize and understand the temperature-driven first-order transition from planar-FIM to a distorted-triple-spiral order upon warming through T_spiral = 92(1) K, showing that spins in the Er layers remain magnetically ordered above T_spiral rather than their previously reported paramagnetic behavior²⁵. Using additional results from inelastic neutron scattering and magnetization measurements, we determine the key microscopic magnetic interactions and predict the emergence of various collinear, noncollinear, and noncoplanar magnetic phases as functions of temperature and magnetic field. Importantly, we find that the stability of the magnetic phases is controlled by thermal fluctuations of the Er spins which act to weaken the effective Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy.

Results and discussion

Experimental results

The magnetization M of Er166 for a weak magnetic field of μ₀H = 0.005 T applied perpendicular to the crystalline c axis is shown in Fig. 2a and the zero-field neutron diffraction pattern for (0, 0, l) reciprocal-lattice points is shown in Fig. 2b for increasing temperature. The main features in both figures agree with previously published data^22,26; with cooling, a peak in [M/H](T) in Fig. 2a at the Néel temperature of T_N = 348(1) K coincides with the emergence of satellite magnetic-Bragg peaks in Fig. 2b surrounding the (0, 0, 2) structural-Bragg peak; a large jump in [M/H](T) at T_spiral accompanies the disappearance of the satellite peaks. The primary satellites [those satellite peaks closest to (0, 0, 2)] correspond to a temperature-dependent AFM propagation vector of τ = (0, 0, τ) which indicates the presence of magnetic ordering with a temperature-dependent structure that is modulated along c. Analysis of satellites for different values of the h and l reciprocal-lattice coordinates indicates that the spins primarily lie in the crystalline ab plane. Second-harmonic (2τ) and third-harmonic (3τ) satellites are also present in the figure and are addressed below.

Fig. 2: Temperature-driven distorted-triple-spiral to ferrimagnetic ordering.

a Magnetization divided by field versus temperature for a magnetic field of μ₀H = 0.005 T applied perpendicular to the (c) crystalline axis. The inset shows a zoomed-in view of the high-temperature peak. b Diffraction pattern for (0, 0, l) reciprocal-lattice points and increasing temperature. Data for T < 105 K and T > 105 K are from two different experiments. A log scale is used for the intensity and white circles are the fitted centers of the (0, 0, 2 ± τ) magnetic-Bragg peaks. Similar data for (1, 0, l) are given in Supplementary Section 1.1. c The temperature evolution of the integrated intensities of the (1, 0, 0) and $(1,0,\bar{\tau })$ Bragg peaks. Darker (lighter) symbols correspond to measurements made on cooling (warming).

Figure 2 c shows that the integrated intensity of the $(1,0,\bar{\tau })$ primary satellite monotonically increases while cooling down to T_spiral and that a jump in the integrated intensity of the (1, 0, 0) Bragg peak accompanies the disappearance of the satellite at T_spiral. This jump signals the emergence of a magnetic-Bragg peak on top of the (1, 0, 0) structural-Bragg peak and is due to a transition to planar-FIM order with the spins continuing to lie in the ab plane (FIM-ab). The sharp jump and temperature hysteresis of the integrated intensity of the (1, 0, 0) peak around T_spiral are both consistent with a first-order transition. Figure 3d shows that the lattice parameters experience a small change in slope at T_spiral but there is no conclusive sign of an accompanying structural transition.

Fig. 3: Higher harmonic magnetic-Bragg peaks.

a Diffraction patterns along (0, 0, l) for the indicated temperatures demonstrating the presence of higher-harmonic satellite peaks. Vertical lines indicate the centers of the τ, 2τ, and 3τ peaks. b, c Diffraction patterns showing the splitting of the lineshapes of the (1, 0, τ) b and (1, 0, 2τ) c satellite peaks. Dashed green and orange lines show individual gaussian components of the lineshapes and vertical dotted lines indicate the center of each component. Δτ₁ and Δτ₂ show the distances between the centers of the components. d The a and c lattice parameters plotted versus temperature as determined from single-crystal x-ray diffraction. e–g Single-crystal x-ray diffraction data for T = 140 K taken across the (0, 0, 4−2τ) e, (0, 0, 4) f, and (0, 0, 4+2τ) g positions. The color bar scales are counts per second. The absence of Bragg peaks in e and g is evidence that the 2τ peaks are not structural in origin. Error bars indicate one standard deviation.

Previous reports attributed the transition at T_spiral to a loss of Er magnetic ordering for T > T_spiral²², leaving a Mn only double-spiral structure similar to that found for Y166^9,10,19. The double-spiral order is described by a small angle δ between the spin directions of the Mn bilayers [e.g. layers (1) and (2) in Fig. 1b and c], which are strongly coupled via the exchange interaction ${{{{\mathcal{J}}}}}_{2}^{{{{\rm{MM}}}}}$ indicated in Fig. 1a. A larger angle Φ = 2πτ describes a compound rotation of the spins between bilayers and encapsulates the periodicity of the spiral. For Er166, however, the temperature dependence of τ that we observe is much stronger than it is for Y166^9,20 and is similar to observations for the triple-spiral order of TmMn₆Sn_5.8Ge_0.2²⁷. In addition, we find that if we assume double-spiral order, the integrated intensity of the $(1,0,\bar{\tau })$ satellite would give a Mn ordered magnetic moment of μ_Mn ≈ 8μ_B at T = 108 K. This value is too large for Mn-only magnetic ordering. Furthermore, as described below, the temperature dependence of the integrated intensity is in reasonable agreement with the mean-field analysis which predicts triple-spiral magnetic order.

The ideal triple-spiral magnetic order is similar to double-spiral order but with the Er spins participating in the ordering and pointing antiparallel to the bisector of (Φ − δ). This is shown in Fig. 1b and c. The spins of the FM Er layer rotate in phase with the Mn spiral order, with the total triple spiral having a period of c/τ. As we show below, the mean-field results in combination with the single-crystal neutron diffraction data strongly support the presence of triple-spiral ordering between T_N and T_spiral. We also have performed a Rietveld refinement using FULLPROF²⁸ to powder neutron diffraction data taken at T = 200 K. Supplementary Section 1.2 shows that the refinement yields good agreement with the triple-spiral structure, with τ = 0.1876(6) and δ = 14.0(2)^∘, and reasonable values for the ordered Er and Mn magnetic moments of μ_Er = 3.9(3)μ_B and μ_Mn = 2.0(1)μ_B, respectively.

In addition to the temperature-dependent primary satellites, two other remarkable features appear in the neutron diffraction data. First, much weaker satellite peaks are evident in Fig. 2b that are 2τ and 3τ away from (0, 0, 2). These higher-harmonic satellite peaks are highlighted in the constant-temperature cuts in Fig. 3a. The second remarkable feature is the splitting of the lineshapes of individual satellites, as shown in Fig. 4. Both of these features suggest that the triple-spiral order is distorted in Er166 and we next address each feature in turn.

Fig. 4: Temperature-dependent splitting of the magnetic-Bragg peaks.

a (1, 0, l) neutron diffraction patterns for temperatures crossing the Néel temperature of T_N = 348(1) K. b The $(1,0,\bar{\tau })$ magnetic-Bragg peak for T = 345 and 98 K. Gaussian components of the lineshapes are shown by dashed lines, where the higher temperature lineshape has three gaussian components and the lower temperature lineshape contains two gaussian components. c (1, 0, l) neutron diffraction patterns for temperatures within the triple-spiral (T > T_spiral) and ferrimagnetic-ab (T < T_spiral) phases, where T_spiral = 92(1) K. d Temperature evolution of the center of the (1, 0, 1) Bragg peak and the centers of the gaussian components of the lineshapes for the (1, 0, ±τ) magnetic-Bragg peaks. Error bars indicate one standard deviation.

Odd-harmonic magnetic satellites are usually associated with squaring up of the AFM order²⁹ (such as “bunched-spiral” order) whereas it is much less common to observe even-harmonic magnetic satellites. Possible sources for even-harmonic satellites include distortions of the chemical lattice, magnetoelastic distortions of the ideal triple-spiral magnetic order, or the existence of fan-type magnetic order similar to that which occurs when applying an in-plane magnetic field to a magnetic spiral^30,31,32. Figure 3e–g show that we do not observe the 2τ peaks in single-crystal x-ray diffraction data, which suggests that the 2τ satellites arise from magnetic diffraction.

Regarding the possibility of fan-type magnetic order, we hypothesize that a net magnetization must exist within a magnetic domain in order for it to occur with no applied magnetic field. This could happen if the orientation of the Er spins is pinned by a strong CEF potential. However, both our neutron diffraction and magnetization measurements do not find evidence for a net magnetization existing between T_spiral and T_N, as we do not conclusively measure any magnetic contributions at integer (h, k, l) positions in the diffraction data and zero-field-cooled and field-cooled magnetization measurements made with μ₀H = 0.1 T do not reveal a domain population imbalance. Our mean-field modeling, described below, is also dismissive of zero-field fan-type order. Thus, we associate the 2τ peaks with an unknown distortion of the ideal triple-spiral magnetic order.

Next, the splitting of the lineshapes of the magnetic-Bragg peaks is also suggestive of a distortion of the ideal triple spiral. The cuts in Fig. 4a to c show that the lineshape of a primary-satellite peak is not described by a gaussian lineshape. Rather, depending on temperature, the lineshape is better fit by the sum of two (T ≤ 256 K) or three (T > 256 K) gaussian peaks and a background. This is exemplified in Fig. 4b for the $(1,0,\bar{\tau })$ primary satellite at 345 K and 98 K where the individual gaussian components are indicated by dashed lines.

A similar splitting of the primary-satellite lineshape has been reported for Y166^9,10,18 and Ga-substituted Y166^33,34, with the splitting for Y166 being characterized by Δτ₁ = τ⁺ − τ⁻ ≈ 0.05 at room temperature. Here, τ⁺ and τ⁻ refer to the centers of the two gaussian components contributing to the lineshape. As shown in Fig. 4b and d, we find that three rather than two gaussian components contribute to the lineshapes of the (1, 0, ± τ) primary satellites of Er166 around room temperature, with a somewhat smaller splitting of Δτ₁ ≈ 0.03 between components. Similar to Y166, Fig. 4d shows that the splitting is reduced with decreasing temperature, becoming barely resolvable at intermediate temperatures. However, further cooling results in an increase of the splitting where the third gaussian component never reemerges.

Like the double-spiral ordering of Y166⁹, the origin of the splitting of the primary-satellite lineshapes for the triple-spiral ordering of Er166 is not yet completely clear. We show in Supplementary Sections 1.2 and 1.3 that the lineshape splitting is observed for different single-crystal samples and even in the powder neutron diffraction data. This points to the splitting being an intrinsic property. The lineshape splitting could arise from different magnetic domains with slightly different spiral periods or it could result from beating of the spiral order within a single magnetic domain that results in long-period (∝1/Δτ₁) modulations of the magnetic structure. The single-domain origin of the lineshape splitting gains some credence from the observed broadening of the 2τ and 3τ satellites, which is consistent with splitting of their lineshapes. However, Fig. 3b and c surprisingly show that the splittings of the (1, 0, τ) and (1, 0, 2τ) lineshapes are unequal, with Δτ₂ ≈ 1.5Δτ₁. Though not completely understood, this observation provides strong evidence supporting the magnetic nature of the 2τ satellites and lends weight to the single-domain hypothesis for the lineshape splitting.

Magnetic Hamiltonian

The competition between exchange coupling and magnetic anisotropy makes the development of a microscopic magnetic model for Er166 challenging, but necessary to understand the magnetic phase stability. We define a magnetic Hamiltonian comprised of isotropic-exchange interactions (${{{{\mathcal{H}}}}}_{ex}$) between Mn-Er and Mn-Mn spins, a Zeeman term (${{{{\mathcal{H}}}}}_{{{{\rm{Z}}}}}$) for coupling spins to an externally applied magnetic field, and single-ion terms for the Er (${{{{\mathcal{H}}}}}_{{{{\rm{Er}}}}}$) and Mn ($6{{{{\mathcal{H}}}}}_{{{{\rm{Mn}}}}}$) crystallographic sites that capture the magnetic anisotropy.

The exchange Hamiltonian is given by

$${{{{\mathcal{H}}}}}_{ex}=\mathop{\sum}\limits_{k}\mathop{\sum}\limits_{i < j}{{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}{{{{\bf{s}}}}}_{i}\cdot {{{{\bf{s}}}}}_{j}+{{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\mathop{\sum}\limits_{\langle i < j\rangle }{{{{\bf{s}}}}}_{i}\cdot {{{{\bf{S}}}}}_{j},$$

(1)

where we label various intralayer and interlayer interactions between Mn spins s_i as ${{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}$ where k is a layer index. ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\, > \,0$ is the AFM coupling between neighboring Mn and Er spins S_j, and all of the pertinent interlayer interactions are diagrammed in Fig. 1a. The values of ${{{{\mathcal{J}}}}}_{k}^{{{{\rm{MM}}}}}$ are similar to those found for Tb166¹², and whereas the intralayer Mn-Mn interaction ${{{{\mathcal{J}}}}}_{0}^{{{{\rm{MM}}}}}$ is large and FM, it is known that the frustrated Mn-Mn interlayer (k > 0) interactions lead to competition between collinear and spiral phases^17,19. The Zeeman energy is given by

$${{{{\mathcal{H}}}}}_{{{{\rm{Z}}}}}=-{\mu }_{{{{\rm{B}}}}}(6g{{{\bf{s}}}}+{g}_{J}{{{\bf{J}}}})\cdot {\mu }_{0}{{{\bf{H}}}}$$

(2)

where g ≈ 2 for Mn, and g_J = 6/5, J = 15/2 for Er, giving g_JJ = 9.

Complex behavior of the magnetic anisotropy arises due to the action of the CEF potential of neighboring ions on the 4f orbital states of Er. The Hamiltonian for the CEF acting on Er is

$${{{{\mathcal{H}}}}}_{{{{\rm{Er}}}}}={B}_{2}^{0}{{{{\mathcal{O}}}}}_{2}^{0}+{B}_{4}^{0}{{{{\mathcal{O}}}}}_{4}^{0}+{B}_{6}^{0}{{{{\mathcal{O}}}}}_{6}^{0}+{B}_{6}^{6}{{{{\mathcal{O}}}}}_{6}^{6}$$

(3)

where ${B}_{l}^{m}$ are the CEF parameters for Er³⁺ with hexagonal point-group symmetry and ${{{{\mathcal{O}}}}}_{l}^{m}$ are Stevens operators. We find that our CEF parameters lead to uniaxial magnetic anisotropy for Er in the ground state, although barely so. The uniaxial Er anisotropy competes with the simple Mn easy-plane anisotropy given by

$${{{{\mathcal{H}}}}}_{{{{\rm{Mn}}}}}={K}^{{{{\rm{M}}}}}{s}_{z}^{2}$$

(4)

where K^M > 0. Easy-plane Mn anisotropy is consistent with the planar-FIM or helical ground states found in R166 compounds with magnetically isotropic (R = Gd) or non-magnetic (R = Sc, Y, Lu) ions, respectively^18,22.

A representative set of parameters for the magnetic Hamiltonian are shown in Table 1 and their estimation is described in Supplementary Section 1.4. Using these parameters, we next describe a mean-field analysis of the equilibrium states which provides semi-quantitative agreement with observations of the temperature-driven FIM to triple-spiral transition and field and temperature-driven spin-reorientation and spin-flop transitions. The strong intralayer Mn-Mn exchange justifies the assumption that each Mn and Er layer remains FM upon cooling or upon the application of a magnetic field, but both the direction and magnitude of the spins can vary from layer to layer. Thus, determination of the magnetic structure can require minimization of the free energy for spin configurations potentially spanning dozens of unit cells. To facilitate the calculations, we initially ignore planar anisotropy of the Er ion [i.e. the ${B}_{6}^{6}$ term in Eq. (3)] and consider only uniaxial applied fields.

Table 1 Heisenberg and crystal field parameters for ErMn₆Sn₆

Within a mean-field description, the magnetic structure is determined by minimizing the magnetic free energy described by four angles θ_Er, θ_Mn, Φ, and δ. θ_Er (θ_Mn) describes the polar angle of the Er (Mn) spins away from c, and Φ and δ are the spiral angles defined above. Minimization of the ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$ exchange energy requires that μ_Er points antiparallel to the bisector of (Φ − δ), as shown in Fig. 1c. With these constraints, the possible magnetic phases are planar-FIM (FIM-ab), uniaxial-FIM (FIM-c), vertical-plane-canted (VP-canted), planar-spiral, vertical-conical-spiral (VCS), and forced-FM (FF). Further details of the mean-field calculations are given Supplementary Sections 1.5 and 1.6.

Mean-field results and discussion

Competition between the weaker FM ${{{{\mathcal{J}}}}}_{1}^{{{{\rm{MM}}}}}$ and AFM ${{{{\mathcal{J}}}}}_{3}^{{{{\rm{MM}}}}}$ interactions lead to the zero-field spiral phases in the hexagonal R166s with non-magnetic R = Sc, Y, or Lu¹⁹. For R166s with magnetic R ions, the ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$ coupling destabilizes the spiral state in favor of collinear-FIM order, as shown in Fig. 5a. The spiral phase is the preferred T = 0 K state for ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\approx 0$ and as ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$ is increased, Φ → 0 and FIM-ab becomes the preferred ground state above a critical value of ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}\approx 1.1$ meV. Thus, the experimentally determined value of ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}=1.35$ meV for Er166 correctly predicts the FIM-ab ground state.

Fig. 5: Mean-field analysis of the zero-field magnetic phases.

a The evolution from planar-ferrimagnetic (FIM-ab) to ideal-triple-spiral magnetic order as a function of the Mn-Er interlayer interaction ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$. The angles Φ and δ are defined in Fig. 1c, and green arrows indicate the T = 0 K magnetic phases for Y166 (${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}=0$) and Er166 (${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}=1.35$ meV). b The evolution from FIM-ab to ideal-triple-spiral order with temperature. Dots are the experimental values for the spiral periodicity from Fig. 2b. The inset shows the reduction of the Er ordered magnetic moment with temperature due to thermal fluctuations. c The squares of the magnetic structure factors for ideal-triple-spiral and double-spiral order for the $(1,0,\bar{\tau })$ satellite peak calculated using the mean-field values for Φ, δ, and the Mn and Er ordered magnetic moments. The measured integrated intensity versus temperature for the $(1,0,\bar{\tau })$ satellite (left and right triangles) is also shown. Error bars indicate one standard deviation.

We have performed mean-field calculations for T > 0 K to test the stability of the FIM-ab phase of Er166 at higher temperatures. We find that with rising temperature the accompanying increase in thermal fluctuations of the Er spins destabilizes the FIM-ab phase in favor of the ideal triple spiral at T_spiral. This is exemplified by the plots in Fig. 5b which show that the ideal triple spiral becomes the equilibrium state above T = 75 K, which is in good agreement with the experimentally observed value of T_spiral = 92(1) K. Figure 5b also shows that the calculated temperature dependence of Φ aligns almost perfectly with the experimentally determined values of Φ from Fig. 2b. These results give very strong confidence in the chosen interlayer-exchange parameters shown in Table 1. Finally, Supplementary Figure 8 shows that only a small free energy difference exists between the FIM-ab and ideal triple-spiral phase. This is consistent with observations that the FIM-ab phase can be stabilized at T > T_spiral by a very small planar magnetic field³⁵.

Previous reports have ascribed the transition at T_spiral to a decoupling of Mn and Er magnetic sublattices and the complete loss of Er ordering²². Our mean-field and experimental results indicate that Er participates in the spiral, leading to the experimentally observed distorted-triple-spiral phase, although there is a substantial reduction of μ_Er with increasing temperature, as shown in the inset to Fig. 5b. Figure 5c shows the squares of the magnetic structure factors for the double-spiral and ideal triple-spiral orders calculated for the $(1,0,\bar{\tau })$ satellite peak using the mean-field determined parameters. The curve for the ideal triple spiral agrees well with the neutron diffraction data, showing concave-up behavior. The double-spiral curve, on the other hand, shows virtually no temperature dependence below T = 300 K. From these comparisons, we also understand that the continued shortening of the triple-spiral period with increasing temperature is caused by growing thermal fluctuations of the Er spins which progressively reduce μ_Er and the effective Mn-Er coupling.

For more insight into the microscopic details of the magnetism giving rise to a high-temperature triple-spiral phase, we next consider the six-fold planar anisotropy term ${B}_{6}^{6}$ for the Er and whether its proper treatment leads to fan-like phases or distortions of the ideal triple-spiral order that generate the 2τ and 3τ satellite neutron diffraction peaks. We begin by estimating the temperature dependence of the planar MAE by calculating the free-energy difference $\Delta {{{{\mathcal{F}}}}}_{\varphi }$ between FIM-ab phases with spins pointing either along the easy (φ_Er = 30^∘, φ_Mn = 210^∘) or hard (φ_Er = 0^∘, φ_Mn = 180^∘) planar axis. Figure 6a shows that thermal fluctuations of the Er spins leads to a rapid decrease of $\Delta {{{{\mathcal{F}}}}}_{\varphi }$ with increasing temperature, with the planar MAE being reduced by 95% at T_spiral ($\Delta {{{{\mathcal{F}}}}}_{\varphi }\,\approx \,0.035$ meV) and becoming negligible above T = 150 K.

Fig. 6: Role of planar magnetic anisotropy in establishing the magnetic phases.

a Temperature evolution of the planar-magnetic-anisotropy energy estimated as the free-energy difference between easy-axis (φ_Er = 30^∘, φ_Mn = 210^∘) and hard-axis (φ_Er = 0^∘, φ_Mn = 180^∘) spin orientations for a FIM-ab phase in the mean-field approximation. b Average spiral angles adopted by the T = 0 K minimum energy solution for planar spins in a stack of 36 Mn-Er-Mn layers and for different values of the planar-anisotropy parameter ${K}_{3}^{{\prime} }$.

We next examine whether a certain size of the planar-MAE constant ${K}_{3}^{{\prime} }$, where ${K}_{3}^{{\prime} }={J}^{(6)}{B}_{6}^{6}=\Delta {{{{\mathcal{F}}}}}_{\varphi }/2$, can lead to distorted-spiral order using results from classical energy-minimization calculations for T = 0 K and a stack of 36 Mn-Er-Mn layers. Figure 6b shows that larger values of ${K}_{3}^{{\prime} }$ do not lead to fan-like phases, but for moderate values of ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$ larger values of ${K}_{3}^{{\prime} }$ lead to a lock-in of the spiral periodicity at Φ = 60^∘ (τ = 1/6), where the six-fold planar MAE of the Er spins is fully minimized. We also find that nonzero values of ${K}_{3}^{{\prime} }$ lead to a first-order like jump in the spiral periodicity at T_spiral, in agreement with observations.

It is difficult to address whether distortions of the ideal triple-spiral order close to T_spiral can result from moderate values of ${K}_{3}^{{\prime} }\, <\, 0.02$ meV. Such distortions, for example, could lead to even and odd-harmonic satellite peaks caused by the pinning of the Er spin direction by the planar MAE. However, three factors suggest that the planar anisotropy plays no role in producing the higher-harmonic satellites: (1) the higher-harmonic satellites persist well above T = 200 K, where the Er-planar MAE is fully quenched; (2) there is no experimental evidence for a lock-in of the spiral periodicity to 60°; (3) anharmonicity of the triple spiral resulting from Er spin bunching is expected to produce only odd-harmonic satellites^36,37,38.

M(H) data for H∥c measured for various temperatures are shown in Fig. 7a. Similar to previous results²⁶, sharp steps to a plateau are visible for T < T_spiral. These steps correspond to a first-order magnetization process (FOMP) that coherently rotates the planar spins of the FIM-ab phase to lay along c, resulting in the FIM-c state. The critical FOMP field is small at low temperature (μ₀H_FOMP = 0.65 T) and increases with increasing temperature to a maximum of ≈ 4 T close to T_spiral.

Fig. 7: Field-induced magnetic phases and the role of polar magnetic anisotropy.

a Measured magnetization versus magnetic field data for several temperatures with the field applied parallel to (c). b Complementary results from mean-field calculations using the parameters in Table 1. c Results from mean-field calculations for the temperature evolution of the polar magnetic-anisotropy energy, which is estimated as the free-energy difference between uniaxial (θ_Mn = 0°, θ_Er = 180°) and planar (θ_Mn = θ_Er = 90°) spin orientations for a ferrimagnetic phase. K^M is the Mn anisotropy parameter.

We understand the magnetization data using our mean-field model by calculating the polar MAE which we define as the free-energy difference $\Delta {{{{\mathcal{F}}}}}_{\theta }$ between FIM-ab (θ = 90°) and FIM-c phases (θ = 0°). Figure 7c shows that the polar MAE is small at low temperatures, indicating the near degeneracy of FIM-ab and FIM-c phases caused by competing Er uniaxial and Mn easy-plane anisotropies. This is consistent with the small value for H_FOMP at T = 2 K seen by experiment, and our mean-field calculations for M(H) shown in Fig. 7b are similar to the experimental data in Fig. 7a, correctly predicting a small H_FOMP for low temperatures. Upon increasing the temperature, H_FOMP shifts to higher fields, indicating that thermal fluctuations and quenching of the Er anisotropy increase the net easy-plane anisotropy. Above T_spiral, the steps in M(H) broaden with increasing temperature until the curves show linear behavior, rather than a step, before plateauing. This departure from the step-like jump signals an (H, T) region where a gradual canting of the spins occurs with increasing field. The canting creates a VCS magnetic order which terminates in the FIM-c phase at higher field.

The increase of H_FOMP and disappearance of the FOMP above T_spiral is again due to increasing thermal fluctuations of the Er spins with increasing temperature, which in this case results in a temperature-driven decrease of the contribution of the Er uniaxial anisotropy to the MAE. Surprisingly, the single-ion anisotropy of Er attains a planar character above T ≈ 100 K, resulting in the maximum in $\Delta {{{{\mathcal{F}}}}}_{\theta }$ shown in Fig. 7c. At higher temperatures, Er contributions to the MAE become completely quenched by thermal fluctuations and the MAE approaches the classical value for the Mn ions of 6K^M = 1.02 meV. At all temperatures, the mean-field calculations of M(H) show qualitative agreement with the experimental curves which confirms the assignment of the FIM-ab, VCS, and FIM-c magnetic phases. This is despite a likely overestimation of the size of the Er spin thermal fluctuations by the calculations.

Given its success in describing the low-field phases, we employ the mean-field model to predict the magnetic phase diagram for larger values of H∥c. The results are shown in Fig. 8. A low-temperature metamagnetic transition from FIM-c to a VP-canted structure is predicted at μ₀H = 23 T, similar to experimental observations²⁶. This transition is driven by a flop of the Er spins into the plane perpendicular to H (Er-flop). Another metamagnetic transition occurs at 58 T into the FF phase through which the Er spins flip to be parallel to H (Er-flip). The first-order character of both transitions are dictated by the large ${B}_{4}^{0}$ Er CEF term which creates an energy barrier between uniaxial (θ_Er = 0°,180°) and planar (θ_Er = 90°) configurations of the Er spins.

Fig. 8: H∥c mean-field magnetic phase diagram.

The phases are: planar-ferrimagnetic (FIM-ab), uniaxial-ferrimagnetic (FIM-c), triple-spiral (3-spiral), vertical-conical-spiral (VCS), vertical-plane-canted (VP-canted), and forced-ferromagnetic (FF). Metamagnetic transitions are labeled as: first-order magnetization process (FOMP), Er-spin flop (Er-flop), and Er-spin flip (Er-flip). The horizontal dotted line indicates the FIM-c to FF crossover where the ordered Er magnetic moment is completely quenched. Red (blue) arrows show the orientation of the Mn (Er) spins in different layers.

Like the disappearance of the low-field FOMP above T_spiral, weakening of the Er anisotropy by thermal fluctuations leads to the disappearance of the VP-canted phase. The transition from FIM-c to FF also becomes continuous at higher temperature. Notably, the ordered Er moment is completely quenched at the FIM-c to FF crossover. Here, the exchange and Zeeman energies for the Er spins exactly cancel at a crossover field of $\approx 12{{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}({g}_{J}-1)/{g}_{J}{\mu }_{B}=47$ T.

Our experimental results and mean-field analysis have revealed the temperature and magnetic field responses of the magnetic order of ErMn₆Sn₆. Crucial to the compound’s magnetic tunability is competition between the various interlayer interactions and single-ion magnetic-anisotropy energies. At zero-field, the increase in thermal spin fluctuations with increasing temperature decreases μ_Er and results in a weakening of ${{{{\mathcal{J}}}}}^{{{{\rm{ME}}}}}$ and of the uniaxial Er anisotropy. This leads to the first-order transition from the FIM-ab to a distorted-triple-spiral phase at T_spiral. For a nonzero field, weakening of μ_Er by thermal fluctuations is responsible for the increase and eventual disappearance of the low-field FOMP above T_spiral as well as the elimination of the field-induced VP-canted phase. Remarkably, our calculations also reveal that at high temperatures μ_Er is effectively quenched at the crossover between FIM-c and FF order due to an exact cancellation of the exchange and Zeeman energies for the Er spins.

We have also investigated whether the observed splitting of the lineshapes of the magnetic-Bragg peaks in the triple-spiral phase is explained by the in-plane magnetic anisotropy. This splitting is also observed for Y166’s double-spiral order but with a different temperature dependence, begging the question of whether the Er anisotropy explains the differences we observe for Er166. Interestingly, recent work on GdV₆Sn₆ has found evidence for incommensurate amplitude-modulated magnetic order which produces diffraction signatures reminiscent of those seen for the double spiral of Y166³⁹. The amplitude-modulated order is associated with an RKKY mechanism within the Gd triangular sublattice, however, which is quite different than the Mn-Mn and Mn-Er interlayer exchange we discuss for Er166.

We conclude that an unknown distortion of the triple-spiral ordering of Er166 exists that is intrinsic to the material and our mean-field analysis verifies that the single-ion anisotropy of the Er, induced by CEF splitting, cannot be responsible in all instances for the distortions. We suggest that a non-sinusoidal layer-to-layer variation of the main spiral and a long-period modulation (beating) exist that occur within each magnetic domain. Similar distortions are often caused by the competition between isotropic-exchange interactions that favor uniform spiral phases and magnetic anisotropy that favors collinear or bunched-spiral distortions^36,37,38. Neither spin bunching resulting from anisotropy effects nor amplitude-modulated order, however, is expected to produce even-harmonic satellite diffraction peaks³⁸ and would not explain the 2τ satellites that we observe.

The general understanding of the interactions controlling the low-field spin-reorientation transitions, higher field spin-flop transitions, and the development of conical-spiral phases in R166 kagome metals that we have presented gives a roadmap towards studying the behavior of Dirac fermions with control of the Mn spin orientation and real-space spin chirality. In the FIM phases, the low critical field for the FOMP can be a viable source for controlling the Chern gap, which is maximized in the FIM-c phase. For example, magneto-optical transitions between valence and conduction bands of massive Dirac fermions can be switched using small applied fields^3,40,41. Triple-spiral phases in Er166 with tunable periodicity also are attractive for studying the role of vector spin chirality in transport and optical properties^42,43,44. Indeed, the Hamiltonian parameters listed in Table 1 are largely consistent and scalable across the RMn₆Sn₆ series, with the details of the R magnetic anisotropy and effective magnetic coupling between the R and Mn ions being responsible for determining the magnetic ground state^27,45. This latter point is born out by the results of our in-depth study and allows for further predictions of novel magnetic and topological phases across the entire R166 series.

Methods

Single-crystal magnetization and neutron diffraction

Single-crystals of ErMn₆Sn₆ (Er166) were grown from excess Sn flux as described previously¹². The samples were determined to be single-phase by x-ray diffraction. Magnetization M measurements were made on a Quantum Design, Inc., Magnetic Property Measurement System down to a temperature of T = 1.8 K and in magnetic fields up to μ₀H = 7 T. For measurements of the H∥c orientation, the plate-like samples were glued to a plastic disc and held inside a plastic drinking straw. Prior to measuring with the sample, the bare disc was measured for background subtractions.

Single-crystal neutron diffraction measurements were made on the Fixed-Incident-Energy Triple-Axis Spectrometer at the High-Flux Iostope Reactor, Oak Ridge National Laboratory. Neutrons with a wavelength of λ = 2.377 Å were selected by a double-bounce pyrolitic-graphite (PG) monochromator system and a PG analyzer using the PG (0, 0, 2) Bragg reflection. Söller slit collimators with collimations of $4{0}^{{\prime} }$-$4{0}^{{\prime} }$-$4{0}^{{\prime} }$-$8{0}^{{\prime} }$ were placed before the monochromator, between the sample and monochromator, between the sample and analyzer, and between the analyzer and detector, respectively. PG filters placed after each monochromator were used to reduce contamination by higher-order wavelengths. The sample was mounted on an Al sample holder and cooled in a He closed-cycle refrigerator while either immersed in He exchange gas (T < 300 K) or in vacuum. Two different samples were studied with masses of 109.0(1) mg and 282.6(1) mg over two successive experiments. Both samples were aligned with the (h, 0, l) reciprocal-lattice plane set in the scattering plane of the instrument. Diagrams of the chemical structure were made using VESTA⁴⁶.

Powder neutron diffraction

Powder neutron diffraction measurements were made on the time-of-flight diffractometer POWGEN at the Spallation Neutron Source, Oak Ridge National Laboratory. 6.874 g of Er166 powder was loaded into a 6 mm diameter single-wall vanadium can. The powderized sample was obtained by grinding several single crystals. The sample was cooled using a He closed-cycle refrigerator and the automatic sample changer (PAC) was used. Data were collected using the high-resolution setting.

Single-crystal X-ray diffraction

High-resolution single-crystal x-ray diffraction measurements were performed at Ames National Laboratory using a four-circle diffractometer with Cu K_α1 radiation from a rotating-anode source and a Ge (1, 1, 1) monochromator. The sample was attached to a flat Cu mount which was thermally anchored to the cold head of a He closed-cycle refrigerator. Be domes were used as vacuum shrouds and heat shields. A small amount of He exchange gas facilitated thermal equilibrium.