Introduction

With distinguished symmetry and associated geometrical frustration, the kagome lattice can host peculiar states including flat bands1, Dirac fermions2,3, unconventional superconductivity4,5 and spin liquid phases6,7. Magnetic kagome materials are an ideal setting in which strongly correlated topological electronic states may emerge8,9,10,11,12,13,14. In particular, transition-metal-based kagome magnets1,2,3,6,7,8,15,16,17,18,19,20 are emerging as outstanding candidates for such states, as they feature both large Berry curvature fields and unusual magnetic tunability. As an example, we established Co3Sn2S2 as a material that hosts frustrated magnetism in the kagome lattice8 and find that the volume-wise magnetic competition drives the thermodynamic or quantum evolution of the intrinsic anomalous Hall conductivity (AHC), thereby tuning its topological state. A number of nontrivial magnetic phases and a large topological Hall effect was also observed in another rare earth-transition metal based system YMn6Sn620,21,22. A new nematic chirality mechanism, which originates in frustrated interplanar exchange interactions that trigger strong magnetic fluctuations, was discussed as the reason for the topological Hall effect. This suggests that detailed experimental studies to understand the magnetic fluctuations, as well as the static magnetic structure of these systems, is necessary to reveal the coupling between relativistic electrons and the magnetic properties.

In the so-called 166 materials, TbMn6Sn6 contains the heavy rare earth Tb and crystallizes in a HfFe6Ge6-type structure (space group P6/mmm) composed of hexagonal Tb layers containing Sn atoms and Mn kagome nets, stacked in the sequence -Mn-Tb-Mn-Mn-Tb-Mn- along the c-axis (see Supplementary Note 1 and Supplementary Fig. 1)23,24,25,26,27,28,29. Structurally, compared with the Co3Sn2S2 system consisting of a Co3Sn kagome layer and a Sn2 honeycomb layer, the RMn6Sn6 system features a pristine Mn3 kagome layer (Fig. 2a), as the Sn atoms are pushed away from the kagome layer by the chemical pressure from the R atoms. TbMn6Sn6 has different Tb and Mn magnetic sublattices. Previous neutron diffraction studies indicated that this compound exhibits in-plane ferrimagnetic ordering between Tb and Mn sublattices below the Curie temperature TC = 423 K23,26, due to the strong Tb-Mn exchange interaction. It was also reported that at TC2 = 320 K23,26, a spin reorientation transition occurs and the easy magnetization direction changes from the ab-plane at high temperatures to the c-axis at low temperatures. From the electronic point of view, TbMn6Sn6 exhibits Shubnikov-de Haas quantum oscillations with nontrivial Berry phases at relatively low fields (from ~ 7 T), a large AHC (0.14 e2/h per Mn kagome layer) arising from Berry curvature fields, and quasi-linear ( H1.1) magnetoresistance (MR) likely resulting from linearly dispersive electrons13,18. Moreover, TbMn6Sn6 was found to demonstrate a bulk-boundary correspondence between the Chern gap and the topological edge state, as well as Berry curvature field correspondence of Chern-gapped Dirac fermions13,18. Thus, it is identified as a promising topological magnetic system13,18. Other than the identified high static magnetic ordering temperatures of TbMn6Sn6, microscopic studies of spin fluctuations and its tunability remain largely unexplored.

Here we utilize high resolution muon-spin rotation (μSR), a very powerful local magnetic probe, in combination with magnetization, powder and single crystal neutron diffraction to systematically characterize the phase diagram, uncovering intriguing slow magnetic fluctuations in TbMn6Sn6 in a wide temperature range down to 1.7 K. These fluctuations slow down below \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K and form static patches with an ideal out-of-plane ferrimagnetic order below TC1 20 K. TC1 is considered as a magnetic crossover temperature signaling a slowing down of magnetic fluctuations rather than being regarded as a true phase transition temperature. We further show that a hydrostatic pressure of 2.1 GPa stabilises the static out-of-plane topological ferrimagnetic ground state in the whole volume of the sample. We also find that the AHC does not follow the temperature evolution of the magnitude of the ordered Tb and Mn moments, but it is strongly influenced by the observed low-temperature magnetic crossover. We discuss the possible impact of the observed low-T magnetic crossover on the topological transport properties of TbMn6Sn6.

Results

Macroscopic magnetic properties

Temperature- and field-dependent magnetization experiments were performed in zero-field-cooled (ZFC) and field-cooled (FC) conditions for low applied fields of μ0H = 5 mT and 70 mT, as shown in Fig. 1a, b. The field was applied both in-plane (Fig. 1b) and out-of-plane, along the crystallographic c-axis (Fig. 1a). We observe a large and sharp transition occurring at TC2 320 K, corresponding to the spin reorientation transition noted in previous neutron diffraction studies23,26. This marks the transition between a high-temperature in-plane magnetic phase to a low-temperature ferrimagnetic phase with magnetic moments oriented out-of-plane, along the crystallographic c-axis. More importantly, when the magnetic field was applied along the c-axis, we observed a large reduction of zero-field cooled susceptibility χZFC below \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K and settling into a negligibly small slightly diamagnetic plateau below TC1 20 K. On the other hand, the field cooled susceptibility χFC shows a weak temperature dependence across 120 K and down to the base-T. This gives rise to a much larger hysteresis in TbMn6Sn6 below 120 K than in the T-range of 120 K–320 K. The plateau completely disappears with the application of higher magnetic fields, as it is not visible with the application of even a modest field of 70 mT. The reduction of χZFC is also less pronounced in 70 mT. The susceptibility, measured for a field applied in the ab-plane, is largely insensitive to the low-T transition (see Fig. 1b). So, the large difference between the FC and ZFC response is not caused by the appearance of the in-plane structure but, rather, is consistent with the scenario that different out-of-plane ferrimagnetic domains tend to cancel out (anti-align) after ZFC. If we FC the sample even in low fields, then the domains align. This suggests that magnetic states above and below 120 K are c-axis aligned, but slightly different from each other. This conclusion is also substantiated by the field dependent measurements of ZFC magnetization at T = 1.8 K, 250 K, and at 350 K, measured for the field applied along the c-axis (Fig. 1c) and along the ab-plane (Fig. 1d). The critical fields above which the magnetization plateau appears are similar for T = 1.8 K and 250 K. The only difference is that the data collected at 1.8 K show a plateau up to 12 mT, before increasing and then entering the linear regime, implying that the plateau, which is a characteristic feature for the low-T state, is easily destroyed by weak applied fields. For H c, the saturated magnetization does not occur at 350 K until a relatively high applied magnetic field of  1.2 T (inset of Fig. 1c), much higher than 175–200 mT at 5 K and 250 K (Fig. 1c). Conversely, when the field is applied along the ab-plane, saturation is almost immediately achieved upon application of 40 mT at 350 K, while at the base temperature the plateau was never achieved even under maximum applied field (7 T). These results are compatible with the in-plane ferrimagnetic order at 350 K.

Fig. 1: Temperature and field dependent macroscopic magnetic properties.
figure 1

a, b The temperature dependence of the zero-field cooled (sample was cooled down to the base-T in zero magnetic field and the measurements were done upon warming) and field-cooled (the sample was cooled down to the base-T in an applied magnetic field and the measurements were done upon warming) magnetization, measured in an field of 5 mT and 70 mT, applied along the (001) direction (a) and applied within the ab-plane (b). Arrows mark the magnetic transition temperatures TC1, \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) and TC2. c, d The field dependence of the zero-field cooled magnetization, recorded at various temperatures with the field applied along the (001) direction (c) and within the ab-plane (d). The error bars in the plots represent the standard error. The error bars are smaller than the data points.

Determination of magnetic structure

The temperature dependence of the sample magnetization clearly uncovers a magnetic transition/crossover at \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K. In order to characterize this transition, neutron scattering experiments were performed from 2 to 250 K using high resolution neutron powder30,31 and single crystal diffraction (see the Supplementary Note 2 and the Supplementary Figs. 24). From powder diffraction, the crystal structure was well refined by the Rietveld method, employing a hexagonal lattice structure in the space group P6/mmm, No. 191. Lattice constants a and c show a smooth monotonous decrease with decreasing temperature and a plateau below 50 K, as depicted in Fig. 2b. To solve the magnetic structure, the collected diffraction patterns were first refined with powder matching via the LeBail method with FullProf32,33,34, which confirmed the propagation vector at k = 0 found by previous studies23,26. Symmetry analysis shows that the maximal most symmetric subgroup mGM2+ (\(P6/mm^{\prime} m^{\prime}\) No. 191.240, see Fig. 2a) fits the data from 2 K to 250 K best, with a similar χ2 to the fit obtained via the LeBail method, implying that the model can hardly be improved. The temperature dependence of the estimated magnetic moments of Tb and Mn are shown in Fig. 2c. They both increase monotonically with decreasing temperature down to 100 K and stay nearly constant below this temperature.

To further explore the low temperature magnetic structure, single crystal neutron diffraction measurements were performed (see Fig. 2) both in the ordered state and in the paramagnetic regime (T 435K). The ideal out-of-plane ferrimagnetic c-axis aligned \(P6/mm^{\prime} m^{\prime}\) structure has the best fit for the data collected at 150 K and 300 K (see the Supplementary Note 3 and the Supplementary Fig. 4); however, at the lowest temperature, 5 K, the out-of-plane ferrimagnetic P2/m, No. 10.42 achieved a slightly better fit to the data. This lower-symmetry structure allows for three different Mn sites and a mixing of two irreducible representations. Both \(P6/mm^{\prime} m^{\prime}\) and P2/m are characterised by a perfectly c-axis aligned structure. Any incommensurability or out-of-plane canting of the magnetic moments were excluded by additional state-of-the-art Small Angle Neutron Scattering (SANS) measurements over a broad range of momentum transfer.

At 2 K, the magnetic order stays c-axis aligned under the in-plane magnetic field as high as 10 T (see Fig. 2d). At 230 K, the spin reorientation transition from the c-axis to the ab-plane is induced, signaled by the abrupt emergence of the (001) peak at a critical in-plane magnetic field of 4.8 T (see Fig. 2d, see also the Supplementary Note 3 and the Supplementary Fig. 5).

Fig. 2: Crystal and magnetic structures of TbMn6Sn6.
figure 2

a Magnetic structure of TbMn6Sn6. Φ is the deviation angle from the c-axis. Φ = 0 indicates out-of-plane ferrimagnetic order and Φ = 90 indicates the in-plane ferrimagnetic order. The Mn atoms construct a kagome lattice (red middle size circles), while the Tb atoms (turquoise larger circles) form a honeycomb structure. b The temperature dependence of the lattice constants a and c in TbMn6Sn6. c The temperature dependence of the terbium and manganese magnetic moments. d The field dependence of the (001) peak, recorded at 2 K and 130 K. The error bars represent the standard deviation of the fit parameters.

Microscopic details of static and dynamic magnetic state

To gain further insight into the intriguing magnetic properties of TbMn6Sn6, we employed the μSR technique, which serves as an extremely sensitive local probe for detecting microscopic details of the static magnetic order, ordered magnetic volume fraction, and magnetic fluctuations. The local probe feature makes μSR a perfect complementary technique to neutron diffraction and magnetization measurements.

Figure 3 a displays representative zero-field (ZF) μSR time spectra for TbMn6Sn6 taken at various temperatures in the range from 1.8 K to 400 K. ZF μSR spectra reveal coherent oscillations of the muon spin, indicating the existence of a well-defined internal field at the muon stopping sites in the sample in the whole investigated temperature range. This signal is expected for well defined long-range magnetic order. Either a single or a superposition of two distinct precession frequencies can be clearly seen in the μSR spectra. To better visualize the magnetic response, we show the Fourier transform amplitudes of the oscillating components of the μSR time spectra as a function of temperature (Fig. 3b, c), which is a measure of the probability distribution of internal fields sensed by the muon ensemble. We observe only one low field component below 400 K (Component III in Fig. 3c), then a coexistence of a low and high field components (Component III and Component II, respectively) between 330 K and 315 K (Fig. 3c), and the high field component persists down to the base-T (Fig. 3b, c). Below 20 K a second higher component appears (Component I), which becomes the dominant component at the base temperature of 1.7 K (Fig. 2b). The spectra below 20 K are also characterized by loss of a few percent of the initial asymmetry.

Fig. 3: Zero-field muon-spin rotation signals for TbMn6Sn6.
figure 3

a Zero-field μSR time spectra, recorded at various temperatures in the temperature range between 1.8 K to 370 K. The solid lines are the fit of the data using Eq. (2). s. Error bars are the standard error of the mean in about 106 events. The error of each bin count n is given by the standard deviation of n. The errors of each bin in the muon-spin rotation time spectra A(t) are then calculated by the standard error propagation. b, c Fourier transform amplitudes of the oscillating components of the muon-spin rotation time spectra, indicating the internal field distribution P(B), as a function of temperature.

The temperature dependences of the internal magnetic fields and the relative fractions of the three internal field components for the single crystal of TbMn6Sn6 are shown in Fig. 4a, b, respectively. There is a sharp and large (by a factor of six) increase of the internal field across the high-temperature in-plane ferrimagnetic to a low-temperature c-axis ferrimagnetic phase transition temperature TC2 320 K, as shown in Fig. 4a. However, there is a temperature range of about 15 K (marked by a vertical grey line) in which both in-plane and c-axis ferrimagnetic phases coexist in the sample but are macroscopically phase separated (see Fig. 4b), pointing to the first-order nature of the phase transition at TC2. Upon lowering the temperature below 310 K, the internal field for the out-of-plane component monotonously decreases down to the lowest temperature of 1.7 K. The internal field of the additional component monotonously increases below 25 K and saturates below 10 K. As can be clearly seen in Fig. 4b, the fraction of Component I increases at the cost of Component II. Component I eventually attains a volume fraction of 90% at 1.7 K and thus becomes the dominant state. Additionally, we obtain the direction of the internal magnetic field at the muon site by evaluating the data from all four positron detectors surrounding the sample: Forward-Backward (F-B) and Up-Down (U-D) (see Fig. 4e, f) (details are given in the Supplementary Note 4 and the Supplementary Fig. 6). The measured amplitude FUD of the oscillations detected on U-D detectors shows the maximum amplitude (nearly 100%) in the temperature range between 1.7 K and 310 K and no oscillations are found on F-B detectors. This indicates that the static internal field is pointing towards the c-axis and no spin reorientation takes place down to 1.7 K. Above 310 K, the magnitude of FUD is reduced by a factor of two, which indicates that the internal field is pointing somewhere in the ab-plane, which is consistent with the spin reorientation transition from c-axis to the ab-plane.

Fig. 4: Static order and dynamic fluctuations in TbMn6Sn6.
figure 4

a The temperature dependences of the internal magnetic fields for the three magnetic components. Vertical lines mark the critical temperatures TC1 and TC2. TC2 is the transition temperature from high temperature low field to low temperature high field component, while TC1 is the transition temperature, below which two high field components, exhibiting volume wise competition, are observed. The error bars represent the standard deviation of the fit parameters. Right axis depicts the amplitude of the oscillating component of the muon-spin rotation signal from up-down (U-D) positron detectors. b The temperature dependences of the relative volume fractions (f1, f2, f3) of the three magnetically ordered regions. c Zero-field muon-spin rotation signals from Forward-Backward (F-B) positron detectors, recorded at various temperatures. d The temperature dependence of dynamic depolarization rate of the muon-spin rotation signal, measured in F-B positron detectors. Arrows mark the magnetic transition temperature TC1 and the temperature \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) for the onset of visible magnetic fluctuations. Inset of panel (d) shows the muon stopping site within the structure of TbMn6Sn6. e A schematic overview of the experimental setup for the muon spin forming 45 with respect to the c-axis of the crystal. The sample was surrounded by four detectors: Forward (F), Backward (B), Up (U) and Down (D). f Schematic illustration of the muon spin precession around the internal magnetic field for two cases: The field is perpendicular to the c-axis and points towards the U-detector. θ is the angle between the magnetic field and the muon spin polarization at t = 0. The field is parallel to the c-axis of the crystal and points towards the F-detector.

We note one important aspect. If the magnetic order would be fully static with the internal field pointing along the c-axis, only a weak depolarization of the μSR signal would be observed in the F-B detector. In contrast, a fast depolarization of the implanted muons is seen in a wide temperature range (see Fig. 4c). The fast depolarization of the μSR signal in the F-B detector is direct evidence for the involvement of fluctuations in the magnetic state of TbMn6Sn6 in the time window of the technique (10−5 to 10−12 s). That the fluctuations are the cause of the observed muon spin depolarization is also supported by the measurement under 300 mT applied in the F-B direction (longitudinal field geometry), which show a negligible field effect on the depolarization rate at 300 K (see the Supplementary Note 4 and the Supplementary Fig. 6). As seen in Fig. 4c, the depolarization rate λL of the μSR signal in the F-B detectors shows a weak temperature dependence down to \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K, below which λL2 shows a large increase. It reaches its maximum at TC1 20 K and decreases rapidly at lower temperatures. The amplitude of the dynamic signal also decreases below 20 K, but it still exists even at the base-T of 1.7 K. The increase of λL2 below \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) is the signature of a slowing of magnetic fluctuations until a quasi-static order is established below TC1 and a coherent precession with slightly higher frequency than the original is seen. Therefore, the TC1 should be considered as a crossover temperature signaling a slowing down of magnetic fluctuations rather than being regarded as a true phase transition temperature. Moreover, even at the base-T both static patches (which correspond to the higher frequency) and dynamic patches (which correspond to the lower frequency) coexist. Using the moment sizes determined from neutron experiments and the c-axis ferrimagnetic structure, muon stopping site and local field calculations (see Methods for more details) find a dipolar field of [0 0 1.41] T at the only stable muon stopping site, which is in good agreement with the high-frequency Component I. Any deviation from this c-axis-aligned structure will therefore produce a lower internal field; thus, muon stopping site calculations show that the structure within the static patches below 20 K is perfectly c-axis aligned. For fluctuations faster than the μSR time window, the technique probes their time-averaged structure. Therefore, the combined observations of both static coherent oscillations and magnetic fluctuations between 20 K and 310 K can be best understood if the time averaged (ferrimagnetic) structure has a net static internal field aligned with the c-axis, but with smaller time-averaged c-axis moment. This explains the slightly lower (by 15%) internal field at the muon stopping site created by fluctuating patches.

Hydrostatic pressure μSR measurements

For further insight into the magnetic order and the low-temperature magnetic crossover in TbMn6Sn6, ZF-μSR experiments were carried out as a function of hydrostatic pressure. The probability distributions of internal fields sensed by the muon ensemble measured at p = 0 GPa and 2.1 GPa for various temperatures are shown in Fig. 5a, b, respectively. Figure 5c shows the field distribution measured at base-T of 2.5 K as a function of pressure. The fraction of the higher frequency Component I, arising from patches with the static c-axis order, increases with pressure and eventually attains the full volume fraction at 2.1 GPa. This implies that increased pressure stabilises the static out-of-plane ferrimagnetic ground state. From temperature dependent measurements, we also find that under a pressure of 2.1 GPa, the onset temperature below which Component I appears is higher by 5 K in comparison to ambient pressure. These findings demonstrate the volume-wise competition between static and dynamic magnetic patches in this material can be tuned with pressure.

Fig. 5: Hydrostatic pressure promoting the static out-of-plane ferrimagnetic state in TbMn6Sn6.
figure 5

Measured internal field distribution P(B) recorded at various temperatures for the ambient pressure p = 0 GPa (a) and maximum applied pressure of p = 2.1 GPa (b). c P(B) measured at the base-T of 2.5 K at various pressures.

Discussion

The combination of powder and single crystal neutron diffraction establishes the c-axis ferrimagnetic \(P6/mm^{\prime} m^{\prime}\) structure in TbMn6Sn6 below TC2 = 310 K. μSR and macroscopic magnetization measurements uncover a transition in TbMn6Sn6 at \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K. μSR shows that the transition at \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) is related to a slowing down of magnetic fluctuations and is considered a crossover rather than a true phase transition. These magnetic fluctuations only become static below TC1 20 K, as seen by the additional precession frequency in the μSR signal. Considering the fact that neutron diffraction depicts the static order in the whole temperature range, we conclude that the moments fluctuate at a rate of the order of MHz, yet slower than the nearly instantaneous (ns-ps) time scale of neutron scattering. As μSR probes the time averaged structure for fluctuations faster than the time window, the combination of static coherent oscillations and magnetic fluctuations in the μSR signal can be understood if the time averaged structure at high temperatures is such that the net static internal field points along the c-axis. Muon stopping site calculations and local field analysis show that Component I arises from the static ideal c-axis order and Component II stems from dynamic fluctuations with smaller time averaged c-axis moment. The fact that the fraction of Component I increases at the cost of Component II with decreasing temperature indicates that the slowing down process proceeds in the following fashion: when the fluctuations slow down and reach static enough conditions that we start observing the coherently precessing μSR signal below TC1 20 K, the patches of such static regions increase with decreasing temperature and the fraction of the corresponding signal increases. However, even at the base-T of 1.7 K, Component I does not acquire the full fraction and the moments are not yet fully static. This suggests that the establishment of a full volume static c-axis-aligned ferrimagnetic \(P6/mm^{\prime} m^{\prime}\) order might appear only in the zero-temperature limit. The application of hydrostatic pressure supports this ideal c-axis-aligned phase, allowing for us to achieve a 100% volume fraction at finite temperatures with the application of moderate pressure. The presence of magnetic fluctuations can also explain the magnetization data. In the SQUID measurements, which is a slower probe than μSR, a large difference between the FC and ZFC response is observed below \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K, e.g., in the region where fluctuations slow down. Since domains become more static, different out-of-plane ferrimagnetic domains tend to cancel out (anti-align) after ZFC. The domains try to minimize the total magnetization. However, if we FC even in low fields, then the domains align with the external fields instead. At temperatures higher than \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K, the domains are highly dynamic on microsecond time scales and the overall effect is averaged out and no hysteresis is registered in the magnetization data.

The interplay between the observed intricate magnetism and the spin-orbit coupled band structure seems to further induce non-trivial variations of its topological properties, which is characterized by the influence of the low temperature magnetic crossover on the temperature evolution of the anomalous Hall conductivity as well as by the appearance of a large quasi-linear MR and the quantum oscillations (see the Supplementary Note 5, 6 and the Supplementary Fig. 7). The dynamics of the electrons are much faster than the time scale of these slow magnetic fluctuations. Therefore, one can think of the electrons moving in the “static” random field created by the domain distributions that affects the coupling of the Dirac electrons to the local magnetic structure and the effective topological Chern gap13. Moreover, there may be parasitic conduction channels created by the slowly fluctuating domain walls etc. that affect transport. Furthermore, the slow fluctuations and broad transition from \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) to TC1 may involve more of the low energy Dirac fermions, such that they may have a spin component in the plane that do not open a topological mass gap solely by in-plane spin-orbit coupling. In fact, even below TC1, we would expect magnetic fluctuations from the ordered ferrimagnet (such as spin waves etc.), that would still have an effect in reducing the topological response, but the emergence of dense ordered static patches certainly helps protect these from the slow dynamical fluctuations of the domains. Since we show the presence of out-of-plane ferrimagnetic structure (comprised by Tb and Mn moments) with magnetic fluctuations below 320 K, the exciting perspective arises of a magnetic system in which the topological response can be obtained at room temperature just by switching slow fluctuations into a static state e.g., without substantial modification of magnetic structure.

Conclusions

The exploration of topological electronic phases that result from strong electronic correlations is a frontier in condensed matter physics. Kagome lattice systems are an ideal setting in which strongly correlated topological electronic states may emerge. Our key finding is the identification of a low-temperature magnetic crossover in TbMn6Sn6, which seems to be intimately coupled to its topological properties. Namely, a combination of neutron diffraction, μSR and magnetization measurements indicate that the system TbMn6Sn6 exhibits out-of-plane ferrimagnetic \(P6/mm^{\prime} m^{\prime}\) order between Mn and Tb moments with slow magnetic fluctuations in a wide temperature range 1.7 K–315 K. The moments fluctuate at a rate on the order of MHz, yet slower than the nearly instantaneous time window of neutron scattering. The fluctuations start to slow down below \({T}_{{{{{{{{\rm{C1}}}}}}}}}^{* }\) 120 K, forming quasi-static patches only below TC1 20 K and become enhanced in a volume-wise manner upon further lowering of temperature. The established quasi-static magnetic state below TC1 takes the ideal out-of-plane ferrimagnetic structure. The application of pressure increases the volume fraction of such patches and pressure of 2.1 GPa stabilises static out-of-plane ferrimagnetic ground state in the whole volume of the sample. These experiments reveal a complex low-temperature magnetic competition, which will stimulate further theoretical studies to obtain a microscopic understanding of the relation between the slow fluctuations, low-temperature volume-wise magnetic evolution of the static c-axis ferrimagnetic patches and the topological electronic properties.

Methods

General remarks

We concentrate on the high resolution35,36 muon spin relaxation/rotation (μSR) measurements of the temperature dependence of the magnetic moment as well as on the magnetically ordered volume fraction in the single crystals of TbMn6Sn6 and on the high resolution neutron powder diffraction of the magnetic structure at various temperatures. In a μSR experiment, positive muons implanted into a sample serve as an extremely sensitive local probe to detect small internal magnetic fields, ordered magnetic volume fractions and magnetic fluctuations in the bulk of magnetic materials37. Density functional theory calculations were used to explore the electronic band structure and to calculate the Berry curvature-induced anomalous Hall conductivity. Neutron diffraction is used to determine the magnetic structure. The techniques of μSR, neutron diffraction, transport and DFT complement each other ideally as we are able to study the detailed temperature dependence of the magnetic order parameter and ordered volume fractions with μSR experiments, and correlate them with the measured and calculated anomalous Hall conductivity.

Sample preparation

Details of the sample preparation, characterization, and the Hall effect measurements are described elsewhere13,18.

High resolution neutron powder diffraction measurement

The magnetic and crystal structure of TbMn6Sn6 have been studied with neutron powder diffraction (NPD) experiments carried out at the Swiss Neutron Spallation Source (SINQ), at the Paul Scherrer Institute in Villigen, Switzerland. Several single crystal samples were crushed into a fine powder and loaded into a 6 mm diameter vanadium can. The diffraction patterns were collected on the High Resolution Powder diffractometer for Thermal neutrons (HRPT) using wavelengths λ = 1.8857 Å and λ = 2.449 Å (Ge-monochromator, 2θmax = 160, 2θstep = 0.1) in the temperature range from 1.50 to 306 K. High-statistic acquisitions for magnetic structure refinements were made.

Temperature and field dependent single crystal neutron diffraction measurements

Single crystal neutron diffraction was performed on our sample using the thermal single crystal diffractometer ZEBRA, at the SINQ/PSI. A single hexagonal crystal of approximately 4 mm diameter and 0.4 mm thickness, the c-axis was along the thin dimension of the crystal. After the single-crystallinity check and alignment by Laue X-ray diffraction, the sample was mounted to a 0.3 mm-thick rectangular sheet of aluminum with GE varnish, then bound with thin aluminum wires before being wrapped in 0.1 mm aluminum foil. Two different setups were used - with a cooling machine or furnace on a Eulerian cradle and with a cryomagnet with vertical field up to 10 T. For both experiments the incident neutron wavelength of 1.18 Å was selected by the Ge-monochromator. Datasets at four temperatures - 5 K, 150 K, 300 K and 425 K - were measured on the Eulerian cradle. In the magnet experiment the magnetic field was applied along the (1–10) direction, within the ab-plane. The datasets at three temperatures −2 K, 70 K and 250 K - were collected. Field dependence was measured up to 8 T at 2 K for the (110), (002) reflections and at 230 K for the (001), (002), and (110) reflections, respectively. From our collected datasets, we performed refinements with the FullProf suite using the ISODISTORT program online to generate the Shubnikov subgroups. The longer wavelength of λ = 2.3 Å (PG monochromator) and the 2D detector were used to detect any incommensurate feature near or splitting of the 002 reflection on cooling between 225 K and 25 K. No new features have been found.

μSR experiment

In a μSR experiment nearly 100% spin-polarized muons (μ+) are implanted into the sample one at a time. The positively charged μ+ thermalize at interstitial lattice sites, where they act as magnetic microprobes. In a magnetic material the muon spin precesses in the local field Bμ at the muon site with the Larmor frequency 2πνμ = γμ/(2π)Bμ [muon gyromagnetic ratio γμ/(2π) = 135.5 MHz T−1].

Longitudinal-field (LF) and zero-field μSR experiments on the single crystalline samples of TbMn6Sn6 were performed at the πM3 beamline of the Paul Scherrer Institute (Villigen, Switzerland), using the low background GPS instrument35. The specimen was mounted in a He gas-flow cryostat and CCR with the c-axis parallel to the muon beam direction in order to cover the temperature range between 1.7 K and 400 K.

Hydrostatic pressure μSR experiments were performed at the μE1 beamline of the Paul Scherrer Institute using the General Purpose Decay-Channel Spectrometer (GPD) instrument38. An ensemble of many single crystals were compacted into a MP-35N pressure cell, mounted in a He gas-flow cryostat able to achieve a base temperature of 2.5 K.

Analysis of ZF-μSR data

The ZF-μSR spectra from Up-Down positron detectors were fitted using the following model39:

$${A}_{{{{{{{{\rm{ZF,UD}}}}}}}}}(t)=\mathop{\sum }\limits_{j=1}^{3}\left({f}_{T,j}\cos (2\pi {\nu }_{j}t+\phi ){e}^{-{\lambda }_{T,j}t}\right)+{f}_{L,j}{e}^{-{\lambda }_{L,j}t}.$$
(1)

The model (1) is an anisotropic magnetic contribution characterized by an oscillating transverse component and a slowly relaxing longitudinal component. The longitudinal component arises due to the parallel orientation of the muon spin polarization and local magnetic field components. In polycrystalline samples with therefore randomly oriented fields this results in a so-called one-third tail with \({f}_{{{{{{{{\rm{L}}}}}}}}}=\frac{1}{3}\). For single crystals, fL varies between unity and zero as the orientation between field and polarization changes from being parallel to perpendicular. Note that the whole volume of the sample is magnetically ordered in the whole investigated temperature range. Depending on the temperature range, either a single, or two distinct well separated precession frequencies can be clearly seen in the μSR spectra. Spectra near the base-T requires the addition of the third broad oscillating component with the internal field value very close to the one, observed at high temperatures. In addition, the spectra below 20 K are also characterised by a missing small fraction of the initial asymmetry, which points to a more disordered static state below 20 K than the one above.

The ZF-μSR spectra from Forward-Backward positron detectors were fitted using the following dynamic model:

$${A}_{{{{{{{{\rm{ZF,FB}}}}}}}}}(t)={A}_{fast}{e}^{-{\lambda }_{FB,fast}t}+{A}_{slow}{e}^{-{\lambda }_{FB,slow}t}.$$
(2)

The two terms represent a fast and a slow relaxation component, respectively, and λFB,fast and λFB,slow are the muon-spin-relaxation rates for each component.

Calculation of the muon stopping site

The DFT-based computer simulations carried out in this work were performed with the CASTEP40 code. The crystal structure of TbMn6Sn6 used for the computation was obtained from the Inorganic Crystal Structure Database via the CrystalWorks portal. A plane wave cutoff of 800 eV for these calculations was chosen by converging energy and forces using the automated tool CASTEPconv41. As regards the k-point grid size, a high-density 12 × 12 × 6 Monkhorst-Pack k-point grid42 was used. This produced forces accurate well within an error of 0.05 eV/Å, which was used as the limit tolerance for geometry optimization. Geometry optimization on the structure was performed with a LBFGS algorithm, fixing the unit cell parameters to their experimental values, to a tolerance of 0.05 eV/Å for the forces. The LDA exchange-correlation functional was used in combination with auto-generated ultrasoft pseudopotentials. The DFT calculations were spin-polarized calculations, with the quantization axis along the [001] direction and with initial magnetic moments of Tb = –9.0 μB and Mn = 2.4 μB, which are the experimental magnetization values at T 1.8 K.

The determination of the muon stopping sites was performed using the Unperturbed Electrostatic Method (UEP), as implemented in the software package pymuon-suite43, which provides various utilities to estimate the muon stopping sites44,45. The UEP method uses Density Functional Theory (DFT) calculations to estimate the host material’s electrostatic potential plus a combination of mathematical analysis and clustering techniques to estimate potential muon stopping sites. Single stable stopping site was identified. which are separated by potential barriers on the order of at least 0.2 eV, indicating that the muons are locked into these sites after thermalization.

Subsequent calculations of the internal dipolar fields at the potential muon sites were performed using the Python package muesr46. A sphere with a radius large enough to encapsulate a 100 × 100 × 100 supercell was constructed and dipole summation over all moments in the sphere was performed to find the local field at the muon sites. The magnetic structure considered was the experimentally proposed c-axis-aligned magnetic structure, with magnetic moments of Tb = –9.0 μB and Mn = 2.4 μB. The local dipolar field for the muon stopping site indicated in the inset of Fig. 4d is [0.00002 0.00001 1.40588] T, which is in reasonable agreement with the value and direction of the experimentally observed local field for Component I at T < 10 K.