Introduction

The discovery of topological insulators with bulk band gaps but robust topological surface states protected by time reversal symmetry triggered much research into topological phases of matter.1,2 Topological insulators are bulk insulators yet have conducting surface states as a consequence of the topologically non-trivial band structure, where a Dirac cone lies within the bulk gap.3,4 Subsequently, a number of semimetals with non-trivial topologies have also been discovered. In some cases such as Dirac and Weyl semimetals, the electronic structure is gapless and contains Dirac or Weyl points at the positions where the bands cross with linear dispersions and the excitations are well described as Dirac or Weyl fermions.5,6,7,8,9,10,11 Meanwhile in other topological semimetals, the band structure contains similar band inversion features to topological insulators, but the Fermi level does not lie within the band gap.12,13 The role of electronic correlations in topological systems has also become of particular interest after the proposal that SmB6 is a topological Kondo insulator, where the surface state lies within the gap opened due to the Kondo interaction.14,15,16 One striking feature is the presence of quantum oscillations in SmB6 from de Haas-van Alphen effect (dHvA) measurements,17,18 despite the material being a bulk insulator at low temperatures, stimulating debate over whether these arise from the surface,17,19 or if this is a bulk property.18,20 Furthermore, Tan et al.18 found that the dHvA amplitude shows an anomalous increase at low temperatures below 1 K, and the origin of this highly unusual deviation is currently not resolved.

Recently, the X(Sb,Bi)(X = lanthanide) series of semimetals with a cubic rocksalt structure have attracted considerable interest. These materials are commonly found to have an extremely large magnetoresistance (XMR), generally attributed to the nearly perfect compensation of electron and hole carriers.21,22,23,24,25,26,27 Meanwhile, non-trivial band topologies have been theoretically predicted,28,29 and evidence for topological states is found experimentally for some compounds from ARPES and transport measurements,25,27,30,31,32,33,34,35 while other members appear to be topologically trivial.24,36,37 In LaSb, it was suggested that the low-temperature plateau in the temperature dependence of the resistivity in applied magnetic fields was due to a topologically protected surface state, similar to that observed in SmB6.21 However, ARPES measurements show conflicting results over whether the band topology is trivial,34,36 while the resistivity plateau can be explained by a three band model with nearly perfect electron-hole compensation.36 Furthermore, by substituting for lanthanide elements with partially filled 4f electron shells, the effect of tuning the spin-orbit coupling, magnetism and electronic correlations can be studied. For example, CeSb shows evidence for Weyl fermions in the field-induced ferromagnetic state from both theoretical calculations and the measurement of a negative longitudinal magnetoresistance.25 NdSb has also been proposed to be a topological semimetal from the observation of a Dirac-like semi-metal phase using ARPES, as well as from band structure calculations and analysis of the Landau indices.31,38

In this work, we report magnetoresistance and quantum oscillation measurements of the antiferromagnet SmSb.39 Based on the analysis of Shubnikov-de Haas (SdH) oscillations, we found that the phase of all the harmonic oscillations corresponding to the α-band is close to π, while for the other bands it is almost zero. These results provide evidence that the band topology of the α-band is non-trivial. Moreover, striking differences are found between the temperature dependences of the quantum oscillation amplitudes from SdH and dHvA measurements. The amplitudes of the SdH oscillations show anomalous behavior at low temperatures, which may arise due to the presence of multiple Fermi surface sheets or a conduction channel related to the topological state.

Results and discussion

The temperature dependence of the electrical resistivity [ρ(T)] of SmSb is displayed in Fig. 1a, where the data up to 300 K is shown in the inset. The ρ(T) data show a small hump at around 65 K, which is explained as resulting from the splitting of the crystalline electric fields,40 as well as a sharp drop of ρ(T) at the antiferromagnetic transition at TN = 2.2 K.41 While TN changes very little in applied magnetic fields,42 an upturn appears in ρ(T) at low temperatures, which becomes more pronounced as the field is increased. As shown in Fig. 1a, b, there is a strong increase of ρ(T) in-field below TN, before reaching a nearly constant value below around 1 K. Figure 1c shows the field dependence of the magnetoresistance [ρ(H)/ρ(0)] of SmSb up to 15 T at various temperatures down to 0.3 K. At 0.3 K ρ(H)/ρ(0) = 74,000, which decreases rapidly as the temperature is increased. Figure 1d displays measurements performed up to higher fields at 1.8 K, where the magnetoresistance increases quadratically to about 5558 at 60 T. The magnetoresistance of SmSb is one of the largest observed among the XSb family of compounds,25,36,37,38,43 and the quadratic field dependence suggests that the XMR most likely results from electron-hole carrier compensation.44 In this case, the increase of mobility below TN may lead to both the small low temperature values of ρ(T) in zero-field, as well as the very large magnetoresistance and low-temperature plateau of ρ(T) in high fields. While this low temperature plateau has also been observed in other XSb materials,21,25,26,27 the appearance of the plateau at a lower temperature in SmSb is likely due to the comparatively low TN.

Fig. 1
figure 1

Resistivity and magnetoresistance of SmSb. Temperature dependence of the electrical resistivity of SmSb sample S1 at a zero and low applied fields, and b, at high fields. The inset of a displays the resistivity over the full temperature range from 0.3 to 300 K. c Field dependence of the transverse magnetoresistance of SmSb sample S1 at various temperatures up to applied fields of 15 T. d Transverse magnetoresistance at 1.8 K of SmSb samples S2 and S6, up to applied fields of 32 T and 60 T respectively. The solid line shows the B2 dependence

Full size image

To examine the topology of the electronic structure of SmSb, we also analyzed the SdH oscillations in the resistivity for the two samples displayed in Fig. 1d. From the fast Fourier transformation analysis (see Supplementary Fig. 1), the most prominent oscillation frequencies consist of three fundamental frequencies (Fα, Fβ, and Fγ), and two harmonic frequencies (F2α and F3α). In the following, the data are analyzed with five oscillations, corresponding to these frequencies. Note that for the data measured in a pulsed magnetic field, the data measured at 15 K were analyzed, so as to avoid complications arising from strong harmonic oscillations. To obtain the oscillation phase factors (ϕi) corresponding to each fundamental frequency, we have used Lifshitz-Kosevich (LK) theory to fit the data:45

$$\Delta \rho /\rho _0 = \mathop {\sum}\limits_{i = 1}^3 {\mathop {\sum}\limits_r {a_{i,r}} } \sqrt {B/r} R_T^{i,r}R_D^{i,r}R_S^{i,r}{\mathrm{cos}}(2\pi r(F_i/B - 1/2) - \phi ^i).$$
(1)

Here ai,r is a constant corresponding to the rth harmonic of the ith band, ϕi is the corresponding phase factor, and RT, RD and RS are the temperature, Dingle and spin damping factors respectively. These are given by \(R_T^{i,r} = (14.69rm_iT/B)/{\rm sinh}[(14.69rm_iT/B)]\), \(R_D^{i,r} = {\mathrm{exp}}( - 14.69rm_iT_D/B)\), and \(R_S^{i,r} = {\mathrm{cos}}(\pi grm_i/m_e)\), where mi denotes the cyclotron mass for each band, me is the free electron mass and TD is the Dingle temperature. The data were fitted using Eq. (1), and it can be seen in Fig. 2 that this can well describe the SdH oscillations both measured in a static field up to 35 T (S2) and a pulsed field up to 60 T (S6). Note that since the data are analyzed for a fixed temperature and field-angle, \(R_S^{i,r}\) is taken to be a constant value. Here Fi for the five components were fixed to the values from the FFT analysis, and the mi for the α- and β-bands were obtained from the temperature dependence of the oscillation amplitudes from dHvA measurements (see Fig. 5). On the other hand it is difficult to obtain mi from analyzing the temperature dependence of oscillations corresponding to the γ-band, and therefore this was a fitted parameter in the analysis. In addition, TD and ϕi for each band were fitted parameters in the analysis, and the results are displayed in Supplementary Table 1.

Fig. 2
figure 2

SdH oscillations analysis using Lifshitz-Kosevich theory. Analysis of the SdH oscillations of SmSb samples a S2 at 1.8 K and b S6 at 15 K using Lifshitz-Kosevich theory in Eq. (1). The red thin line represents the fitted curves and the thick line corresponds to the experimental results. The corresponding field range for fitting is 15–32 T and 20–60 T respectively

Full size image

The phase factor corresponding to the α-band from both samples is 1.16π. Note that the phase is given by \(\phi ^i = \phi _D^i + \lambda ^i\), where ϕD corresponds to the LK correction, and λ is the sum of geometric and dynamical phases, which includes the Berry phase, the phase factor resulting from the orbital magnetic moment and the Zeeman coupling phase factor.45 For a two-dimensional cylinder-like Fermi surface, ϕD is zero, while for a three-dimensional Fermi surface pocket, ϕD is π/4 or −π/4 for oscillations corresponding to the minimum or maximum Fermi surface cross sections respectively. Since the oscillation frequency is lowest when the field is applied along the [100] direction (see Supplementary Fig. 2), this suggests that ϕD = π/4, and hence λ = 0.91π. This is close to the value of π indicating a π Berry phase, as expected for a topologically non-trivial electronic band,46,47,48 while the small deviation from π suggests the possible influence of dynamical phases.45 On the other hand, for fundamental oscillations corresponding to the β-band, λ of −0.15π and 0.01π are obtained for samples S2 and S6 respectively, while the values for the γ band are −0.24π and −0.17π. These results suggest that both these bands have near zero Berry phase, where the influence of dynamical phase factors leads to a small deviation from zero.

A Landau fan diagram was also utilized to extrapolate the dominant oscillation phase factor (Fig. 3), where Landau indices were assigned to the positions of the SdH oscillations which correspond to the deepest valleys, for both S2 and S6. The uncertainty of the valley positions is small which doesn’t significantly affect the results, and the periodicity of these valleys corresponds to that of the α-band. From linearly fitting the Landau index n as a function of 1/B, the extrapolated residual Landau indices are n0 = 0.53(5) for sample S2 and n0 = 0.52(3) for sample S6. These agree well with the above conclusions that the phase factor of the α-band, which is the strongest frequency component observed in the quantum oscillations, is close to π.

Fig. 3
figure 3

Landau fan diagram analysis of the Berry phase. a SdH oscillations of samples S2 and S6 of SmSb plotted as a function of 1/B. The oscillations were obtained from the field dependence of the magnetoresistance at 1.8 K up to 32 T, and at 4 K up to 60 T, respectively after subtracting the background contribution. b The 1/B dependence of the valley positions in the SdH measurements. The solid lines show linear fits to the data, which both extrapolate to a non-zero residual value close to n0 = 0.5

Full size image

Though these results are consistent with those expected for a non-trivial band topology, in ref. 45 it was suggested that the phase obtained from quantum oscillation experiments cannot directly probe the Berry phase in three-dimensional centrosymmetric materials. In our measurements, the field is applied along the [100] direction, which corresponds to the C4 rotation axis of the cubic crystal structure. Due to the symmetry constraints in this situation (mirror symmetry),45 the spin degeneracy of each band is preserved, and the overall phase factor will be quantized to either π or zero, which is not a linear summation of the phase factors of both spin up and down bands, and thus cannot be used to directly obtain the Berry phase or determine the band topology. Here we note that the same phase factor is found from measurements above and below TN. Since the magnetic structure of SmSb has not yet been reported, we are not able to determine if the same symmetry constraints are present in the magnetically ordered state. Electronic structure calculations for the case of zero applied field suggest that SmSb has a trivial band topology but is close to the boundary between trivial and non-trivial.29 However we note that these calculations assume a Sm valence of 3+, while a mixed valence of about 2.75 was reported from X-ray photoemission spectroscopy.49 As such it is important to perform further studies to reveal the origin of this nontrivial Berry phase. Evidence for a π Berry phase was also found in isostructural LaBi,50 where both an odd number of band inversions and surface states with an odd number of massless Dirac cones were observed in ARPES measurements.32,33,34,35

In addition, we also performed field-dependent ac susceptibility measurements down to low temperatures where the presence of dHvA oscillations is clearly observed. For comparison, SdH and dHvA oscillations are shown for various temperatures down to 0.3 K in Fig. 4a, b respectively, after subtracting the background contribution. The respective FFT are shown in Fig. 4c, d, where two principal frequencies are observed, which remain unchanged below TN, similar to previous reports.42 The observed values are Fα ≈ 334 T and 328 T, and Fβ ≈ 633 T and ≈ 590 T for SdH and dHvA respectively, where the differences may be due to a small misalignment. The SdH oscillation amplitudes as Δρ corresponding to both the α and β bands increase significantly with decreasing temperature below 2.5 K, while in dHvA measurements the change is more gradual. Furthermore, the amplitude of the α-band oscillations is maximum at around 0.8 K, and upon further decreasing the temperature, the amplitude decreases. Meanwhile, the angular dependence of Fα can be fitted with the equation for anisotropic three-dimensional ellipsoidal pockets51 (see Supplementary Fig. 2), which can describe the anisotropic bullet-like Fermi surface pockets around the X-points, corresponding to the bulk Fermi surface of the α-band.23,29

Fig. 4
figure 4

SdH and dHvA oscillations at various temperatures. a SdH oscillations obtained from the field dependence of the magnetoresistance of sample S1 of SmSb at various temperatures. b dHvA oscillations at several different temperatures from field-dependent ac susceptibility measurements of sample S3 of SmSb. The measurements were performed with the magnetic field applied along the principal crystallographic axis. The corresponding FFT spectra are displayed for oscillations in c SdH, and d dHvA

Full size image

Figure 5a, b presents the temperature dependence of the oscillation amplitudes from both SdH and dHvA measurements. Since there is a rapid change of the resistivity both with changing temperature and field, the SdH oscillation amplitudes are displayed as Δρ/ρ0, so as to normalize by the background values. The temperature dependence of the dHvA amplitudes are well fitted across the whole temperature range (0.3–10 K) for both the α and β bands with the amplitude ∝RT (Eq. 1), where the fitted values of the effective cyclotron masses are 0.26 me and 0.28 me respectively. These results are consistent between measurements of both the ac and dc susceptibility (samples S3 and S5 respectively). However, although the SdH and dHvA data coincide well at higher temperatures, below around 2.5 K there is a significant deviation which cannot be accounted for by the LK formula. Moreover, for the α-band, the amplitude reaches a maximum at around 1.6 K, before beginning to decrease. We note that these results are highly repeatable, as shown by the measurements of the two samples displayed in Fig. 5, and when the SdH amplitudes are plotted as Δρ, the deviation from LK behavior is even more pronounced (see Supplementary Fig. 3). In addition, the Dingle temperature analysis does not show a significant change at low temperatures (Supplementary Fig. 4), and as a result this behavior cannot be accounted for by the temperature dependence of TD.

Fig. 5
figure 5

Anomalous temperature dependence of SdH oscillation amplitudes. Temperature dependence of the SdH (filled symbols) and dHvA (empty symbols) amplitudes of the fundamental a α-band and b β-band frequencies, for several different samples, where the SdH amplitudes are displayed as Δρ/ρ0. Samples S1 and S3 were measured in a 3He cryostat in fields up to 15 T, S2 was measured at the High Magnetic Field Laboratory, while S4 and S5 were measured using a Physical Property Measurement System. The field ranges over which the data were analyzed are displayed, while the vertical dashed line marks the position of TN. The solid lines display fits to the dHvA amplitudes using the temperature damping factor of the Lifshitz-Kosevich formula, with effective masses of 0.26 me for the α-band and 0.28 me for the β-band

Full size image

One possible explanation for the deviation between dHvA and SdH measurements is that the unusual behavior of SdH oscillations is related to the topologically non-trivial electronic structure which manifests a π Berry phase. We note that dHvA measurements of floating zone grown crystals of the proposed Kondo insulator SmB6 reveal a steep increase of the oscillation amplitude below 1 K which cannot be explained by LK theory,18,52 while there is also evidence for a non-trivial Berry phase.17 Currently, there is considerable debate over whether the quantum oscillations in SmB6 are from the insulating bulk, or the surface.17,18 Proposals such as the magnetic breakdown scenario give rise to quantum oscillations in the bulk of an insulator which deviate from LK behavior,20 while it has also been suggested that the anomalous increase of dHvA quantum oscillation amplitudes in SmB6 arises due to surface quantum criticality.19 In contrast to SmB6, the amplitudes of our dHvA measurements of SmSb can be well fitted using the LK expression and the anomalous behavior is only seen in SdH measurements. This may suggest that the anomalous behavior of SmSb arises only from certain conduction channels with high mobility, which do not significantly contribute to dHvA measurements. Metallic edge states were proposed to explain the anomalous quantum oscillations of some charge transfer salts, where there is a deviation of SdH amplitudes from conventional behavior, which is not observed in those from the dHvA effect.53,54 However, these systems have highly conducting quasi-two-dimensional planes where the edge states are suggested to arise in the quantum Hall regime, and this is therefore quite a different scenario to that presented by SmSb. Meanwhile, a comparison between SdH and dHvA cannot be made for SmB6, since SdH oscillations in magnetotransport measurements have not been reported.

Another possibility is that the deviation between SdH and dHvA measurements arises due to the difficulty in disentangling the contributions to the conductivity from the different FS sheets. Although it is often assumed that Δρ/ρ0 is proportional to the density of states and that the temperature dependence is well described by the LK formula, for multiband systems the situation can be more complicated due to the different contributions to the conductivity from each band.48 This effect will not influence the dHvA results, which can be well described by the LK formula across the whole temperature range. We note that below around TN, the normalized oscillation amplitude of the α-band is anomalously low relative to the dHvA results, while for the β-band there is an increase. This suggests that with decreasing temperature, there may be changes of the relative contributions to the conductivity from the two bands. Therefore, for systems with multiple FS sheets and large magnetoresistance, analysis of the cyclotron masses using SdH measurements may be difficult.

Another question posed by these results is why this anomalous temperature dependence of SdH amplitudes is not observed in other XSb compounds. This may be related to the fact that in other XSb materials, the rapid increase of the magnetoresistance occurs at considerably higher temperatures. For example, in LaSb and YSb this occurs at around 80 K and 100 K, respectively,21,55 while the SdH oscillations are usually measured below 20 K for these materials, where the magnetoresistance shows relatively little change with temperature. However, as displayed in Fig. 1b, the magnetoresistance of SmSb changes strongly below 2 K, which may influence the oscillation amplitudes at low temperatures. Therefore the origin of the dramatic departure from conventional behavior in the SdH amplitudes is an open question, and in particular since this deviation onsets near TN, the possible relationship between the antiferromagnetic state and the anomalous behavior also needs to be explored.

To summarize, we find evidence for a π Berry phase in the α-band of SmSb from analyzing high field measurements of SdH oscillations. Furthermore, our quantum oscillation measurements show that the amplitudes of dHvA oscillations are well described by the standard LK formula, while those from the SdH effect show anomalous behavior at low temperatures. The origin of this unusual behavior in SdH oscillations remains unclear, and therefore further studies are required to understand the origin of the Berry phase, as well as any relationship with the anomalous SdH oscillations.

Methods

Single crystals of SmSb were synthesized using an Sn flux-method.56 The elements were combined in a molar ratio Sm:Sb:Sn of 1:1:20 in an Al2O3 crucible before being sealed in evacuated quartz ampoules. These were slowly cooled from 1100 °C down to 800 °C, before centrifuging to remove the Sn flux. The resulting crystals are cubic with typical dimensions of around 3 mm. The residual resistivity ratio of RRR = ρ(300 K)/ρ(0.3 K) ≈ 4000, indicates a high sample quality.

Low temperature resistivity and ac susceptibility measurements were performed between 0.27 and 9 K using an Oxford Instruments 3He system, in applied fields up to 15 T. The resistivity data in this system was measured using a Lakeshore 370 resistance bridge, with a current of 0.3 mA across the whole temperature range. Four Pt wires were attached to the sample by spot welding so that the current was applied along the [100] direction. dHvA results were measured using a commercial susceptometer for ac susceptibility measurements, which consists of three sets of coils, including a drive coil, a pick-up coil, and a coil for compensation. Additional resistivity measurements in fields up to 9 T, and dc-susceptibility measurements in fields up to 13 T, were also performed using a Quantum Design Physical Property Measurement System (PPMS) from 300 to 2 K, where the susceptibility was measured using a vibrating sample magnetometer insert. The high field magnetoresistance measurements were performed up to 60 T using a pulsed field magnet at the Los Alamos National Laboratory, USA, while measurements up to 32 T were carried out using the Cell 5 Water-Cooling Magnet at the High Magnetic Field Laboratory of the Chinese Academy of Sciences.