## Introduction

Applying a strong magnetic field to an electron gas confines the electrons motion in cyclotron orbits with a set of discrete eigenenergies—the Landau levels. In two-dimensional (2D) systems, this quantization leads to a fully gapped energy spectrum and to the emergence of the quantum Hall effect (QHE)1. In the limit where only the lowest Landau level (LLL) is occupied (the so-called quantum limit), electron–electron interactions can play a significant role, leading to the appearance of correlated states, such as the fractional quantum Hall effect2. In contrast, the Landau level spectrum of a three-dimensional (3D) electron gas is not fully gapped and becomes like that of a one-dimensional system. As a consequence, the electrons can still move along the field direction, which in turn destroys the quantization of the Hall effect3,4,5. Nevertheless, it has been predicted that a generalized version of the QHE can emerge in 3D electron systems that exhibit a periodically modulated superstructure6,7,8. Analogous to as in two dimensions, in the vicinity of the quantum limit, 3D electron systems are also prone to form a variety of correlated electron states, including Luttinger liquids; charge, spin and valley density waves; excitonic insulators; Hall and Wigner crystals; or staging transitions in the case of highly anisotropic layered systems3,4,6,9,10,11,12,13. It has been theoretically pointed out that some of these states are related to quantum Hall physics in three dimensions and likewise could manifest themselves in a Hall response that should be observable in the quantum limit of 3D semimetals10,13,14.

Inspired by these ideas, the possibility of finding a 3D QHE has been explored in several material systems. For example, signatures of the integer quantum Hall effect (IQHE) have been found in quasi-2D semiconducting multilayer lattices15, Bechgaards salts16,17, η-Mo4O1118, n-doped Bi2Te319, and EuMnBi220, in which the layered crystal structure itself supplies the stack of 2D systems. Very recently, the QHE has also been observed in 3D graphite films5, bulk ZrTe521, and HfTe5 samples22. In graphite, the imposed periodic superstructure has been attributed to the formation of standing electron waves. In ZrTe5 and HfTe5, the IQHE was originally believed to arise from a charge density wave (CDW), due to the scaling of plateau height with the Fermi wavevector. This scenario is, however, in contrast with thermodynamic and thermoelectric measurements on ZrTe5 that did not reveal any signatures of a field-induced CDW transition. Instead, it was proposed that ZrTe5 should be considered a stack of weakly interacting Dirac 2DEGs with the plateau height scaling originating from the interplay of small carrier density and the peculiarities of Landau quantization of the Dirac dispersion23. In parallel to the search for the 3D QHE, there has been a long-standing experimental effort to observe field-induced correlated states in the quantum limit of three-dimensional materials. Although those studies have provided signatures of field-induced states in the longitudinal electrical resistivity of Bi24,25, ZrTe521,26 and graphite5,27,28,29,30, correlated states with Hall responses have yet to be observed.

In this work, we present measurements of the low-temperature longitudinal and Hall resistivities of the 3D semimetals HfTe5 and ZrTe5. Previous studies have shown that HfTe5 is an isostructural counterpart of ZrTe531. Both materials share an orthorhombic crystal structure and a single elliptical 3D Fermi surface, comprising less than 1% of the Brillouin zone and hosting massive Dirac Fermions with almost linearly dispersing bands in the vicinity of the Fermi level (see Supplementary Information for details). These specific properties have been considered essential for the observation of the 3D IQHE in both materials21,22,23. Moreover, recent progress in ZrTe521 and HfTe532 single-crystal growth has enabled a Hall mobility μ that exceeds 100,000 cm2 V−1 s−1 at low temperatures (<4 K) (see “Methods” section and Supplementary Information S1). The quality of these crystals is comparable to that of graphene samples, which have previously proven appropriate for observing the FQHE in two dimensions33. While the 3D band structure of ZrTe5 and HfTe5 is very similar21, hafnium has a higher atomic number than zirconium and hence naturally introduces stronger spin–orbit coupling (SOC)31, which has been previously shown to stabilize correlated states in the quantum limit of 2D electron systems34. Therefore, HfTe5 is the more promising candidate for the observation of unconventional Hall responses of correlated ground states in its quantum limit.

## Results

HfTe5 and ZrTe5 typically grow as millimeter-long ribbons with an aspect ratio of approximately 1:3:10, reflecting their crystalline anisotropy. Details of the growth process, crystal structure, and first transport characterization of our samples can be found in refs. 21,32. We have measured the longitudinal electrical resistivity ρxx and Hall resistivity ρxy (see “Methods” section) of three HfTe5 samples (A, B, C) and three ZrTe5 samples (D, E, F) as a function of magnetic field B and temperature T, with the electrical current applied along the a-axis of the crystals.

At T = 300 K, ρxx is around 0.5 mΩ cm (see Fig.1a, Supplementary Fig. S1 and ref. 32) with an electron density of n = 1.3 × 1019 cm−3 and μ = 10,000 cm2 V−1 s−121,23,32. Upon cooling in zero magnetic field, ρxx increases with decreasing T until it reaches a maximum at TL = 70 K (Fig.1a). Such a maximum has previously been observed in HfTe532 and ZrTe521, and it is attributed to a Lifshitz transition, here inducing a change in charge-carrier type. Consistently, the slope of ρxy(B) changes sign at TL, indicating electron-type transport for T < TL32. At 3 K, we find n = 8.7 × 1016 cm−3 and μ = 250,000 cm2V−1s−1 (see Supplementary Note 1 and ref. 21). All investigated samples show similar electrical transport properties. In the main text, we focus on data obtained from HfTe5 sample A and ZrTe5 sample D. Additional data of samples B, C, E, and F can be found in the Supplementary Information and ref. 23.

To characterize the Fermi-surface (FS) morphology of our pentatelluride samples, we have measured Shubnikov-de Haas (SdH) oscillations with respect to the main crystal axes at 3 K. For this purpose, we followed the analysis of ref. 21 and rotated B in the zy and zx planes, while measuring ρxx (B) at a series of different angles (Fig. 1, Supplementary Fig. S3 and ref. 23). The SdH frequency BF,i is directly related to the extremal cross-section of the Fermi surface SF,i, normal to the applied B direction via the Onsager relation BF,i = SF,i(/2πe). Examples of the SdH oscillations for which the magnetic field was aligned along the three principal crystallographic directions (x, y, and z axes) are shown in Fig. 1b–d (upper panels). In each field direction, we have observed maxima in ρxx that are periodic in 1/B, each of which corresponds to the onset of a Landau level. In the associated minima, ρxx(B) does not vanish, which is a consequence of the remaining dispersion in z direction in 3D systems and Landau level-broadening due to disorder (see Supplementary Note 4, Supplementary Fig. S4S7 and ref. 23 for details). To determine the SdH oscillation frequency, we have subtracted the smooth high-temperature (50 K)—ρxx(B) from the low-T-data, obtaining the oscillating part of the longitudinal resistivity Δρxx(B). Employing a standard Landau-index fan diagram analysis to Δρxx(B) (Fig. 1h, Supplementary Figs. S3, S8, S9, Supplementary Note 2 and ref. 23), we have found only a single oscillation frequency for all rotation angles measured, consistent with a single electron pocket at the Fermi energy. The extracted BF,i of sample A for B along the three principal directions are BF,x = (9.9 ± 0.1) T, BF,y = (14.5 ± 0.5) T, and BF,z = (1.3 ± 0.1) T. Here, the errors denote the standard deviation of the corresponding fits.

In contrast to 2D materials, HfTe5 and ZrTe5 show in-plane SdH oscillations when B is aligned with x and y, indicating a 3D Fermi-surface pocket. The shape of the FS is further determined by the analysis of the rotation angle-dependence of BF. As shown in Fig. 1i, j, the angle-dependent SdH frequency is well represented by a 3D ellipsoidal equation $$B_{{\mathrm{F}},3D} = B_{{\mathrm{F}},z}B_{{\mathrm{F}},i}/\sqrt {(B_{{\mathrm{F}},z}\sin \theta )^2 + (B_{{\mathrm{F}},i}\cos \theta )^2}$$, where θ is the rotation angle in the zi plane. As a cross-check, we also plot the formula of a 2D cylindrical Fermi surface BF,2D = BF,z/cosθ, which deviates significantly from the experimental data for θ > 80°. Hence, the ellipsoid equations can be used to obtain the Fermi wave vectors $$k_{F,x} = \sqrt {S_{{\mathrm{F}},y}S_{{\mathrm{F}},z}} /\sqrt {{\uppi}S_{{\mathrm{F}},x}}$$ = (0.005 ± 0.001) Å−1, $$k_{{\mathrm{F}},y} = \sqrt {S_{{\mathrm{F}},x}S_{{\mathrm{F}},z}} /\sqrt {{\uppi}S_{{\mathrm{F}},y}}$$ = (0.008 ± 0.001) Å−1 and $$k_{{\mathrm{F}},z} = \sqrt {S_{{\mathrm{F}},x}S_{{\mathrm{F}},y}} /\sqrt {{\uppi}S_{{\mathrm{F}},z}}$$ = (0.058 ± 0.006) Å−1 that span the 3D FS of HfTe5 sample A in the x, y, and z direction, respectively (Fig. 1k). The errors in kF,i originate from the errors of the BF,i. The preceding analysis indicates that for our HfTe5 and ZrTe5 samples, the quantum limit with the field along the z is achieved already for the field of BC = 1.8 and 1.2 T21,23, respectively. Further details of our band-structure analysis can be found in Supplementary Fig. S6, Supplementary Table S1, refs. 21,23. Above 6 T, we find that for both materials, ρxx(B) steeply increases with the magnetic field (Fig. 2b). Such a steep increase has previously been observed in ZrTe5 and has been attributed to a field-induced metal-insulator transition21.

For the field-aligned with the z axis (B||z), we additionally observed in both studied compounds pronounced plateaus in Hall resistance ρxy(B) that appear at the minima of the SdH oscillations in ρxx(B)—features commonly related to the QHE (Fig. 2a and ref. 23). The height of the last integer plateau is given by (h/e2) π/kF,z, similar to as reported in the literature for the 3D IQHE21,22. The plateaus are most pronounced at low temperatures, but still visible up to T = 30 K (Fig. 2b, c and ref. 23).

We note that the observed quantization of ρxy is not immediately obvious from the predicted quantization in σxy. The Hall resistivity tensor is given by ρxy = σxy/(σxxσyy + σxy2) with a magnetic field in z direction, where σxx and σxx are the longitudinal component of the conductivity tensor in x and y direction, respectively. Vice versa, the Hall conductivity tensor element is given by σxy = ρxy/(ρxx ρyy + ρxy2) with a magnetic field in z direction. However, in our samples at low temperatures σxx < σxy (Supplementary Figs. S11 and S12) and thus σxy−1 ≈ ρxy, enabling the direct observation of the quantization. Due to the geometry of the HfTe5 crystals (elongated needles) and their mechanical fragility, performing reliable measurements of ρyy is not possible. Instead, we estimate the error of the σxy using the ratio of Drude resistivities $$\rho _{yy}/\rho _{xx} = (n_{xy}e^2t_x/m_x^ \ast )/({\it{n}}_{xy}e^2t_y/m_y^ \ast )$$ given by the quantum lifetimes and effective masses obtained from Shubnikov-de Haas oscillations on sample A (Supplementary Table S1). nxy is the charge-carrier concentration in the xy plane. Based on this analysis, we find ρyy/ρxx ≈ 0.4, which results in an error of below 8 % in the estimated σxy for the investigated field range owing to ρxx (B) < ρxy (B). Both these errors lay within the estimated error of kF,z of 10 %.

Figure 2d shows the angular-dependence of the Hall plateaus, which we find to scale with the rotation angle. This behavior is very similar to the sister compound ZrTe5.21 In both materials, the height of the Hall plateaus and its position in |B| depends only on the field component that is perpendicular to the xy plane B = |B|cosθ21,23.

So far, our analysis focused on similarities between the Hall effects observed in ZrTe5 and HfTe5. Upon cooling the samples to 50 mK, an obvious difference emerges, as shown in Fig. 3 and Supplementary Figs. S11S15. At low fields below the quantum limit (B < BC), both compounds exhibit signatures of new peaks and plateaus in ρxx(B) and ρxy(B). Such features have been observed in the past and are related to spin splitting of the Landau levels23,35. However, at high fields (B > BC)—in the quantum limit, HfTe5 exhibits an additional peak in ρxx(B), accompanied by a plateau-like feature in ρxy(B). This is in sharp contrast to ZrTe5, in which ρxx(B) and ρxy(B) smoothly increase. Using the Landau-index fan diagram obtained at 3 K and gauging the indexing of Landau bands with respect to the N = 1 band, we find that the additional peak in ρxx(B), in the quantum limit of HfTe5 is situated at N = 3/5. This indexing is confirmed by corresponding maxima in ρxx(B) and/or σxx(B) of all three HfTe5 samples investigated (compare Supplementary Figs. S11 and S12), despite being less pronounced in some of them.

Although the relation between the magnitude of the plateau-like feature and its corresponding Landau index is not obvious from ρxy(B), a comparison of the respectively calculated conductivity reveals that the magnitude of the plateau-like feature in the quantum limit scales as 3/5-times with respect to the plateau related to the LLL. A close investigation of the plateau height in conductivity (Supplementary Fig. S16) reveals that both the plateau-like features at N = 1 and N = 3/5 are well developed in the conductivity, within 1 and 2% of N·(e2/h)kF,z/π in the range of 0.5 T around the plateau center.

In order to verify whether the observed features can be explained by invoking the presence of a second pocket at the Fermi energy, we have performed additional magneto-transport measurements up to 70 T (Supplementary Fig. S16 and ref. 23). The measurements did not reveal any additional quantum oscillations, which is consistent with band-structure calculations31 and a previous ARPES study on our samples36: The Fermi level, obtained from the analysis of the Shubnikov-de Haas oscillations is (9 ± 2) meV (Supplementary Information), which is in agreement with the ARPES experiment. According to the ARPES data, at 15 K, the nearest additional band is located ~5 meV above the Fermi level (lowest temperature measured in the ARPES study) as compared to the Fermi function broadening of kB·15 K ≈ 1 meV. Below 15 K, the Fermi level stays constant with respect to the band edges, as indicated by the temperature-independent Shubnikov-de Haas frequency in our experiments. Hence, the next nearest band in our samples is ~kB·60 K away from the Fermi level and does not contribute to the low-temperature transport experiments. Our data can, therefore, be analyzed in terms of a single electron-type Dirac pocket.

Further insight into the possible origin of the N = 3/5 state in the quantum limit can be obtained from the line shape of Δρxx(B), which resembles the line shape of σxx(B), (Fig. 4e, f) a common feature of canonical 2D QHE systems37 A related empirical observation is that in both fractional and IQHE in 2D systems the longitudinal resistance is ρxx(B) is connected to ρxy(B) via ρxx(Bz) = γB·dρxy(B)/dB, where γ is a dimensionless parameter of the order of 0.01–0.05, which measures the local electron concentration fluctuations38,39. Comparison of σxx(B) (Fig. 4d) and γB·dρxy(B)/dB (Fig. 4e, upper panel) as a function of B−1 reveals that both quantities show maxima and minima at the same field positions as Δρxx(B). In particular, the derivative relation is well fulfilled with γ = 0.04, which is in the expected range reported for 2DESs. These results suggest that the observed plateau-like feature observed in the quantum limit in HfTe5 is related to quantum Hall physics.

To gain quantitative insights into the states that cause the features in the Hall effect in HfTe5, we have estimated the gap energies of the x–y plane ΔN associated with N = 1 and N = 2 below and N = 3/5 in the quantum limit. We have fitted the T-dependence of the ρxx(B) minima (Fig. 4f) in the thermally activated regime ρxx(B) α exp(−ΔN/2kBT), where kB is the Boltzmann constant (Supplementary Fig. S17). For integer N, we find Δ1 = (40 ± 2) K at N = 1 and Δ2 = (9 ± 1) K at N = 2. The gap energy of the N = 3/5 state in the quantum limit is two orders of magnitude lower: Δ3/5 = (0.49 ± 0.09) K. The deviations given for the gaps are the errors obtained from the thermally activated fits in Supplementary Fig. S13. In spite of considerable Landau level-broadening, both the size of the gaps of the integer and the N = 3/5 states compare well with the gaps obtained for integer and correlated quantum Hall states in 2DESs37,40. The different ΔN are also in agreement with the T-dependence of the corresponding Hall features (Fig. 4g). While the integer plateaus are observable up to tens of Kelvin, the plateau-like feature in the quantum limit vanishes at around 0.5 K.

Those considerations suggest that the Hall feature observed in the quantum limit of HfTe5 is associated with physics in the LLL only (as pointed out above, this Landau level is non-degenerate since HfTe5 is like ZrTe5 a gapped Dirac semimetal). The finite value of ρxx at N = 3/5 implies the absence of a fully established bulk gap, which in turn means that a truly quantized Hall effect as in 2D systems without the kF,z-scaling cannot be expected. Nevertheless, the emergence of the plateau-like feature in the quantum limit of a single-band system at low temperatures calls for an explanation beyond a simple single-particle picture: The Hall conductivity of a non-interacting single band in which the chemical potential adjusts to keep the particle number fixed simply decreases as 1/B, as observed in the isostructural ZrTe5 (Fig.3b).

## Discussion

Although possible scenarios for the emergence of a plateau in the Hall resistance in the quantum limit of electron plasma include the formation of a CDW, Luttinger liquid, Wigner crystallization, or the so-called Hall crystal41,42, a favorable scenario builds on the notion that ZrTe5 and HfTe5 can be thought of as a stack of interacting 2DEGs. Based on a Hartree–Fock analysis, it was proposed13 that in a layered structure the gain in exchange energy can exceed the energy cost for distributing electrons unequally between layers. The electrons then undergo spontaneous staging transitions in which only every i-th layers is occupied, while all other layers are emptied (the number i depends on the average electron density and the state formed)—some of which are only stabilized due to the interplay of electron interaction and spin–orbit coupling43. Depending on layer separation, electron density, and the strength of electron–electron interactions, various types of layered Laughlin states or Halperin states can then be formed3,13,14. These states are naturally associated with Hall responses. While staging transitions are unlikely in isotropic three-dimensional materials at high electron densities, HfTe5 has a very anisotropic band structure with small tunneling amplitudes along z, and hosts only a relatively small number of electrons in its Dirac pocket. Our data are thus consistent with strong interactions stabilizing a correlated state that gives a Hall response in HfTe5 in the quantum limit.

In conclusion, our measurements reveal an unconventional correlated electron state manifested in the Hall conductivity of the bulk semimetal HfTe5 in the quantum limit, adjacent to the 3D IQHE at lower magnetic fields. The observed plateau-like feature is accompanied by a Shubnikov-de Haas minimum in the longitudinal electrical resistivity and its magnitude is approximately given by 3/5(e2/h)kF,z/π. Analysis of derivative relations and estimation of the gap energies suggest that this feature is related to quantum Hall physics. The absence of this unconventional feature in the quantum limit of isostructural single-band ZrTe5 samples with similar electron mobility and Fermi wavevector indicates the presence of a correlated state that may be stabilized by spin–orbit coupling. However, further experimental and theoretical efforts in determining the real interactions and texture of the field-induced correlated states in HfTe5 are necessary to settle the puzzle of the unconventional Hall response in the quantum limit. In particular, experiments directly probing the density of states and the real space charge distribution such as Scanning Tunneling Spectroscopy and in-field X-ray diffraction could shed additional light on the nature of the observed feature.

## Methods

### Single-crystal sample growth and pre-characterization

Single crystals of HfTe5 were obtained via a chemical vapor transport method. Stoichiometric amounts of Hf (powder, 3 N) and Te (powder, 5 N) were sealed in a quartz ampoule with iodine (7 mg ml−1) and placed in a two-zone furnace. A temperature gradient in the range of 400–500 °C was applied. After ca. 1-month, long ribbon-shaped HfTe5 single crystals were extracted from the ampule with a typical size of the single crystals 1 mm × 0.5 mm × 3 mm (width × height × length). High-quality needle-shaped (about 0.1 × 0.3 × 20 mm3) single crystals of ZrTe5 were synthesized using the tellurium flux method and high-purity elements (99.9999% zirconium and 99.9999% tellurium). The lattice parameters of the crystals were confirmed by single-crystal X-ray diffraction. The samples used in this work are of the same batch as the ones reported in refs. 21,23,32,36 and have similar Fermi level positions. As shown in these papers, in our HfTe5 and ZrTe5 samples a three-dimensional topological Dirac semimetal state emerges only at around Tp ≈ 65 K (at which the resistivity shows a pronounced peak), manifested by a large negative magnetoresistance. This Dirac semimetal is a critical point between two distinct topological insulator phases: weak (T > Tp) and strong (T < Tp). At high temperatures, the extracted band gap is around 30 meV (185 K), and at low temperatures 10 meV (15 K)36. However, we note that the Fermi level at these temperatures is not located in the gap, but several meV in the valance band for T > Tp and in the conduction band for T < Tp. Hence, our HfTe5 and ZrTe5 samples are metallic at both high and low temperatures.

### Electrical transport measurements

Electrical contacts to the HfTe5 and ZrTe5 single crystals were defined with an Al hard mask. Ar sputter etching was performed to clean the sample surface prior to the sputter deposition of Ti (20 nm) and Pt (200 nm) with a BESTEC UHV sputtering system. Subsequently, Pt wires were glued to the sputtered pads using silver epoxy. All electrical transport measurements up to ±9 T were performed in a temperature-variable cryostat (PPMS Dynacool, Quantum Design), equipped with a dilution refrigerator inset. To avoid contact-resistance effects, only four-terminal measurements were carried out. The longitudinal ρxx and Hall resistivity ρxy were measured in a Hall-bar geometry with standard lock-in technique (Zurich instruments MFLI and Stanford Research SR 830), applying a current of 10 μA with a frequency of f = 1 kHz across a 100 kΩ shunt resistor. The electrical current is always applied along the a-axis of the crystal.

The pulsed magnetic field experiments up to 70 T were carried out at the Dresden High Magnetic Field Laboratory (HLD) at HZDR, a member of the European Magnetic Field Laboratory (EMFL).