## Abstract

Interacting electrons confined to their lowest Landau level in a high magnetic field can form a variety of correlated states, some of which manifest themselves in a Hall effect. Although such states have been predicted to occur in three-dimensional semimetals, a corresponding Hall response has not yet been experimentally observed. Here, we report the observation of an unconventional Hall response in the quantum limit of the bulk semimetal HfTe_{5}, adjacent to the three-dimensional quantum Hall effect of a single electron band at low magnetic fields. The additional plateau-like feature in the Hall conductivity of the lowest Landau level is accompanied by a Shubnikov-de Haas minimum in the longitudinal electrical resistivity and its magnitude relates as 3/5 to the height of the last plateau of the three-dimensional quantum Hall effect. Our findings are consistent with strong electron-electron interactions, stabilizing an unconventional variant of the Hall effect in a three-dimensional material in the quantum limit.

## Introduction

Applying a strong magnetic field to an electron gas confines the electrons motion in cyclotron orbits with a set of discrete eigenenergies—the Landau levels. In two-dimensional (2D) systems, this quantization leads to a fully gapped energy spectrum and to the emergence of the quantum Hall effect (QHE)^{1}. In the limit where only the lowest Landau level (LLL) is occupied (the so-called quantum limit), electron–electron interactions can play a significant role, leading to the appearance of correlated states, such as the fractional quantum Hall effect^{2}. In contrast, the Landau level spectrum of a three-dimensional (3D) electron gas is not fully gapped and becomes like that of a one-dimensional system. As a consequence, the electrons can still move along the field direction, which in turn destroys the quantization of the Hall effect^{3,4,5}. Nevertheless, it has been predicted that a generalized version of the QHE can emerge in 3D electron systems that exhibit a periodically modulated superstructure^{6,7,8}. Analogous to as in two dimensions, in the vicinity of the quantum limit, 3D electron systems are also prone to form a variety of correlated electron states, including Luttinger liquids; charge, spin and valley density waves; excitonic insulators; Hall and Wigner crystals; or staging transitions in the case of highly anisotropic layered systems^{3,4,6,9,10,11,12,13}. It has been theoretically pointed out that some of these states are related to quantum Hall physics in three dimensions and likewise could manifest themselves in a Hall response that should be observable in the quantum limit of 3D semimetals^{10,13,14}.

Inspired by these ideas, the possibility of finding a 3D QHE has been explored in several material systems. For example, signatures of the integer quantum Hall effect (IQHE) have been found in quasi-2D semiconducting multilayer lattices^{15}, Bechgaards salts^{16,17}, *η*-Mo_{4}O_{11}^{18}, *n*-doped Bi_{2}Te_{3}^{19}, and EuMnBi_{2}^{20}, in which the layered crystal structure itself supplies the stack of 2D systems. Very recently, the QHE has also been observed in 3D graphite films^{5}, bulk ZrTe_{5}^{21}, and HfTe_{5} samples^{22}. In graphite, the imposed periodic superstructure has been attributed to the formation of standing electron waves. In ZrTe_{5} and HfTe_{5}, the IQHE was originally believed to arise from a charge density wave (CDW), due to the scaling of plateau height with the Fermi wavevector. This scenario is, however, in contrast with thermodynamic and thermoelectric measurements on ZrTe_{5} that did not reveal any signatures of a field-induced CDW transition. Instead, it was proposed that ZrTe_{5} should be considered a stack of weakly interacting Dirac 2DEGs with the plateau height scaling originating from the interplay of small carrier density and the peculiarities of Landau quantization of the Dirac dispersion^{23}. In parallel to the search for the 3D QHE, there has been a long-standing experimental effort to observe field-induced correlated states in the quantum limit of three-dimensional materials. Although those studies have provided signatures of field-induced states in the longitudinal electrical resistivity of Bi^{24,25}, ZrTe_{5}^{21,26} and graphite^{5,27,28,29,30}, correlated states with Hall responses have yet to be observed.

In this work, we present measurements of the low-temperature longitudinal and Hall resistivities of the 3D semimetals HfTe_{5} and ZrTe_{5}. Previous studies have shown that HfTe_{5} is an isostructural counterpart of ZrTe_{5}^{31}. Both materials share an orthorhombic crystal structure and a single elliptical 3D Fermi surface, comprising less than 1% of the Brillouin zone and hosting massive Dirac Fermions with almost linearly dispersing bands in the vicinity of the Fermi level (see Supplementary Information for details). These specific properties have been considered essential for the observation of the 3D IQHE in both materials^{21,22,23}. Moreover, recent progress in ZrTe_{5}^{21} and HfTe_{5}^{32} single-crystal growth h**a**s enabled a Hall mobility *μ* that exceeds 100,000 cm^{2} V^{−1} s^{−1} at low temperatures (<4 K) (see “Methods” section and Supplementary Information S1). The quality of these crystals is comparable to that of graphene samples, which have previously proven appropriate for observing the FQHE in two dimensions^{33}. While the 3D band structure of ZrTe_{5} and HfTe_{5} is very similar^{21}, hafnium has a higher atomic number than zirconium and hence naturally introduces stronger spin–orbit coupling (SOC)^{31}, which has been previously shown to stabilize correlated states in the quantum limit of 2D electron systems^{34}. Therefore, HfTe_{5} is the more promising candidate for the observation of unconventional Hall responses of correlated ground states in its quantum limit.

## Results

HfTe_{5} and ZrTe_{5} typically grow as millimeter-long ribbons with an aspect ratio of approximately 1:3:10, reflecting their crystalline anisotropy. Details of the growth process, crystal structure, and first transport characterization of our samples can be found in refs. ^{21,32}. We have measured the longitudinal electrical resistivity *ρ*_{xx} and Hall resistivity *ρ*_{xy} (see “Methods” section) of three HfTe_{5} samples (A, B, C) and three ZrTe_{5} samples (D, E, F) as a function of magnetic field ** B** and temperature

*T*, with the electrical current applied along the

*a*-axis of the crystals.

At *T* = 300 K, *ρ*_{xx} is around 0.5 mΩ cm (see Fig.1a, Supplementary Fig. S1 and ref. ^{32}) with an electron density of *n* = 1.3 × 10^{19} cm^{−3} and *μ* = 10,000 cm^{2} V^{−1} s^{−1} ^{21,23,32}. Upon cooling in zero magnetic field, *ρ*_{xx} increases with decreasing *T* until it reaches a maximum at *T*_{L} = 70 K (Fig.1a). Such a maximum has previously been observed in HfTe_{5}^{32} and ZrTe_{5}^{21}, and it is attributed to a Lifshitz transition, here inducing a change in charge-carrier type. Consistently, the slope of *ρ*_{xy}(** B**) changes sign at

*T*

_{L}, indicating electron-type transport for

*T*<

*T*

_{L}

^{32}. At 3 K, we find

*n*= 8.7 × 10

^{16}cm

^{−3}and

*μ*= 250,000 cm

^{2}V

^{−1}s

^{−1}(see Supplementary Note 1 and ref.

^{21}). All investigated samples show similar electrical transport properties. In the main text, we focus on data obtained from HfTe

_{5}sample A and ZrTe

_{5}sample D. Additional data of samples B, C, E, and F can be found in the Supplementary Information and ref.

^{23}.

To characterize the Fermi-surface (FS) morphology of our pentatelluride samples, we have measured Shubnikov-de Haas (SdH) oscillations with respect to the main crystal axes at 3 K. For this purpose, we followed the analysis of ref. ^{21} and rotated **B** in the *z*–*y* and *z*–*x* planes, while measuring *ρ*_{xx} (**B**) at a series of different angles (Fig. 1, Supplementary Fig. S3 and ref. ^{23}). The SdH frequency *B*_{F,i} is directly related to the extremal cross-section of the Fermi surface *S*_{F,i}, normal to the applied **B** direction via the Onsager relation *B*_{F,i} = *S*_{F,i}(ℏ/2π*e*). Examples of the SdH oscillations for which the magnetic field was aligned along the three principal crystallographic directions (*x*, *y*, and *z* axes*)* are shown in Fig. 1b–d (upper panels). In each field direction, we have observed maxima in *ρ*_{xx} that are periodic in 1/**B**, each of which corresponds to the onset of a Landau level. In the associated minima, *ρ*_{xx}(**B**) does not vanish, which is a consequence of the remaining dispersion in *z* direction in 3D systems and Landau level-broadening due to disorder (see Supplementary Note 4, Supplementary Fig. S4–S7 and ref. ^{23} for details). To determine the SdH oscillation frequency, we have subtracted the smooth high-temperature (50 K)—*ρ*_{xx}(**B**) from the low-*T*-data, obtaining the oscillating part of the longitudinal resistivity Δ*ρ*_{xx}(**B**). Employing a standard Landau-index fan diagram analysis to Δ*ρ*_{xx}(**B**) (Fig. 1h, Supplementary Figs. S3, S8, S9, Supplementary Note 2 and ref. ^{23}), we have found only a single oscillation frequency for all rotation angles measured, consistent with a single electron pocket at the Fermi energy. The extracted *B*_{F,i} of sample A for ** B** along the three principal directions are

*B*

_{F,x}= (9.9 ± 0.1) T,

*B*

_{F,y}= (14.5 ± 0.5) T, and

*B*

_{F,z}= (1.3 ± 0.1) T. Here, the errors denote the standard deviation of the corresponding fits.

In contrast to 2D materials, HfTe_{5} and ZrTe_{5} show in-plane SdH oscillations when **B** is aligned with *x* and *y*, indicating a 3D Fermi-surface pocket. The shape of the FS is further determined by the analysis of the rotation angle-dependence of *B*_{F}. As shown in Fig. 1i, j, the angle-dependent SdH frequency is well represented by a 3D ellipsoidal equation \(B_{{\mathrm{F}},3D} = B_{{\mathrm{F}},z}B_{{\mathrm{F}},i}/\sqrt {(B_{{\mathrm{F}},z}\sin \theta )^2 + (B_{{\mathrm{F}},i}\cos \theta )^2}\), where *θ* is the rotation angle in the *z*–*i* plane. As a cross-check, we also plot the formula of a 2D cylindrical Fermi surface *B*_{F,2D} = *B*_{F,z}/cos*θ*, which deviates significantly from the experimental data for *θ* > 80°. Hence, the ellipsoid equations can be used to obtain the Fermi wave vectors \(k_{F,x} = \sqrt {S_{{\mathrm{F}},y}S_{{\mathrm{F}},z}} /\sqrt {{\uppi}S_{{\mathrm{F}},x}}\) = (0.005 ± 0.001) Å^{−1}, \(k_{{\mathrm{F}},y} = \sqrt {S_{{\mathrm{F}},x}S_{{\mathrm{F}},z}} /\sqrt {{\uppi}S_{{\mathrm{F}},y}}\) = (0.008 ± 0.001) Å^{−1} and \(k_{{\mathrm{F}},z} = \sqrt {S_{{\mathrm{F}},x}S_{{\mathrm{F}},y}} /\sqrt {{\uppi}S_{{\mathrm{F}},z}}\) = (0.058 ± 0.006) Å^{−1} that span the 3D FS of HfTe_{5} sample A in the *x*, *y*, and *z* direction, respectively (Fig. 1k). The errors in *k*_{F,i} originate from the errors of the *B*_{F,i}. The preceding analysis indicates that for our HfTe_{5} and ZrTe_{5} samples, the quantum limit with the field along the *z* is achieved already for the field of *B*_{C} = 1.8 and 1.2 T^{21,23}, respectively. Further details of our band-structure analysis can be found in Supplementary Fig. S6, Supplementary Table S1, refs. ^{21,23}. Above 6 T, we find that for both materials, *ρ*_{xx}(**B**) steeply increases with the magnetic field (Fig. 2b). Such a steep increase has previously been observed in ZrTe_{5} and has been attributed to a field-induced metal-insulator transition^{21}.

For the field-aligned with the *z* axis (**B**||*z*), we additionally observed in both studied compounds pronounced plateaus in Hall resistance *ρ*_{xy}(**B**) that appear at the minima of the SdH oscillations in *ρ*_{xx}(**B**)—features commonly related to the QHE (Fig. 2a and ref. ^{23}). The height of the last integer plateau is given by (*h*/*e*^{2}) *π*/*k*_{F,z}, similar to as reported in the literature for the 3D IQHE^{21,22}. The plateaus are most pronounced at low temperatures, but still visible up to *T* = 30 K (Fig. 2b, c and ref. ^{23}).

We note that the observed quantization of *ρ*_{xy} is not immediately obvious from the predicted quantization in *σ*_{xy}. The Hall resistivity tensor is given by *ρ*_{xy} = *σ*_{xy}/(*σ*_{xx}*σ*_{yy} + *σ*_{xy}^{2}) with a magnetic field in *z* direction, where *σ*_{xx} and *σ*_{xx} are the longitudinal component of the conductivity tensor in *x* and *y* direction, respectively. Vice versa, the Hall conductivity tensor element is given by *σ*_{xy} = *ρ*_{xy}/(*ρ*_{xx} *ρ*_{yy} + *ρ*_{xy}^{2}) with a magnetic field in *z* direction. However, in our samples at low temperatures *σ*_{xx} < *σ*_{xy} (Supplementary Figs. S11 and S12) and thus *σ*_{xy}^{−1} ≈ *ρ*_{xy}, enabling the direct observation of the quantization. Due to the geometry of the HfTe_{5} crystals (elongated needles) and their mechanical fragility, performing reliable measurements of *ρ*_{yy} is not possible. Instead, we estimate the error of the *σ*_{xy} using the ratio of Drude resistivities \(\rho _{yy}/\rho _{xx} = (n_{xy}e^2t_x/m_x^ \ast )/({\it{n}}_{xy}e^2t_y/m_y^ \ast )\) given by the quantum lifetimes and effective masses obtained from Shubnikov-de Haas oscillations on sample A (Supplementary Table S1). *n*_{xy} is the charge-carrier concentration in the *x*–*y* plane. Based on this analysis, we find *ρ*_{yy}/*ρ*_{xx} ≈ 0.4, which results in an error of below 8 % in the estimated *σ*_{xy} for the investigated field range owing to *ρ*_{xx} (*B*) < *ρ*_{xy} (*B*). Both these errors lay within the estimated error of *k*_{F,z} of 10 %.

Figure 2d shows the angular-dependence of the Hall plateaus, which we find to scale with the rotation angle. This behavior is very similar to the sister compound ZrTe_{5}.^{21} In both materials, the height of the Hall plateaus and its position in |**B**| depends only on the field component that is perpendicular to the *x*–*y* plane *B*_{⊥} = |**B**|cos*θ*^{21,23}*.*

So far, our analysis focused on similarities between the Hall effects observed in ZrTe5 and HfTe5. Upon cooling the samples to 50 mK, an obvious difference emerges, as shown in Fig. 3 and Supplementary Figs. S11–S15. At low fields below the quantum limit (*B* < BC), both compounds exhibit signatures of new peaks and plateaus in *ρ*_{xx}(*B*) and *ρ*_{xy}(*B*). Such features have been observed in the past and are related to spin splitting of the Landau levels^{23,35}. However, at high fields (*B* > BC)—in the quantum limit, HfTe5 exhibits an additional peak in *ρ*_{xx}(*B*), accompanied by a plateau-like feature in *ρ*_{xy}(*B*). This is in sharp contrast to ZrTe5, in which *ρ*_{xx}(*B*) and *ρ*_{xy}(*B*) smoothly increase. Using the Landau-index fan diagram obtained at 3 K and gauging the indexing of Landau bands with respect to the *N* = 1 band, we find that the additional peak in *ρ*_{xx}(*B*), in the quantum limit of HfTe5 is situated at *N* = 3/5. This indexing is confirmed by corresponding maxima in *ρ*_{xx}(*B*) and/or *σ*_{xx}(*B*) of all three HfTe_{5} samples investigated (compare Supplementary Figs. S11 and S12), despite being less pronounced in some of them.

Although the relation between the magnitude of the plateau-like feature and its corresponding Landau index is not obvious from *ρ*_{xy}(*B*), a comparison of the respectively calculated conductivity reveals that the magnitude of the plateau-like feature in the quantum limit scales as 3/5-times with respect to the plateau related to the LLL. A close investigation of the plateau height in conductivity (Supplementary Fig. S16) reveals that both the plateau-like features at *N* = 1 and *N* = 3/5 are well developed in the conductivity, within 1 and 2% of *N*·(*e*^{2}/*h*)*k*_{F,z/}π in the range of 0.5 T around the plateau center.

In order to verify whether the observed features can be explained by invoking the presence of a second pocket at the Fermi energy, we have performed additional magneto-transport measurements up to 70 T (Supplementary Fig. S16 and ref. ^{23}). The measurements did not reveal any additional quantum oscillations, which is consistent with band-structure calculations^{31} and a previous ARPES study on our samples^{36}: The Fermi level, obtained from the analysis of the Shubnikov-de Haas oscillations is (9 ± 2) meV (Supplementary Information), which is in agreement with the ARPES experiment. According to the ARPES data, at 15 K, the nearest additional band is located ~5 meV above the Fermi level (lowest temperature measured in the ARPES study) as compared to the Fermi function broadening of *k*_{B}·15 K ≈ 1 meV. Below 15 K, the Fermi level stays constant with respect to the band edges, as indicated by the temperature-independent Shubnikov-de Haas frequency in our experiments. Hence, the next nearest band in our samples is ~*k*_{B}·60 K away from the Fermi level and does not contribute to the low-temperature transport experiments. Our data can, therefore, be analyzed in terms of a single electron-type Dirac pocket.

Further insight into the possible origin of the *N* = 3/5 state in the quantum limit can be obtained from the line shape of Δ*ρ*_{xx}(*B*), which resembles the line shape of *σ*_{xx}(*B*), (Fig. 4e, f) a common feature of canonical 2D QHE systems^{37} A related empirical observation is that in both fractional and IQHE in 2D systems the longitudinal resistance is *ρ*_{xx}(**B**) is connected to *ρ*_{xy}(**B**) via *ρ*_{xx}(*B*_{z}) = *γB·*d*ρ*_{xy}(**B**)*/*d**B**, where *γ* is a dimensionless parameter of the order of 0.01–0.05, which measures the local electron concentration fluctuations^{38,39}. Comparison of *σ*_{xx}(**B**) (Fig. 4d) and *γB·*d*ρ*_{xy}(**B**)*/*d**B** (Fig. 4e, upper panel) as a function of **B**^{−1} reveals that both quantities show maxima and minima at the same field positions as Δ*ρ*_{xx}(**B**). In particular, the derivative relation is well fulfilled with *γ* = 0.04, which is in the expected range reported for 2DESs. These results suggest that the observed plateau-like feature observed in the quantum limit in HfTe_{5} is related to quantum Hall physics.

To gain quantitative insights into the states that cause the features in the Hall effect in HfTe_{5}, we have estimated the gap energies of the *x–y* plane *Δ*_{N} associated with *N* = 1 and *N* = 2 below and *N* = 3/5 in the quantum limit. We have fitted the *T*-dependence of the *ρ*_{xx}(*B*) minima (Fig. 4f) in the thermally activated regime *ρ*_{xx}(**B**) *α* exp(−Δ_{N}/2*k*_{B}*T*), where *k*_{B} is the Boltzmann constant (Supplementary Fig. S17). For integer *N*, we find Δ_{1} = (40 ± 2) K at *N* = 1 and Δ_{2} = (9 ± 1) K at *N* = 2. The gap energy of the *N* = 3/5 state in the quantum limit is two orders of magnitude lower: Δ_{3/5} = (0.49 ± 0.09) K. The deviations given for the gaps are the errors obtained from the thermally activated fits in Supplementary Fig. S13. In spite of considerable Landau level-broadening, both the size of the gaps of the integer and the *N* = 3/5 states compare well with the gaps obtained for integer and correlated quantum Hall states in 2DESs^{37,40}. The different Δ_{N} are also in agreement with the *T*-dependence of the corresponding Hall features (Fig. 4g). While the integer plateaus are observable up to tens of Kelvin, the plateau-like feature in the quantum limit vanishes at around 0.5 K.

Those considerations suggest that the Hall feature observed in the quantum limit of HfTe_{5} is associated with physics in the LLL only (as pointed out above, this Landau level is non-degenerate since HfTe_{5} is like ZrTe_{5} a gapped Dirac semimetal). The finite value of *ρ*_{xx} at *N* = 3/5 implies the absence of a fully established bulk gap, which in turn means that a truly quantized Hall effect as in 2D systems without the *k*_{F,z}-scaling cannot be expected. Nevertheless, the emergence of the plateau-like feature in the quantum limit of a single-band system at low temperatures calls for an explanation beyond a simple single-particle picture: The Hall conductivity of a non-interacting single band in which the chemical potential adjusts to keep the particle number fixed simply decreases as 1/**B**, as observed in the isostructural ZrTe_{5} (Fig.3b).

## Discussion

Although possible scenarios for the emergence of a plateau in the Hall resistance in the quantum limit of electron plasma include the formation of a CDW, Luttinger liquid, Wigner crystallization, or the so-called Hall crystal^{41,42}, a favorable scenario builds on the notion that ZrTe_{5} and HfTe_{5} can be thought of as a stack of interacting 2DEGs. Based on a Hartree–Fock analysis, it was proposed^{13} that in a layered structure the gain in exchange energy can exceed the energy cost for distributing electrons unequally between layers. The electrons then undergo spontaneous staging transitions in which only every *i*-th layers is occupied, while all other layers are emptied (the number *i* depends on the average electron density and the state formed)—some of which are only stabilized due to the interplay of electron interaction and spin–orbit coupling^{43}. Depending on layer separation, electron density, and the strength of electron–electron interactions, various types of layered Laughlin states or Halperin states can then be formed^{3,13,14}. These states are naturally associated with Hall responses. While staging transitions are unlikely in isotropic three-dimensional materials at high electron densities, HfTe_{5} has a very anisotropic band structure with small tunneling amplitudes along *z*, and hosts only a relatively small number of electrons in its Dirac pocket. Our data are thus consistent with strong interactions stabilizing a correlated state that gives a Hall response in HfTe_{5} in the quantum limit.

In conclusion, our measurements reveal an unconventional correlated electron state manifested in the Hall conductivity of the bulk semimetal HfTe_{5} in the quantum limit, adjacent to the 3D IQHE at lower magnetic fields. The observed plateau-like feature is accompanied by a Shubnikov-de Haas minimum in the longitudinal electrical resistivity and its magnitude is approximately given by *3/5*(*e*^{2}/*h*)*k*_{F,z/}π. Analysis of derivative relations and estimation of the gap energies suggest that this feature is related to quantum Hall physics. The absence of this unconventional feature in the quantum limit of isostructural single-band ZrTe_{5} samples with similar electron mobility and Fermi wavevector indicates the presence of a correlated state that may be stabilized by spin–orbit coupling. However, further experimental and theoretical efforts in determining the real interactions and texture of the field-induced correlated states in HfTe_{5} are necessary to settle the puzzle of the unconventional Hall response in the quantum limit. In particular, experiments directly probing the density of states and the real space charge distribution such as Scanning Tunneling Spectroscopy and in-field X-ray diffraction could shed additional light on the nature of the observed feature.

## Methods

### Single-crystal sample growth and pre-characterization

Single crystals of HfTe5 were obtained via a chemical vapor transport method. Stoichiometric amounts of Hf (powder, 3 N) and Te (powder, 5 N) were sealed in a quartz ampoule with iodine (7 mg ml^{−1}) and placed in a two-zone furnace. A temperature gradient in the range of 400–500 °C was applied. After ca. 1-month, long ribbon-shaped HfTe5 single crystals were extracted from the ampule with a typical size of the single crystals 1 mm × 0.5 mm × 3 mm (width × height × length). High-quality needle-shaped (about 0.1 × 0.3 × 20 mm^{3}) single crystals of ZrTe5 were synthesized using the tellurium flux method and high-purity elements (99.9999% zirconium and 99.9999% tellurium). The lattice parameters of the crystals were confirmed by single-crystal X-ray diffraction. The samples used in this work are of the same batch as the ones reported in refs. ^{21,23,32,36} and have similar Fermi level positions. As shown in these papers, in our HfTe5 and ZrTe5 samples a three-dimensional topological Dirac semimetal state emerges only at around *T*_{p} ≈ 65 K (at which the resistivity shows a pronounced peak), manifested by a large negative magnetoresistance. This Dirac semimetal is a critical point between two distinct topological insulator phases: weak (*T* > *T*_{p}) and strong (*T* < *T*_{p}). At high temperatures, the extracted band gap is around 30 meV (185 K), and at low temperatures 10 meV (15 K)^{36}. However, we note that the Fermi level at these temperatures is not located in the gap, but several meV in the valance band for *T* > *T*_{p} and in the conduction band for *T* < *T*_{p}. Hence, our HfTe5 and ZrTe5 samples are metallic at both high and low temperatures.

### Electrical transport measurements

Electrical contacts to the HfTe_{5} and ZrTe_{5} single crystals were defined with an Al hard mask. Ar sputter etching was performed to clean the sample surface prior to the sputter deposition of Ti (20 nm) and Pt (200 nm) with a BESTEC UHV sputtering system. Subsequently, Pt wires were glued to the sputtered pads using silver epoxy. All electrical transport measurements up to ±9 T were performed in a temperature-variable cryostat (PPMS Dynacool, Quantum Design), equipped with a dilution refrigerator inset. To avoid contact-resistance effects, only four-terminal measurements were carried out. The longitudinal *ρ*_{xx} and Hall resistivity *ρ*_{xy} were measured in a Hall-bar geometry with standard lock-in technique (Zurich instruments MFLI and Stanford Research SR 830), applying a current of 10 μA with a frequency of *f* = 1 kHz across a 100 kΩ shunt resistor. The electrical current is always applied along the *a*-axis of the crystal.

The pulsed magnetic field experiments up to 70 T were carried out at the Dresden High Magnetic Field Laboratory (HLD) at HZDR, a member of the European Magnetic Field Laboratory (EMFL).

## Data availability

All data generated or analyzed during this study are available within the paper and its Supplementary Information file. Reasonable requests for further source data should be addressed to the corresponding author.

## References

- 1.
Klitzing, K. V., Dorda, G. & Pepper, M. New method for high-accuracy determination of the fine-structure constant based on quantized hall resistance.

*Phys. Rev. Lett.***45**, 494–497 (1980). - 2.
Tsui, D. C., Stormer, H. L. & Gossard, A. C. Two-dimensional magnetotransport in the extreme quantum limit.

*Phys. Rev. Lett.***48**, 1559 (1982). - 3.
Burnell, F. J., Bernevig, B. A. & Arovas, D. P. Scenario for fractional quantum Hall effect in bulk isotropic materials.

*Phys. Rev. B***79**, 155310 (2009). - 4.
Chalker, J. T. in

*Supersymmetry and Trace Formulae*75–83 (Springer, 1999). - 5.
Yin, J. et al. Dimensional reduction, quantum Hall effect and layer parity in graphite films.

*Nat. Phys.***15**, 437–442 (2019). - 6.
Halperin, B. I. Possible states for a three-dimensional electron gas in a strong magnetic field.

*Jpn. J. Appl. Phys.***26**, 1913 (1987). - 7.
Kohmoto, M., Halperin, B. I. & Wu, Y.-S. Diophantine equation for the three-dimensional quantum Hall effect.

*Phys. Rev. B***45**, 13488 (1992). - 8.
Bernevig, B. A., Hughes, T. L., Raghu, S. & Arovas, D. P. Theory of the three-dimensional quantum Hall effect in graphite.

*Phys. Rev. Lett.***99**, 146804 (2007). - 9.
Celli, V. & Mermin, N. D. Ground state of an electron gas in a magnetic field.

*Phys. Rev.***140**, A839 (1965). - 10.
Pan, Z. & Shindou, R. Ground-state atlas of a three-dimensional semimetal in the quantum limit.

*Phys. Rev. B***100**, 165124 (2019). - 11.
Takada, Y. & Goto, H. Exchange and correlation effects in the three-dimensional electron gas in strong magnetic fields and application to graphite.

*J. Phys. Condens. Matter***10**, 11315 (1998). - 12.
Akiba, K. et al. Possible excitonic phase of graphite in the quantum limit state.

*J. Phys. Soc. Jpn.***84**, 54709 (2015). - 13.
MacDonald, A. H. Staging transitions in multiple-quantum-well systems.

*Phys. Rev. B***37**, 4792 (1988). - 14.
Qiu, X., Joynt, R. & MacDonald, A. H. Phase transitions in a multiple quantum well in strong magnetic fields.

*Phys. Rev. B***42**, 1339 (1990). - 15.
Störmer, H. L., Eisenstein, J. P., Gossard, A. C., Wiegmann, W. & Baldwin, K. Quantization of the Hall effect in an anisotropic three-dimensional electronic system.

*Phys. Rev. Lett.***56**, 85 (1986). - 16.
Hannahs, S. T., Brooks, J. S., Kang, W., Chiang, L. Y. & Chaikin, P. M. Quantum Hall effect in a bulk crystal.

*Phys. Rev. Lett.***63**, 1988 (1989). - 17.
Cooper, J. R. et al. Quantized Hall effect and a new field-induced phase transition in the organic superconductor (TMTSF) 2 PF 6.

*Phys. Rev. Lett.***63**, 1984 (1989). - 18.
Hill, S. et al. Bulk quantum Hall effect in η− Mo

_{4}O_{11}.*Phys. Rev. B***58**, 10778 (1998). - 19.
Cao, H. et al. Quantized Hall effect and Shubnikov–de Haas oscillations in highly doped Bi 2 Se 3: Evidence for layered transport of bulk carriers.

*Phys. Rev. Lett.***108**, 216803 (2012). - 20.
Masuda, H. et al. Quantum Hall effect in a bulk antiferromagnet EuMnBi

_{2}with magnetically confined two-dimensional Dirac fermions.*Sci. Adv.***2**, e1501117 (2016). - 21.
Tang, F. et al. Three-dimensional quantum Hall effect and metal–insulator transition in ZrTe

_{5}.*Nature***569**, 537–541 (2019). - 22.
Wang, P. et al. Approaching three-dimensional quantum Hall effect in bulk HfTe

_{5}.*Phys. Rev. B***101**, 161201 (2020). - 23.
Galeski, S. et al. Origin of the quasi-quantized Hall effect in ZrTe

_{5}. Preprint at https://arxiv.org/abs/2005.12996 (2020). - 24.
Yang, H. et al. Phase diagram of bismuth in the extreme quantum limit.

*Nat. Commun.***1**, 47 (2010). - 25.
Behnia, K., Balicas, L. & Kopelevich, Y. Signatures of electron fractionalization in ultraquantum bismuth.

*Science***317**, 1729–1731 (2007). - 26.
Yu, W. et al. Quantum oscillations at integer and fractional Landau level indices in single-crystalline ZrTe

_{5}.*Sci. Rep.***6**, 35357 (2016). - 27.
Yaguchi, H. & Singleton, J. A high-magnetic-field-induced density-wave state in graphite.

*J. Phys. Condens. Matter***21**, 344207 (2009). - 28.
Esquinazi, P. D. & Lysogorskiy, Y.

*Basic Physics of Functionalized Graphite*(Springer, 2016). - 29.
Arnold, F. et al. Charge density waves in graphite: towards the magnetic ultraquantum limit.

*Phys. Rev. Lett.***119**, 136601 (2017). - 30.
LeBoeuf, D. et al. Thermodynamic signatures of the field-induced states of graphite.

*Nat. Commun.***8**, 1–6 (2017). - 31.
Weng, H., Dai, X. & Fang, Z. Transition-metal pentatelluride ZrTe

_{5}and HfTe_{5}: a paradigm for large-gap quantum spin Hall insulators.*Phys. Rev. X***4**, 11002 (2014). - 32.
Zhao, L.-X. et al. Anomalous magneto-transport behavior in transition metal pentatelluride HfTe

_{5}. Chinese.*Phys. Lett.***34**, 37102 (2017). - 33.
Bolotin, K. I., Ghahari, F., Shulman, M. D., Stormer, H. L. & Kim, P. Observation of the fractional quantum Hall effect in graphene.

*Nature***462**, 196 (2009). - 34.
Ito, T., Nomura, K. & Shibata, N. Spin–orbit interaction enhanced fractional quantum Hall states in the second Landau level.

*J. Phys. Soc. Jpn.***79**, 73003 (2010). - 35.
Liu, Y. et al. Zeeman splitting and dynamical mass generation in Dirac semimetal ZrTe

_{5}.*Nat. Commun.***7**, 1–9 (2016). - 36.
Zhang, Y. et al. Temperature-induced Lifshitz transition in topological insulator candidate HfTe

_{5}.*Sci. Bull.***62**, 950–956 (2017). - 37.
Dean, C. R. et al. Multicomponent fractional quantum Hall effect in graphene.

*Nat. Phys.***7**, 693 (2011). - 38.
Chang, A. M. & Tsui, D. C. Experimental observation of a striking similarity between quantum Hall transport coefficients.

*Solid State Commun.***56**, 153–154 (1985). - 39.
Simon, S. H. & Halperin, B. I. Explanation for the resistivity law in quantum Hall systems.

*Phys. Rev. Lett.***73**, 3278 (1994). - 40.
Clark, R. G., Mallett, J. R., Haynes, S. R., Harris, J. J. & Foxon, C. T. Experimental determination of fractional charge e/q for quasiparticle excitations in the fractional quantum Hall effect.

*Phys. Rev. Lett.***60**, 1747 (1988). - 41.
Tešanović, Z., Axel, F. & Halperin, B. I. “Hall crystal” versus Wigner crystal.

*Phys. Rev. B***39**, 8525 (1989). - 42.
Wallace, P. R. The band theory of graphite.

*Phys. Rev.***71**, 622 (1947). - 43.
Mishra, A. & Lee, S. Stacking sensitive topological phases in a bilayer Kane-Mele-Hubbard model at quarter filling.

*Phys. Rev. B***100**, 45146 (2019).

## Acknowledgements

We thank Andrei Bernevig for fruitful discussions. T.M. acknowledges financial support by the Deutsche Forschungsgemeinschaft via the Emmy Noether Program ME4844/1-1 (project id 327807255), the Collaborative Research Center SFB 1143 (project id 247310070), and the Cluster of Excellence on Complexity and Topology in Quantum Matter ct.qmat (EXC 2147, project id 390858490). C.F. acknowledges the research grant DFG-RSF (NI616 22/1): Contribution of topological states to the thermoelectric properties of Weyl semimetals and SFB 1143. G.F.C. was supported by the Ministry of Science and Technology of China through Grant No. 2016YFA0300604, and the National Natural Science Foundation of China through Grant No. 11874417. T.F. and J.W. acknowledge support from the DFG through the Würzburg-Dresden Cluster of Excellence on Complexity and Topology in Quantum Matter ct:qmat (EXC 2147, project-id 39085490), the ANR-DFG grant Fermi-NESt, and by Hochfeld-Magnetlabor Dresden (HLD) at HZDR, member of the European Magnetic Field Laboratory (EMFL). P.M.L., Q.L., and G.G. acknowledge support from the Office of Basic Energy Sciences, U.S. Department of Energy (DOE) under Contract No. DE-SC0012704. J.G. acknowledges support from the European Union’s Horizon 2020 research and innovation program under Grant Agreement ID 829044 “SCHINES”.

## Funding

Open Access funding enabled and organized by Projekt DEAL.

## Author information

### Affiliations

### Contributions

S.G. and J.G. conceived the experiment. X.Z., W.Z., and G.F.C. synthesized and pre-characterized the single-crystal HfTe_{5} bulk samples. P.M.L., Q.L., and G.G. synthesized and pre-characterized the single-crystal ZrTe_{5} bulk samples. S.G. and S.H. fabricated electrical transport devices. A.M. and C.F. sputtered the electrical contacts on the samples. S.G. carried out the low-field transport measurements with the help of S.H., J.G., and R.W. S.G., J.G., T.F., and J.W. carried out the high-field transport experiments. T.E. and T.M. provided the model of the three-dimensional quantum Hall effect. S.G., N.L., T.M., T.F., and J.G. analyzed the data. All authors contributed to the interpretation of the data and to the writing of the manuscript.

### Corresponding authors

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Additional information

**Peer review information** *Nature Communications* thanks Liyuan Zhang and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

**Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Supplementary information

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Galeski, S., Zhao, X., Wawrzyńczak, R. *et al.* Unconventional Hall response in the quantum limit of HfTe_{5}.
*Nat Commun* **11, **5926 (2020). https://doi.org/10.1038/s41467-020-19773-y

Received:

Accepted:

Published:

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.