## Introduction

Quantum spin Hall insulators (QSHIs) constitute an important class of topological systems having a gapped insulating bulk and gapless helical edge states. Importantly, helical edge states, where the helical locking of spin and momentum suppresses backscattering of charge carriers, are robust against interactions and nonmagnetic disorders, making QSHIs possess promising applications from low power electronics to quantum computing1,2,3,4,5,6,7,8,9. After the first experimental realization of a QSHI in the form of a HgTe/CdTe quantum well at cryogenic temperatures4,10, a quantum spin Hall state has subsequently been identified in exfoliated 1T′ phase of transition metal dichalcogenides (e.g., WTe2) by scanning tunneling microscopy11 and charge transport measurements12. Despite their massive fundamental interest and their prospective technological applications, a major challenge is the identification of large gap QSHI materials, which would enable room temperature dissipationless transport of their edge states.

Recently, a robust QSHI with a gap of up to 0.5 eV, which is one order of magnitude larger than that in WTe2, has also been predicted in monolayer Pt2HgSe313. The ternary compound Pt2HgSe3, so called Jacutingaite14,15,16,17,18,19,20, has a “sandwich-like” structure reminiscent of transition metal dichalcogenides, with a platinum layer between selenium and mercury. In the case of the monolayer, it was argued that the competition between large spin-orbit coupling, associated with Hg and Pt atoms, and sublattice symmetry breaking leads to a QSHI state robust at room temperature and switchable by external electric fields13. Furthermore, recent theoretical work found that bulk Pt2HgSe3 is one of only a few known dual-topological semimetals and may host different surface states protected by symmetries that are unrelated to the QSHI state21,22,23.

Specifically, Wu et al. predicted that the monolayer of Pt2HgSe3 hosts different phases of unconventional superconductivity for finite hole and electron doping24. High pressure can effectively modify lattice structures and the corresponding electronic states in a systematic fashion25,26,27. Indeed, superconductivity has been induced by the use of pressure in some topological compounds27,28,29,30,31,32,33,34. Here, we systematically investigate the high-pressure behavior of bulk Pt2HgSe3. Through ab initio band structure calculations, we find that the application of pressure does not qualitatively change the electronic and topological nature of the material until the structural phase transition is observed in the high-pressure regime. Interestingly, superconductivity appears beyond the structural phase transition and the maximum critical temperature, Tc, of 4.4 K at 88.8 GPa is observed. The results demonstrate that Pt2HgSe3 compounds with nontrivial topology of electronic states display a ground states upon compression.

## Results and discussion

### Crystal structure and electronic properties of Pt2HgSe3

Jacutingaite (Pt2HgSe3) is a layered platinum-group mineral, which has a centrosymmetric trigonal structure, belonging to the space group $$P\bar 3m1$$ (No.164). As shown in Fig. 1a, b, Hg atoms form a buckled honeycomb lattice surrounded by triangles of Pt and Se. There are two inequivalent platinum positions indicated by Pt1 and Pt2. The Pt1 atoms show an octahedral coordination with six selenium atoms, while Pt2 are surrounded by two mercury atoms in a trans position with respect to one another and four selenium atoms in a square planar coordination. The Pt1 and Pt2 octahedra are Se-Se edges shared and form layers oriented parallel to (001), which is further AA-type stacking along the c axis. The $$P\bar 3m1$$ phase is known to be topological at ambient pressure. In our ab initio calculations, we found that, at all pressured studied in this work, Pt2HgSe3 remains topological with the location of topological surface states being slightly modified. The details of the bulk, surface states and their evolutions under pressure will be discussed in later sections. Here, we show in Fig. 1c, d the bulk and surface electronic structures at an arbitrary pressure 15.9 GPa. In Fig. 1c, the dashed blue lines and the solid red lines correspond to the bulk band structures without/with spin-orbital coupling. As is clearly shown, under pressure, Pt2HgSe3 remains metallic and topological, which holds as long as the space group is unchanged.

### Electrical resistivity at high pressure

Pt2HgSe3 possesses a typical layered structure, which is principally sensitive to external pressure. Hence, we measured ρ(T) for Pt2HgSe3 using a nonmagnetic diamond anvil cell (DAC). Fig. 2a shows the plots of temperature versus resistivity of Pt2HgSe3 for pressures up to 88.8 GPa. It reveals a metallic behavior in the whole pressure range. In a low-pressure region, increasing the pressure initially induces a weak but continuous suppression of the overall magnitude of ρ with a minimum occurring at Pmin = 12 GPa. Upon further increasing pressure, the resistivity starts to increase gradually. At a pressure of 54.9 GPa, a small resistivity drop presents at 1.9 K, indicating superconducting phase transition. It should be noted that Mauro et al. performed transport measurements on individual flakes of Pt2HgSe3 down to 250 mK and found no superconducting transition18. However, it cannot rule out that superconductivity appears in the material at pressures below 54.9 GPa, at temperatures lower than measured here. As shown in Fig. 2b, the resistivity drop becomes more visible and the critical temperature Tc increases to the maximum of 4.4 K at 88.8 GPa. The measurements on different samples of Pt2HgSe3 for three independent runs provide the consistent and reproducible results (Supplementary Fig. 1), confirming the intrinsic superconductivity under pressure. To gain insights into the superconducting transition, we applied the magnetic field for Pt2HgSe3 subjected to 88.8 GPa. When increasing μ0H, the resistivity drop is continuously shifted to a lower temperature (Fig. 2c). The upper critical field, μ0Hc2, is determined using 90% point on the resistivity transition curves, and plots of Hc2(T) are shown in Fig. 2d. A simple estimate using the conventional one-band Werthamer–Helfand–Hohenberg approximation35, neglecting the Pauli spin-paramagnetism effect and spin-orbit interaction, yields a value of 2.37 T for Pt2HgSe3. By using the Ginzburg–Landau formula $$\mu _0H_{c2}\left( T \right) = \mu _0H_{c2}\left( 0 \right)\left( {1 - \left( {T/T_c} \right)^2} \right)/\left( {1 + \left( {T/T_c} \right)^2} \right)$$ to fit the data, the estimated μ0Hc2(0) value is 3.07 T at 88.8 GPa. These fields are much lower than the Pauli limiting fields, HP(0) = 1.84Tc ~ 8.10 T, respectively, indicating that Pauli pair breaking is not relevant.

### Crystal structure evolution at high pressure

To further identify the pressure-induced electron structure transition, in situ x-ray diffraction (XRD) measurements have been performed on Pt2HgSe3 to analysis the structure evolution under various pressures. Fig. 3a displays the high-pressure synchrotron XRD patterns of Pt2HgSe3 measured at room temperature up to 96.3 GPa. A representative refinement at 1 atm is displayed in Supplementary Fig. 2. All the diffraction peaks can be indexed well to a trigonal structure with space group $$P\bar 3m1$$ based on Rietveld refinement with General Structure Analysis System (GSAS) software package36. As shown in Fig. 3b, both a-axis and c-axis lattice constants decrease with increasing pressure. The structure of Pt2HgSe3 is robust until 50 GPa. However, when the pressure increases up to 54.2 GPa, a set of new peaks emerges and dominates on further compression, indicating the occurrence of a structural phase transition. It should be noted that the superconductivity is observed beyond this pressure.

The pressure-induced structure evolution of Pt2HgSe3 was also confirmed by in situ Raman spectroscopy measurements. According to group theory analysis, there are seven Raman-active modes (3A1g+4Eg) that can be observed experimentally for Pt2HgSe337. Fig. 3c shows the Raman spectra of Pt2HgSe3 at various pressures. The assignments of the modes of Pt2HgSe3 at 1.9 GPa are given as follows: $$A_{1g}^3$$ = 96.3 cm−1, $$E_g^3$$ = 137.2 cm−1, $$E_g^2$$ = 184.3 cm−1, $$A_{1g}^2$$ = 202.2 cm−1, $$A_{1g}^1$$ = 210.7 cm−1, $$E_g^1$$ = 213.1 cm−1. The $$E_g^4$$ mode has not been observed in ambient condition due to its low scattering efficiency. With increasing pressure, the profile of the spectra remains similar to that at ambient pressure, whereas the observed modes exhibit blue shift, thus showing the normal pressure behavior (Fig. 3d). An abrupt disappearance of Raman peaks for pressure near to 50 GPa indicates the structural phase transition to a high-pressure phase (Supplementary Fig. 3). The evolution of the Raman spectra is consistent with our synchrotron XRD patterns. In summary, the Raman study provides further evidence for pressure-induced structural phase transitions.

### Pressure-induced phase transition and high-pressure structure

To identify the high-pressure phase, we performed extensive structure searches of Pt2HgSe3 by using our developed structure search method38,39 (Supplementary Figs. 46). Structure searches are carried out with simulation cells ranging from one to four formula units and successfully predicted a stable phase (space group Pc, Phase II) at 68 GPa. The enthalpy difference curves for the predicted phases are shown in Supplementary Fig. 4a. The Phase II has lower enthalpy than that of the ambient phase (space group $$P\bar 3m1$$, Phase I)15, when the pressure is above 45 GPa, indicating the energetic stability of the newly predicted phase. The experimentally observed phase transition is in apparent agreement with our theoretical prediction. Dynamical structural stabilities of the predicted structure were further investigated by calculating phonon dispersion curves. As shown in Supplementary Figs. 4 and 7, no imaginary frequency was found for the new structure, indicating dynamical stability of predicted structures.

We emphasize that by only relying on the experimental data, the structural evolution of the high-pressure phase is not possible because the XRD peaks are rather weak and broad. However, we have the predicted structure at hand, allowing us to refine the observed XRD data from 58.0 to 96.3 GPa by using the predicted structures. It is remarkable that the uses of the predicted structures gave excellent Rietveld fittings, therefore leading to the unambiguous determination of the high pressure as the predicted structure. The high-pressure phase (Phase II) of Pt2HgSe3 possesses a monoclinic structure with space group Pc. The details of crystallographic data are shown in Table 1. The predicted crystal structure of Pt2HgSe3 at 68 GPa is shown in Fig. 4a. Accompanying the structure transition, the bonding feature changes dramatically. In the Phase II, Hg exists as a dimer with Hg-Hg distance of 2.52 Å, which is much smaller than that shortest value (4.5 Å) in ambient phase. In addition, compared with Phase I, the Pt-Pt, Se-Se distances decrease from 3.67, 3.37 Å to 2.81, 2.54–2.77 Å, respectively, while the Hg-Pt, Hg-Se and Pt-Se distances become diverse.

### Electronic structure and topological properties under high pressure

To theoretically understand the evolution of the topology under pressures, we have performed DFT calculations of Pt2HgSe3 in $$P\bar 3m1$$ phase. Fig. 5a shows the bulk electronic structure of Pt2HgSe3 at ten different pressures. At all pressures, Pt2HgSe3 are metallic with electron pockets at K and H, and hole pockets between A-L, A-H, K-H, etc. With the increase of the applied pressure, the electron pockets become larger with the Dirac point at K and H moving to higher binding energy. Meanwhile, the hole pockets between A-L, A-H become larger as well by extending to higher energy. However, the hole pocket between K-H becomes smaller.

We further calculated the electronic states at (001) surface of Pt2HgSe3 to gain insight into the pressure influence on the topological nature, shown in Fig. 5b. We used the selected columns of the density matrix method to automatically generate the Wannier orbitals40,41, which were subsequently used to construct the tight-binding model to reproduce the 144 Bloch bands around the Fermi level (EF) and further calculated the surface states with the iterative Green’s function approach42. As shown in Fig. 5b, at all pressures studied where the $$P\bar 3m1$$ structure is preserved, the topological nature of Pt2HgSe3 is unaffected by the external pressure. Both surface states locating above and below the Fermi level survive at all ten pressures, indicating a robust topological nature of the system, confirming the topology is crystalline symmetry protected.

We further calculated the electronic band structure and the density of states (DOS) for the newly predicted phase. Fig. 4b indicates typical metallic feature of phase II (Pc phase) at 68 GPa, with significant contributions from Pt t2g and Se py orbitals around the Fermi level. Our results demonstrate that superconductivity observed here comes from phase II. This is obviously different from previous prediction where unconventional superconductivity in monolayer jacutingaite could induce at van Hove filling for electron and hole doping24. We also characterized the phase II in terms of the topology by calculating the elementary band representations of the bands below the band gap around the Fermi level using VASP2Trace43, and we found it is trivial (Supplementary Fig. 8). The transition from $$P\bar 3m1$$ to Pc phase not only induces the sharp change of resistivity, but also accompanies the topological phase transition originating from the loss of mirror protection present in $$P\bar 3m1$$.

### Phase diagram of Pt2HgSe3

The pressure dependence of the resistivity at 1.8 K and the critical temperature of superconductivity for Pt2HgSe3 are summarized in Fig. 6. It is seen that the high pressure dramatically alters the electronic properties in Pt2HgSe3. The resistivity first decreases with pressure to a minimum at approximate 11 GPa and then displays the opposite trend with further increasing pressure. The non-monotonic evolution of ρ(T) is also observed in other topological materials32,44. Here, we emphasize that this peculiar behavior of the resistivity in Pt2HgSe3 is not associated with a structure phase transition based on in situ high-pressure XRD measurements. However, the lattice parameter ratio c/a presents a discontinuous trend at the same pressure, indicating the anisotropy is changed at this pressure (Fig. 3b). Our calculations clearly indicate that the topological phase transition cannot explain the resistivity change in Pt2HgSe3, as the topological nature persists to the structure change at 60 GPa. The evolution of the surface states is strongly affected by the surface potential, which does not monotonically change with the increase of pressure. Thus, we do not observe a uniform behavior of the surface state under the evolution of the pressure. The surface states only take a smaller weight as compared to the bulk states in contribution to the fermi surface. The overall fermi surface enlarges with the increase of pressure. This is consistent with the resistivity measurement at pressures smaller than 11 GPa. However, when pressure is greater than 11 GPa, the experimental resistivity displays an upper turn and further increases with pressure, which cannot be solely explained by the electronic structure predicted by DFT. Meanwhile, the Raman spectra show that $$A_{1g}^2$$ mode becomes weak and $$E_g^4$$ mode emerged at around 10.7 GPa (Fig. 3d). Thus, other mechanisms, including the intrinsic electron–phonon interactions, extrinsic defects/vacancies reaction to external pressure, are more likely responsible for this interesting behavior.

At a further increase of pressure, structural phase transition observed accompanying 3.4% volume drops at a critical pressure of 54.2 GPa (as shown on the upper panel of Fig. 6). It is clearly seen that the superconducting state emerges beyond the phase transition, and then the superconducting transition temperature increases further with applied pressure. The Tc of Pt2HgSe3 rises to 4.4 K at the pressure of 88 GPa and still does not exhibit the trend of saturation. As superconductivity occurs among electronic states at EF, we investigated the DOS at EF in the Phase II for various pressures (Supplementary Figs. 9 and 10). The upper panel of Fig. 6 also shows DOS at the EF from 65 to 90 GPa. It is clear that the DOS increase monotonically with increasing pressure. In addition, the electron–phonon coupling (EPC) parameter (λ) and Tc is estimated from the Allen–Dynes formula of the Pc phase under high pressures. λ increases from 0.41 to 0.43 and theoretical Tc increases from 0.68 to 0.87 K monotonically when pressure increase from 45 to 90 GPa, which agree with our experimental results (Supplementary Fig. 9 and Supplementary Tables 1 and 2).

In conclusion, the evolution of the electrical transport properties in QSHI Pt2HgSe3 is investigated under high pressure. A non-monotonic evolution of ρ(T) is observed under high pressure. The nontrivial topology in Pt2HgSe3 is robust and persists to around 55 GPa according to the calculations. The appearance of superconductivity is accompanied by a structural phase transition. Considering effectively tunable of electronic properties and crystal structure in this QSHI, Pt2HgSe3 offers a platform for exploring exotic physics upon compression.

## Methods

### Crystal growth

The high-quality Pt2HgSe3 sample used in this work was synthesized from three individual elements by high-temperature solid state reactions23.

### Experimental details of high-pressure measurements

In situ high-pressure resistivity measurements were performed in a nonmagnetic DAC. A cubic BN/epoxy mixture layer was inserted between BeCu gaskets and electrical leads. Four Pt foils were arranged in a van der Pauw four-probe configuration to contact the sample in the chamber for resistivity measurements. Pressure was determined by the ruby luminescence method45. An in situ high-pressure Raman spectroscopy investigation of Pt2HgSe3 was performed using a Raman spectrometer (Renishaw inVia, UK) with a laser excitation wavelength of 532 nm and low-wavenumber filter. A symmetric DAC with anvil culet sizes of 250 μm was used, with silicon oil as pressure transmitting medium (PTM). In situ high-pressure XRD measurements were performed at beamline BL15U of Shanghai Synchrotron Radiation Facility (x-ray wavelength λ = 0.6199 Å). Symmetric DACs with anvil culet sizes of 250 μm and Re gaskets were used. Silicon oil was used as the PTM and pressure was determined by the ruby luminescence method45. The two-dimensional diffraction images were analyzed using the FIT2D software46. Rietveld refinements on crystal structures under high pressure were performed using the GSAS and the graphical user interface EXPGUI36,47.

### DFT calculations

Structure prediction was performed through a swam-intelligence-based CALYPSO method and its same-name code38,39,48. Density functional total energy calculations and structure relaxation were performed using the VASP plane-wave code49,50. We have adopted the Perdew–Burke–Ernzerhof (PBE) generalized gradient approximation density functional51 and frozen-core all-electron projector-augmented wave potentials52 in our calculations. The electronic wave functions are expanded in a plane-wave basis set with a kinetic energy cutoff of 350 eV. The Brillouin zone sampling is performed on k-meshes with a reciprocal space resolution of 2π × 0.03 Å−1 to ensure that energies are converged to several meV/atom. The bulk and surface electronic structures under the evolution of pressures were calculated with VASP, while EPC parameter λ was calculated within the framework of linear response theory through the EPC module of the QUANTUM ESPRESSO code53. We have adopted the PBE generalized gradient approximation density functional and ultrasoft pseudopotential in our calculations. The kinetic energy cutoff with 70 Ry, a 3 × 3 × 3 k-mesh, and a 3 × 3 × 3 q-mesh are used for the EPC calculations.