Article | Open

# Spin excitations and the Fermi surface of superconducting FeS

• npj Quantum Materialsvolume 2, Article number: 14 (2017)
• doi:10.1038/s41535-017-0019-6
Revised:
Accepted:
Published online:

## Abstract

High-temperature superconductivity occurs near antiferromagnetic instabilities and the nematic state. Debate remains on the origin of nematic order in FeSe and its relation with superconductivity. Here, we use transport, neutron scattering and Fermi surface measurements to demonstrate that hydrothermo grown superconducting FeS, an isostructure of FeSe, is a tetragonal paramagnet without nematic order and with a quasiparticle mass significantly reduced from that of FeSe. Only stripe-type spin excitations are observed up to 100 meV. No direct coupling between spin excitations and superconductivity in FeS is found, suggesting that FeS is less correlated and the nematic order in FeSe is due to competing checkerboard and stripe spin fluctuations.

## Introduction

High-transition temperature superconductivity in copper oxides and iron-based materials occurs near checkerboard and stripe antiferromagnetic (AF) instabilities, respectively.1,2,3 Although there is also ample evidence for the existence of a nematic order, where a translationally invariant metallic phase spontaneously breaks rotational symmetry,4,5,6,7,8 and for a nematic quantum critical point near optimal superconductivity in iron-based superconductors,9, 10 much remains unclear concerning its microscopic origin and its relationship to superconductivity.2, 3 In particular, recent debates focus on whether nematic order in superconducting FeSe below the tetragonal-to-orthorhombic transition temperature Ts = 91 K without static AF order11,12,13 is due to competing magnetic instabilities or orbital ordering.14,15,16,17,18,19,20,21,22 Here, we use transport, neutron scattering and Fermi surface measurements to demonstrate that superconducting FeS, an isostructure of FeSe,23, 24 is a tetragonal paramagnet without nematic order and with a quasiparticle mass significantly reduced from that of FeSe. Our neutron scattering experiments in the energy regime below 100 meV reveal only stripe-type spin fluctuations in FeS that are not directly coupled to superconductivity. These properties suggest that FeS is a weakly correlated analog of FeSe and, moreover, that the nematic order in FeSe is due to the frustrated magnetic interactions underlying the competing checkerboard and stripe spin fluctuations.16,17,18

A key to understanding the physics of the iron-based superconductors is to determine the role played by magnetism and by electronic nematicity to superconductivity.1,2,3, 5,6,7 In a typical AF ordered iron-pnictide, a tetragonal-to-orthorhombic lattice distortion Ts occurs at temperatures above or at the AF ordering temperature TN,2 and the nematic phase is observed in the paramagnetic orthorhombic phase between Ts and TN.5,6,7 Although iron chalcogenide FeSe single crystals [Fig. 1a, b] also undergo a nematic transition at Ts and become superconducting at T c  = 9.3 K,11 the low-temperature static AF ordered phase is absent.12, 13 This has fueled debates concerning the role of AF order and spin fluctuations to the nematic phase and superconductivity.12,13,14,15,16,17,18,19,20,21,22 Initially, nuclear magnetic resonance experiments on FeSe suggested that magnetism plays no role in its nematic transition.12, 13 However, subsequent neutron scattering measurements reveal strong low-energy spin fluctuations at the stripe AF ordering wave vector and a resonance coupled to superconductivity,14, 15 similar to spin fluctuations in the iron pnictides.2 In addition, recent spin excitation measurements suggest that the nematic transition in FeSe is due to a competition between the checkerboard and the stripe spin fluctuations at AF wave vectors (1, 1) and (1, 0), respectively [Fig. 1c, d],16 consistent with the frustrating magnetic interactions.17, 18 In this picture, one would expect that S-substituted FeSe1−x S x , which reduces Ts and lattice orthorhombicity,25, 26 should have reduced spin fluctuations associated with the checkerboard order. As FeS single crystals are isostructural to FeSe but with a reduced Tc = 4.3 K, it should allow a direct comparison with FeSe,14,15,16 and thus elucidate the role of spin fluctuations to the nematic phase and to superconductivity.

## Results

Here, we use transport (Fig. 1), neutron scattering (Figs. 2 and 3), quantum oscillation experiments (Fig. 4), as well as density function theory (DFT)27 and DFT combined with dynamical mean field theory (DMFT) calculations28, 29 to study single crystals of FeS (see supplementary information for additional data and analysis). To search for the presence of a nematic phase in FeS, we performed elastoresistance measurements on single crystals of FeS and BaFe1.97Ni0.03As230 using a piezo electric device (Fig. 1g) (see ref. 10).

Figure 1h compares the strain dependence of the elastoresistance at different temperatures for FeS, FeSe, and BaFe1.97Ni0.03As2, respectively. While there is a clear resistivity anisotropy for FeSe and BaFe1.97Ni0.03As2, indicative of a nematic phase, FeS reveals no anisotropy in measurements of the elastoresistance from 5 to 105 K. We therefore conclude that FeS has no nematic order, which is consistent with the previous reports on FeS23, 24 and with the notion that the nematic phase vanishes for FeSe1−x S x for x ≥ 0.17.31,32,33 The results from the transport measurements are complemented by those from elastic neutron scattering measurements, which reveal that the system is paramagnetic at all temperatures, (see supplementary information for additional data and analysis) suggesting that the previous observation of magnetic order in FeS is likely due to impurity phases.34, 35

Having established the absence of any nematic order in FeS, we turn to probing the spin excitation spectrum by inelastic neutron scattering experiments. Figure 2 summarizes our neutron time-of-flight measurements on FeS to determine the overall wave vector and energy dependence of the spin fluctuations.30 For these measurements, we use the orthorhombic unit cell notation, and define momentum transfer Q in the three-dimensional (3D) reciprocal space in Å−1 as Q = Ha* + Kb* + Lc*, where H, K, and L are Miller indices and $a * = a ^ 2π∕a$, $b * = b ^ 2π∕b$, $c * = c ^ 2π∕c$ [Fig. 1c, d]. Our single crystals are aligned with the c-axis along the incident beam and with the a-axis in the horizontal plane. In this geometry, we expect that the checkerboard and stripe AF correlations occur at (±1, ±1) and (±1, 0) in-plane wave vectors, respectively. Figure 2a–d shows the spin excitations of FeS at energy transfers of E = 20 ± 4, 40 ± 5, 50 ± 7, and 59 ± 7 meV, respectively. In all cases, we see transversely elongated spin excitations centered around the stripe wave vector (1, 0) with no obvious magnetic signal at the checkerboard wave vector (1, 1). Since magnetic scattering is normalized to absolute units using a vanadium standard,2 we can quantitatively compare the results with those of FeSe.15, 16 Figure 2e–h shows the transverse cuts for FeS (solid circles) and FeSe (solid lines) corresponding to energies in Fig. 2a–d along the [1, k] direction [see red dashed lines in Fig. 2a for scan direction]. Integrating the scattering over the same energy interval, we see that the FeS scattering is much weaker, and we do not observe magnetic scattering associated with the checkerboard correlations for energies below 100 meV, in contrast with the clear magnetic scattering of FeSe at (1, 1) as marked by the vertical arrows in Fig. 2e–h. Figure 2i compares the energy dependence of the local dynamic susceptibility χ′′(E), defined as the dynamic susceptibility integrated over the dashed white box in Fig. 2a (see ref. 2) for both FeS and FeSe.16 Within the energy region probed, χ′′(E) increases with increasing energy but has about a quarter of the intensity of FeSe (Fig. 2i).

To determine if spin excitations in FeS couple to superconductivity, we carried out temperature dependence measurements of the low-energy spin fluctuations near the stripe ordering wave vector (1, 0). For this purpose, single crystals of FeS were aligned in the [H, 0, L] scattering plane, and maps of the scattering intensity at different energies above and below Tc were measured using a cold neutron spectrometer. Figure 3a–d shows background subtracted scattering maps at E = 0.75, 2, 4 and 6 meV, respectively, well below Tc at T = 1.5 K. In all cases, we see rod-like scattering centered at (1, 0, L) with extended scattering along the L direction, consistent with short-range c-axis spin correlations. In the case of FeSe, a neutron spin resonance coupled to superconductivity was found near Er = 4 meV, which corresponds to approximately 5.3kBTc where kB is the Boltzmann constant, at (1, 0).14, 15 Since the Tc of FeS is about half of that of FeSe, the resonance in FeS should be present around Er ≈ 2 meV. To accurately determine the temperature dependence of the dynamic susceptibility near (1, 0), we integrate the scattering around (1, 0) along the L direction, and then fit the profile to a Gaussian on a linear background [see inset in Fig. 3e]. After correcting the Bose factor, we show in Fig. 3e the temperature dependence of the dynamic susceptibility χ′′(E) near the wave vector (1, 0). The energy dependence of χ′′(E) is weakly temperature-dependent below about 10 meV ,and shows no evidence for a neutron spin resonance expected around E r  ≈ 2 meV.

The contrast in the spin dynamics between FeS and FeSe is striking and provides the clue to the physics of both systems. We start from the observation that, as in the case of P-for-As substitution,9 the reduction of Fe-pnictogen distance on moving from FeSe to FeS facilitates electron hopping, and thus reduces the electron correlations [Fig. 1b], as seen in spin excitations of BaFe2(As0.7P0.3)2.36 The notion that FeS is a less correlated analog of FeSe, is qualitatively consistent with our conclusion that the spin spectral weight at low energies is much reduced in FeS, compared to FeSe (Fig. 2i).

The stoichiometric nature of FeS facilitates both quantum oscillation measurements and electronic structure calculations, thereby providing the opportunity to address the correlation physics in a more quantitative way. We therefore turn to the understanding of both the Fermi surface and the effective quasiparticle mass. Figure 1e shows the calculated Fermi surfaces of FeS using combined DFT and DMFT (see supplementary information for additional data and analysis). Comparing with schematics of the measured Fermi surfaces of FeSe in Fig. 1f (see ref. 25) substituting S for Se in FeSe induces the d xy orbital hole pocket near (1, 1), and changes the properties of the hole pockets near the Γ point (0, 0) [Fig. 1e]. To quantitatively determine the differences in the Fermi surfaces of FeS and FeSe, we performed torque magnetometry and resistivity measurements under high magnetic fields. Figure 4 summarizes the quantum oscillatory phenomena observed on FeS investigated through torque magnetometry and resistivity measurements under fields as high as μ0H = 35T in resistive Bitter magnets equipped with either a 3He refrigerator or 4He cryostat. Resistivity measurements were performed on a sample characterized by a residual resistivity ratio (RRR = R300K/R6K) of 41, using a standard four wire technique, while torque was measured through a cantilever beam set-up whose deflection was determined capacitively (see supplementary information for additional data and analysis). We were able to observe well pronounced Shubnikov-de Haas (SdH) and de Haas-van Alphen (dHvA) oscillations in the resistance and in torque measurements, respectively. Typical dHvA and SdH oscillations, and their respective fast fourier transformations (FFTs) for $H∥c-$axis are shown in Fig. 4a, b, respectively. Although their amplitudes differ, most of the SdH frequencies observed below $1kT$, which are indicated by the peaks labeled as α, β, κ, δ and $ϵ$, are reproduced in the dHvA spectrum. Only $ν$ and γ are not visible in the dHvA data. Furthermore, the prominent dHvA peaks at F = 370T and 400T seem to be suppressed in the SdH data, which is attributable to the lower temperature for the torque measurements. Here, it is important to emphasize that the SdH-effect is superimposed onto an electrical transport quantity (resistivity), which is driven by scattering processes, while the dHvA one is superimposed onto a thermodynamic variable (magnetic susceptibility), which in a metal is dependent upon the density of states at the Fermi level. Therefore, it is not surprising that the relative amplitude between peaks observed in the FFT spectra is technique-dependent. In addition, different crystals from a given synthesis batch are likely to display variations in mobility. This should affect the detection of some of the orbits, and hence also produce comparative differences in the FFT spectra collected from the different crystals, as seen in our experiments.

The effective mass μ of the different orbits can be extracted from the temperature dependence of the FFT amplitude as depicted in Fig. 4c. The decrease of the FFT amplitude with increasing temperature is described by the Lifshitz–Kosevich damping factor $R T =πλ∕sinh ( π λ )$. Considering only the first harmonic, one gets λ = 2πkBT/βH, where β 1/μ. This analysis yields effective masses of 1.1(1)m0 for the α, β and κ orbits as well as 1.7(1), 1.8(2), 1.9(2) and 1.8(2)m0 for the δ, $ϵ$, ν and γ orbits. Thus charge carriers in FeS have lower effective masses than those of FeSe whose masses range from 1.9 to 7.2m0.37 Notice that we obtain somewhat heavier masses for the $α$ and β orbits than the values reported in ref. 38. We re-analyzed our data by, for instance, extracting the effective masses from different field windows. However, we found that this does not explain the difference between the effective mass values extracted from both studies.

This is consistent with our DFT + DMFT calculations with mass enhancement m*/mband of 1.9/1.6 for t2g /eg orbitals in FeS, which is much smaller than that in FeSe.28 The whole angular dependence of the SdH and dHvA frequencies as a function of θ is shown in Fig. 4d, where θ denotes the angle between H and the crystallographic c-axis. Based on the dHvA measurements, we observe a multitude of frequencies especially in the region between 0.3 and 0.6kT, as well as at least three additional Fermi surface pockets with F ≥ 1kT. While tracking the individual frequencies that belong to certain Fermi surface sheets is a difficult task in the dHvA data, the picture seems to become clearer for the SdH oscillations. Nevertheless, we were not able to observe SdH oscillations for θ > 30°. The lines depicted in Fig. 4d are intended to provide a hint on the evolution of the frequencies as a function of θ. However, a precise comparison with band structure calculations is required to associate the observed frequencies with specific Fermi surface sheets (see supplementary information for additional data and analysis). Band structure calculations find that the Fermi surface consists of two-dimensional (2D) cylindrical Fermi surface sheets at the center and at the corners of the Brillouin zone, respectively.27 2D orbits would lead to a $F∝ ( cos ( θ ) ) - 1$ dependence, which are not clearly observed here. Although the angular dependence of some of the frequencies (e. g., α and β) could match a cylindrical Fermi surface, the bulk of the observed frequencies are clearly 3D in character and cannot be described by the currently available band structure calculations. A recent report on the SdH on FeS crystals detected only the two main peaks observed in our FFT spectra, probably because the measurements were performed at much lower fields.38 However, the authors conclude that the Fermi surface of FeS has a 2D character in contrast to our observations. Nevertheless, in their study, the SdH oscillations were observed in a quite narrow angular range, i.e. Δθ~ ± 30° with respect to the c-axis, which is not a wide enough range to reach a definitive conclusion on the dimensionality of its Fermi surface. On the other hand, the observation of two of the same frequencies, or cross sectional areas, in samples grown by different groups further confirms that we are detecting the intrinsic Fermi surface of FeS.

## Discussion

In an attempt to further understand the observed quantum oscillations in Fig. 4a–d, we carried out first-principles DFT plus single-site DMFT calculations in the paramagnetic phase of FeS, using the experimentally determined FeS crystal structure (see supplementary information for additional data and analysis) and Hubbard U = 5.0 eV and Hund's J = 0.8 eV. When computing the 3D Fermi surface and the dHvA frequencies, we further incorporated the corrections from the long-range exchange interaction by shifting the hole (electron) Fermi surface down (up) by 50 meV. The calculated 3D Fermi surfaces are shown in Fig. 4e. In particular, the middle hole Fermi surface and the two electron Fermi surfaces are quite 3D-like, with large variation of the pocket size (cross section along the [0, 0, 1] direction) along the kz direction. As shown in Fig. 4f, the DFT + DMFT-calculated dHvA frequencies agree well with experimental values. We further assign each dHvA frequency to its corresponding position on the 3D Fermi surface (see supplementary information for additional data and analysis).

The reduced strength of the electrons correlations in FeS compared to FeSe also provides the understanding of the contrast in the spin dynamics of FeS to those of FeSe. Figure 4g, h shows the energy dependence of the ground state magnetic scattering S(Q, E) for FeS and FeSe, respectively, calculated through a combination of DFT and DMFT methods.28, 29 The main conclusion from these calculations is that the spin excitations are much more energetic for FeS than for FeSe, with the strongest scattering for FeSe occurring below 170 meV, while for FeS, they extend to energies well beyond 400 meV similar to the case of iron phosphites.29

It is also instructive to compare the spin dynamics of the superconducting state in FeS with the results on FeSe and iron pnictide superconductors. For most iron-based superconductors, the appearance of superconductivity is coupled with changes in the spin excitations with the opening of a spin gap, and inducing a neutron spin resonance near the stripe AF wave vector.2 The presence of a resonance has mostly been interpreted as due to quasiparticle excitations between the hole-Fermi surfaces near the Γ-point and the electron Fermi surfaces near (1, 0) as a consequence of Fermi surface nesting.2 Given the hole and electron Fermi surfaces in FeS [Fig. 1e] and FeSe [Fig. 1f], one would expect the presence of spin fluctuations in both materials at the commensurate stripe AF wave vector (1, 0). Our finding that FeS is a weakly correlated analog of FeSe, provides a natural understanding of the lack of a neutron resonance. More quantitatively, from magnetic and transport measurements, it was argued that FeSe is deep inside Bardeen–Cooper–Schrieffer (BCS) and Bose–Einstein-condensate (BCS-BES) cross-over regime, where the ratio of superconducting gap Δ to Fermi energy εF is of the order of unity.39, 40 From the experimentally obtained values for the SdH frequencies F and the effective masses μ in FeS, we can estimate the Fermi energy $ϵ F$ by using: $ϵ F = ℏ 2 k F 2 ∕2μ$, $A=π k F 2$ and F = ħA/2πe. Assuming that the superconducting gap Δ can be estimated by using the BCS formula for a weakly coupled superconductor for FeS: Δ(T→0) = 1.764kBTc = 0.65 meV with Tc = 4.3 K, we can calculate the ratio of superconducting gap to Fermi energy as shown in the Table 1. It clearly shows that the electron pairing in FeS is much closer to a BCS superconductor, again in line with our finding of a correlation strength in FeS that is considerably reduced than that of FeSe.

To summarize, our inelastic neutron scattering experiments below 100 meV indicate that the spin excitations in FeS occur at the stripe AF wave vector (1, 0) with no observable signal at the checkerboard ordering wave vector (1, 1), and are much weaker than those of FeSe (Fig. 2). The weaker correlations in FeS is confirmed by our observation, via quantum oscillations measurements, of a minute enhancement in the effective mass relative to its non-interacting counterpart, and lead to a reduction of the low-energy spin spectral weight while increasing the energy scale for the (1,1) excitations. The weaker correlations also imply that FeS is much closer to a BCS superconductor, which allow us to understand why the low-energy spin excitations do not directly respond to superconductivity (Fig. 3). These results for the isostructural and stoichiometric FeS highlight the strongly correlated nature of FeSe. Indeed, the electron spectral weight in FeSe mainly resides in the incoherent part, which induces quasi-local moments. The ensuing physics of frustrated magnetism not only yields the nematic order but also is manifested in the co-existing spin excitations at (1, 0) and (1, 1) wave vectors.17, 18 The strong correlations in FeSe also enhance the effective quasiparticle interactions in its superconducting state, giving rise to a resonance spin excitation in FeSe.14, 15 As such, our findings elucidate both the origin of the nematic order and the nature of the superconductivity in FeSe.

## Methods

Magnetic torque and transport measurements on FeS were performed at the National High Magnetic Field Laboratory in Tallahassee, Florida (see supplementary information for additional data and analysis). Our inelastic neutron scattering measurements were carried out at the Fine-Resolution Fermi Chopper Spectrometer (SEQUOIA) at the Spallation Neutron Source, Oak Ridge National Laboratory and at the Multi Axis Crystal Spectrometer (MACS) at NIST Center for Neutron Research (NCNR), National Institute of Standards and Technology. Sample alignment for MACS and initial charactization is done at Spin-Polarized Inelastic Neutron Spectrometer, National Institute of Standards and Technology. We have also performed neutron powder diffraction experiments on the BT-1, NCNR. Single crystals of FeS (6.0 g for SEQUOIA and 6.5 g for MACS) were grown using hydro-thermo method and characterizations of our samples are discussed in the supplementary information for additional data and analysis. Pieces with size larger than 3 × 3 mm2 were used in the neutron scattering experiment. The elasto-resistance measurements were carried out using PPMS with a strain gauge attached on the piezo stack to measure strain at different temperatures. Measurements were performed by changing voltage on piezo stack and results presented here were scaled to actual strain in the sample. To facilitate an easy comparison with the results on FeSe,14,15,16 we used the orthorhombic notation with a = b ≈ 5.19 Å and c = 5.03 Å for FeS. In this notation, the stripe AF spin excitations for FeS occur at (±1, 0, L) positions in reciprocal space. Samples are co-aligned in the [H, 0, L] scattering plane with a mosaic of 8°. In the SEQUOIA experiment, the incident beam with E i  = 80,150 meV was along the c-axis of the crystals. In the MACS experiment, Ef = 5 meV was used for excitations above 1.6 meV and E f  = 3.7 meV was used for excitations below 1.25 meV. Details of DFT + DMFT calculations are described in the supplementary information for additional data and analysis).

## References

1. 1.

Scalapino, D. J. A common thread: the pairing interaction for unconventional superconductors. Rev. Mod. Phys. 84, 1383–1417 (2012).

2. 2.

Dai, P. C. Antiferromagnetic order and spin dynamics in iron based superconductors. Rev. Mod. Phys. 87, 855–896 (2015).

3. 3.

Si, Q., Yu, R. & Abrahams, E. High temperature superconductivity in iron pnictides and chalcogenides. Nat. Rev. Mater. 1, 16017 (2016).

4. 4.

Fradkin, E., Kivelson, S. A., Lawler, M. J., Eisenstein, J. P. & Mackenzie, A. P. Nematic Fermi fluids in condensed matter physics. Annu. Rev. Condens. Matter Phys. 1, 153–178 (2010).

5. 5.

Fisher, I. R., Degiorgi, L. & Shen, Z. X. In-plane electronic anisotropy of underdoped 122 Fe-arsenide superconductors revealed by measurements of detwinned single crystals. Rep. Prog. Phys. 74, 124506 (2011).

6. 6.

Fernandes, R. M., Chubukov, A. V. & Schmalian, J. What drives nematic order in iron-based superconductors? Nat. Phys. 10, 97–104 (2014).

7. 7.

Böhmer, A. E. & Meingast, C. Electronic nematic susceptibility of iron-based superconductors. C. R. Phys. 17, 90–112 (2016).

8. 8.

Lu, X. Y. et al. Nematic spin correlations in the tetragonal state of uniaxial-strained BaFe2x Ni x As2. Science 345, 657–660 (2014).

9. 9.

Dai, J., Si, Q., Zhu, J.-X. & Abrahams, E. Iron pnictides as a new setting for quantum criticality. Proc. Natl. Acad. Sci. USA 106, 4118–4121 (2009).

10. 10.

Kuo, H.-H., Chu, J.-H., Palmstrom, J. C., Kivelson, S. A. & Fisher, I. R. Ubiquitous signatures of nematic quantum criticality in optimally doped Fe-based superconductors. Science 352, 958–962 (2016).

11. 11.

McQueen, T. et al. Tetragonal-to-orthorhombic structural phase transition at 90 K in the superconductor Fe1.01Se. Phys. Rev. Lett. 103, 057002 (2009).

12. 12.

Baek, S. H. et al. Orbital-driven nematicity in FeSe. Nat. Mater. 14, 210–214 (2015).

13. 13.

Böhmer, A. E. et al. Origin of the tetragonal-to-orthorhombic phase transition in FeSe: A combined thermodynamic and NMR study of nematicity. Phys. Rev. Lett. 114, 027001 (2015).

14. 14.

Rahn, M. C., Ewings, R. A., Sedlmaier, S. J., Clarke, S. J. & Boothroyd, A. T. Strong (π,0) Spin fluctuations in βFeSe observed by neutron spectroscopy. Phys. Rev. B 91, 180501(R) (2015).

15. 15.

Wang, Q. et al. Strong interplay between stripe spin fluctuations, nematicity and superconductivity in FeSe. Nat. Mater. 15, 159–163 (2016).

16. 16.

Wang, Q. et al. Magnetic ground state of FeSe. Nat. Commun. 7, 12182 (2016).

17. 17.

Yu, R. & Si, Q. Antiferroquadrupolar and Ising-nematic orders of a frustrated bilinear-biquadratic Heisenberg model and implications for the magnetism of FeSe. Phys. Rev. Lett. 115, 116401 (2015).

18. 18.

Wang, F., Kivelson, S. A. & Lee, D.-H. Nematicity and quantum paramagnetism in FeSe. Nat. Phys 11, 959–963 (2015).

19. 19.

Glasbrenner, J. K. et al. Effect of magnetic frustration on nematicity and superconductivity in iron chalcogenides. Nat. Phys 11, 953–958 (2015).

20. 20.

Cao, H.-Y., Chen, S., Xiang, H. & Gong, X.-G. Antiferromagnetic ground state with pair-checkerboard order in FeSe. Phys. Rev. B 91, 020504 (2015).

21. 21.

Chubukov, A. V., Fernandes, R. M. & Schmalian, J. Origin of nematic order in FeSe. Phys. Rev. B 91, 201105 (2015).

22. 22.

Yamakawa, Y., Onari, S. & Kontani, H. Nematicity and magnetism in FeSe and other families of Fe-based superconductors. Phys. Rev. X 6, 021032 (2016).

23. 23.

Lai, X. et al. Observation of superconductivity in tetragonal FeS. J. Am. Chem. Soc. 137, 10148 (2015).

24. 24.

Borg, Ch. K. H. et al. Strong anisotropy in nearly ideal tetrahedral superconducting FeS single crystals. Phys. Rev. B 93, 094522 (2016).

25. 25.

Watson, M. D. et al. Suppression of orbital ordering by chemical pressure in FeSe1−x S x . Phys. Rev. B 92, 121108 (2015).

26. 26.

Wang, L. R. et al. Superconductivity-enhanced nematicity and ‘s+d′ gap symmetry in Fe(Se1−x S x ). Phys. Status Solidi B 254, 1600153 (2017). doi:10.1002/pssb.201600153.

27. 27.

Subedi, A., Zhang, L., Singh, D. J. & Du, M. H. Density functional study of FeS, FeSe, and FeTe: electronic structure, magnetism, phonons, and superconductivity. Phys. Rev. B 78, 134514 (2008).

28. 28.

Yin, Z. P., Haule, K. & Kotliar, G. Kinetic frustration and the nature of the magnetic and paramagnetic states in iron pnictides and iron chalcogenides. Nat. Mater. 10, 932–935 (2011).

29. 29.

Yin, Z. P., Haule, K. & Kotliar, G. Spin dynamics and orbital-antiphase pairing symmetry in iron-based superconductors. Nat. Phys 10, 845–850 (2014).

30. 30.

Lu, X. Y. et al. Impact of uniaxial pressure on structural and magnetic phase transitions in electron-doped iron pnictides. Phys. Rev. B 93, 134519 (2016).

31. 31.

Hosoi, S. et al. Nematic quantum critical point without magnetism in FeSe1−x S x superconductors.  Proc. Natl. Acad. Sci. USA 113, 8139–8143 (2016).

32. 32.

Watson, M. D. et al. Emergence of the nematic electronic state in FeSe. Phys. Rev. B 91, 155106 (2015).

33. 33.

Tanatar, M. A. et al. Origin of the resistivity anisotropy in the nematic phase of FeSe. Phys. Rev. Lett. 117, 127001 (2016).

34. 34.

Holenstein, S. et al. Coexistence of low-moment magnetism and superconductivity in tetragonal FeS and suppression of T c under pressure. Phys. Rev. B 93, 140506 (2016).

35. 35.

Kirschner, F. K. K. et al. Robustness of superconductivity to competing magnetic phases in tetragonal FeS. Phys. Rev. B 94, 134509 (2016).

36. 36.

Hu, D. et al. Spin excitations in optimally P-doped BaFe2(As0.7P0.3)2 superconductor. Phys. Rev. B 94, 094504 (2016).

37. 37.

Terashima, T. et al. Anomalous Fermi surface in FeSe seen by Shubnikovâ€“de Haas oscillation measurements. Phys. Rev. B 90, 144517 (2014).

38. 38.

Terashima, T. et al. Upper critical field and quantum oscillations in tetragonal superconducting FeS. Phys. Rev. B 94, 100503 (2016).

39. 39.

Kasahara, S. et al. Field-induced superconducting phase of FeSe in the BCS-BEC cross-over. Proc. Natl. Acad. Sci. USA 111, 16309–16313 (2014).

40. 40.

Kasahara, S. et al. Giant superconducting fluctuations in the compensated semimetal FeSe at the BCS-BEC crossover. Nat. Commun. 7, 12843 (2016).

## Acknowledgements

The single crystal growth and neutron scattering work at Rice is supported by the U.S. DOE, BES under contract no. DE-SC0012311 (P.D.). A part of the materials work at Rice is supported by the Robert A. Welch Foundation Grants No. C-1839 (P.D.). The theoretical work at Rice is supported by the NSF Grant No. DMR-1611392 and the Robert A. Welch Foundation Grant No. C-1411 (Q.S.). Z.P.Y acknowledges financial support by the National Natural Science Foundation of China, Grant No. 11674030, the National Key Research and Development Program of China under contract No. 2016YFA0302300. L.B. is supported by DOE-BES through award DE-SC0002613. The NHMFL is supported by NSF through NSF-DMR-1157490 and the State of Florida. The use of ORNL's SNS was sponsored by the Scientific User Facilities Division, Office of BES, US DOE. The work at JHU was supported by the US DOE, BES, Division of Materials Sciences and Engineering under award DE-FG02-08ER46544 (CB).  I.E. was supported by the joint DFG-ANR Project (ER 463/8-1) and DAAD PPP USA N57316180.

## Author information

### Affiliations

1. #### Department of Physics and Astronomy and Center for Quantum Materials, Rice University, Houston, TX, 77005, USA

• Haoran Man
• , Jiangang Guo
• , Rui Zhang
• , Yu Song
• , Weiyi Wang
• , Qimiao Si
•  & Pengcheng Dai
2. #### Department of Physics and National High Magnetic Field Laboratory, Florida State University, Tallahassee, FL, 32310, USA

• Rico Schönemann
•  & Luis Balicas
3. #### Center for Advanced Quantum Studies and Department of Physics, Beijing Normal University, Beijing, 100875, China

• Zhiping Yin
•  & Pengcheng Dai
4. #### Institute for Quantum Matter and Department of Physics and Astronomy, The Johns Hopkins University, Baltimore, MD, 21218, USA

• Mingxuan Fu
•  & Collin Broholm
5. #### Quantum Condensed Matter Division, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA

• Matthew B. Stone
6. #### NIST Center for Neutron Research, National Institute of Standards and Technology, Gaithersburg, MD, 20899, USA

• Qingzhen Huang
• , Leland Harriger
• , Jeffrey W. Lynn
•  & Collin Broholm
7. #### Department of Physics and Astronomy, University of Missouri, Columbia, MO 65211-7010, USA

• David J. Singh
8. #### Institut für Theoretische Physik III, Ruhr-Universität Bochum, Bochum, D-44801, Germany

• Felix Lochner
•  & Ilya Eremin
9. #### Max-Planck Institut für Eisenforschung, Düsseldorf, Germany

• Felix Lochner
•  & Tilmann Hickel

### Contributions

Single crystal growth and neutron scattering experiments were carried out by H.M., J.G., R.Z. with assistance from M.F., M.S., Q.H., Y.S., J.W.L., C.B. and P.D. W.Y.W., and Y.S. performed ICP measurement. Quantum oscillation measurements and analysis were performed by R.U.S. and L.B. Theoretical understandings were performed by Z.P.Y. (DFT + DMFT), D.J.S, T.H., I.E. (DFT), and Q.S. (electron correlations and magnetic frustration). P.D. oversees the entire project. The paper was written by P.D., H.M., Z.P.Y., L.B., I.E., Q.S., and all authors made comments.

### Competing interest

The authors declare that they have no competing interests.

### Corresponding authors

Correspondence to Zhiping Yin or Luis Balicas or Pengcheng Dai.