Emergent charge order near the doping-induced Mott-insulating quantum phase transition in Sr3Ru2O7

Search for novel electronically ordered states of matter emerging near quantum phase transitions is an intriguing frontier of condensed matter physics. In ruthenates, the interplay between Coulomb correlations among the 4d electronic states and their spin-orbit interactions, lead to complex forms of electronic phenomena. Here we investigate the double layered Sr3(Ru1−xMnx)2O7 and its doping-induced quantum phase transition from a metal to an antiferromagnetic Mott insulator. Using spectroscopic imaging with the scanning tunneling microscope, we visualize the evolution of the electronic states in real- and momentum-space. We find a partial-gap at the Fermi energy that develops with doping to form a weak Mott insulating state. Near the quantum phase transition, we discover a spatial electronic reorganization into a commensurate checkerboard charge order. These findings bear a resemblance to the universal charge order in the pseudogap phase of cuprates and demonstrate the ubiquity of charge order that emanates from doped Mott insulators. Quantum phase transitions and emergent electronic ordered states are intriguing phenomena in condensed matter physics. Using a ruthenate material system, the authors employ scanning tunnelling microscopy and spectroscopy to visualise the transition from a metal to a Mott insulator via doping and find evidence for emergent charge order.

T he bilayer ruthenate, Sr 3 Ru 2 O 7 , has a complex quasi-twodimensional electronic structure due to the rotation of the bulk RuO 6 octahedra that leads to the reconstruction of the Fermi surface (Fig. 1a). As a result, multiple electronic bands cross the Fermi level, revealed by angle resolved photoemission spectroscopy (ARPES) 1 and de Haas-van Alphen (dHvA) 2 studies. While naively, the extended 4d Ru-orbitals as compared to 3d orbitals are expected to make it a weakly correlated metal, Sr 3 Ru 2 O 7 is one of the most strongly renormalized heavy delectron material systems, demonstrated by the heavy flat electronic bands in previous ARPES 1 and scanning tunneling microscopy (STM) 3 experiments. Signatures of the inherent electronic correlations are further emphasized by the impact of minute perturbations on its electronic ground state. The most prominent is the magnetic field-tuned quantum critical behavior and emergent electronic nematic order [4][5][6] with spin density wave instability 7 seen in transport and neutron scattering.
Doping acts as another pertinent non-thermal tuning parameter of the electronic states of Sr 3 Ru 2 O 7 . At a few percent of Mn replacing Ru, Sr 3 (Ru 1−x Mn x ) 2 O 7 undergoes a metal to Mottinsulating quantum phase transition (QPT), accompanied at lower temperatures by an E-type antiferromagnetic (AFM) order with a wave vector Q AFM = (π/2a, π/2a) (Fig. 1b) [8][9][10] analogous to the AFM structure of FeTe 11 . Resonant X-ray scattering (REXS) indicates the intensity and correlation length (not exceeding 160 nm) of the AFM order to decrease at lower temperatures (T « T AFM ), perhaps providing indications of a competing order 10,12 . While the origin of the insulating state remains unclear, the Coulomb correlations between the localized 3d Mn orbitals and the relatively extended 4d Ru orbitals may play a crucial role in the metal to insulator transition. X-ray diffraction indicates the RuO 6 octahedral-rotation to decrease with doping and be completely suppressed near x = 0.2 13,14 . On the other hand, X-ray absorption experiments 15 reveal an unexpected 3 + valence of the Mn suggesting that Mn doping does not introduce holes to the Fermi surface, rather tunes the electronic states likely by structural distortions as well as enhanced Coulomb correlations with increased doping. Such a structural distortion clearly impacts the electronic structure and makes it susceptible towards magnetic instabilities 16 . Indeed, previous electron microscopy and spectroscopy experiments have shown, in real space, the extreme sensitivity of the Mott-type metal-insulator transition to applied mechanical stress/strain 17 .
At the border of AFM and Mott insulating QPT, emergent electronic instabilities are frequently discovered. Using spectroscopic imaging with the STM, we investigate the doping-induced metallic to AFM and Mott-insulating QPT in Sr 3 (Ru 1−x Mn x ) 2 O 7 to explore whether broken electronic symmetry states emerge near the QPT. We find a partial gap at low Mn concentration that evolves with doping into an inhomogeneous full gap. Near the Ref. 18

Ru
Sr O  10 . The blue circles represent the doping dependence of the spectroscopic local density of states at E F measured at T = 13 K (left axis) normalized to value at 200 meV. The blue square is extracted from ref. 18 . c Constant current topographic image of a 5% Mn sample taken at −100 mV and −500 pA showing an atomically ordered surface (Scale bar is 30 Å). The inset shows a magnified image of the same area. d Fourier transform (plotted as the modulus) of the topography in c. Both the Bragg peaks at (±2π/a, 0) and (0, ±2π/a) and satellite reflections at (±π/a, ±π/a) are seen. e Topographic image taken on the x = 1% (1 V, 150 pA), x = 5% (1 V, 150 pA), and x = 10% (1 V, 2500 pA) samples respectively (scale bar is 50 Å). At higher bias voltages, the Mn impurity sites appear as bright spots showing the progression of impurity density from 1 to 10% QPT, we find a spatial reorganization of the electronic states into a commensurate checkerboard charge order. These findings provide further evidence for the ubiquity of these electronically ordered states in doped Mott insulators.

Results
Real-space crystal and electronic structure of Sr 3 (Ru 1−x Mn x ) 2 O 7 . Figure 1c shows a topographic STM image of single crystal Sr 3 (Ru 1−x Mn x ) 2 O 7 with x = 5% cleaved in situ in our ultrahighvacuum STM exposing the SrO surface. The Fourier transform (FT) of the topography reveals atomic Bragg peaks at the corners of the map corresponding to (±2π/a, 0) and (0, ±2π/a) (Fig. 1d). Note that, throughout this paper, we use tetragonal notation with lattice constant a~3.9 Å. Additional satellite reflections at (±π/a, ±π/a) correspond to the doubling of the bulk unit cell owing to the rotational distortion of the RuO 6 octahedra (Fig. 1a, d) as previously observed 3,14 . The Mn dopants can be identified by their bias-dependent signatures in the topographic images ( Fig. 1c). They can be best visualized at a sample bias V~1 V as bright spots for different doped samples (Fig. 1e). Counting the number of Mn dopants on the surface reveals a systematic downshift (~2%) with a lower surface doping as compared to the expected bulk (see Supplementary Figures 1-3 and Supplementary Note 1).
The local impact of Mn-dopants on the spatial electronic structure, over a range of doping spanning the QPT, can be visualized through the spectroscopic imaging shown in Fig. 2. At low Mn concentration (x = 1%; much below the QPT) the local electronic density of states (LDOS) reveal a particle-hole asymmetric V-shaped partial-gap of~± 30 meV near the Fermi energy at T = 13 K (Fig. 2a-c). The density of states is similar to what has been previously observed in the undoped parent compound 18 . In particular, a small hump at E F previously identified as a van-Hove singularity 1,18 , is also present in our spectra obtained away from Mn dopants ( Fig. 2c and Supplementary Figure 4 and Supplementary Note 2). The effect of the Mn, at this low concentration, shows modifications of the local gap predominantly near E F (see Fig. 2b, c and see Supplementary Figure 4 and Supplementary Note 2). Conductance maps dI/dV (r, V) ≡ g(r, V) locally at locations r and at a constant sample bias V, displayed in Fig. 2b, reveal that, in this dilute limit the Mndopants spatially modify the LDOS within a length scale of 20 Å.
Increasing the Mn-doping towards the QPT reveals a striking spatial reorganization of the local electronic states. Figure 2d-f displays a topographic image and the LDOS, showing the evolution of the partial-gap in the x = 5% sample. Conductance maps g(r, V) reveal a bi-directional long-range checkerboard charge modulations (Fig. 2e); particularly V = −40 meV and V = −100 meV maps embedded within the long-wavelength inhomogeneity. The absence of this charge ordering in the real space images of the x = 1% sample (Fig. 2b), as well as in earlier STM experiments on the undoped 3,18 and Mn-doped compound at 100 K 14 , indicate its origin not to be related to any surface reconstruction, rather to an electronic instability induced by Mndoping at low temperatures. Further indication of the electronic nature of the ordering phenomenon can be concluded by the absence of any structural peaks, corresponding to the observed modulations, in neutron scattering 8 , in recent low energy electron diffraction (LEED) 14 , and in non-resonant hard X-ray scattering 12  At even higher doping (x = 10%) ( Fig. 2g-i), the STM spectra reveals strong inhomogeneity with areas of V-shaped partial gaps mixed with a complete, yet inhomogeneous, gapping of the Fermi surface into a U-shaped insulating gap of~100 meV ( Fig. 2i and Supplementary Figure 5). This is indicative of a close proximity to the metal-to-Mott insulator transition 19 . The evolution of the gap with doping resembles the opposite trend of what is seen in the hole-doped, spin-orbit-driven, Mott insulator Sr 3 (Ir 1−x Ru x ) 2 O 7 19 . The conductance maps g(r, V) (Fig. 2h) reveal inhomogeneous short-range modulations, equally populated along the two atomic directions with their periodicity locked to 2a. The glassy nature of the observed charge order could be a consequence of the larger electronic inhomogeneity introduced by the Mn-dopants and the transition into the Mott insulating state.
It is important to emphasize that the doping-induced metal to insulator transition evaluated from the surface LDOS at T = 13 K (blue points in Fig. 1b) closely follows that extracted from bulk resistivity measurements (gray area in Fig. 1b) 10 . The slight offset may be a result of the lower doping on the surface as compared to the bulk. This may also explain the slight enhancement of the octahedral distortion on the surface of Sr 3 (Ru 1−x Mn x ) 2 O 7 as compared to the bulk 20 .
Momentum-space electronic structure of Sr 3 To visualize the momentum-space structure of these modulations and their doping dependence, we display in Fig. 3a-f (raw data in Supplementary Figure 6) the real-space conductance maps and selected FTs at several energies taken on a larger surface area for optimal momentum-space resolution. For x = 1%, in addition to the Bragg peaks at (±2π/a, 0) and (0, ±2π/a) and the octahedral rotation satellite peaks at (±π/a, ±π/a), the FTs show dispersive features (q 2 , q 7 , and q 8 ) along the high symmetry directions, corresponding to quasiparticle interference (Fig. 3a, b) (here we follow the same notations as reference 3 ), similar to those previously seen in the STM studies of 1% Ti-doped Sr 3 Ru 2 O 7 3 . The extracted dispersions along (±π/a, ±π/a) are shown in Fig. 3g (more detail in Supplementary Figure 7 and Supplementary Note 3). While no clear peak is observed at the Q* location in this 1% sample, there is a faint and broad intensity in the FT around this wave vector (q 1 in Fig. 3b, whose dispersion is difficult to extract due to its proximity to q 7 and their weak intensity), which locally originates only from areas with Mn dopants (more detail in Supplementary Note 2).
In contrast, for x = 5%, the FTs indicate distinct sharp and non-dispersive peaks at Q* = (±π/a, 0) and (0, ±π/a) that correspond to a real-space modulation with a periodicity of 2a (Fig. 3d, h). In addition to the Q* modulations, we also observe strong non-dispersive modulations coinciding with the AFM ordering wavevector, Q AFM = (±π/2a, ±π/2a) (Fig. 3d, g  These experiments, unsurprisingly, indicate strong interplay between spin and charge instabilities. Close to the QPT (x~5%) the system is susceptible to long-range charge ordering at Q* and Q AFM (Fig. 3) as well as a competing short-range AFM ordering at Q AFM (as revealed by previous REXS 10,12 ). Signs of such a competition can also be seen in the suppressed AFM correlation length with decreasing temperature in REXS 10,12 . This presents an exotic electronic phase in which charge and spin orders simultaneously coincide at the same wavevector. With further doping, however, the charge ordering dominates at Q*, whereas a long-range E-type AFM order (with negligible charge contribution, as seen in our STM data (Fig. 3f)) dominates at Q AFM 8,12 . It is important to note that although STM is primarily sensitive to charge order, some forms of orbital orders can also be detected through the STM tip sensitivity to different orbital wave functions 21,22 and therefore orbital contribution to Q AFM cannot be omitted as a possible scenario.
Origin and structure of the observed charge order. To provide insights into the origins of the emergent broken symmetry states at Q* and Q AFM , we compute the bare susceptibility χ(q, ω) using a 12 orbital tight-binding model 23 , fit to match the Fermi surface measured by ARPES on Sr 3 Ru 2 O 7 (see Methods and Supplementary Figure 11). The calculations of the bare susceptibility, χ(q, 0), whose peaks correspond to the wavevectors with strong Fermi surface nesting, reveal broad features around Q* = (π/a, 0) and Q AFM = (π/2a, π/2a), which are enhanced with increased Mndoping (see Supplementary Figure 11). As previously shown, the effect of Mn-dopants on the Sr 3 Ru 2 O 7 band structure can be partly accounted for by the suppression of the RuO 6 octahedral rotation in the tight-binding band structure 24 . As doping increases, the suppressed rotational-distortion (modeled as suppression in the hopping parameter 24 ) reduces the coupling between the d xz and d yz orbitals making the α 2 band more square-like. This itself enhances the nesting at Q * and Q AFM , leading to an increase in χ (q, 0) (see Supplementary Figure 11). Our theoretical analysis therefore indicates that a broad Fermi surface nesting does exist near Q* and Q AFM near x~5%. However, we would like to emphasize that nesting alone cannot explain the emergent charge order at Q* and it will require enhanced anisotropic electron-phonon (e-p) coupling and/or strong Coulomb correlations to induce a commensurate charge ordering instability.
Recently, strong e-p coupling have been observed at the charge ordering wavevectors in YBa 2 Cu 3 O 6.6 25 , Bi 2 Sr 2 CaCu 2 O 8+x 26 , and NbSe 2 27 . In addition, a significant softening in the temperature dependence of the B 1g phonon mode has also been observed to coincide with the metal to Mott insulator transition in Ca 3 Ru 2 O 7 pointing towards the possibility of similar physics in Mn-doped Sr 3 Ru 2 O 7 . Inelastic resonant X-ray or neutron scattering 28,29 can probe the momentum dependence of the e-p coupling and such future experiments may offer a more complete picture of the origin of the observed instabilities in Sr 3 (Ru 1−x Mn x ) 2 O 7 . Finally, we examine the structure and symmetry of the charge ordering at Q*. Figure 4a shows a zoom-in of the conductance map for the x = 10% sample (white box in Fig. 3e). While the charge order can be clearly observed, the long-wavelength inhomogeneity in the conductance map obscures the detail of the ordering patterns. To visualize the real space character of these patterns, we Gaussian-filter the Q* peaks in the raw unsymmetrized FTs (see Supplementary Figure 12) and inverse Fourier transform back into real space. Figure 4b shows the charge order map corresponding to Fig. 4a. The figure displays a mixture of spatially inhomogeneous checkerboard and stripe patterns. To quantitatively distinguish between the two glassy  Fig. 4 Real-space structure of the emergent charge order. a Real space conductance map for x = 10% sample, corresponding to the area in the white box of Fig. 3e. b Inverse Fourier transform (FT) of the FT (preserving the phase) for x = 10% sample after isolating the Q* peaks using a Gaussian profile filter (see Supplementary Note 5). The process enhances the glassy checkerboard charge order seen in a by suppressing the long wavelength inhomogeneity. c, d Spatial amplitude of the charge order along the two orthogonal directions (Y and X), respectively. Higher intensity corresponds to higher strength of the charge order along the specific (Y or X) direction (scale bar is 20 Å). e Cross-correlation of the two orthogonal charge order amplitude maps (see Supplementary Figure 13) as a function of bias. The positive correlation is an indication of a glassy checkerboard order orders, we compute the amplitudes of the two orthogonal components of the charge order (Fig. 4c, d) and calculate their cross-correlation (Fig. 4e) (see Supplementary Figure 13 and Supplementary Note 5). In a checkerboard order, the two orthogonal components are locally correlated (positive crosscorrelation). A stripe order leads to a local anti-correlation between the two orthogonal charge ordering amplitudes. The positive cross-correlation observed as a function of bias (Fig. 4e) indicates globally a four-fold symmetric bi-directional charge order.

Methods
Sample preparation and characterization. The single crystals of Sr 3 (Ru 1−x Mn x ) 2 O 7 used in our experiments were grown at RIKEN laboratory using the floating zone technique. These single crystals were obtained under 1 MPa atmosphere of argon gas containing 10% oxygen. Small flat samples were cut down to~(2 × 2 × 0.5 mm 3 ) and attached to aluminum plates using H20E conducting epoxy. Aluminum cleaving posts were then attached to samples perpendicular to the a-b cleaving plane using H74F nonconducting epoxy. Samples were cleaved at room temperature in ultrahigh vacuum by knocking the post off, and then immediately transferred in situ to the custom RHK Pan STM head, which had been cooled down to T~13 K. STM topographies were taken in constant current mode and dI/dV measurements were performed using a standard lock-in technique with a reference frequency of 0.921 kHz. PtIr tips were used in all experiments. Tips were prepared prior to each experiment on a Cu(111) surface that had been treated with several rounds of sputtering and annealing, and then placed into the microscope head to cool. The sample and Cu were placed next to each other inside the microscope head so as to reduce exposure and preserve tip structure when moving from one to the other. The data presented in this paper were collected from approximately 3 successfully cleaved samples for each doping percentage (1, 5, 7.5, and 10%). There were negligible differences measured between different cleaves and between different areas on a sample.
Crystal structure. All representations of the crystal structure seen here or in the accompanying supplementary material were produced using VESTA 3 39 .
Tight binding calculations. To calculate the bare susceptibility, we start from a 12 orbital tight-binding model that contains t 2g orbitals, the bilayer structure, and the spin-orbit coupling. The hopping parameters are determined by fitting the Fermi surfaces observed in ARPES. Details can be found in ref. 23 . The resulting Hamiltonian in the momentum space can be expressed as where Φ k ¼ d yz k;s;k z ; d xz k;s;k z ; d xy k;Às;k z ; d yz kþQ;s;k z ; d xz kþQ;s;k z ; d xy kþQ;Às;k z are the electron annihilation operators, k is the in-plane momentum, k z is the out-of-plane momentum associated with the bilayer structure, Q = (π, π) is the wavevector associated with the octahedral rotation, and s is the spin index. The matrix element h k (k z ) can be found in ref. 23 . The retarded bare susceptibility in a multiorbital system can in general be written as a tensor of χ s;s′ l 1 ;l 2 ;l 3 ;l 4 q; ω ð Þ¼ 1 N X kαβ W α;β l 1 ;l 2 ;l 3 ;l 4 where W α;β l 1 ;l 2 ;l 3 ;l 4 ¼ ψ αÃ l 1 s ðkÞψ β l 2 s′ ðk þ qÞψ βÃ l 3 s′ ðk þ qÞψ α l 4 s ðkÞ, and ψ α ls ðkÞ is the component of the eigenvector of the eigenstate α projected on the state with orbital l and spin s. This bare susceptibility expressed in the form of Lindhard function with extra matrix elements takes into account the scatterings between different orbitals in a 12 orbital model. To perform the momentum sum over the Lindhard function, we have used a k-mesh of (Nx × Ny = 250 × 250 points) in all the calculations. In the normal state, the system still has the spin rotational symmetry in the presence of the spin-orbit coupling. As a result, χ s;s′ l 1 ;l 2 ;l 3 ;l 4 q; ω ð Þ is the same for any combination of (s, s′), and consequently we can omit (s, s′). Finally, we compute χðq; 0Þ ¼ X l 1 ;l 2 ;l 3 ;l 4 χ l 1 ;l 2 ;l 3 ;l 4 ðq; 0Þ ð3Þ which exhibits peaks at wavevectors corresponding to the Fermi surface nesting. As the doping is introduced, the octahedral rotation is reduced. The leading effect of this octahedral rotation is to enable hopping between different orbitals on nearest neighbor sites. The details of this model are discussed in ref. 23 . The reduced octahedral rotation results in less hybridization between the d xz and d yz orbitals. Note that the current model cannot capture the insulating behavior in the paramagnetic state.
Data availability