Acoustic frequency atomic spin oscillator in the quantum regime

Quantum noise reduction and entanglement-enhanced sensing in the acoustic frequency range is an outstanding challenge relevant for a number of applications including magnetometry and broadband noise reduction in gravitational wave detectors. Here we experimentally demonstrate quantum behavior of a macroscopic atomic spin oscillator in the acoustic frequency range. Quantum back-action of the spin measurement, ponderomotive squeezing of light, and virtual spring softening are observed at oscillation frequencies down to the sub-kHz range. Quantum noise sources characteristic of spin oscillators operating in the near-DC frequency range are identified and means for their mitigation are presented.


I. INTRODUCTION
Quantum mechanics implies that the measurement of a specific observable, e.g., position or a spin projection, is accompanied by the injection of noise in the canonically conjugate variable, e.g., momentum or another spin projection.This noise, resulting from quantum backaction (QBA) [1], together with the imprecision noise (shot noise), determines the precision bounds in quantum metrology tasks.The performance achieved with balanced (and uncorrelated) QBA and imprecision noise sources is referred to as the standard quantum limit (SQL).The microscopic mechanism behind the QBA depends on the physical platform.In the case of interferometric displacement measurements (such as in gravitational wave detectors), it is due to the shot noise of light, and manifests itself as fluctuations in the laser radiationpressure force.In spin-polarized systems the QBA mechanism is attributable to the light shift caused by quantum fluctuations of the Faraday probe polarization [2].Recently, QBA has been observed in various quantum systems [3][4][5][6].
Atomic spin ensembles have become a rich resource for quantum sensing and for engineering macroscopic quantum states with applications in ultra-sensitive magnetometry, search for new physics, and interferometry [7][8][9][10][11][12].A remarkable feature of spin ensembles is the ability to implement an effective negative-mass oscillator, demonstrated in several protocols, such as entanglementassisted magnetometry [13,14] and quantum memory for a set of two-mode-squeezed states [15].A central application of such an oscillator is the broadband QBA evasion in hybrid systems proposed in Refs.[16,17].
To date, quantum sensing beyond the SQL based on atomic spins has been predominantly performed in the MHz frequency range.QBA-free sensing in the acoustic frequency range would enable new sensing applications beyond SQL.It has also become increasingly important in current and future gravitational wave detectors (GWDs) [18] as they approach SQL-limited performance in the acoustic frequency band [19][20][21].As proposed in Refs.[22,23], combining a GWD with a negative-mass spin oscillator with the help of a recently demonstrated two-color source of entangled light [24] allows for cancel- The spin ensemble is probed by linearly polarized off-resonant light with a top-hat spatial mode shape.The probe polarization angle α with respect to the x-axis is adjusted for the QND measurement (Methods, Sec.F) of the collective atomic spin.A quarter-and a half-wave plate define the quadrature phase ϕ detected by the polarization homodyning.[b]: when prepared in highly polarized (coherent spin) state, the atomic ensemble can be described as two-level system, thus exhibiting the behavior of a harmonic oscillator.Specifically, we can prepare the atomic oscillator with the effective negative mass, creating inverted spin population.[c]: The effect of ponderomotive squeezing, originating from cross-correlations between QBAN and SN, can be interpreted as the virtual shift of resonance frequency.lation of both shot noise and QBA noise, enabling broadband sensitivity beyond the SQL.
Here we demonstrate the QBA-limited performance of a spin oscillator in the audio-frequency band.Analogously to optomechanics [25], the spin ensemble can generate ponderomotive squeezing of light, i.e., reduction of noise via correlations between amplitude and phase quadrature fluctuations.We demonstrate ponderomotive squeezing tunable in its frequency down to 700 Hz.The correlations between the light quadratures also lead to another crucial element of low-frequency sensing that we present here: the virtual oscillator-frequency downshift, which is, for example, necessary for matching the spin response to that of a GWD [23] as well as for other sensing applications in the acoustic frequency range [26].Furthermore, we observe and model the residual lowfrequency noise sources limiting the present performance and outline ways to overcome them.

arXiv:2303.11029v3 [quant-ph] 18 Aug 2023
A spin-polarized atomic ensemble precessing at frequency Ω S ∼ |B| in a magnetic field B acts as an oscillator with an effective positive or negative mass depending on the orientation of the collective spin Ĵ with respect to B [17].The ensemble is probed by light (Fig. 1) with the interaction defined by the quantum nondemolition (QND) Hamiltonian Ĥint ∼ a 1 Ŝz Ĵz [27], where a 1 is the vector polarizability and Ŝz is a component of the Stokes vector operator Ŝ [6].The collective spin state is read out by measuring the quadrature of the probe optical field QL (ϕ)= PL cos(ϕ)+ XL sin(ϕ), where ϕ is the homodyne phase and XL ( PL ) are the normalized Stokes operators representing the amplitude (phase) quadrature, respectively.The power spectral density (PSD) S S for the detected optical field normalized to the shot noise is [28,29] The terms in Eq. ( 1) are the contributions from imprecision shot noise (SN), QBA noise (QBAN), crosscorrelations between the QBAN and SN, atomic thermal fluctuations (thermal noise, TN), and broadband spin-response noise.The nominal imprecision noise level is represented by unity, the strength of the QBA noise term , where g cs is the photon-atom coupling rate and d is the optical depth of the spin ensemble [29][30][31].The spectral response of the oscillator is governed by the susceptibility func- , where the spin damping rate γ S = γ S,0 + γ S,pb is decomposed into a probe power-broadening part γ S,pb ∝ Γ S and an intrinsic linewidth γ S,0 .The term containing the correlations between QBAN and SN, S corr = Γ S Re [χ S (Ω)], present at ϕ ̸ = 0, π/2, induces an effective frequency downshift (virtual spring softening) of the spin response to external forces as it appears in the light field [23], whose effect on the observed spectrum is discussed in the Results section.It is analogous to the virtual rigidity effect in quantum optomechanics [18].
The term 1) is the response of the spin oscillator to the stochastic force ζ that has the spectrum S ζ = (n S + 1/2), where n S is the thermal occupancy of the spin oscillator.Finally, the contribution of S bb arises from extraneous, fast-decaying atomic modes coupling to the probe light [32].In the present work, it is minimized by employing a top-hat probe beam with a high cell filling factor (Methods, Sec.B).The measurement precision of the indicated noise contributions except the nominal shot noise can be improved with a better overall detection efficiency η.
A proper choice of ϕ allows for destructive interference between SN and QBAN.As a result, the output light noise drops below the shot noise level in a certain frequency range, provided that the thermal contribution ∝ S TN is sufficiently small.Besides its practical util-ity in various applications, such ponderomotive squeezing [25] allows us to calibrate the QBAN as discussed below.Analogously to the ponderomotive squeezing in optomechanics [33], the maximal degree of squeezing induced by the atomic ensemble in the limit of γ S ≪ Γ S , Ω S is and is achieved in a narrow frequency range around Ω ≈ Ω opt when the optimal phase ϕ opt of the detection quadrature is selected and the broadband noise is ignored.The quantum cooperativity is the integrated area ratio between the QBAN and the thermal noise.

II. RESULTS
The ensemble of N S ≈ 10 10 −10 11 Cesium-133 atoms is contained in an antirelaxation-coated vapor cell (2 × 2 × 80 mm 3 ) heated by a low-noise heater to 40 • C providing a large optical depth and cooperativity [32] (Fig. 1).To minimize the optical losses, both input and output surfaces are anti-reflection coated with an overall transmission of 96%.The PSD of the output probe light (∼ 1 mW) is measured by polarization homodyne detection [15,30] with an overall detection efficiency of η ≈ 92% and more than 14 dB shot noise clearance above the electronic noise for analysis frequencies down to 100 Hz.The homodyne phase ϕ is controlled by wave plates.
The spin oscillator is prepared by optical pumping of the atomic ensemble either to the lowest (|F = 4, m F = −4⟩) or to the highest (|F = 4, m F = 4⟩) Zeeman sublevel with a degree of spin polarization of ≲ 98% (Methods, Sec.C).Low electro-magnetic noise, as required to reach quantum-limited performance, is achieved by a combination of magnetic coils operated with ultra low current noise and magnetic shielding (Methods, Sec.A).The widely tunable resonance frequency Ω S of the spin oscillator is controlled by the magnitude of the applied magnetic field B, scaling as 0.35 MHz/G.The sign of the effective oscillator frequency Ω S , equivalent to the sign of the effective mass, can be set by the direction of B or, alternatively, by the direction of circular polarization of the pump fields.The probe beam is linearly polarized at an angle α relative to the magnetization axis x.The frequency detuning ∆ of the optical field from the transition 6S 1/2 , F = 4 ↔ 6P 3/2 , F ′ = 5 (see Fig. 1) is adjustable and was initially set to 1.6 GHz.
From the analysis of the spin noise spectrum, we extract the parameters of the collective spin oscillator system appearing in Eq. (1); cross-validations of the readout [a]: The homodyne phase is set to ϕ = 0, corresponding to the detection of the phase quadrature of probe light (red curve).The fitting of experimental traces using noise model Eq. ( 1) is described in the text.Reconstructed quantum back-action noise (QBAN) and thermal noise (TN, defined by thermal occupation nS = 3.5) are shown as the light red shaded area and the light blue shaded area, respectively.The ratio between QBAN and TN results in the quantum cooperativity Cq = 3. [b]: The homodyne phase is adjusted to produce maximum ponderomotive squeezing (green curve) SS ≲ −5 dB (also shown in the inset) below the shot noise level (black curve).The yellow curve shows the spin noise at ϕ ≈ −0.25π detection quadrature.Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.[c]: Total force-normalized quantum noise of light (SN and QBAN) exhibiting the virtual tuning of effective resonance frequency ΩS, whose absolute value corresponds to the position of the minimum for each curve.The shift depends on the homodyne detection phase ϕ, see Eq. ( 4), and is accompanied by a decreased effective readout rate ΓS = ΓS cos 2 ϕ.In particular, the choice ϕ ≈ −0.25π provides ∆ΩS,1/(2π) ≈ −2.1 kHz, whereas observation of maximized ponderomotive squeezing (ϕopt ≈ −0.45π) yields the smaller downshift ∆ΩS,2/(2π) ≈ −1.2 kHz.Apart from that, such forcenormalized quantum noise leads to a decrease of the vertical offset (better sensitivity to an external signal) together with an increase of the steepness (reduced quantum-enhanced bandwidth) [see SI for details].
rate Γ S are performed using the coherent induced Faraday rotation technique (CIFAR, see Ref. [34] and Methods, Sec.D).The thermal occupancy is found from the atomic spin polarization using the magneto-optical resonance method (MORS, [35], Sec.C).The reconstructed distribution of Zeeman sublevel populations allows for distinguishing between the positive-and negative-mass configurations (see Methods, Sec.E).
We begin with characterization of the system in the upper part of the acoustic spectral range, setting the Larmor frequency |Ω S |/(2π) = 18 kHz.Importantly, we explore the configuration of an effective negative mass for the spin oscillator.Performing the fits of the spin noise spectra at phase quadrature PL and the quadrature QL (ϕ opt ) yielding the strongest ponderomotive squeezing (Fig. 2[a,b]), we extract the essential parameters of the atomic spin ensemble.The readout rate Γ S /(2π) = 3.8 kHz is in reasonable agreement with the results of the CIFAR calibration, whereas the amount of thermal noise, encoded in the thermal occupation n S = 3.5, is larger than the value n S ≈ 0.6 obtained from MORS.This is likely due to noise sources not accounted for in the model of Eq. (1), for example, the ubiquitous intensity fluctuations of the probe laser, that are absent in MHz frequency range, but grow significantly towards the audioband.Consequently, we estimate the cooperativity C q ≈ 3. QBAN-dominated spin dynamics (Fig. 2[a]) is further confirmed by observation of strong ponderomotive squeezing S SS ≲ −5.0 dB (Fig. 2[b]).This value matches well the retrieved C q linked to the level of quantum noise reduction by means of Eq. ( 2).
Virtual frequency downshift of the observed spin oscillator response.
As noted in the discussion below Eq. ( 1), correlations between the SN and QBAN can alter the spectrum of the light noise in a manner that mimics a probe system with a downshifted resonance frequency.Invoking this technique is of particular interest for sensing in the audio band, as straightforward engineering of a quantum-limited probe system with a low resonance frequency is challenging due to thermal and technical noise sources.The virtual shift is also a crucial element of the broadband quantum-noise reduction scheme for GWD beyond the SQL presented in Ref. [23].The frequency response of the GWD is close to the free-mass susceptibility χ I ∝ −1/Ω 2 .The idea of [23] is to engineer an effective spin oscillator with the same susceptibility, χ S ∝ 1/Ω 2 , as for the GWD (except for an overall sign flip), which can be accomplished by the virtual frequency downshift of the spin oscillator.
To explain how this virtual shift arises, we start by noting that the light spectrum resulting from a measurement of a spin oscillator is modified when ϕ ̸ = 0, π/2 due to the cross-correlations between SN and QBAN, as captured by Eq. ( 1).However, such a squeezing spectrum does not readily reveal the performance of the spin oscillator in the aforementioned applications.Instead, the squeezing spectrum (e.g., Fig. 2[b]) should be rescaled to forcenoise normalization (e.g., Fig. 2[c]), which directly shows the sensitivity of the measurement to forces acting on the spin oscillator.The renormalization is performed according to the Fourier-frequency-dependent transfer function that maps a force acting on the oscillator into the output light (the procedure is detailed in the SI).An elucidating analytical description of the force-normalized spectra is achieved by changing to a new basis of uncorrelated SN and QBAN light quadratures (see SI), yielding the effective susceptibility of the spin oscillator (assuming The virtual spring softening arises from the term ∝ Γ S in Eq. ( 4) and results in the effective oscillator frequency ΩS = Ω S 1 + Γ S sin(2ϕ)/Ω S defining the minimum point in the force-normalized spectrum [23].Whenever −π/2 < ϕ sign(Ω S ) < 0, an effective frequency downshift is implemented.We observe the frequency shift of the initial 1 kHz with its sign depending on the sign of the effective mass of the oscillator.The maximal ∆Ω S is obtained at the homodyne detection phase set to ϕ = ±sign(Ω S )π/4.The size of the shift matches well the extracted experimental parameters of the system, mainly meaning the readout rate Γ S .The ideal regime for application to GWD noise evasion is when Γ S exceeds Ω S , as it opens up the possibility to reduce the effective resonance frequency down to zero, ΩS = 0, which occurs at Γ S sin(2ϕ) = −Ω S .Based on the present demonstration, we can envision a realistic spin oscillator with bare frequency Ω S /(2π) in the kHz range whose susceptibility is modified by the virtual frequency shift so as to match the susceptibility of a free mass, characteristic of the GWDs.
Quantum spin oscillator in the low-frequency acoustic range.Suppression of the near-DC noise.
Having investigated the atomic spin oscillator in the upper audioband, we now target the lower acoustic range down to sub-kHz range.We find that a straightforward reduction of Larmor frequency down towards DCfrequencies by reducing the external magnetic field is accompanied by drastic reduction of ponderomotive squeezing that entirely disappears at |Ω|/(2π) ∼ 10 kHz.If the model Eq. ( 1) is exploited, the compromised performance of the spin oscillator is explained by a boost of thermal occupation n S , consequently affecting S TN and reducing quantum cooperativity C q .Searching for a rationale from physical point of view, we envision incompleteness of the spin noise model Eq. ( 1) due to the deviation of the lightspin interaction from the QND Hamiltonian ∼ a 1 Ŝz Ĵz in the near-DC frequency range.The description of the ground-state multiplet F = 4 of Cesium atoms requires extension beyond the two-level (spin-1/2) model [36] implied by the QND Hamiltonian.Such expansion involves alignment operators ĵ2 x − ĵ2 y , { ĵx , ĵy } ≡ ĵx ĵy + ĵy ĵx that couple to a probe field through the atomic tensor component proportional to the tensor polarizability a 2 [37].Accordingly, the following amendment to the QND interaction Hamiltonian must be included The effect of the first term in the square brackets is centered around the Larmor frequency Ω S and can be adjusted by the input polarization of light as presented in   [38].We observe both the Ω = 2Ω S and Ω = 0 spectral components (Fig. 10[d] in Methods, Sec.F) [39], but mainly focus on the latter, which we will refer to as 'DC noise'.The zero-frequency component amplified by the intensity noise of the probe laser spans up to |Ω|/(2π) ≲ 10 − 20 kHz, as shown in Fig. 3[a].Consequently, the contribution of the DC noise to the noise budget leads to deterioration of the ponderomotive squeezing in the low audio-frequency band (see Methods, Sec.F2).
Crucially, we find that such DC noise can be strongly suppressed by minimizing the alignment term in the Hamiltonian, Eq. (5).In particular, one can increase the optical detuning ∆ and benefit from the fast decline of a 2 [37] which defines the strength of the alignment noise (see Fig. 3[a]).However, it should be taken into account that QBAN and thermal noise also depend on the detuning (Methods, Sec.E).Analyzing each term as a function of ∆ (shown on Fig. 3[b]), we predict the existence of an optimal detuning ∆ opt yielding the best ponderomotive squeezing (see Methods, Sec.F2 for details).We confirm it experimentally for the spin oscillator with the resonance frequency |Ω S |/(2π) = 3 kHz (see Fig. 3[c]).For such oscillator the increase of the detuning from initial ∆ in /(2π) = 1.6 GHz up to ∆ opt /(2π) ∈ 3.0−3.5GHz has The contribution of QBAN to the dynamics of the spin oscillator remains substantial down to the lowest acoustic frequency, although being reduced, as quantified by the extrapolated C q indicated in Fig. 4 (top panel).

III. DISCUSSION
We have experimentally demonstrated a macroscopic quantum spin oscillator in the acoustic frequency range.Quantum-backaction-dominated performance has been achieved for the oscillator with a negative effective mass.We have shown effective spring softening, an effect critical for the implementation of broadband quantum noise reduction in the acoustic and near-DC frequency bands relevant for various applications including gravitational wave detection beyond the SQL.We have identified the deleterious effect of the tensor spin polarizability on the low-frequency spin quantum noise and have found a way to minimize it by an optimal choice of detuning ∆ of the probe light.
The reported results constitute an important milestone towards the implementation of the proposal [22,23] for suppression of the quantum noise in interferometer-type GWDs using a negative-mass atomic oscillator as a ref- erence.Combining the spin oscillator at |Ω S |/(2π) ≲ 2 kHz dominated by QBA with an effective downshift of the Larmor frequency |∆Ω S |/(2π) ≳ 2 kHz demonstrated in the upper audioband, we expect to emulate the motion of a free-mass object, operating the negative-mass spin oscillator with ΩS approaching zero.Fig. 5 illustrates the expected broadband noise reduction in the GWD signal below the SQL obtained by combining the spin ensemble and the entangled light source demonstrated in Ref. [24].The dark red curve presents the case of C q = 40, corresponding approximately to the ratio Γ S /γ S,pb in the present experiment, while assuming the absence of thermal noise n S = 0, suppressed tensor noise, negligible optical losses and the power-broadening-dominant regime (γ S,0 ≪ γ S,pb ).The effect of a moderate thermal noise n S = 2, which reduces C q and adds extra uncorrelated noise, is shown by the light red curve.The orange dashed vertical line indicates the initial resonance frequency of the spin oscillator |Ω S,GWD |/(2π) ≈ 76 Hz which is optimal for the implementation of the virtual frequency shift in the presented frequency range.The reduction of the intrinsic atomic linewidth γ S,0 together with the mitigation of DC noise will make it possible to reach a sensitivity improvement of GWDs comparable to the predicted performance of other quantum-noise-evasion protocols [41].The advantages of our approach in comparison to, e.g., achieving frequency-dependent squeezing by means of a long filter cavity [42,43], include the tun-ability of the quantum noise evasion (via Γ S , Ω S and ϕ) and its small physical footprint.Another possible advantage is the reduced effect of optical losses in the GWDs, which is due to the fact that only one of the two entangled modes propagates in the GWD, whereas the other mode interacts with the relatively low-loss spin ensemble [22,23].
In a broader perspective, the reported results are relevant for quantum sensing of particle mobility [44] or magnetic fields [45] in the acoustic range of sideband frequencies.The squeezed light source in the acoustic frequency range reported here has certain advantages compared to more traditional sources based on nonlinear optics [46].It does not require powerful lasers and nonlinear crystals and is characterized by intrinsic phase stability due to collinear propagation of the coherent carrier and quantum fluctuations.The robust and tunable squeezed light source reported here is relevant for quantum magnetometry [47], especially for biomedical applications where signals in the sub-kHz range often prevail [26].In the field of hybrid optomechanics, coupling of the atomic spin oscillator to a trapped dielectric nanoparticle would allow the optical backaction-evading measurement of mechanical forces in the ∼ 1 − 200 kHz frequency range [48].

A. Atomic vapor cell and PCB coils
The spin ensemble of N S ≈ 10 10 ∼ 10 11 Cesium-133 atoms is contained in an antirelaxation-coated (C30+) rectangular channel (2 × 2 × 80 mm 3 ) as shown in Fig. SI1[a,c] providing a good balance between large quantum cooperativity C q [32] and low-frequency quantum-noise-dominated performance for our experiment.The spin-preserving coating grants a room temperature dark decoherence rate of ∼ 50 Hz during the experiment and the connection to a Cesium atom reservoir allows adjusting the vapor density ρ based on the operational temperature.The vapor cell is placed in magnetic fields provided by PCB coils as in Fig. SI1[a].The inner bias magnetic field B is generated by a PCB coils system in Fig. SI1[b] (combination of a linear-gradient and a parabolic magnetic field) driven by an ultra-low-ACnoise current source which provides the inhomogeneity < 0.1 within the cell volume [49].The setup is positioned in a 5-layer magnetic shield protecting the spins from perturbations from the external DC and RF magnetic fields.The setup with freely adjustable PCBs coils system allows tuning the Larmor frequency from a few Hz up to 1MHz without obviously affecting the intrinsic line-width γ S,0 /(2π).

B. Broadband noise reduction (BNR)
When a linearly polarized probe light interacts with a spin ensemble and records the dynamic of the collective spin system, the measured spin noise spectrum (SNS) in Fig. 6[a] would be affected by various dephasing mechanisms, such as wall collision, the probe beam size, and atomic motion diffusion characteristics [32,[50][51][52].Therefore, the SNS from an atomic vapor cell is a combination of Lorentzians (S total (Ω) → Γ i χ i (γ Si , Ω) with the individual weights (Γ i , γ i ) which correspond to the overlap of the probe beam spatial profile (e.g., a Gaussian mode) with each of the spin diffusion modes.The pronounced narrowband noise spectrum in Fig. 6[c] (orange area) originated from the sum of the slowly decaying modes, and the broadband spin response floor [b] (green area) is due to the modes which decay rapidly due to the motion of atoms in and out of the probe beam during the measurement.With the help of a diffractive beam shaper and a telescope system, we could produce a 1.65 × 1.65 mm 2 square top-hat beam (as shown in Fig. 6[d]) collimated along 8 cm (corresponds to a cell filling factor of 72%), The increased filling factor for the rectangular cell channel helps to reduce the broadband noise down to less than 0.3 in shot noise units [SN] and to improve the relative amplitude ratio between the narrowband and broadband response up to ∼ 600, making the contribution of the S bb term in Eq. ( 1) negligible.

C. Preparation and characterization of atomic state
The Hamiltonian for an ensemble of atomic spins with a collective angular momentum Ĵ = N k=1 ĵk in the ex- , where m F denotes the Zeeman sublevel within the hyperfine manifold F .The ensemble is then polarized along x-axis (see Fig. 1), so that the component Ĵx becomes a macroscopic variable Ĵx → J x = ℏF N/2.Within the Holstein-Primakov approximation, the spin precesses in the yz-plane ∼ Ω S Ĵ2 z + Ĵ2 y at the Larmor frequency Ω S ∼ |B|.The collective spin can be co-oriented (J x > 0) or counter-oriented (J x < 0) with respect to B. This leads to opposite directions of rotation of the Ĵy(z) -components, or equivalently, to the opposite signs of Ω S (see Fig. 9).This situation is commonly referred to a spin oscillator with a negative or positive effective mass [17].
The detailed configuration of atomic levels without Zeeman splitting is depicted in Fig. 7[a] which outlines the pumping scheme.Circularly polarized pump and repump lasers are tuned to the 6S 1/2 , F = 4 ↔ 6P 1/2 , F ′ = 4 and 6S 1/2 , F = 3 ↔ 6P 3/2 , F ′ = 4 transitions respectively as in Fig. 7, which corresponds to in the D1 and D2 lines.The applied method of atomic polarization characterization is based on magneto-optical resonance spectroscopy (MORS, [35]).The spacing between adjacent Zeeman sublevels on the ground hyperfine level follows the equation where Ω QZS ∼ Ω 2 S refers to Quadratic Zeeman splitting effect.Consequently, Zeeman resonances can be resolved provided their small linewidth compared to Ω QZS .This condition turns out to be fulfilled if the bias magnetic field is boosted and the resonance frequency Ω S /(2π) is set to MHz range (|B| ∼ 3 G).The Zeeman transitions are excited by applying AC-magnetic field, the resulting spin response is recorded onto the probing optical field and is then read out by means of balanced polarimetry.The strength of the transitions between Zeeman sublevels depends on their populations.Therefore, the orientation of spin ensemble (also named as spin polarization P) can be characterized using the MORS signal.
We extract the spin polarization P ≈ 98% in Fig. 7[b] (equivalent to the thermal occupation of n S ∼ 0.15) by the pulsed MORS measurement with a (≈ 1mW) probe light in this experiment.The orientation goes down to P = 89% (Fig. 7[c], yielding n S ∼ 0.6) in the regime of continuous probing under the same optical power [32].The repump power P re ≈ 5mW was conditioned upon the maximum available laser power, whereas the pump power P p ≈ 50µW was chosen after the optimization of ponderomotive squeezing at |Ω S |/(2π) ∼ 1 MHz.The power broadening from the pump and repump lasers contribute < 100Hz decoherence to the spin linewidth.From the pulsed MORS we estimate the intrinsic linewidth γ S0 /(2π) ≈ 150 Hz which contains all decay contributions except for the power broadening induced by the probe field.Using Eq. ( 1), we fit the spectra of light probing the spin ensemble at Larmor frequency |Ω S |/(2π) = 18 kHz.The retrieved thermal occupation n S = 3.5 is larger than the result obtained from the calibration by MORS in the continuous regime.The n S extracted from the full spin model might then be treated as an effective thermal occupancy that includes additional noise sources not accounted for in Eq. ( 1), for example, intensity noise of the probe laser.To calibrate the spin measurement rate Γ S and damping rate γ S , we investigate the atomic response to strong modulation of the probe light polarization.The outlined technique is referred to as Coherently induced Faraday rotation (CIFAR) [34].The experimental setup is shown in Fig. 8[a].A weak linearly polarized optical field denoted as 'drive' is phase-modulated at frequency Ω mod using piezo-electric transducer PZT 1 and subsequently overlapped with the orthogonal polarized Local Oscillator (LO) on a polarizing beam-splitter PBS 2 .One of the output modes of PBS 2 thus contains the state Qmod L,in (θ) ∼ ( XL,in sin θ + PL,in cos θ) sin(Ω mod t) with an arbitrary modulated polarization quadrature.The phase angle θ is set by the phase lock loop between LO and drive fields with a feedback signal applied to the piezo element PZT 2 in one of the interferometer arms.The optical field Qmod L,in (θ) probes the atomic oscillator and is then detected with the same balanced polarimetry detection setup.Scanning the modulation frequency Ω mod around Larmor frequency Ω S , one obtains the characteristic shape of the measured spectrum signal S CIFAR (Ω mod ) that provides information about Γ S and γ S .However, the correctness of extracted parameters strongly depends on the precise knowledge of the modulation phase θ.To account for that, we perform the fit of S CIFAR (Ω mod ) at several points of locked θ and obtain results for Γ S and γ S as shown on the Fig. 8[b,c].The uncertainty ∼ 15% on both parameters is mainly attributed to imperfect calibration of θ, which is limited by the software of a FPGA board in the phase lock loop.This circumstance might also address the discrepancy between the values of the readout rate from CIFAR technique and from the fit of the full spin noise model Eq. ( 1).Therefore, we consider the CIFAR calibration as a rough estimation of measurement and damping rates and use them as initial parameters for the full spin noise model.

E. Spin noise spectra with effective masses
The main goal of a measurement reported in this section is to reveal the difference between spin oscillator with an effective negative and positive mass.We operate the atomic ensemble in the Zeeman resolved regime (magnetic field is set to |B| ≈ 3G, giving |Ω S |/(2π) ≈ 1 MHz, as for MORS calibration, Sec.C) and study spin noise spectra (presented on the Fig. 9[c,d]), when the system is driven by quantum noise limited light without applied AC magnetic field.It is then possible to see the consequences of finite spin polarization, and hence the populations of Zeeman sublevels different from |m F = 4⟩ on the ground F = 4. Specifically, we observe several peaks around |Ω|/(2π) ≈ 960 kHz.Using Eq.( 6), we identify two peaks centered at Ω S1a (Ω S1d  In addition, we also observe a third peak in the spectra, centered at frequency ΩS2 that is always higher than ΩS1i, regardless of the sign of the mass of the oscillator at F = 4.This component presumably arises due to inhomogeneity of magnetic field across the atomic cell and represents unresolved Zeeman structure.Insets: the sign of the resonance frequency defines the orientation of rotation in phase space.[e, f ]: Adjustment of homodyne detection phase ϕ = ϕopt allows for the observation of ponderomotive squeezing.Green and red curves correspond to the choice ϕ = −|ϕopt| and ϕ = +|ϕopt| respectively and compared to SN level (gray trace).The negative-mass oscillator displays reduced ponderomotive squeezing, allegedly due to an increased spin damping rate caused by extra magnetic inhomogeneous broadening with a sub-optimal current ratio for the magnetic coils.Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.

F. Spin alignment noise
An atomic spin ensemble driven by Hamiltonian (5) demonstrates the distinctive features of linear birefringence.At the quantum level, the composite dynamics of the spin alignment interaction causes several phenomena, such as a tensor-induced Stark shift of the oscillator's Larmor frequency, cooling or amplification of the spin state, and even spin dynamics beyond the oscillation frequency.In this section, we will give an overview of the influence of each alignment operator on the atomic spin dynamics.
The presence of the Ŝ⊥ Ĵy term added to the Faraday rotation Ŝz Ĵz means that the interaction has deviated from the QND-interaction.It affects the response of the atomic system recorded onto the phase light quadrature PL,out (see Fig. 10, [a]).Such impact might be seen as an effective change of the QND readout rate Γ S and inducing a dynamic contribution to the damping rate γ ′ S /2 ∼ γ S /2 + E S Γ S .Consequently, the maximal level of ponderomotive squeezing is altered (Fig. 10, [b]), when QL (ϕ opt ) is selected.Finally, the amplitude output light quadrature XL,out , being a QND-variable otherwise, is now also disturbed.This is manifested in a characteristic dip/peak as demonstrated in Fig. 10, [c].At the same time, we notice that the strength of the Ŝ⊥ Ĵy term is controlled by the angle α of the probe input polarization.In the present experiment we wish to work at the QND configuration, which is set by rotating a halfwave plate in front of the cell and making the spectrum of XL,out flat around Ω S corresponding to E S ≈ 0 (also depicted in Fig. 10, [c]).We note that the alignment operator studied here is also responsible for the tensor Stark shift effect moving the resonance frequency Ω S (clearly seen in Fig. 10, [a]).It has to be taken into account when estimating the size of the virtual frequency shift by cross correlations between SN and QBAN.

DC alignment noise
It is conceivable that the spin ensemble can sense fluctuations of the probe laser via a mechanism responsible for the tensor interaction (∼ a 2 ) [53].In particular, coupling through the alignment operator ĵ2 x − ĵ2 y explains the abrupt rise of noise centered at zero frequency, being clearly separated from Larmor peak, as shown in Fig. 10[d] (|Ω S |/(2π) = 43 kHz).However, the DC noise component has a tangible overlap with the Larmor peak shifted down to the acoustic range.In this case the QBAN dominated dynamics and the ponderomotive squeezing are compromised.
We study the detrimental influence of DC noise on the ponderomotive squeezing and introduce the term S DC that should be included in the spin model Eq.(1) in addition to defined above contributions.We then explore S DC as a function of the detuning ∆ in a manner it was done for QBAN and thermal noise.Having in mind tensor interaction, we expect S DC ∼ (a 2 /a 1 ) 2 (Γ S ) 2 |χ S,DC (Ω)| 2 , where χ S,DC (Ω) is the susceptibility function that defines the spectral shape of DC noise.We model χ S,DC (Ω) by the Lorenz peak with center frequency located at Ω = 0. Consequently, one may surmise |χ S,DC (Ω)| 2 dΩ ∼ |χ S (Ω)| 2 dΩ if the mechanisms forming decay rate γ S are still valid for S DC .Finally, we obtain the expression Ω S DC dΩ ∼ (a 2 /a 1 ) 2 Ω S QBA dΩ ∼ 1/ ∆ 4 γ S,0 ∆ 2 + C for the integral area of DC noise, using the approximations a 2 ∼ 1/∆ and a 1 ∼ 1.Such dependence on the detuning is validated on the Fig. 3[b] for the experimental data.The next step is to exploit the approximation given by Eq. ( 2) for the optimized ponderomotive squeezing and add S DC .This leads to the formula where and C q ∼ A/ C + γ S,0 ∆ 2 as was deduced in Sec.E.Note that we simplify the expression for DC noise and use S DC = D/∆ r (r ∈ [4,6]) in order to reduce the number of parameters in the model of spin noise budget.The expression Eq. ( 8) states that there exists an optimal point ∆ opt which minimizes S S .The ∆ opt is defined by the actual values of all coefficients in Eq. ( 8) and appears to be ∆ opt /(2π) ∈ 3 − 4 GHz for a spin oscillator in low acoustic range and chosen set of parameters (the example for Ω S /(2π) = 3 kHz is shown in Fig. 3[c]).Exceeding this level brings us to the regime where reduction of the DC-noise term cannot compensate for the decline of S QBA /S T N due to the significance of the intrinsic spin linewidth γ S,in .As a final remark, we note that the amount of DC noise depends on the phase of the detection quadrature.In particular, S DC is maximized in the amplitude Stokes quadrature, thus having a direct impact on the ponderomotive squeezing spectrum.In contrast, the DC noise is not present when the phase Stokes quadrature is observed.Also, it seems to be independent of the input light polarization (angle α).Those effects require further investigation.

Figure 1 .
Figure 1.Schematics of the experimental setup.[a]:The spin ensemble is probed by linearly polarized off-resonant light with a top-hat spatial mode shape.The probe polarization angle α with respect to the x-axis is adjusted for the QND measurement (Methods, Sec.F) of the collective atomic spin.A quarter-and a half-wave plate define the quadrature phase ϕ detected by the polarization homodyning.[b]: when prepared in highly polarized (coherent spin) state, the atomic ensemble can be described as two-level system, thus exhibiting the behavior of a harmonic oscillator.Specifically, we can prepare the atomic oscillator with the effective negative mass, creating inverted spin population.[c]: The effect of ponderomotive squeezing, originating from cross-correlations between QBAN and SN, can be interpreted as the virtual shift of resonance frequency.

Figure 2 .
Figure 2. Spin noise spectra at Larmor frequency |ΩS|/(2π) = 18 kHz.[a]:The homodyne phase is set to ϕ = 0, corresponding to the detection of the phase quadrature of probe light (red curve).The fitting of experimental traces using noise model Eq.(1) is described in the text.Reconstructed quantum back-action noise (QBAN) and thermal noise (TN, defined by thermal occupation nS = 3.5) are shown as the light red shaded area and the light blue shaded area, respectively.The ratio between QBAN and TN results in the quantum cooperativity Cq = 3. [b]: The homodyne phase is adjusted to produce maximum ponderomotive squeezing (green curve) SS ≲ −5 dB (also shown in the inset) below the shot noise level (black curve).The yellow curve shows the spin noise at ϕ ≈ −0.25π detection quadrature.Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.[c]: Total force-normalized quantum noise of light (SN and QBAN) exhibiting the virtual tuning of effective resonance frequency ΩS, whose absolute value corresponds to the position of the minimum for each curve.The shift depends on the homodyne detection phase ϕ, see Eq. (4), and is accompanied by a decreased effective readout rate ΓS = ΓS cos 2 ϕ.In particular, the choice ϕ ≈ −0.25π provides ∆ΩS,1/(2π) ≈ −2.1 kHz, whereas observation of maximized ponderomotive squeezing (ϕopt ≈ −0.45π) yields the smaller downshift ∆ΩS,2/(2π) ≈ −1.2 kHz.Apart from that, such forcenormalized quantum noise leads to a decrease of the vertical offset (better sensitivity to an external signal) together with an increase of the steepness (reduced quantum-enhanced bandwidth) [see SI for details].

Figure 3 .
Figure 3.[a]: Spectra of the light probing the spin ensemble reveal the strong near-DC component (Ω/(2π) ≲ 20 kHz, being clearly separated from the response at ΩS (set to 1 MHz), leading to the reduction of ponderomotive squeezing in the acoustic frequency range.The DC-noise contribution decreases as the optical detuning ∆ is increased.[b]: Comparison of QBAN, thermal noise (TN), and DC-noise areas as a function of ∆. [c]: Influence of probe detuning ∆ on the degree of ponderomotive squeezing measured at |ΩS|/(2π) ≈ 3 kHz, where DC noise has a significant contribution to the noise budget.At the detuning optimal for (∆opt/(2π) ∈ 3.0 − 3.5 GHz the ratio between QBAN and uncorrelated noise sources (including DC noise) is maximized and the best squeezing SS ≈ −3 dB is observed.[d]: Spin noise spectra at different detuning ∆ in the optimal for ponderomotive squeezing detection phase ϕopt.

Fig. 10 [
Fig.10[a,b,c] (Methods, Sec.F1).The second term affects the spin noise at Ω = 0 and Ω = 2Ω S since the matrix element ⟨F, m F,f | ĵ2x − ĵ2 y |F, m F,i ⟩ is non-zero for |m F,f − m F,i | = 0, 2, respectively[38].We observe both the Ω = 2Ω S and Ω = 0 spectral components (Fig.10[d]in Methods, Sec.F)[39], but mainly focus on the latter, which we will refer to as 'DC noise'.The zero-frequency component amplified by the intensity noise of the probe laser spans up to |Ω|/(2π) ≲ 10 − 20 kHz, as shown in Fig.3[a].Consequently, the contribution of the DC noise to the noise budget leads to deterioration of the ponderomotive squeezing in the low audio-frequency band (see Methods, Sec.F2).

Figure 4 .
Figure 4. Observation of the atomic response recorded onto the probe optical field, when the spin oscillator is moved to the lower part of the audioband.The spectra of the phase quadrature (ϕ = 0) are shown on the top panel [a, b, c], whereas the bottom panel [d, e, f ] displays the case of the homodyne phase adjusted to produce the strongest squeezing induced by the atomic ensemble.The level of ponderomotive squeezing is optimized by adjusting the optical detuning for each Larmor frequency, being gradually increased from ∆/(2π) = 3 GHz for |ΩS|/(2π) = 3 kHz up to ∆/(2π) = 4 GHz for |ΩS|/(2π) = 1 kHz.See comments in the text.Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.

Figure 5 .
Figure5.The strain-referenced quantum noise of GWD with characteristic coupling rate ΩqI /(2π) = 100 Hz[23,40] for a standard quantum-noise-limited interferometer (magenta curve) and the configuration with an injected frequencyindependent 10 dB phase-squeezed vacuum state of light (blue dashed curve) is compared with the projected sensitivity of a joint measurement in the reference frame of a negative-mass spin oscillator linked to the GWDs by utilizing an entangled state of light (10 dB two-mode-squeezed vacuum state).Results for two different configurations of parameters of the joint system are presented by the dark red and light curves respectively, both expected to overcome the SQL (dashed black curve).See comments in the text.

Figure 7 .
Figure 7. Level scheme of optical pumping [a].The spin polarization used for the calibration of thermal noise is prepared at 98% [b] in the pulsed MORS measurement and goes down to 89% [c] with a (≈ 1mW) probe light.

Figure 8 .
Figure 8. Implementation of CIFAR technique for calibration of the readout rate ΓS.[a]: the experimental layout.Results of the CIFAR amp [b] and phase signal [c] at different modulation phases θ.

Figure 9 .
Figure 9.The left [a, c, e] and right [b,d, f] columns display the configurations of the spin system with positive and negative mass respectively.[a, b]: Atomic ensemble is described as a harmonic oscillator within the 2-level-system approximation (either of mF ±4 and adjacent Zeeman sublevels hyperfine level F = 4).If a single excitation lowers the energy of the system [b], then the oscillator has an effective negative frequency (mass).[c, d]: the spectra of the optical field after probing the atomic spin oscillator (|ΩS|/(2π) = 0.96 MHz), of which the phase quadrature is detected (ϕ = 0).We distinguish the positive-[c] and negative-mass [d] configurations, comparing the frequency of the strongest transition ΩS1a (Ω S1d ) to the other transitions from the F = 4 multiplet [only Ω S1b (ΩS1c) can be identified].In addition, we also observe a third peak in the spectra, centered at frequency ΩS2 that is always higher than ΩS1i, regardless of the sign of the mass of the oscillator at F = 4.This component presumably arises due to inhomogeneity of magnetic field across the atomic cell and represents unresolved Zeeman structure.Insets: the sign of the resonance frequency defines the orientation of rotation in phase space.[e, f ]: Adjustment of homodyne detection phase ϕ = ϕopt allows for the observation of ponderomotive squeezing.Green and red curves correspond to the choice ϕ = −|ϕopt| and ϕ = +|ϕopt| respectively and compared to SN level (gray trace).The negative-mass oscillator displays reduced ponderomotive squeezing, allegedly due to an increased spin damping rate caused by extra magnetic inhomogeneous broadening with a sub-optimal current ratio for the magnetic coils.Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.

Figure 10 .
Figure 10.Effect of the varying tensor alignment term (ES controlled by the input polarization angle α) on the spin noise spectrum [a] and ponderomotive squeezing [b].The QND interaction is chosen whereby the tensor contribution is minimized in the amplitude quadrature of light [c].The spectrum of SS| QL,out (ϕ opt ) (ϕopt yields the strongest ponderomotive squeezing) when |ΩS|/(2π) is set to 43 kHz [d].Apart from the dispersive signal centered at |ΩS|/(2π), the peak at twice Larmor frequency is visible together with noise enhancing towards |ΩS|/(2π) = 0.The last two effects originate from the tensor interaction term given by Eq. (5).Axes normalized to the shot noise of light [SN], represented in linear or decibel scale.

Figure SI1 .
Figure SI1.Atomic experimental setup [a].The cesium vapor cell is surrounded by a PCBs coil system [b] inside a 5-layer magnetic shield.[c] The different cell cross sections (1 × 1 − 5 × 5 mm 2 ) allow to choose the dark decoherence rate γ0 from 100 to 6 Hz.