Measurement-induced, spatially-extended entanglement in a hot, strongly-interacting atomic system

Quantum technologies use entanglement to outperform classical technologies, and often employ strong cooling and isolation to protect entangled entities from decoherence by random interactions. Here we show that the opposite strategy—promoting random interactions—can help generate and preserve entanglement. We use optical quantum non-demolition measurement to produce entanglement in a hot alkali vapor, in a regime dominated by random spin-exchange collisions. We use Bayesian statistics and spin-squeezing inequalities to show that at least 1.52(4) × 1013 of the 5.32(12) × 1013 participating atoms enter into singlet-type entangled states, which persist for tens of spin-thermalization times and span thousands of times the nearest-neighbor distance. The results show that high temperatures and strong random interactions need not destroy many-body quantum coherence, that collective measurement can produce very complex entangled states, and that the hot, strongly-interacting media now in use for extreme atomic sensing are well suited for sensing beyond the standard quantum limit. It’s still unclear whether entanglement can be generated, survive, and be observed in hot environments dominated by random collisions. Here, the authors use quantum non-demolition measurement on a hot alkali vapor to put more than ten trillion atoms in a long-lived and spatially extended entangled state.

E ntanglement is an essential resource in quantum computation, simulation, and sensing 1 , and is also believed to underlie important many-body phenomena such as high-T c superconductivity 2 . In many quantum technology implementations, strong cooling and precise controls are required to prevent entropy-whether from the environment or from noise in classical parameters-from destroying quantum coherence. Quantum sensing 3 is often pursued using low-entropy methods, for example, with cold atoms in optical lattices 4 . There are, nonetheless, important sensing technologies that operate in a highentropy environment, and indeed that employ thermalization to boost coherence and thus sensor performance. Notably, vaporphase spin-exchange-relaxation-free (SERF) techniques 5 are used for magnetometry 6,7 , rotation sensing 8 , and searches for physics beyond the standard model 9 , and give unprecedented sensitivity 10 . In the SERF regime, strong, frequent, and randomly-timed spin-exchange (SE) collisions dominate the spin dynamics, to produce local spin thermalization. In doing so, these same processes also decouple the spin degrees of freedom from the bath of centre-of-mass degrees of freedom, which increases the spin coherence time 5 . Whether entanglement can be generated, survive, and be observed in such a high entropy environment is a challenging open question 11 .
Here, we study the nature of spin entanglement in this hot, strongly-interacting atomic medium, using techniques of direct relevance to extreme sensing. We apply optical quantum nondemolition (QND) measurement 12,13 -a proven technique for both generation and detection of non-classical states in atomic media-to a SERF-regime vapor. We start with a thermalized spin state to guarantee the zero mean of the total spin variable and use a [1, 1, 1] direction magnetic field (see Fig. 1a) to achieve QND measurements on three components of the total spin variable. We track the evolution of the net spin using the Bayesian method of Kalman filtering 14 , and use spin squeezing inequalities 15,16 to quantify entanglement from the observed statistics. We observe that the QND measurement generates a macroscopic singlet state 17 -a squeezed state containing a macroscopic number of singlet-type entanglement bonds. This shows that QND methods can generate entanglement in hot atomic systems even when the atomic spin dynamics include strong local interactions. The spin squeezing and thus the entanglement persist far longer than the spin-thermalization time of the vapor; any given entanglement bond is passed many times from atom to atom before decohering. We also observe a sensitivity to gradient fields that indicates the typical entanglement bond length is thousands of times the nearest-neighbor distance. This is experimental evidence of long-range singlet-type entanglement bonds. These experimental observations complement recent predictions of coherent inter-species quantum state transfer by spin collision physics 18,19 .

Results
Material system. We work with a vapor of 87 Rb contained in a glass cell with buffer gas to slow diffusion, and housed in magnetic shielding and field coils to control the magnetic environment, see Fig. 1a. The density is maintained at n Rb = 3.6 × 10 14 atoms/cm 3 , and the magnetic field, applied along the [1,1,1] direction, is used to control the Larmor precession frequency ω L /2π. At this density, the spin-exchange collision rate is 325 × 10 3 s −1 . For ω L below about 2π × 5 kHz, the vapor enters the SERF regime, characterized by a large increase in spin coherence time.
Spin thermalization. The spin dynamics of such dense alkali vapors 5 is characterized by a competition of several local spin interactions, diffusion, and interaction with external fields, buffer gases, and wall surfaces. While the full complexity of this scenario has not yet been incorporated in a quantum statistical model, in the SERF regime an important simplification allows us to describe the state dynamics in sufficient detail for entanglement detection, as we now show.
If j (l) and i (l) are the lth atom's electron and nuclear spins, respectively, the spin dynamics, including sudden collisions, can  87 Rb vapor and 100 torr of N 2 buffer gas, which is housed in a low-noise magnetic enclosure. The transmitted light is detected with a shot-noise-limited polarimeter (a Wollaston prism plus differential detector), which indicates F z , the projection of the collective spin F on the probe direction, plus optical shot noise. A static magnetic field along the [1, 1, 1] direction causes the spin components to precess as F z ! F x ! F y every onethird of a Larmor cycle. In this way the polarimeter record contains information about all three components 17 . b Representative sample of the Stokes parameter S out ð Þ y t ð Þ showing raw data (blue dots), optimal estimate for the atomic spin gF z t ð ÞS x (red line), and ±4σ confidence interval (pale red region), as computed by Kalman filter (KF). Signal clearly shows atomic spin coherence over ms time-scales. c Expanded view of early signal, colors as in (b), bars show ±4σ confidence regions of gF z t ð ÞS x for a thermal spin state (TSS) and the standard quantum limit (SQL) for a spin-polarized state. The KF acquires a sub-SQL estimate for F z in 20 μs, far less than the coherence time. Probe power = 2 mW, Larmor frequency = 1.3 kHz, cell temperature = 463 K. be described by the time-dependent Hamiltonian where the terms describe the hyperfine interaction, SE collisions, spin-destruction (SD) collisions, and Zeeman interaction, respectively. A hf is the hyperfine (HF) splitting and t l;l 0 ð Þ n is the (random) time of the n-th SE collision between atoms l and l 0 , which causes mutual precession of j (l) and j l 0 ð Þ by the (random) angle θ n . We indicate with R SE the rate at which such collisions move angular momentum between atoms. Similarly, the third term describes rotations about the random direction d m by random angle ψ m , and causes spin depolarization at a rate R SD . γ e = 2π × 28 GHz T −1 is the electron spin gyromagnetic ratio. We neglect the much smaller i·B coupling. We note that short-range effects of the magnetic dipole-dipole interaction (MDDI) are already included in R SE and R SD , and that long-range MDDI effects are negligible in an unpolarized ensemble, as considered here.
The SERF regime is defined by the hierarchy A hf ≫ R SE ≫ γ e |B|, R SD . Our experiment is in this regime, as we have A hf ≈ 10 9 s −1 , R SE ≈ 10 5 s −1 , γ e |B| ≈ 10 4 s −1 and R SD ≈ 10 2 s −1 . The hierarchy implies the following dynamics: on short times, the combined action of the HF and SE terms rapidly thermalizes the spin state, i.e., generates the maximum entropy consistent with the ensemble total angular momentum F, which is conserved by these interactions (see "Methods", "Spin thermalization" section). We indicate this F-parametrized max-entropy state by ρ th ð Þ F . We note that entanglement can survive the thermalization process; for example, ρ th ð Þ F¼0 is a singlet and thus necessarily describes entangled atoms. On longer time-scales, F experiences precession about B due to the Zeeman term and diffusive relaxation due to the depolarization term.
Non-destructive measurement. We perform a continuous nondestructive readout of the spin polarization using Faraday rotation of off-resonance light. On passing through the cell the optical polarization experiences rotation by an angle gF z ðtÞ ( π, where z is the propagation axis of the probe, g is a light-atom coupling constant and F ≡ F a − F b , where F α is the collective spin orientation from atoms in hyperfine state α ∈ {1, 2} (see "Methods", "Observed spin signal" section).
For thermalized spin states 〈F 〉/ F h i, so that the observed polarization rotation gives a view into the full spin dynamics. The optical rotation is detected by a balanced polarimeter (BP), which gives a signal proportional to the Stokes parameter where S x is the Stokes component along which the input beam is polarized 20 . S in ð Þ y t ð Þ is a zero-mean Gaussian process, whose variance is dictated by photon shot-noise and is characterized by a power-spectral analysis of the BP signal 21 .
Spin dynamics and spin tracking. The evolution of F t ð Þ is described by the Langevin equation (see "Methods", "Spin dynamics" section) where γ = γ e /q is the SERF-regime gyromagnetic ratio, i.e., that of a bare electron reduced by the nuclear slowing-down factor 5 , which takes the value q = 6 in the SERF regime 22 . Γ is the net relaxation rate including diffusion, spin-destruction collisions, and probe-induced decoherence, Q is the equilibrium variance (see below) and dW h , h ∈ {x, y, z} are independent temporal Wiener increments. Based on Eqs. (3) and (2), we employ the Bayesian estimation technique of Kalman filtering (KF) 14 to recover F t ð Þ, which is shown as gF z t ð ÞS x to facilitate comparison against the measured S out ð Þ y t ð Þ in Fig. 1b). The KF (see "Methods", "Kalman filter" section) gives both a best estimate and a covariance matrix Γ F t ð Þ for the components of F t ð Þ, which gives an upper bound on the variances of the post-measurement state. Figure 1c shows that the F z component of Γ F t ð Þ is suppressed rapidly, to reach a steady state value which is below the SQL. The other components are similarly reduced in variance by the measurement, and the total variance ΔF j j 2 Tr Γ F ½ can be compared against spin squeezing inequalities 15,16 to detect and quantify entanglement: Defining the spin-squeezing parameter ξ 2 ΔF j j 2 =SQL, where SQL ≡ N A 13/8 is the standard quantum limit, ξ 2 < 1 detects entanglement, indicating a macroscopic singlet state 17 . The minimum number of entangled atoms 15 is N A (1 − ξ 2 )13/16 (see "Methods", "Entanglement witness" section).
Experimental results. The cell temperature was stabilized at 463 K to give an alkali number density of n Rb = 3.55(6) × 10 14 atoms cm −3 , calibrated as described in "Methods", "Density calibration" section, and thus N A = 5.32(12) × 10 13 atoms within the 3 cm × 0.0503(8) cm 2 effective volume of the beam. At this density, the SE collision rate is R SE ≈ 325 × 10 3 s −1 . By varying B we can observe the transition to the SERF regime, and the consequent development of squeezing. Figure 2a shows spin-noise spectra (SNS) 21 , i.e., the power spectra of detected signal from BP, for different values of B, from which we determine the resonance frequency ω L = γB, relaxation rate Γ and the number density. Using these as parameters in the KF (see "Methods", "Kalman filter" section), we obtain ΔF j j 2 as shown in Fig. 2b, including a transition to squeezed/entangled states as the system enters the SERF regime.
At a Larmor frequency of 1.3 kHz, we observe ξ 2 = 0.650(2) or 1.88(1) dB of spin squeezing at optimal probe power 2 mW (see "Methods", "Kalman filter" section), which implies that at least 1.52(4) × 10 13 of the 5.32(12) × 10 13 participating atoms have become entangled as a result of the measurement. This greatly exceeds the previous entanglement records: 5 × 10 5 cold atoms in singlet states using a similar QND strategy 17 and a Dicke state involving 2 × 10 11 impurities in a solid, made by storing a single photon in a multi-component atomic ensemble 23 . This is also the largest number of atoms yet involved in a squeezed state; see Bao et al. for a recent record for polarized spin-squeezed states 24 . We use this power and field condition for the experiments described below, and note that the spin-relaxation time greatly exceeds the spin-thermalization time. In this condition, the entanglement bonds are rapidly distributed amongst the atoms by SE collisions without being lost.
We now study the spatial distribution of the induced entanglement. As concerns the observable F , the relevant dynamical processes, including precession, decoherence, and probing, are permutationally-invariant: Eqs. (3) and (2) are unchanged by any permutation of the atomic states. This suggests that any two atoms should be equally likely to become entangled, and entanglement bonds should be generated for atoms separated by Δz ∈ [0, L], where L = 3 is the length of the cell. Indeed, such permutational invariance is central to proposals 25,26 that use QND measurement to interrogate and manipulate many-body systems. There are other possibilities, however, such as optical pumping into entangled sub-radiant states 27 , that could produce localized singlets.
We test for long-range singlet-type entanglement by applying a weak gradient B 0 d B j j=dz during the cw probing process. A magnetic field gradient, if present, causes differential Larmor precession that converts low-noise singlets into high-noise triplets, providing evidence of long-range entanglement. For example, singlets with separation Δz will convert into triplets and back at angular frequency 28 Ω ¼ γB 0 Δz. The range δΔz of separations then induces a range δΩ ¼ γB 0 δΔz of conversion frequencies, which describes a relaxation rate. In Fig. 3 we show the KF-estimated ΔF z j j 2 as a function of B 0 and of time since the last data point, which clearly shows faster relaxation toward a thermal spin state with increasing B 0 . The observed additional relaxation for B 0 = 57.2 nT mm −1 (relative to B 0 = 0) is δΩ = 1.54 × 10 3 s −1 , found by an exponential fit. For Δz on the order of a wavelength, as would describe sub-radiant states, we would expect δΩ~1 s −1 at this gradient, which clearly disagrees with observations. The observed r.m.s. separation δΔz is about one millimeter, which is thousands of times the typical nearestneighbor distance n À1=3 Rb % 0:14 μm.

Discussion
Our observation of complex, long-lived, spatially-extended entanglement in SERF-regime vapors has a number of implications.
First, it is a concrete and experimentally tractable example of a system in which entanglement is not only compatible with, but, in fact, stabilized by entropy-generating mechanisms-in this case strong, randomly-timed spin-exchange collisions. It is particularly intriguing that the observed macroscopic singlet state shares several traits with a spin liquid state 2 , which is conjectured to underlie high-temperature superconductivity, a prime example of quantum coherence surviving in an entropic environment. Second, the results show that optical quantum non-demolition measurement can efficiently produce complex entangled states with long-range entanglement. This confirms a critical assumption of QND-based proposals 25,26 for QND-assisted quantum simulation of exotic antiferromagnetic phases. Third, the results show that SERF media are compatible with both spin squeezing and QND techniques, opening the way to quantum enhancement of what is currently the most sensitive approach to lowfrequency magnetometry and other extreme sensing tasks.

Methods
Density calibration. In the SERF regime, and in the low spin polarization limit, decoherence introduced by SE collisions between alkali atoms is quantified by 5,29,30 where for 87 Rb atomic samples the nuclear spin I = 3/2, and ω L = γ e |B|/q. In Eq. (4) the spin-exchange collision rate R SE ¼ σ SE n Rb V is proportional to the alkali density n Rb with proportionality dictated by the SE collision cross-section σ SE and the relative thermal velocity between two colliding 87 Rb atoms V. Using the reported value 31 of σ SE = 1.9 × 10 −14 cm 2 and V ¼ 4:75 10 4 cm s À1 , which is computed for 87 Rb atoms at a temperature of 463 K, we then calibrate the alkali density by fitting the measured linewidth Δν as a function of ω L . The model uses Δν = Δν 0 + Δν SE , where Δν SE is given by Eq. (4), and Δν 0 describes density-independent broadening due to power broadening and transit effects. n Rb and Δν 0 are free parameters found by fitting, with results shown in Fig. 4.
Observed spin signal. For a collection of atoms, we define the collective total atomic spin F P l f l ð Þ , where f (l) is the total spin of the lth atom. We identify the contributions of the two hyperfine ground states F a = 1 and F b = 2, defined as  of the rotating-frame spin component F z as a function of delay since F z was last aligned along the laboratory z axis, and thus subject to measurement by Faraday rotation. Error bars show ±4σ uncertainty, originating in the uncertainty of atom number. As seen in these data, increased gradient B 0 causes a faster relaxation toward the thermal spin state (TSS) value, as is expected for a gas of singlets with a range of separations (along the z axis) in the mm range. The blue and gray dashed lines are the standard quantum limit (SQL) (2.88(7) × 10 13 spins 2 ) and TSS (7.99(18) × 10 13 spins 2 ) noise levels, respectively. Probe power = 2 mW.
variables F a,z , F b,z through the input-output relation 20,32-34 is the positive-frequency (negative-frequency) part of the quantized electromagnetic field with polarization β = ± for sigma-plus (sigmaminus) polarized light. The factor (g a F a,z − g b F b,z ) ≡ Θ FR plays the role of a Faraday rotation angle, which in this small-angle regime can be seen to cause a displacement of S y (t) from its input value. It should be noted that Θ FR is operatorvalued, enabling entanglement of the spin and optical polarizations, and that the hyperfine ground states F a , F b contribute differentially to it.
The coupling constants are 14,33,35 where r e = 2.82 × 10 −13 cm is the classical electron radius, f osc = 0.34 is the oscillator strength of the D 1 transition in Rb, c is the speed of light, and ν − ν α is the optical detuning of the probe-light. ϒ ¼ 2:4 GHz is the pressure-broadened fullwidth at half-maximum (FWHM) linewidth of the D 1 optical transition for our experimental conditions of 100 Torr of N 2 buffer gas. For a far-detuned probe beam, such that |ν − (ν a + ν b )/2| ≫ |ν a − ν b | (as in this experiment) one can approximate g ≡ g a ≈ g b , such that Spin thermalization. In a local region containing a mean number of atoms N A , the SE and HF mechanisms will rapidly produce a thermal state ρ. We note that this process conserves F, and thus also conserves the statistical distribution of F, including possible correlations with other regions. ρ is then the maximum-entropy state consistent with a given distribution of F. Partitioning arguments then show that, for weakly polarized states such as those used in this experiment, the mean hyperfine populations are N a h i=N A ¼ 3=8 and N b h i=N A ¼ 5=8, and the polarisa- The same relations must hold for spin observables that sum F over larger regions, including the region of the beam, which determines which atoms contribute to the observed signal.
Entanglement witness. We can construct a witness for singlet-type entanglement 16 as follows: we define the total variance Separable states of N A atoms will obey a limit ΔF j j 2 ≥ N A C, where C is a constant, meaning that ΔF j j 2 <N A C witnesses entanglement. To find C, we note that a product state of N a atoms in state F a and N b atoms in state F b has ΔF j j 2 ≥ P α N α F α . Separable states are mixtures of product states. For such states, due to the concavity of the variance, ΔF j j 2 ≥ P α N α h iF α holds 16 . In light of the 3:5 ratio resulting from spin thermalization, this gives or C ¼ 13=8. Therefore, the standard quantum limit (SQL) is N A 13/8. We define the degree of squeezing ξ 2 ΔF j j 2 = P α N α h iF α À Á . Meanwhile the "thermal spin state (TSS)," i.e. the fully-mixed state, has ΔF j j 2 ¼ N A 9=2. Our condition provides also a quantitative measure of the number of entangled atoms. We consider a pure entangled quantum state of the form where jψ l ð Þ i are single particle states. Here, N p particles are in a product state, while N e = N A − N p particles are in an entangled state denoted by jΦ e i. For the collective variances of jκi we can write that for h = x, y, z. Let us try to find a lower bound on (11). Let us assume that all atoms are in state F α . Then, we know that ΔF h ð Þ 2 can even be zero, if the entangled state jΦ e i is a perfect singlet. Hence, Based on these, the number of entangled atoms in this case is bounded from below as N e ≥ (1 − ξ 2 )N A , where ξ 2 ¼ ΔF j j 2 =SQL and the standard quantum limit SQL in this case is F α N A .
Let us now consider the case when some atoms have F 1 others have F 2 In particular, let us consider a state of the type (10) such that N α particles have spin F α with α = 1, 2 such that F 1 ≤ F 2 . Then, for such a pure state, holds, where Note that the bound in Eq. (13) is sharp, since it can be saturated by a quantum state of the type (10). In order to minimize the left-hand side of Eq. (13), the particles corresponding to the product part must have as many spins in F 1 as possible, since this way we can obtain a small total variance. In particular, if N p ≥ N 1 then all atoms in the product part must have an F 1 spin, otherwise at least N 1 atoms of the N p atoms.
It is instructive to rewrite Eq. (10) with a piece-wise linear bound as The bound in Eq. (15) is plotted in Fig. 5a. So far, we have been discussing a bound for a pure state of the form (10). The results can be extended to a mixture of such states straightforwardly, since the bound in Eq. (13) is convex in (N 1 , N p ). Then, in our formulas N 1 , must be replaced by N 1 h i. We also have to define the number of entangled particles N e for the case of a mixed state. A mixed state has N e entangled particles, if it cannot be constructed as a mixture of pure states, which all have fewer than N e entangled particles 15 .
We know that in our experiments F 1 = 1, F 2 = 2, and N 1 h i ¼ 3=8N A . From these, we obtain the minimum number of entangled atoms as The bound in Eq. (16) is plotted in Fig. 5b. Here, again ξ 2 ¼ ΔF j j 2 =SQL and the standard quantum limit SQL in this case is N A × 13/8. For our experiment, the ξ 2 ≥ 3/13 case is relevant.
The macroscopic singlet state gives a metrological advantage in estimating gradient fields 28,36 and in detecting displacement of the spin state, e.g. by optical pumping 14,37 .
Balanced polarimeter signal. The photocurrent I(t) of the balanced polarimeter shown in Fig. 1a is where the detector's responsivity is < ¼ q e η=E ph in terms of the detector quantum efficiency η, charge of the electron q e , and photon energy E ph . To account for its spatial structure in Eq. (17) the integral is carried over the area of the probe. From Eq.
(2) and Eq. (17) one obtains the differential photocurrent increment where g 0 ¼ gq e , the stochastic increment dw sn (t), due to photon shot-noise, is given representing a differential Wiener increment. In our experiments the photocurrent I(t) is sampled at a rate Δ −1 = 200 k Samples/s. To formulate the discrete-time version of Eq. (17) we consider the sampling process as a short-term average of the continuous-time measurement. The photocurrent I(t k ) recorded at t k = kΔ, with k being an integer, can then be expressed as where the Langevin noise ξ D (t k ) obeys E ξ D t ð Þξ D t 0 ð Þ ½ ¼ δ t À t 0 ð Þηq 2 e _ N=Δ, with Δ −1 quantifying the effective noise-bandwidth of each observation.
Spin dynamics. We model the dynamics of the average bulk spin of our hot atomic vapor in the SERF regime, 5,29,38 and in the presence of a magnetic field B in the where the matrix A includes dynamics due to Larmor precession and spin relaxation. It can be expressed as A ij = −γB h ε hij + Γ ij , where h, i, j = x, y, z. The relaxation matrix Γ has eigenvalues T À1 1 and T À1 2 ¼ T À1 1 þ T À1 SE for spin components parallel and transverse to B, respectively. We note that in the SERF regime the decoherence introduced by SE collisions between alkali atoms is quantified by Eq. (4) 5,29 .
To account for fluctuations due to spin noise in Eq. (20) we add a stochastic term ffiffiffi σ p dW where dW h , h ∈ {x, y, z) are independent Wiener increments. Thus the statistical model for spin dynamics reads where the strength of the noise source σ, the matrix A, and the covariance matrix in from which we obtain σ = 2ΓQ.
Kalman filter. Kalman filtering is a signal recovery method that provides continuously-updated estimates of all physical variables of a stochastic model, along with uncertainties for those estimates. For linear dynamical systems with gaussian noise inputs, e.g. the spin dynamics of Eq. 3 with the readout of Eq. 5, the Kalman filter estimates are optimal in a least-squares sense. The KF estimates, e.g. those shown in Fig. 1b and c, indicate our evolving uncertainty about the values of the physical quantities, e.g. F z . As such, they provide an upper bound on the intrinsic uncertainty of these same quantities due to, e.g. quantum noise. As information accumulates, the uncertainty bounds on F x , F y and F z contract toward zero, implying the production of squeezing and entanglement. This is measurement-induced, rather than dynamically-generated entanglement. The measured signal, i.e. the optical polarization rotation, indicates a joint atomic observable: the sum of the spin projections of many atoms. For an unpolarized state such as we use here the physical back-action-which consists of small random rotations about the F z axis induced by quantum fluctuations in the ellipticity of the probe-has a negligible effect. We construct the estimator e F t of the macroscopic spin vector using the continuous-discrete version of Kalman filtering 14 . This framework relies on a twostep procedure to construct the estimatex t , and its error covariance matrix , of the state x t of a continuous-time linear-Gaussian process, in our case F t ð Þ, that is observed at discrete-time intervals Δ = t k − t k−1 . Measurement outcomes are described by the observations vector z k , in our case the scalar I k , which is assumed to be linearly related to x t via the coupling matrix H k and to experience independent stochastic Gaussian noise as described previously 14 .
In the first step of the Kalman filtering framework, also called the prediction step, the values at t = t k , e F kjkÀ1 and Σ kjkÀ1 , are predicted conditioned on the process dynamics and the previous instance, e F kÀ1jkÀ1 and Σ kÀ1jkÀ1 , as follows: is the state transition matrix describing the evolution of the dynamical model Eq. (20) within the time interval Δ, and is then the effective covariance matrix of the system noise 14 .
In the second step, or update step, the information gathered through the fresh photocurrent observation I k is incorporated into the estimate: where H k ¼ ηg _ N; 0; 0 Â Ã and the Kalman gain K k is defined as with sensor covariance R Δ = R/Δ dictated by the power-spectral-density, R, of the photocurrent noise, i.e. due to photon shot-noise, and the sampling period, Δ. As dicussed in previous work 14 the KF is initialised according to a distribution that represents our prior knowledge about the system at time t = t 0 and fixes e F 0j0 $ N μ 0 ; Σ 0 À Á , where μ 0 , and Σ 0 are the mean value and total variance of the observed data. After initialization KF estimates for the covariance matrix Σ k|k undergo a transient and once this transient has decayed they converge to a steady state value Σ ss .
In Fig. 6 we observe this behavior for the total variance ΔF j j 2 Tr Γ F ð Þ as a function of time t = t k , where Γ F ¼ Σ kjk . After about 0.8 ms, the total variance reaches steady state value which is used to compare with SQL and indicates squeezing degree. Figure 7 shows squeezing degree at different probe power, and presents the optimal probe power we observed is 2 mW.

Validation.
To validate the sensor model we employ three validation techniques sensitive to both the statistics of the optical readout and spin noise. First, we analyse the statistics of the sensor output innovation, i.e., the difference between observations I k (data) and Kalman estimates (ỹ k ¼ I k À H k e F kjkÀ1 ). In Fig. 8, we show theỹ k histogram with the sensor output estimation error, which is described by zero-mean Gaussian process with variance equal to R Δ þ H k Σ kjkÀ1 H T k . We find 94% ofỹ k data lie within a two-sided 95% confidence region of the expected Gaussian distribution, thus indicating a very close agreement of the model and observed statistics. We note that while being a standard technique in the validation of Kalman filtering 39 , this technique for our experimental conditions is more sensitive to photon shot noise than to spin noise. Therefore, to further validate our estimates we also include two other validation techniques, designed to be sensitive to the atomic statistics on a range of time-scales.
Particularly, we perform Monte Carlo simulations based on the model described by Eqs. (2), (3) and (17) and fed with the operating conditions of our experiments and compare the power spectral density (PSD) of the simulated sensor output (Simulation) to the observed PSD of the measurements (Data), as shown in Fig. 9a. The observed agreement between Data and Simulation suggests the validity of the statistics of the spin dynamics model. Finally, we employ the Kalman filter to identify the evolution of the atomic state variables based on the Simulation. We can then compare the distribution of Kalman spin-estimates from the Data versus that from Simulation. The results are shown in Fig. 9b and c, respectively. The similarity in the statistics of these two spin estimates validates the spin dynamics model. Together with the above validations, it provides a full validation of both the optical and spin parts of the model. Gradient field tests. A weak gradient magnetic field is applied along the probe (z) direction by coils implemented inside the magnetic shields. In Fig. 10  observe that the variance of each component increases towards the TSS noise level with gradient field. We note that due to the bias field along the [1,1,1] direction, the current (t = t k ) sensor reading indicates F z at that time, while F x and F y describe components that were measured 1/3 and 2/3 Larmor cycles earlier, respectively. The combined variance is used to compute ΔF z j j 2 , as in Fig. 3. We note that the Stern-Gerlach (SG) effect, in which a gradient causes wave-functions components to separate in accordance with their magnetic quantum numbers, also contributes to the loss of coherence. The SG contribution is negligible, however, due to the weak gradients used here and the rapid randomization of momentum caused by the buffer gas.   (18)×10 13 spins 2 ) and standard quantum limit (SQL) (2.88(7) × 10 13 spins 2 ) noise levels, respectively for one spin component F h ; h 2 x; y; z f g.

Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request. Open-access datasets from this work available at https:// doi.org/10.5281/zenodo.3694692.