Introduction

The second law of thermodynamics is of fundamental and practical importance1. On the one hand, it allows one to predict which transformations are possible in nature. On the other hand, it offers a method for determining the efficiency of a given process by comparing it to its ideal, reversible limit. According to the standard formulation of the second law, no work can be cyclically extracted from a system coupled to a single reservoir at temperature T0, that is, the power output has to be negative, \({\dot{{\mathcal{W}}}}_{\text{ext}}\ \le \ 0\)1. However, in the presence of measurement and feedback, this statement of the second law breaks down and positive work can be produced, as exemplified by Maxwell’s and Szilard’s thought experiments2,3. For Markovian feedback protocols a refined version of the second law reads \({\dot{{\mathcal{W}}}}_{\text{ext}}\ \le \ {k}_{\text{B}}{T}_{0}{\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\), where T0 is the bath temperature and \({\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\) is the information flow to the detector, defined as the time variation of the mutual information between a variable and its measured value4,5,6. This inequality has been experimentally verified with colloidal particles7,8 and single electrons9,10. When the information rate \({\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\) is positive, more work can be extracted from the system than permitted by the usual second law of thermodynamics.

The fact that a control signal cannot be applied instantaneously implies that feedback circuits inevitably exhibit memory effects and are thus non-Markovian. The Markovian approximation is only valid when the delay, i.e., the time between measurement and feedback, is much smaller than the typical timescales of the system. Delayed feedback is widespread in many areas, from chaotic systems to biology11,12,13,14,15, emphasizing the crucial need to expand the second law to account for finite memory. However, such generalization is nontrivial. Because of the non-Markovian nature of the feedback, the conventional approach of stochastic thermodynamics16 cannot be applied and the usual condition of local detailed balance does not hold. As a result, new contributions to the nonequilibrium entropy production occur, leading to the extended second law for continuous, non-Markovian feedback, \({\dot{{\mathcal{W}}}}_{\text{ext}}\ \le \ {k}_{\text{B}}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{pump}}\), where \({\dot{{\mathcal{S}}}}_{\text{pump}}\) is the entropy pumping rate, which incorporates the effect of the time delay17,18,19,20. Since \({\dot{{\mathcal{S}}}}_{\text{pump}}\ \le \ {\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\), this is the tightest second-law inequality to date18. Despite the omnipresence of delay in feedback processes, a dedicated experimental investigation of this non-Markovian generalization of the second-law inequality is still lacking.

Here, we report the experimental study of the thermodynamics of continuous, non-Markovian feedback control applied to the underdamped center-of-mass (CM) motion of a levitated microsphere21. Levitated particles are an ideal experimental platform to explore thermodynamics in small systems22,23,24,25,26,27. We confirm the validity of the generalized second law for time delays spanning two decades and observe the breakdown of the high-quality-factor (high Q) approximation18. We establish that the efficiency of the feedback is enhanced when non-Markovian effects are included. We further explore the relation between Markovian and non-Markovian feedback by analyzing how the delay affects the correlations between measurement outcome and velocity of the particle. We finally explore the limitations of feedback cooling and the saturation of the effective CM temperature of the system above the bath temperature for very large delay.

Results

Generalized second law

We consider a harmonic oscillator in contact with a heat bath at temperature T0, and subjected to a delayed feedback control that acts as an information reservoir (Fig. 1). Delayed feedback control means here that we acquire information about the oscillator position xt at time t and apply a force \({F}_{\text{fb}}(t)\propto {x}_{t-{t}_{\text{fb}}}\) to manipulate its motion based on the position measured at time t − tfb. The stochastic dynamics of the oscillator is governed by the underdamped Langevin equation28,

$${\ddot{x}}_{t}+{\Gamma }_{0}{\dot{x}}_{t}+{\Omega }_{0}^{2}{x}_{t}-g{\Gamma }_{0}{\Omega }_{0}{x}_{t-{t}_{\text{fb}}}=\sqrt{\frac{2{\Gamma }_{0}{k}_{\mathrm{{B}}}{T}_{0}}{m}}{\xi }_{t},$$
(1)

with m the particle mass, Ω0 its natural frequency, and Γ0 the damping coefficient. The quantity ξt is a centered Gaussian white noise with \(\langle \xi (t)\xi (t^{\prime} )\rangle =\delta (t-t^{\prime} )\). The linear feedback is applied via the force \({F}_{\text{fb}}=-gm{\Gamma }_{0}{\Omega }_{0}{x}_{t-{t}_{\text{fb}}}\) with feedback gain g > 018. The mechanical quality factor of the resonator is given by Q0 = Ω0∕Γ0 and the feedback damping rate by Γfb = gΓ0. It is convenient to introduce the normalized delay τ = tfbΩ0 and the position of the oscillator normalized to its standard deviation in equilibrium17,18,19,20. The dynamics of the particle is then fully characterized by a set of dimensionless parameters (g, Q0, τ) (Methods). Both the harmonic and Brownian terms in Eq. (1) are Markovian. Memory effects enter only via the delayed feedback force Ffb.

Fig. 1: Non-Markovian delayed feedback control.
figure 1

The system consists of a particle (white) in contact with a single heat bath (red) and coupled to a feedback reservoir (green). The feedback loop continuously detects the position xt of the particle and applies a delayed feedback force \({F}_{\text{fb}}({x}_{t-{t}_{\text{fb}}})\) proportional to the position at time t − tfb. Depending on the time delay tfb, the feedback cools or heats the system to a steady-state effective temperature Teff. The blue arrows in the figure correspond to cooling, when the feedback control pumps the entropy \({{\mathcal{S}}}_{\text{pump}}\) out of the system. Energy flows as heat \({\mathcal{Q}}\) from the bath to the particle and is extracted as work \({\mathcal{W}}\) by the feedback circuit (blue arrows). In the case of heating, the energy and entropy flows change direction. In both regimes the generalized second law holds in the form, \({\dot{{\mathcal{W}}}}_{\text{ext}}=-\dot{{\mathcal{W}}}\ \le \ {k}_{\text{B}}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{pump}}\).

Full size image

When the feedback force acts to cool the harmonic oscillator, the extracted work \({{\mathcal{W}}}_{\text{ext}}=-{\mathcal{W}}\) is taken to be positive. According to the first law, the energy balance reads \(\Delta U={\mathcal{W}}-{\mathcal{Q}}\). As a result, the heat dissipated into the heat bath in the steady state is \({\mathcal{Q}}=-{{\mathcal{W}}}_{\text{ext}}\). On the other hand, the steady-state entropy balance is17,18,19,20

$$-\frac{\dot{{\mathcal{Q}}}}{{k}_{\mathrm{B}}{T}_{0}}=\frac{{\dot{{\mathcal{W}}}}_{\text{ext}}}{{k}_{\mathrm{B}}{T}_{0}}\,{\le}\, {\dot{{\mathcal{S}}}}_{\text{pump}},$$
(2)

where \({\dot{{\mathcal{S}}}}_{\text{pump}}\) is the entropy pumping rate, an additional contribution to the entropy production that stems from the feedback control and depends on the delay for non-Markovian protocols. The entropy pumping rate is computed by coarse-graining the harmonic and feedback forces over the position variable17,18,19,20 (Methods). For a harmonic potential and a linear feedback force, the velocity distribution is a Gaussian (Fig. 2b), \(P(v)=\exp \left(-{v}^{2}/2{\sigma }_{v}^{2}\right)/\sqrt{2\pi {\sigma }_{v}^{2}}\) with variance \({\sigma }_{v}^{2}\). The non-Markovian feedback control leads to a cooling of the microparticle, corresponding to negative entropy pumping and extracted work rates, when \({\sigma }_{v}^{2}\,{<}\,1\). By contrast, heating occurs for \({\sigma }_{v}^{2}\,{> }\, 1\).

Fig. 2: Experimental setup.
figure 2

a Two counterpropagating laser beams (red) are coupled into a hollow-core photonic crystal fiber (HCPCF). They form a standing wave that allows for trapping a silica microparticle at an antinode (inset, white arrow). The fiber ends are placed inside vacuum chambers (vac) to control the pressure inside the HCPCF. The particle position along the fiber axis is detected with an interferometric readout. The feedback loop is implemented by delaying the detected signal by a time tfb. To apply the feedback force, a feedback laser (green) is modulated with the delayed signal via an acousto-optic modulator (AOM). b Phase-space distribution P(x, v) of the microparticle derived from the position measurements at Teff = T0 (red circles) and Teff < T0 (blue circles). The respective marginals for the distributions of position and velocity are shown on the top and right panels.

Full size image

The validity of inequality (2) can be assessed by comparing the entropy pumping to Markovian bounds. The usual Markovian velocity feedback (VFB) cooling29,30, with a feedback force proportional to the instantaneous velocity of the particle, Ffb ∝ − v, is recovered in the limit of Q0 ≫ τ and for τ = π∕2 +  2πn, with n an integer. The entropy pumping rate corresponds in this case to \({\dot{{\mathcal{S}}}}_{\text{vfb}}=g/{Q}_{0}\)31, which we identify as the Markovian information flow \({\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\) (Methods). The high-quality-factor approximation (Q0 ≫ 1) allows one to map the non-Markovian dynamics to an effectively Markovian Langevin equation, because the motion of the oscillator is essentially coherent on that timescale18. The effect of the feedback is then incorporated in a modified damping \(\Gamma ^{\prime} ={\Gamma }_{0}(1+g\sin \tau )\) and mechanical frequency \(\Omega {^{\prime} }^{2}={\Omega }_{0}^{2}[1-({\Gamma }_{\text{fb}}/{\Omega }_{0})\cos \tau ]\) of the resonator (Supplementary Note 2). The Markovian second law remains valid with these modified parameters. In the high Q approximation, the entropy pumping is given by \({\dot{{\mathcal{S}}}}_{\text{highQ}}=(g/{Q}_{0})\sin \tau\)18. The approximation is expected to breakdown for larger τ, when the Brownian force noise leads to dephasing between the oscillator motion and the feedback signal. This regime can only be correctly described using the generalized second law (2).

Experimental setup and results

In our experiment, we use an optically levitated microparticle to implement the dynamics of Eq. (1), which holds for any harmonic system with linear feedback control. A standing wave is formed by two counterpropagating laser beams (λ = 1064 nm) inside a hollow-core photonic crystal fiber (HCPCF) (Fig. 2a and Methods)21. A silica microsphere (969-nm diameter) is trapped at an intensity maximum of the standing wave. The amplitude of the particle motion is sufficiently small to allow for a harmonic approximation of the potential with frequency Ω0∕2π = 404 kHz (Supplementary Note 7). The damping coefficient Γ0 as well as the bath temperature T0 = 293 K are determined by the surrounding gas. In our setup, the linear dependence of the damping coefficient Γ0 on the environmental pressure allows simple and systematic tuning of this parameter along with the mechanical quality factor Q0 = Ω0 ∕ Γ0. The particle motion along the x-axis is detected by interferometric readout of the light scattered by the particle21. The signal is fed into a delay line that is digitally implemented, and the output signal serves to control the power of a feedback laser. This laser exerts a radiation pressure force in one direction, accelerating the microparticle proportional to the delayed particle position. The overall amplification of the signal sets the proportionality constant, which is given by gΩ0Γ0, where the gain in our experiment is g = 0.36 (Supplementary Note 5). The whole feedback circuit has a minimal delay of tfb = 2.6 μs, i.e., τ = 2.04π.

We first test the extended second law (2) by varying the delay over two decades. Figure 3a demonstrates the validity of the non-Markovian inequality (2) over all relevant timescales. The non-Markovian entropy pumping rate \({\dot{{\mathcal{S}}}}_{\text{pump}}\) (green) is a much more precise upper bound to the extracted work rate (black) than the Markovian pumping rate \({\dot{{\mathcal{S}}}}_{\text{vfb}}\) (horizontal dashed line). In particular, the Markovian result fails to capture the oscillations of the extracted work rate, as well as the heating phases induced by the delay. The high Q approximation (dashed-dotted line) correctly describes the oscillatory behavior of \({\dot{{\mathcal{S}}}}_{\text{pump}}\) for short delays. Yet, it does not account for the oscillation decrease induced by the Brownian force noise for long delays. We already observe significant deviations for a delay of only three oscillation periods with a mechanical quality factor of Q0 = 55. We have also verified the second law (2) by varying the dissipation via Q0 (Supplementary Note 9).

Fig. 3: Test of the generalized second law.
figure 3

a Experimental entropy pumping (green squares) and extracted work (black circles) rates are plotted as a function of time delay for Q0 = 55 and g = 0.36. The shaded area corresponds to the analytical prediction including experimental parameter drifts (Supplementary Note 8). The error bars represent statistical uncertainties. Theoretical predictions for Markovian entropy bounds to the extracted work are shown for velocity feedback (\({\dot{{\mathcal{S}}}}_{\text{vfb}}\), horizontal dashed line) and for the high-quality-factor approximation (\({\dot{{\mathcal{S}}}}_{\text{highQ}}\), dashed-dotted line). The blue (red) region corresponds to cooling (heating) of the particle motion. For all delays, the entropy pumping is larger than the extracted work. At the same time, it represents a tighter bound to the extracted work than the Markovian predictions. b Feedback efficiencies plotted in the cooling regions only. Experimental pumping efficiency \({\eta }_{\text{pump}}={\dot{{\mathcal{W}}}}_{\text{ext}}/({k}_{\mathrm{B}}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{pump}})\) (green squares) of the generalized second law is compared with the experimental Markovian velocity feedback efficiency \({\eta }_{\text{vfb}}={\dot{{\mathcal{W}}}}_{\text{ext}}/({k}_{\mathrm{B}}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{vfb}})\) (black circles), where \({\dot{{\mathcal{S}}}}_{\text{vfb}}=g/{Q}_{0}\). Theoretical Markovian predictions for ηvfb (dashed) and \({\eta }_{\text{highQ}}={\dot{{\mathcal{W}}}}_{\text{ext}}/({k}_{\mathrm{B}}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{highQ}})\) (dashed-dotted) with \({\dot{{\mathcal{S}}}}_{\text{highQ}}=(g/{Q}_{0})\sin \tau\) are shown for comparison. Neglecting the non-Markovian behavior leads to a strongly underestimated efficiency of the feedback process.

Full size image

The generalized second law (2) is crucial to properly estimate the performance of the feedback cooling. In analogy to heat engines and refrigerators, one may define the efficiency of work extraction \({\eta }_{\text{pump}}={\dot{{\mathcal{W}}}}_{\text{ext}}/({k}_{B}{T}_{0}{\dot{{\mathcal{S}}}}_{\text{pump}})\), which characterizes the conversion of information into extracted work32. As shown in Fig. 3b, the corresponding Markovian efficiencies for velocity feedback (ηvfb) and for the high Q approximation (ηhighQ) vastly underestimate the feedback efficiency. We note that the pumping efficiency ηpump and cooling power \({\dot{{\mathcal{W}}}}_{\text{ext}}\) exhibit a trade-off similar to that of heat engines: one is maximal when the other is minimal, and vice versa. By contrast, the velocity feedback efficiency ηvfb exhibits an opposite dependence on τ.

We may gain physical insight on the breakdown of the standard second law as shown in Fig. 3 by analyzing the correlations between particle velocity and feedback force, as well as the effective temperature of the system. Cooling is efficient when the feedback force counteracts the motion of the oscillator, in other words, when the velocity vt of the oscillator and the feedback force (F ∝ xt−τ) are anticorrelated. Heating occurs when they are correlated. Figure 4 shows the correlation function between the two quantities, c(τ) =  1∕(σqσv)∫∫ytvtP(yt, vt)dytdvt with yt = qt−τ (Supplementary Note 4). The delay τ has two effects. First, it changes the phase between the mechanical system and the feedback signal deterministically, resulting in the oscillatory behavior of the correlations, and thus the difference between heating and cooling. Second, it allows for stochastic dephasing of the mechanical motion with respect to the feedback signal, which translates into a reduction of the correlations for increasing delay. These correlations do not vanish, however, but asymptotically approach a finite value. For long delays, the oscillator thermalizes due to the damping. The action of the feedback circuit can then be seen as an independent force noise with the spectrum of a white-noise driven harmonic oscillator. The positive correlations occur as a result of the resonant driving of the mechanical motion by the feedback signal.

Fig. 4: Correlation and colored noise for long delays.
figure 4

a Experimental (black) and theoretical (red) correlation functions of the microparticle CM motion as a function of τ for g = 0.36. Zoom in for short (b) and long (c) delays. (d) Experimental probability distribution P(qt–τ, vt) for different values of time delay in b, c indicated by black arrows. The system is still correlated for very long delays (τ  = 88π) due to resonant driving by colored force noise created by the feedback loop.

Full size image

Figure 5 displays the ratio of the effective steady-state temperature, \({T}_{\text{eff}}={T}_{0}{\sigma }_{q}^{2}\)18, and the bath temperature T0. Figure 5a clearly shows how the action of the feedback is reduced when the mechanical quality factor Q0 is decreased or the time delay τ increased. For all values of Q0, there is a certain delay τ, beyond which cooling is no longer possible (black line). For even longer delays, the effective temperature reaches a constant value, \({T}_{\,\text{eff}\,}^{\infty }/{T}_{0}\approx 1+{g}^{2}/2\), that is independent of Q0 for weak coupling g ≪ Q0 (Supplementary Note 3). This is in line with the second law that predicts an asymptotic negative work extraction, \({{\mathcal{W}}}_{\,\text{ext}\,}^{\infty }\approx -{g}^{2}/(2{Q}_{0})\) for very long delays. Figure 5b provides a cut for constant Q0 = 55 through Fig. 5a. The gray-shaded area on the right shows the region where Teff > T0 for our gain g = 0.36. Note that this region can be reduced by decreasing the feedback gain. Excellent agreement between theory (red) and data (black) is observed.

Fig. 5: Steady-state effective temperature.
figure 5

a Calculated 2D color map of the ratio Teff∕T0 of the microparticle motion as a function of τ and Q0 for g = 0.36. The black solid line represents the border beyond which cooling is no longer possible. Note that this border is gain dependent. b Experimental (black) and calculated (red) normalized effective temperature for Q0 = 55, corresponding to the dashed horizontal line shown in a. The gray-shaded area shows the region with Teff > T0 for very long delays.

Full size image

Discussion

In summary, we have performed an extensive experimental study of the second law of thermodynamics in the presence of continuous non-Markovian feedback. Our results constitute an important step toward bridging theoretical developments in stochastic information thermodynamics and more technical applications like continuous feedback. Possible generalizations include the consideration of measurement noise31 and the extension to nonlinear potentials and nonlinear feedback (e.g., parametric cooling of levitated nanoparticles33). Nonlinear delayed feedback control has thus already proven to be a more robust and effective method for synchronization compared with its linear counterpart34. Another avenue of future research is the use of optimal control via Kalman–Bucy filters20. Intuitively, one might hope that elaborate filtering methods, like Kalman filters, may overcome the impact of delay that we observe in our simple feedback scenario by an optimal prediction of the instantaneous velocity. However, while this will help to reduce the effect of measurement noise, the effect of stochastic Brownian force noise that occurs during the delay is fundamentally unpredictable. We therefore anticipate no improvement compared with the long-delay results presented in our work.

Methods

Normalized form of the equation of motion

After time contraction t → tΩ0 and position normalization x → q = x∕xth with \({x}_{\text{th}}=\sqrt{{k}_{\text{B}}{T}_{0}/m{\Omega }^{2}}\) the thermal root mean square amplitude, the Langevin equation can be rewritten in the dimensionless form,

$${\ddot{q}}_{t}+\frac{1}{{Q}_{0}}{\dot{q}}_{t}+{q}_{t}-\frac{g}{{Q}_{0}}{q}_{t-\tau }=\sqrt{\frac{2}{{Q}_{0}}}{\xi }_{t},$$
(3)

with Q0 = Ω0∕Γ0 the quality factor, g = Γfb∕Γ0 the feedback gain, τ = tfbΩ0 the normalized delay and ξt representing a centered Gaussian white noise force with \(\langle \xi (t)\xi (t^{\prime} )\rangle =\delta (t-t^{\prime} )\).

Experimental setup

The particle is trapped in an intensity maximum of the standing wave and oscillates in a harmonic trap. For a laser power of 400 mW in each trapping beam, the mechanical frequency is Ω0∕2π = 404 kHz. The particle CM motion is recorded using a balanced photodiode and roughly 10% of the light transmitted through the HCPCF and that scattered by the particle. The feedback control is implemented via radiation pressure of a second laser beam which is orthogonally polarized and frequency shifted with respect to the trapping laser to avoid interference effects. The feedback force Ffb is realized in the following steps. The readout signal of the CM motion is bandpass filtered (bandwith: 600 kHz, center frequency: Ω0∕2π) to get rid of technical noise in the detector signal. Then, the signal can be amplified and delayed by an arbitrary time tfb with a field programmable gate array. This signal is then used as modulation input for the AOM. A more detailed description of the experimental setup and the feedback control can be found in Supplementary Note 6 and in ref. 21

Thermodynamic quantities

The non-Markovian entropy pumping rate \({\dot{{\mathcal{S}}}}_{\text{pump}}\) can be computed by coarse-graining the harmonic (h) and feedback (fb) forces over the position variable: \({\dot{{\mathcal{S}}}}_{\text{pump}}=-\int \text{d}\,v\left[\overline{{F}_{\text{fb}}(v)}+\overline{{F}_{\text{h}}(v)}\right]{\partial }_{v}P(v)\), where P(v) is the velocity distribution and \(\overline{{F}_{\text{i}}(v)}=\int dx\overline{{F}_{\text{i}}(x,v)}P(x| v)\) is the corresponding coarse-grained forces17,18,19,20. On the other hand, the Markovian information flow \({\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}\,}\) in the case of VFB is given by the time variation of the mutual information between the velocity v and the measured value of the velocity y: \({\dot{{\mathcal{I}}}}_{\,\text{flow}}^{\text{mar}}=\int \text{d}v\text{d}\,y{\partial }_{v}J(v)\,\text{ln}\,[P(v,y)/P(v)P(y)]\) with the velocity probability current J(v)18,19,20. This quantity diverges for error-free feedback31. We thus identify the Markovian entropy bound with the Markovian limit of \({\dot{{\mathcal{S}}}}_{\text{pump}}\to {\dot{{\mathcal{S}}}}_{\text{vfb}}=g/{Q}_{0}\)18,19,20. For a harmonic potential and a linear feedback force, the velocity distribution is a Gaussian, \(P(v)=\exp \left(-{v}^{2}/2{\sigma }_{v}^{2}\right)/\sqrt{2\pi {\sigma }_{v}^{2}}\), with variance \({\sigma }_{v}^{2}\). The entropy pumping rate is then explicitly \({\dot{{\mathcal{S}}}}_{\text{pump}}=(1-{\sigma }_{v}^{2})/({Q}_{0}{\sigma }_{v}^{2})\) and the work extraction rate reads \({\dot{{\mathcal{W}}}}_{\text{ext}}/({k}_{\mathrm{B}}{T}_{0})=(1-{\sigma }_{v}^{2})/{Q}_{0}\) (Supplementary Note 1). We experimentally obtain the velocity variance needed to verify the generalized second law as follows: for a given time delay τ, a position time trace x(t) is recorded, filtered with a bandwidth of 3Γ0 via post processing, and normalized with the standard deviation of the feedback beam turned off, to find the normalized position q(t). The velocity \(v=\dot{q}\) is then numerically calculated with the finite difference approximation and the variance \({\sigma }_{v}^{2}\) computed.