Introduction

A finite heat flow is a generic feature of systems out of thermal equilibrium. In the last decades, special interest has been devoted to heat exchange and other thermodynamic properties of small (mesoscopic) systems coupled to a bath, which are noisy per se1,2. Stochastic thermodynamics (ST) has emerged as an elegant and consistent framework to generalize thermodynamic notions to the level of noisy trajectories and to systems far from equilibrium3, with numerous applications to soft matter4, biological5,6, and quantum systems7,8. Many fundamental concepts, however, are based on the Markov assumption, although in real-world systems memory effects are in fact often not negligible. While extensions towards several non-Markovian systems have been carried out in the past9,10,11,12,13,14,15,16,17,18, the application to continuous systems with discrete time-delays (i. e., delta-like memory kernels in the equations of motion) is still in its infancy. Delays can be of intrinsic nature as in neural systems19,20 and laser networks21,22, or be generated externally, e. g., by a feedback protocol with time lag between signal detection and action of control23,24,25,26. Moreover, since time-delay is known to enhance control strategies, it is often included intentionally, for example in Pyragas control21. Despite their major importance, delayed systems are still little understood from a thermodynamic perspective, especially in regard to the interplay of delay and nonlinearities.

Recent research27,28,29,30 has revealed that ST of delayed continuous systems is indeed quite involved, even in the absence of nonlinearities. For linear cases, it has been explicitly shown that in the long-time limit a non-equilibrium steady state (NESS) with finite entropy production30 is approached (in the absence of time-dependent forces), as the delay pushes the system out of equilibrium. Nonequilibrium inequalities have been found28 that generalize the second law and provide bounds to the extractable work. Furthermore, the fluctuations of work, heat and entropy production have been investigated29. One crucial issue in the thermodynamic description of delayed systems is the acausality of time-reversed processes appearing in the path integral representation of fluctuating heat q and entropy production27,28,29. Contrary to the Markovian case2, the total average entropy production ΔStot of a delayed system in a NESS differs from the medium entropy \({\rm{\Delta }}{S}_{{\rm{m}}}{\mathscr{T}}={\langle q\rangle }_{{\rm{ss}}}\) (where \({\mathscr{T}}\) is the heat bath’s temperature and \({\langle \cdot \cdot \rangle }_{{\rm{ss}}}\) denotes NESS ensemble averages). In particular, the second law does not impose nonnegativity on ΔSm alone27,28,29. However, while these statements are generic, explicit expressions for the thermodynamic quantities are, so far, only available for systems governed by linear forces27,28,29,30, thus excluding wide classes of physically interesting processes which exclusively arise in nonlinear systems.

As a step in this direction, we here apply ST to investigate the heat rate \(\dot{Q}={\langle \delta q/{\rm{d}}t\rangle }_{{\rm{ss}}}\) of a classical nonlinear delayed system. By considering \(\dot{Q}={\dot{S}}_{{\rm{m}}}{\mathscr{T}}\) rather than \({\dot{S}}_{{\rm{tot}}}\), we avoid the above-mentioned problem induced by acausality, and at the same time consider a key thermodynamic quantity and nontrivial part of the total entropy production, which already provides important physical insight into the thermodynamics of delayed systems. This strategy enables us to address several fundamental questions: What is the impact of time-delayed feedback on heat exchange and entropy production? Has the overdamped limit consequences for the thermodynamic description? Do thermodynamic quantities reflect delay-induced dynamical behaviour? An example are spontaneous oscillations occurring in many nonlinear delayed systems due to their infinite dimensionality31,32.

To this end, we consider a simple exemplary system composed of an overdamped particle subject to nonlinear static forces and a deterministic (i. e., error-free), continuous linear feedback force with a discrete time delay τ ≥ 0. Such a force is imposed, e. g., by optical tweezers33,34. The idealized heat bath is assumed to remain at equilibrium, unaffected by the feedback. Our calculation explicitly predicts that delay alone induces a finite heat flow whose direction is tunable. This means, in particular, that the delayed force can generate a steady heat flow from the bath to the particle, i. e., feedback cooling. We moreover unravel an important universal heat flow caused by the nontrivial interplay between delay and noise, which also occurs for underdamped motion. We discuss its physical origin in detail and propose that this heat flow is linked to the entropy pumping known for underdamped systems with velocity-dependent feedback, i. e., “molecular refrigerators”27,35,36. We further detect discontinuous behaviour at τ → 0. Discussing the application to a paradigmatic bistable system37, we evaluate the heat rate via several approximations and by numerical simulations. In particular, we consider the medium entropy production near coherence resonance32,38,39,40 (CR), that is, the appearance of regular positional oscillations at a finite thermal energy caused by the interplay of nonlinearity, noise, and delay41,42,43,44. Importantly, in contrast to stochastic resonance, there is no periodic external driving. Combining the discretized (Master) equation approach of41 with ST, we show that the medium entropy production has a maximum at CR, and provide an analytical explanation for this maximum, which has, so far, only been detected numerically44.

Model

We consider stochastic processes described by the overdamped Langevin equation (LE)45

$$\begin{array}{cc}\gamma {\rm{d}}X(t)= & {F}_{{\rm{c}}{\rm{o}}{\rm{n}}}[X(t)]{\rm{d}}t+{F}_{{\rm{d}}}[X(t-\tau )]{\rm{d}}t+\gamma \sqrt{2{D}_{0}}\xi (t){\rm{d}}t\\ = & -\sum _{i=1}^{m}{a}_{i}X{(t)}^{i}{\rm{d}}t-bX(t-\tau ){\rm{d}}t+\gamma \sqrt{2{D}_{0}}\xi (t){\rm{d}}t,\end{array}$$
(1)

with \({a}_{i},b\in {\mathbb{R}}\), where the total deterministic force F(x, xτ) = Fcon(x) − bxτ depends on the instantaneous, X(t), and on the delayed particle position X(t − τ). We assume that the conservative force Fcon can be expressed as a polynomial (which holds indeed for a wide class of nonlinear potentials), while the feedback control Fd is chosen to be linear corresponding, e. g., optical tweezers33. ξ denotes Gaussian white noise with \(\langle \xi (t)\rangle =0\) and 〈ξ(t)ξ(t′)〉 = δ(t − t′), while γ and D0 are the friction and diffusion coefficients satisfying46 \(\gamma {D}_{0}\,=\,{k}_{{\rm{B}}}T\), with kB being the Boltzmann constant. Due to the appearance of the delayed position in (1), the corresponding Fokker-Planck equation (given at the end of this report in the Technical aspects section) for the probability density function ρ(x, t) (PDF) is an infinite hierarchy45,47,48,49. We consider natural boundary conditions, i. e., ρ = ∂xρ → 0 for x → ±∞, and focus on NESSs, where ∂tρ(x, t) = 0. Following the ST framework of Sekimoto1,30, the fluctuating heat δq flowing to the reservoir during the infinitesimal time dt, the increment of internal energy du, and the work δw done by the nonconservative (delayed) forces, are given by

$$\delta q(t)=\gamma [{\rm{d}}X(t)/{\rm{d}}t-\sqrt{2{D}_{0}}\,\xi (t)]\circ {\rm{d}}X(t),$$
(2)
$${\rm{d}}u(t)=-\,{F}_{{\rm{con}}}[X(t)]\circ {\rm{d}}X(t),$$
(3)
$$\delta w(t)={\rm{d}}u(t)+\delta q(t)=-bX(t-\tau )\circ {\rm{d}}X(t\mathrm{).}$$
(4)

The ○-symbol indicates usage of Stratonovich calculus. Plugging the LE (1) into Eqs (3, 4) results in the NESS ensemble averages

$$\dot{U}\equiv {\langle \frac{{\rm{d}}{\rm{u}}}{{\rm{d}}{\rm{t}}}\rangle }_{{\rm{s}}{\rm{s}}}=\frac{1}{\gamma }\,\sum _{i=1}^{m}\,\{\,{a}_{i}\gamma \sqrt{2{D}_{0}}{\langle {X}^{i}\xi \rangle }_{{\rm{s}}{\rm{s}}}-{a}_{i}b\,{C}_{i}(\tau )-{a}_{i}^{2}{\langle {X}^{2i}\rangle }_{{\rm{s}}{\rm{s}}}-\sum _{j > i}^{m}2{a}_{i}{a}_{j}{\langle {X}^{i+j}\rangle }_{{\rm{s}}{\rm{s}}}\,\},$$
(5)
$$\dot{W}\equiv {\langle \frac{\delta w}{{\rm{d}}{\rm{t}}}\rangle }_{{\rm{s}}{\rm{s}}}=\frac{b}{\gamma }\{\sum _{i=1}^{m}{a}_{i}{C}_{i}(\tau )+b{\langle {X}^{2}\rangle }_{{\rm{s}}{\rm{s}}}-\gamma \sqrt{2{D}_{0}}{\langle X(t-\tau )\xi (t)\rangle }_{{\rm{s}}{\rm{s}}}\}.$$
(6)

Due to the time delay, the thermodynamic quantities depend on the spatial autocorrelation functions at time difference τ, \({C}_{i}(\tau )={\langle X{(t)}^{i}X(t-\tau )\rangle }_{{\rm{ss}}}\), mirroring the non-Markovian nature of (1).

Heat rate and medium entropy production

For nonlinear systems, the linear response function method used in30, which bases on the Laplace-transformation, cannot be applied. In the following, we present an alternative approach for the heat rate. In particular, we derive exact expressions for (5, 6) which only involve positional moments at one time. To this end, we utilize two relations that can be derived by projecting the Fokker-Planck equation onto the positional moments and subsequently inserting the LE50.

First, the Ci(τ)-terms can be substituted by the relation50 (valid \(\forall n\ge 1\))

$$b\,{C}_{n}(\tau )=\gamma {D}_{0}\,n{\langle {X}^{n-1}\rangle }_{{\rm{s}}{\rm{s}}}-\,\sum _{i=1}^{m}{a}_{i}{\langle {X}^{n+i}\rangle }_{{\rm{s}}{\rm{s}}}.$$
(7)

Second, the instantaneous noise–position cross correlations in (5) can be replaced by positional moments via50

$$\langle X{(t)}^{n}\xi (t)\rangle =n\,\sqrt{{D}_{0}/2}\,\langle X{(t)}^{n-1}\rangle ,\,{\rm{\forall }}\,n\ge 1.$$
(8)

A derivation of Eqs (7, 8) can be found at the end of this report in the Technical aspects section. Finally, we combine Eq. (8) with a causality argument to evaluate \(\langle X(t-\tau )\xi (t)\rangle \) at τ ≠ 0. Because no physical quantity can be influenced by future noise, statistical independence follows and \(\langle X(t)\xi (t^{\prime} )\rangle =\langle X(t)\rangle \langle \xi (t^{\prime} )\rangle \equiv 0\) must hold \(\forall \,t^{\prime} > t\). Hence,

$$\langle X(t-\tau )\xi (t)\rangle =\sqrt{\frac{{D}_{0}}{2}}{\delta }_{\tau }=\{\begin{array}{cc}\sqrt{{D}_{0}/2}, & \tau =0\\ 0, & \tau > 0\end{array}.$$
(9)

Substituting Eqs (7) and (8) at n = 1, 2, .., m, respectively, into Eq. (5) yields \({\langle {\rm{d}}u/{\rm{d}}t\rangle }_{{\rm{ss}}}=0\). This is expected, since the net impact of conservative forces should vanish in a NESS2. Thus \(\dot{W}={\langle \delta q/{\rm{d}}t\rangle }_{{\rm{ss}}}\equiv \dot{Q}\). Substituting (7) at n = 1, 2, .., m into Eq. (6) (please note that this simplification step is possible due to the polynomial form of force Fcon) and using (9) further yields

$$\begin{array}{cc}\dot{Q} & =\sum _{i=1}^{m}i{a}_{i}{D}_{0}{\langle {X}^{i-1}\rangle }_{{\rm{s}}{\rm{s}}}-\sum _{i=1}^{m}\sum _{j=1}^{m}\,\frac{{a}_{i}{a}_{j}}{\gamma }{\langle {X}^{i+j}\rangle }_{{\rm{s}}{\rm{s}}}+\frac{{b}^{2}}{\gamma }{\langle {X}^{2}\rangle }_{{\rm{s}}{\rm{s}}}\,-b{D}_{0}{\delta }_{\tau }\\ & =\dot{W}={\dot{S}}_{{\rm{m}}}\gamma {D}_{0}/{k}_{{\rm{B}}}.\end{array}$$
(10)

Equation (10) is an exact expression only involving one-time ensemble averages over Xn. \(\dot{Q}\) and \({\dot{S}}_{{\rm{m}}}\) can therefore be computed directly from the steady-state one-time PDF, and hence, on the basis of several approximations known from the literature. The δτ-term suggests discontinuous behaviour of \(\dot{Q}\) at τ → 0. But in order to study this limit properly, one also has to investigate the behaviour of the PDF to clarify whether the moments behave continuously. For nonlinear systems, this is a nontrivial task on its own, as, in fact, no exact solutions for the one-time PDF in the presence of delay are known. However, we can nevertheless address this question analytically, since the approximative PDFs become exact in the Markovian limits45,47, rendering exact results from Eq. (10).

A detailed description of the approximation schemes is beyond the scope of this report. We refer the interested reader to45 and references therein, and here only review some main aspects in brief. The small τ approximation47 is obtained by a first-order Taylor expansion around τ = 0 in the LE, effectively rendering a Markovian system (with exponentially decaying correlations). Two other schemes start from the Fokker-Planck equation for the one-time PDF, which involves the two-time PDF47 ρ2(x, t; x′, t − τ) (see Technical aspects section). The force-linearisation closure45 (FLC) approximates ρ2 by the one from the corresponding linear delayed system. The perturbation theory48,49 (PT) approximates it with ρ2 of the corresponding Markovian process without delay force. For a doublewell potential, the PT requires an additional approximation, since the corresponding two-time PDF is not known. As in45, we use the small-time propagator (PT-st), or the Ornstein-Uhlenbeck (PT-OU) approximation. In the following, we will investigate the heat rate in the two (Markovian) limits, where the approximations become exact.

Limit of Vanishing Feedback Strength

In the limits where Markovianity is recovered, one expects that the system equilibrates (because of the absence of a driving force), and therefore \(\dot{Q}={\dot{S}}_{{\rm{m}}}=0\). The first limit corresponds to b → 0, i. e., vanishing feedback strength. By construction, the one-time steady-state PDFs from the FLC and from the PT become exact at b → 0. They converge in a continuous manner to the equilibrium (Boltzmann) distribution \({\rho }_{{\rm{ss}}}^{b=0}\), as can be easily seen from the PT-st result45 \({\rho }_{{\rm{ss}}}^{{\rm{PT}}}\):

$${\rho }_{{\rm{s}}{\rm{s}}}^{{\rm{P}}{\rm{T}}}(x)=Z\exp \{-\frac{[1-(b\tau /\gamma )][{V}_{{\rm{s}}}(x)+(b/2){x}^{2}]}{\gamma {D}_{0}}\}\mathop{\longrightarrow }\limits^{b\to 0}{\rho }_{{\rm{s}}{\rm{s}}}^{b=0}(x)=Z\exp [-\frac{{V}_{{\rm{s}}}(x)}{\gamma {D}_{0}}],$$
(11)

with \({V}_{{\rm{s}}}^{^{\prime} }(x)=-\,{F}_{{\rm{con}}}(x)={\sum }_{i=1}^{m}{a}_{i}{x}^{i}\) and normalization constant Z. After performing a partial integration step and plugging in the natural boundary conditions, the (i − 1)st positional moment at τ = 0 can therewith be expressed as

$$\begin{array}{cc}{\langle {X}^{i-1}\rangle }_{{\rm{s}}{\rm{s}}}= & {\int }_{-{\rm{\infty }}}^{{\rm{\infty }}}\,Z\exp [-\frac{{V}_{{\rm{s}}}(x)}{\gamma {D}_{0}}]{x}^{i-1}{\rm{d}}x\\ = & {[{\rho }_{{\rm{s}}{\rm{s}}}^{b=0}(x)\frac{{x}^{i}}{i}]}_{-{\rm{\infty }}}^{{\rm{\infty }}}-{\int }_{-{\rm{\infty }}}^{{\rm{\infty }}}\,\frac{{F}_{{\rm{c}}{\rm{o}}{\rm{n}}}(x)}{\gamma {D}_{0}}Z\exp [-\frac{{V}_{{\rm{s}}}(x)}{\gamma {D}_{0}}]\frac{{x}^{i}}{i}{\rm{d}}x\\ = & \sum _{j=1}^{m}\,\frac{{a}_{j}}{i\gamma {D}_{0}}{\int }_{-{\rm{\infty }}}^{{\rm{\infty }}}{x}^{j}{\rho }_{{\rm{s}}{\rm{s}}}^{b=0}(x){x}^{i}{\rm{d}}x=\sum _{j=1}^{m}\,\frac{{a}_{j}}{i\gamma {D}_{0}}{\langle {X}^{j+i}\rangle }_{{\rm{s}}{\rm{s}}},\,{\rm{\forall }}\,i\ge 1\end{array}$$
(12)
$$\Rightarrow \sum _{i=1}^{m}i{a}_{i}{D}_{0}{\langle {X}^{i-1}\rangle }_{{\rm{ss}}}=\sum _{i=1}^{m}\sum _{j=1}^{m}\frac{{a}_{j}{a}_{i}}{\gamma }{\langle {X}^{j+i}\rangle }_{{\rm{ss}}}.$$
(13)

Plugging the last expression into Eq. (10), we finally conclude that \({\mathrm{lim}}_{b\to 0}{\dot{S}}_{{\rm{m}}}=0\) in a continuous manner.

Limit of vanishing delay time

The other relevant limit is that of vanishing delay time, τ → 0. The one-time PDF45 from the small τ expansion47

$${\rho }_{{\rm{s}}{\rm{s}}}^{s\tau }(x)=Z\exp [-\frac{{V}_{{\rm{s}}}(x)+(b/2){x}^{2}}{(\gamma +b\tau ){D}_{0}}]\mathop{\longrightarrow }\limits^{\tau \to 0}{\rho }_{{\rm{s}}{\rm{s}}}^{\tau =0}(x)=Z\exp [-\frac{{V}_{{\rm{s}}}(x)+(b/2){x}^{2}}{\gamma {D}_{0}}]$$
(14)

clearly becomes exact for τ → 0, where it continuously converges to the Boltzmann distribution \({\rho }_{{\rm{ss}}}^{\tau =0}\). Performing steps analogously to Eq. (12), we obtain

$$\begin{array}{cc}{\langle {X}^{i-1}\rangle }_{{\rm{s}}{\rm{s}}} & ={[Z{\rho }_{{\rm{s}}{\rm{s}}}^{\tau =0}(x)\frac{{x}^{i}}{i}]}_{-{\rm{\infty }}}^{{\rm{\infty }}}-{\int }_{-{\rm{\infty }}}^{{\rm{\infty }}}\frac{{F}_{{\rm{c}}{\rm{o}}{\rm{n}}}(x)-bx}{i\gamma {D}_{0}}{x}^{i}{\rho }_{{\rm{s}}{\rm{s}}}^{\tau =0}(x){\rm{d}}x\\ & =\sum _{j=1}^{m}\frac{{a}_{j}}{i\gamma {D}_{0}}{\langle {X}^{j+i}\rangle }_{{\rm{s}}{\rm{s}}}+\frac{b}{i\gamma {D}_{0}}{\langle {X}^{i+1}\rangle }_{{\rm{s}}{\rm{s}}},\end{array}$$
(15)

for all i ≥ 1 at τ → 0, which results in the identities

$$\begin{array}{cc}\sum _{i=1}^{m}i\,{a}_{i}{D}_{0}{\langle {X}^{i-1}\rangle }_{{\rm{s}}{\rm{s}}} & =\sum _{i=1}^{m}\sum _{j=1}^{m}\frac{{a}_{j}{a}_{i}}{\gamma }{\langle {X}^{j+i}\rangle }_{{\rm{s}}{\rm{s}}}+\sum _{i=1}^{m}\frac{b{a}_{i}}{\gamma }{\langle {X}^{i+1}\rangle }_{{\rm{s}}{\rm{s}}},\\ {\langle {X}^{0}\rangle }_{{\rm{s}}{\rm{s}}} & \equiv 1=\frac{b}{\gamma {D}_{0}}{\langle {X}^{2}\rangle }_{{\rm{s}}{\rm{s}}}+\sum _{j=1}^{m}\frac{{a}_{j}}{\gamma {D}_{0}}{\langle {X}^{j+1}\rangle }_{{\rm{s}}{\rm{s}}}.\end{array}$$
(16)

Plugging both into Eq. (10) in an iterative manner, yields the discontinuous limit

$$\mathop{{\rm{l}}{\rm{i}}{\rm{m}}}\limits_{\tau \to 0}{\dot{S}}_{{\rm{m}}}=\mathop{{\rm{l}}{\rm{i}}{\rm{m}}}\limits_{\tau \to 0}\frac{{k}_{{\rm{B}}}b}{\gamma }(1-{\delta }_{\tau })=\{\begin{array}{cc}0, & \tau =0\\ {k}_{{\rm{B}}}b/\gamma , & \tau > 0\end{array}.$$
(17)

This is our first main result. This ubiquitous jump-discontinuity at τ → 0 (see Fig. 1 for an example) indicates an abrupt qualitative change of the thermodynamics when non-Markovianity sets in. It arises due to the discontinuity of the noise–position cross correlations (9) at the onset of causal relationship. Remarkably, the apparent offset of kBb/γ is independent of the details of the potential landscape. We note that the apparent offset has already been observed and discussed in the context of linear systems27,28,29,30. In30 it has been considered as a consequence of inconsistent usage of Ito and Stratonovich calculus. However, as shown here, it is not a mathematical error but also arises within consistently applied Stratonovich calculus. The underlying reason is the interplay of white noise and delay below the short (ballistic) relaxation timescale51, which becomes relevant as τ → 0. To further elucidate the behaviour at τ → 0, we also briefly consider the limit in the underdamped case, where the LE

$$m\ddot{X}(t)=-\,\gamma \dot{X}(t)+{F}_{{\rm{c}}{\rm{o}}{\rm{n}}}[X(t)]-bX(t-\tau )+\gamma \sqrt{2{D}_{0}}\,\xi (t)({\rm{u}}{\rm{n}}{\rm{d}}{\rm{e}}{\rm{r}}{\rm{d}}{\rm{a}}{\rm{m}}{\rm{p}}{\rm{e}}{\rm{d}}\,{\rm{L}}{\rm{E}}),$$
(18)

additionally involves an inertial term with mass m, yielding ballistic motion below the velocity-relaxation timescale, γ/m. For this system, the heat can, in principle, be calculated using the same expression as in the overdamped case, see Eqs (2, 4). As has been shown previously27 for linear systems, \(\dot{Q}\) smoothly decays to zero with τ. However, the response function method used in27 is only applicable to linear systems. For a better comparison, we here consider the correlation \(\langle X(t-\tau )\xi (t)\rangle \) appearing in Eq. (6), which, in our framework, “causes” the discontinuity from a mathematical point of view [compare Eqs (9, 17)]. As opposed to the overdamped case, this correlation is indeed continuous at τ = 0,

$$\langle X(t-\tau )\xi (t)\rangle =0,\,{\rm{\forall }}\,\tau \ge 0$$
(19)
Figure 1
figure 1

Scaled NESS mean heat rate \({\dot{Q}}^{\ast }=\dot{Q}{x}_{0}^{2}/(\gamma {D}_{0}^{2})\) and medium entropy production rate \({\dot{S}}_{{\rm{m}}}^{\ast }={\dot{S}}_{{\rm{m}}}{x}_{0}^{2}/({k}_{{\rm{B}}}{D}_{0})\) vs. delay time, in the delayed bistable system at V0 = 6γD0, k = 0.2γD0obtained from numerics (BD) and from several approximations; and in the corresponding linearised system from Eq. (20). (a) logarithmic x-axis, b. + c. linear plots: (b) zoom out, (c) magnification.

Full size image

(not only for linear systems), indicating that \(\dot{Q}\) behaves continuously. The derivation of (19) is given in the Technical aspects section.

Turning back to the overdamped system, apart from the discontinuous limit, we find from Eq. (17) that for τ > 0, \({\dot{S}}_{{\rm{m}}}\) and \(\dot{Q}\) are nonzero, proving the true non-equilibrium nature of this steady state. According to (17), the rates have negative values at small τ, if b < 0, which is always the case in an optical tweezers setup (see below). The negative signs indicate a steady heat flow from the bath to the particle (feedback cooling). This is a delay-induced phenomenon, which would be impossible in the Markovian counterpart of this system due to the second law36. The fact that here \({\dot{S}}_{{\rm{m}}} < 0\) (the bath constantly loses entropy) underlines that further entropic terms must contribute to the (nonnegative) total entropy production (as discussed for underdamped dynamics in27,28,29). In the cooling regime, the amount of medium entropy loss thus provides a lower bound to the entropic cost27,28,29 of the feedback. Analytical evaluation of the heat rate for (linear and nonlinear) example systems, which we will provide below, reveals that the (negative) heat flow is actually quite pronounced in the regime of small τ (compare with Fig. 1). Notably, this is also true for underdamped linear systems27 (where \(\dot{Q}\) only decays for τ below the velocity-relaxation timescale, compare with Fig. 1 in27). Before proceeding with the concrete examples, we here propose an explanation of this phenomenon.

Discussion of the behaviour for small delay times

When τ is so small that \(X(t-\tau )\approx X(t)-\tau \dot{X}(t-\tau )\), the feedback force Fd = −bX(t − τ) gets a contribution proportional to the strongly fluctuating velocity. This changes the induced steady-state heat flow \(\dot{Q}=\langle {F}_{{\rm{d}}}\,\circ \,\dot{X}(t)\rangle \) significantly. For b < 0, \({F}_{{\rm{d}}}\sim \tau b\dot{X}(t-\tau )\) is a friction-like force. As is well-known for underdamped systems with velocity-dependent (delayed27 or non-delayed35,36) feedback, such a control amounts to medium entropy reduction due to entropy pumping. (Please note that a non-delayed velocity-dependent feedback \({F}_{{\rm{d}}}\sim b\dot{X}(t)\) drives a system out of equilibrium35,36, contrary to a non-delayed position-dependent control). The reason is that the additional friction-like force reduces the thermal fluctuations of the particle, inducing a net energy transfer from the bath to the particle, hence, a net heat flow. (Reversely, the particle fluctuations are enhanced for b > 0, yielding a positive heat flow.) This means, an additional “entropy pumping” contribution to the heat flow arises, explaining the enlarged heat flow at small delay times. Please note that this effect is independent of the question whether the overdamped limit is used, or not, and should be measurable in experimental setups.

When τ gets below the velocity-relaxation time (m/γ), both system behave differently. For underdamped dynamics, the velocities are correlated, implying that the delay-induced heat flow \(\dot{Q}\sim \tau \,b\langle \dot{X}(t-\tau )\dot{X}(t)\rangle \sim \tau \,{e}^{-\gamma \tau /m}\) eventually decays smoothly to zero (resulting in a maximum of \(|\dot{Q}|\) around τ = m/γ, see Fig. 1 in27). For overdamped dynamics, the velocities are, in contrast, uncorrelated (because the white noise directly acts on it \(\dot{X}\sim \xi \)), giving rise to the nontrivial limit τδ(τ) of type “0 × ∞”. This yields a finite value as τ → 0, as we know from Eq. (17). Hence, in an experimental setup, a position-dependent feedback is expected to induce heat flow, unless the (typically unavoidable) delay is well-below the velocity-relaxation timescale.

Linear delayed system

For linear systems, where the force has the form F = −a1x − bxτ, the exact NESS PDFs are known52, and (10) simplifies to the exact and closed expression

$$\frac{\dot{Q}}{{D}_{0}}=\frac{\gamma {\dot{S}}_{{\rm{m}}}}{{k}_{{\rm{B}}}}={a}_{1}-b{\delta }_{\tau }-({a}_{1}^{2}-{b}^{2})\frac{1+b\sinh (\frac{\tau }{\gamma }\sqrt{{a}_{1}^{2}-{b}^{2}})/\sqrt{|{a}_{1}^{2}-{b}^{2}|}}{{a}_{1}+b\cosh (\frac{\tau }{\gamma }\sqrt{{a}_{1}^{2}-{b}^{2}})}.$$
(20)

A plot of \(\dot{Q}\) as a function of τ is given in Fig. 1. As expected, \(\dot{Q}\) has an apparent offset at τ = 0 and \(|\dot{Q}|\) grows while τ decays to zero (until a saturation value is reached). Equation (20) is consistent with30 (where a different approach is used that only applies to linear systems), apart from the additional δτ-term. We stress that, consequentially, only Eq. (20) correctly predicts \({\dot{S}}_{{\rm{m}}}(\tau \to \mathrm{0)}=0\).

Application to a delayed bistable system

We now consider a bistable system composed of a doublewell potential \({V}_{{\rm{s}}}={V}_{{\rm{0}}}[(x/{x}_{0}{)}^{4}-\mathrm{2(}x/{x}_{0}{)}^{2}]\) with minima at ±x0 and potential barrier height V0, supplemented by the delayed (optical tweezers33) potential Vd = (k/2)[(x/x0) − (xτ/x0)]2 with 0 < k < 4V0. The potentials apply a polynomial force \(F=-\,{\rm{d}}({V}_{{\rm{s}}}+{V}_{{\rm{d}}})/{\rm{d}}x=-{a}_{3}{x}^{3}-{a}_{1}x-b{x}_{\tau }\) with coefficients \(b=-\,k{x}_{0}^{-2}\), \({a}_{1}=(\,-\,4{V}_{{\rm{0}}}+k){x}_{0}^{-2}\), \({a}_{3}=4{V}_{{\rm{0}}}{x}_{0}^{-4}\). This is a prototypical nonlinear noisy system which exhibits both: a nontrivial intrawell dynamics within the asymmetric potential wells, and a noise- and delay-induced dynamical state, i. e., positional oscillations between the wells41,42,43,44. For this system, Eq. (10) involves the even moments up to 6th order. Performing Brownian dynamics (BD) simulations (see Technical aspects section for details) to compute the NESS heat rate, both, from Eqs (2) and (10), we indeed find perfect agreement. In the following, we compare the BD results to those obtained from our analytical approach, i. e., Eq. (10) combined with established approximations. Since the approximations are known to perform best when the particle is likely to stay around a potential minimum45, this approach seems appropriate in the low thermal energy regime (\(\gamma {D}_{0}\ll {V}_{0}\)). At the end, we will introduce a complementary approach for larger noise levels.

Low thermal energy – intrawell dynamics

Figure 1 shows \(\dot{Q}=\dot{W}\) and \({\dot{S}}_{{\rm{m}}}\), as functions of the delay time τ, for an exemplary parameter setting in the low noise regime. The simulation results confirm the discontinuous τ-limit and apparent offset as given by Eq. (17), and the predicted feedback cooling. The PT-OU predictions are quantitatively very similar to the ones obtained by the FLC, while the PT-st yields very similar results to the small τ expansion. We will therefore mainly discuss the FLC and the small τ expansion in the following.

While the small τ expansion fails outside the Markovian limit, our analytical approach with FLC makes quantitatively correct predictions for all delay times considered in Fig. 1. The thermodynamic quantities grow with τ, until they approach constant values. Interestingly, the saturation occurs at about \(\tau \approx {x}_{0}^{2}\mathrm{/(2}{D}_{0})\), i. e., on the timescale where the mean-squared displacement 2D0t of a freely diffusing particle is in the range of the extent of a potential well ≈x0 (i. e., the particle has explored the whole well within τ). \({x}_{0}^{2}\mathrm{/(2}{D}_{0})\) is thus an estimate of the intrawell relaxation time trel in the delayed system. Figure 1 also displays the results from the corresponding linear system, F = −a1x − bxτ [with a1 = (8V0 + k)/x0 and b = −k/x0, as given by a second order Taylor expansion of Vs + Vd around a (deterministically) stable fix point, i. e., (x, xτ) = ±(x0, x0)]. Interestingly, \(\dot{Q}\) is not only equivalent to the nonlinear case for very small τ [as expected from (17)], but also saturates on the same timescale.

The agreement between BD and FLC persists at larger barrier heights V0 or smaller k. However, for larger k/(γD0), also the FLC approach breaks down, as can be seen in Fig. 2. The inset shows that the FLC nevertheless captures the qualitative behaviour. Upon increase of k (i. e., the laser intensity in case of optical tweezers33), one may switch from feedback cooling to heating in the nonlinear case.

Figure 2
figure 2

Scaled NESS mean heat rate \({\dot{Q}}^{\ast }=\dot{Q}{x}_{0}^{2}/(\gamma {D}_{0}^{2})\) and medium entropy production rate \({\dot{S}}_{{\rm{m}}}^{\ast }={\dot{S}}_{{\rm{m}}}{x}_{0}^{2}/({k}_{{\rm{B}}}{D}_{0})\) vs. delay force strength k over thermal energy γD0; at V0 = 4γD0, \(\tau ={x}_{0}^{2}/{D}_{0}\). Inset: zoom out. Colour code as in Fig. 1.

Full size image

High thermal energy – interwell dynamics

When the thermal energy is sufficiently high, such that jump processes between the potential wells dominate the dynamics, the discussed approximations all break down by construction. The reason is that it is then the “memory of a jump” expressed in the non-Markovian ρ2, which dominates the process and, e. g., triggers subsequent jumps, and precisely ρ2 is what is (over)simplified in all approaches45. In particular, the FLC effectively assumes an harmonic potential well out of which no jumps can occur, while the PT and small τ expansion render Markovian systems which don’t exhibit spontaneous oscillations, as the latter are a delay-induced phenomenon. This includes situations, where the interplay of noise and delay leads to spontaneous oscillations of X. However, we can still treat this regime via an alternative strategy, which bases on the discretised approach for multistable systems proposed by Tsimring and Pikovsky41. To this end, the X-dynamics is reduced to switching processes between two discrete states s = ±x0 (corresponding to the two potential wells). There are two different transition rates corresponding to the two possible scenarios that the delayed and instantaneous state have the same, or opposite sign, respectively. These transition rates are approximated by the Kramers formula41,45,53,54 within a quasistatic approximation. The latter is justified when the delay time is small compared to the intrawell relaxation time, \(\tau \gg {t}_{{\rm{rel}}}\), and γD0 < V0. Further details including the calculation of the transition rates are given at the end of this report in the Technical aspects section.

The simplification of Eq. (6) due to the 2-state reduction is twofold. First, the spatial correlation function \({C}_{1}({\rm{\Delta }}t)={\langle {X}^{i}(t)X(t-{\rm{\Delta }}t)\rangle }_{{\rm{ss}}}\) can now be calculated analytically41. Second, all other spatial autocorrelation functions are significantly simplified, since the intrawell-dynamics are omitted, yielding for τ > 0

$$\langle X{(t)}^{2}\rangle \to \langle s(t)s(t)\rangle =\langle {x}_{0}^{2}\rangle ={x}_{0}^{2}$$
(21)
$${C}_{i}(\tau )\to \langle {s}^{i}(t)s(t-\tau )\rangle =\{\begin{array}{cc}{x}_{0}^{i}\langle s(t-\tau )\rangle =0, & {\rm{\forall }}\,{\rm{e}}{\rm{v}}{\rm{e}}{\rm{n}}\,i\\ {x}_{0}^{i-1}\langle s(t)s(t-\tau )\rangle ={C}_{1}(\tau )\,{x}_{0}^{i-1}, & {\rm{\forall }}\,{\rm{o}}{\rm{d}}{\rm{d}}\,i.\end{array}$$
(22)

[Please note that the correlation \(\langle X(t-\tau )\xi (t)\rangle \to 0\) at τ > 0 due to causality]. Consequentially, Eq. (6) simplifies to

$$\dot{W}=(b/\gamma )({a}_{1}+{x}_{0}^{2}{a}_{3}){C}_{1}(\tau )+({b}^{2}/\gamma ){x}_{0}^{2}={k}^{2}/(\gamma {x}_{0}^{2})[1-{C}_{1}(\tau )/{x}_{0}^{2}]\mathop{=}\limits^{!}\dot{Q}={\dot{S}}_{{\rm{m}}}\gamma {D}_{0}/{k}_{{\rm{B}}},$$
(23)

where we have used that the mean change of internal energy must vanish in the NESS. Using the explicit formula for C1(τ) [see Eqs (23, 34)], can readily be evaluated. The results are plotted in Fig. 3.

Figure 3
figure 3

Analysis of the dependency on the thermal energy γD0 scaled with the barrier height V0, in the delayed bistable system at k = 0.4V0, \(\tau =25\,\gamma {x}_{0}^{2}/{V}_{0}\), and in corresponding linearised system. (a) Scaled NESS heat rate \({\dot{Q}}^{\ast }=\dot{Q}(\gamma {x}_{0}^{2}/{V}_{0}^{2})\) (the inset gives a magnification), (b). Medium entropy production rate \({\dot{S}}_{{\rm{m}}}^{\ast }={\dot{S}}_{{\rm{m}}}(\gamma {x}_{0}^{2}/{V}_{0}{k}_{{\rm{B}}})\). Magenta lines and y-axes: 2-state model. Vertical cyan lines: maximum of CR order parameter (see Fig. 4).

Full size image

The numerical data in Fig. 3 reveals that, when the thermal energy is small compared to V0, \(\dot{Q}\) increases linearly with γD0, as it also does for a linear delayed system [see Eq. (20)]. In this regime, approximating the doublewell by its linearised version (dashed line) even renders the correct slope. By contrast, in the 2-state model \(\dot{Q}=0\), as expected, since the intrawell dynamics is neglected. At larger γD0/V0, the slope of \(\dot{Q}\) abruptly changes and nonlinear behaviour sets in. This occurs at the onset of the delay-induced oscillations, as reflected by a sudden increase of the CR order parameter from the 2-state model41 and as confirmed by numerical simulations (see Fig. 4). The order parameter measures the height of the main peak at a frequency of about 2π/τ in the power spectrum of C1(Δt). Hence, a nonzero order parameter indicates the occurrence of subsequent escape events with period τ.

Figure 4
figure 4

Detecting coherence resonance (CR) in the bistable system. (a) Analytical power spectrum (from 2-state model) given by the Fourier transform of C1(Δt) =  \({\langle X(t)X(t-{\rm{\Delta }}t)\rangle }_{{\rm{ss}}}\) [Eq. (34)] at three thermal energies at and around the value related to CR (0.36). (b) Order parameter: Normalized main peak at ω ≈ 2π/τ of the power spectrum of C1, from BD and from the 2-state model. The vertical line at 0.36γD0/V0 indicates the maximum around which the system is coherence-resonant.

Full size image

By further increasing γD0/V0, one enters the regime where delay-induced oscillations dominate the dynamics. At a certain finite thermal energy (of about 0.36 γD0/V0 in Figs 3 and 4, see cyan vertical lines and cyan plot in Fig. 4a.), the peak in the power spectrum is highest, resulting in a maximum of the order parameter and indicating that the delay-induced oscillations with mean period ≈τ are most pronounced. In this range of thermal energies, the escape times are comparable with the delay time τ, and the pronounced peak in the power spectrum signals the resonant response of the system to the noise. Please note that the escape rates can be estimated by the Kramers rates in the corresponding system without delay (τ = 0) (see Technical aspects section). We find that in this range of γD0/V0, the 2-state reduction renders a good approximation. In particular, it predicts accurately that a region of steep slope is followed by a lower slope of \(\dot{Q}\) accompanied by a maximum of \({\dot{S}}_{{\rm{m}}}\), and that the latter lies in the regime of CR. This is our second main result.

Analytical explanation for the maximum of entropy production at CR

The behaviour of \(\dot{Q}\) and \({\dot{S}}_{{\rm{m}}}\) can be understood on the basis of the 2-state model. The delay-induced oscillations set in and pause randomly. They have mean period τ, such that C1(τ) = 1 for a perfect oscillation (as in the case of no jumps). However, due to their stochastic nature, C1(τ) is lowered with the occurrence of oscillatory events. When the timescales of noise-induced escapes and oscillatory motion become comparable, the particle dynamics responds resonantly to the noise. Slightly increasing γD0/V0 then significantly increases the number of occurring oscillation periods, yielding a strong reduction of C1(τ). \(\dot{Q}\) thus steeply increases and \({\dot{S}}_{{\rm{m}}}\) reaches high values [Eq. (23)]. At the CR maximum (vertical cyan lines in Figs 3, 4), this effect saturates resulting in a reduced slope of \(\dot{Q}\) and a maximum of \({\dot{S}}_{{\rm{m}}}\) at CR.

For even higher γD0, the superimposing noise generates irregular, low correlated motion with C1(τ) → 0, hence \({\dot{S}}_{{\rm{m}}}\to 0\). In contrast, \({\dot{S}}_{{\rm{m}}}\) of the full system approaches a (nonzero) constant (not shown here). The breakdown of the 2-state approximation in this limit is indeed expected since the state discretisation then becomes meaningless. We have tested several parameter settings confirming that the \({\dot{S}}_{{\rm{m}}}\) is indeed maximal at CR conditions as predicted by the 2-state model. Quantitatively, we always observe an overestimation of the thermodynamic quantities by a factor of about three. This is somewhat surprising since the phase-space reduction inherent to the discretisation is rather expected to yield an underestimation of the steady state quantities3. A possible explanation lies in the quasistatic approximation, but the precise reason is subject of future investigations.

Conclusions

In this work we put forward analytical approaches for the heat rate in a prototypical nonlinear delayed system. We have shown that the heat rate can be evaluated based on positional moments, implying that, despite the inherent memory, no temporal correlations are needed. Our formula can be combined with established approximations for the one-time PDF. We have presented analytical results for the Markovian limits and thereby observed a growing heat flow for vanishing delay time. We have further uncovered discontinuous behaviour at the onset of memory, which can also be found for underdamped motion when the delay appears in the velocity. This finite heat flow at small τ-values above and around the (ballistic) velocity-relaxation timescale, is a consequence of the interplay of noise and delay, while the discontinuity at τ → 0 is a consequence of the white noise assumption and the overdamped limit. For experimental realisations of feedback traps this theoretical result implies that the (unavoidable) delay causes a finite steady-state heat flow unless the delay is significantly smaller than the (ballistic) relaxation timescale.

Although the here presented investigations are restricted to static energy landscapes, we indeed expect the results to also hold for time-dependent external potentials, at least, if their changes are slow compared to the other dynamics. Feedback loops offer a way to precisely control the effective temperature of a particle by suppressing its thermal fluctuations55. When this technique is used to change temperature, e. g., during the cyclic process of a heat engine56, the here discussed heat flow, which is caused even by tiny delays, might play a non-negligible role and should therefore be taken into account in the theoretical calculations of efficiency and entropy production.

Moreover, we have presented an approach capturing jump processes, thereby predicting that the medium entropy production due to the feedback is maximal, when the delay-induced oscillations are coherence-resonant. This work is an important step towards an understanding of thermodynamic notions in delayed systems. Future work will focus on the (non-Gaussian) heat distributions P(q), which appear to violate fluctuation relations29 and on developing approaches for the total entropy production, possibly via Markovian embedding techniques57. We hope that our current findings will stimulate experimental investigations on passive (or even active) colloidal systems, e. g., a validation of the predicted switching from feedback cooling to heating by adjusting the delay force strength, which is tunable in experimental setups.

Technical Aspects

Fokker-Planck equation

The Fokker-Planck equation of the considered non-Markovian system (1) is an infinite hierarchy of coupled equations45,47,48, whose first member reads

$$\gamma {{\rm{\partial }}}_{t}{\rho }_{1}(x,t)=-\,{{\rm{\partial }}}_{x}{\int }_{-{\rm{\infty }}}^{{\rm{\infty }}}F(x,{x}_{\tau }){\rho }_{2}({x}_{\tau },t-\tau ;x,t){\rm{d}}{x}_{\tau }+\gamma {D}_{0}{{\rm{\partial }}}_{xx}{\rho }_{1}(x,t).$$
(24)

Apart from the one-time PDF, ρ1, Eq. (24) also involves the two-time joint PDF, ρ2, and is hence not self-sufficient.

Derivation of relations between correlation functions (7,8)

In order to derive (7), we project the Fokker-Planck equation (24) onto its moments by multiplying it with xn+1, n ≥ 0, and integrating over the spatial domain x ∈ (−∞,∞). Several partial integrations, (where the boundary terms vanish) yield

$$\gamma \frac{{\rm{d}}}{{\rm{d}}t}\frac{\langle X{(t)}^{n+1}\rangle }{(n+1)}=\gamma {D}_{0}n\langle X{(t)}^{n-1}\rangle -\sum _{i=1}^{m}{a}_{i}\langle X{(t)}^{n+i}\rangle -b\langle X{(t)}^{n}X(t-\tau )\rangle .$$
(25)

In a NESS, the left side vanishes and (25) reduces to a relation between positional moments and the spatial autocorrelation function, as given in (7). On the other hand, one can deduce directly by plugging in the LE (1)

$$\begin{array}{cc}\frac{{\rm{d}}}{{\rm{d}}t}\frac{\langle X{(t)}^{n+1}\rangle }{(n+1)} & =\langle X{(t)}^{n}\frac{{\rm{d}}X(t)}{{\rm{d}}t}\rangle \\ & \mathop{=}\limits^{{\rm{L}}{\rm{E}}}-\sum _{i=1}^{m}\frac{{a}_{i}}{\gamma }\langle X{(t)}^{i+n}\rangle +\sqrt{2{D}_{0}}\langle X{(t)}^{n}\xi (t)\rangle -\frac{b}{\gamma }\langle X{(t)}^{n}X(t-\tau )\rangle .\end{array}$$
(26)

A comparison of Eqs (26) and (25) readily provides the surprisingly simple and generic relation (8), which, in fact, generally holds for any (nonlinear) force (as can be shown analogously).

Derivation of noise–position (19) and noise–velocity cross correlations for underdamped dynamics

We aim to calculate the correlations \(\langle X(t-\tau )\xi (t)\rangle \) and \(\langle \dot{X}(t-\tau )\xi (t)\rangle \) for underdamped motion with position-dependent feedback. To this end, we first we use again the causality argument58 that the noise cannot influence past position or past velocity. This implies that both correlations must vanish at τ > 0. Second, we rewrite the correlations at τ = 0 as follows, starting from a formal integration over the LE (18)

$$m\dot{X}(t)=-\,\gamma X(t)+\sum _{i}{a}_{i}{\int }_{0}^{t}X{(s)}^{i}{\rm{d}}s-b{\int }_{0}^{t}X(s-\tau ){\rm{d}}s+\gamma \sqrt{2{D}_{0}}{\int }_{0}^{t}\xi (t){\rm{d}}s+{\mathscr{C}}$$
(27)
$$\begin{array}{cc}\Rightarrow m\langle \dot{X}(t)\xi (t)\rangle = & -\gamma \langle X(t)\xi (t)\rangle +\sum _{i}{\alpha }_{i}{\int }_{0}^{t}\langle X{(s)}^{i}\xi (t)\rangle {\rm{d}}s\\ & -b{\int }_{0}^{t}\langle X(s)\xi (t)\rangle {\rm{d}}s+\gamma \sqrt{2{D}_{0}}{\int }_{0}^{t}\langle \xi (s)\xi (t)\rangle {\rm{d}}s\\ = & -\gamma \langle X(t)\xi (t)\rangle +\sum _{i}{a}_{i}{\int }_{0}^{t}\langle X{(t)}^{i}\xi (t)\rangle {\delta }_{s,t}{\rm{d}}s\\ & -b{\int }_{0}^{t}\langle X(t)\xi (t)\rangle {\delta }_{s,t}{\rm{d}}s+\gamma \sqrt{{D}_{0}/2},\end{array}$$
(28)

where we have simplified the integrals of types \({\int }_{0}^{t}\langle X{(s)}^{i}\xi (t)\rangle {\rm{d}}s\) by using the causality argument, which implies that the integrands are zero for all s < t The constant \({\mathscr{C}}\) amounts for the initial conditions. Third, using the identity \(X(t)={\int }_{0}^{t}\dot{X}(s){\rm{d}}s+X\mathrm{(0)}\), we find

$$\langle X(t)\xi (t)\rangle ={\int }_{0}^{t}\langle \dot{X}(s)\xi (t)\rangle {\rm{d}}s={\int }_{0}^{t}\langle \dot{X}(t)\xi (t)\rangle {\delta }_{s,t}{\rm{d}}s.$$
(29)

Fourth, we establish that Eqs (28, 29) only allows for finite solutions. This can be shown by contradiction, starting with the assumption \(\langle \dot{X}(t)\xi (t)\rangle \to \infty \). Eq. (29) then yields a finite value of \(\langle X(t)\xi (t)\rangle \), which, on the other hand, yields a finite value of \(\langle \dot{X}(t)\xi (t)\rangle \) from Eq. (29), thus a contradiction. Using this finiteness, we finally obtain for τ ≥ 0

$$\langle X(t-\tau )\xi (t)\rangle =0,$$
(30)
$$\langle \dot{X}(t-\tau )\xi (t)\rangle =(\gamma /m)\sqrt{{D}_{0}/2}\,{\delta }_{\tau }.$$
(31)

Numerical simulations

We perform Brownian dynamics simulations with Euler-Maruyama integration scheme. The temporal discretisation is varied between Δt = 10−6 and 10−4, such that the typical time scales are all properly resolved, in particular, the delay time τ > 1000Δt and the intrawell relaxation time trel > 1000Δt. Furthermore, steady-state ensemble averages are obtained after appropriate transient times (at least 5 × 106 time steps) have been cut off, and by using a sufficiently high number of realisations (at least 104). Both is checked by insensitivity against further increase of simulated time or sample size. Numerical integrations are performed in Stratonovich calculus.

Escape times

The mean escape time τK over a barrier of height ΔU of a Markovian system with potential U can estimated by the Arrhenius formula41,45,53,54

$${\tau }_{{\rm{K}}}^{-1}=p({x}_{min})={(2\pi \gamma )}^{-1}\sqrt{|{U}^{{\rm{^{\prime} }}{\rm{^{\prime} }}}({x}_{max})|{U}^{{\rm{^{\prime} }}{\rm{^{\prime} }}}({x}_{min})}\exp [-{\rm{\Delta }}U/(\gamma {D}_{0})],$$
(32)

where p(xmin) denotes the escape rate out of xmin. When using the corresponding system without delay (τ = 0) to estimate the escape times in the controlled bistable system, one finds from (32) \({\tau }_{K}=2\pi \gamma {x}_{0}^{2}(\sqrt{32}{V}_{0}{)}^{-1}{e}^{{V}_{0}/{D}_{0}\gamma }\), which means τK = τ at γD0/V0 = 0.32. Hence, around the CR peak, the escape times are comparable with the delay time, as confirmed by our BD results.

Reduction to 2-state model

Within the 2-state model, the particle being in one of the two potential wells is represented by one discrete state s = ±x0. The transition rates pj∈{1,2} between the states dependent on the delayed position, with p1 if s(t)s(t − τ) > 0, and p2 if s(t)s(t − τ) < 0. Within a quasistatic approximation (appropriate if \(\tau \gg {t}_{{\rm{rel}}}\) and γD0 < V0), we consider the corresponding quasistatic potential Vqs(x) = Vs(x) + Vd(x, xτ = ±x0), obtained by fixing xτ at one of the two deterministically stable solutions ±x0. The transition rates can then be approximated by the two Kramers rates associated with Vqs, given by (32) with pj = p(xmin,j) for escapes out of the two minima, xmin,1 = ±x0 and \({x}_{{\rm{\min }},{\rm{2}}}=\mp ({x}_{0}/2)[1+\sqrt{1-(k/{V}_{0})}]\approx \mp {x}_{0}\), over the barrier at \({x}_{{\rm{\max }}}=\mp ({x}_{0}\mathrm{/2)[1}-\sqrt{1-(k/{V}_{0})}]\approx 0\). In the last step we have used twice \(\sqrt{1-(k/{V}_{0})}\approx 1\) for \(k\ll {V}_{0}\). The corresponding second derivatives can be calculated exactly as \({V}_{{\rm{qs}}}^{^{\prime\prime} }({x}_{{\rm{\max }}})=-\mathrm{(4}{V}_{0}-k)/{x}_{0}^{2}\) and \({V}_{{\rm{qs}}}^{^{\prime\prime} }({x}_{\min ,j\in \{{\rm{1}},{\rm{2}}\}})=\mathrm{(8}{V}_{0}+k)/{x}_{0}^{2}\), while the potential barrier heights read ΔVqs,1 = V0 + k/2, and ΔVqs,2 = V0 − 3k/2. This finally yields the transition rates

$${p}_{j\in \{1,2\}}={(2\pi \gamma {x}_{0}^{2})}^{-1}\sqrt{(8{V}_{0}+k)(4{V}_{0}-k)}\exp [\,-\,{\rm{\Delta }}{V}_{{\rm{q}}{\rm{s}},j}/(\gamma {D}_{0})],$$
(33)

which are the only ingredients needed to calculate C1(Δt) from the formula41

$${C}^{1}({\rm{\Delta }}t)=\frac{(\sqrt{{p}_{1}}+\sqrt{{p}_{2}})\,{e}^{-2\sqrt{{p}_{1}{p}_{2}}{\rm{\Delta }}t}+(\sqrt{{p}_{1}}-\sqrt{{p}_{2}})\,{e}^{-2\sqrt{{p}_{1}{p}_{2}}(\tau -{\rm{\Delta }}t)}}{\sqrt{{p}_{1}}+\sqrt{{p}_{2}}+(\sqrt{{p}_{1}}-\sqrt{{p}_{2}})\,{e}^{-2\sqrt{{p}_{1}{p}_{2}}\tau }}.$$
(34)