Inferring broken detailed balance in the absence of observable currents

Identifying dissipation is essential for understanding the physical mechanisms underlying nonequilibrium processes. In living systems, for example, the dissipation is directly related to the hydrolysis of fuel molecules such as adenosine triphosphate (ATP). Nevertheless, detecting broken time-reversal symmetry, which is the hallmark of dissipative processes, remains a challenge in the absence of observable directed motion, flows, or fluxes. Furthermore, quantifying the entropy production in a complex system requires detailed information about its dynamics and internal degrees of freedom. Here we introduce a novel approach to detect time irreversibility and estimate the entropy production from time-series measurements, even in the absence of observable currents. We apply our technique to two different physical systems, namely, a partially hidden network and a molecular motor. Our method does not require complete information about the system dynamics and thus provides a new tool for studying nonequilibrium phenomena.

I rreversibility is the telltale sign of nonequilibrium dissipation 1,2 . Systems operating far-from-equilibrium utilize part of their free energy budget to perform work, while the rest is dissipated into the environment. Estimating the amount of free energy lost to dissipation is mandatory for a complete energetics characterization of such physical systems. For example, it is essential for understanding the underlying mechanism and efficiency of natural Brownian engines, such as RNA-polymerases or kinesin molecular motors, or for optimizing the performance of artificial devices [3][4][5] . Often the manifestation of irreversibility is quite dramatic, signaled by directed flow or movement, as in transport through mesoscopic devices 6 , traveling waves in nonlinear chemical reactions 7 , directed motion of molecular motors along biopolymers 8 , and the periodic beating of a cell's flagellum 9,10 or cilia 11 . This observation has led to a handful of experimentally validated methods to identify irreversible behavior by confirming the existence of such flows or fluxes 3,[12][13][14] . However, in the absence of directed motion, it can be challenging to determine if an observed system is out of equilibrium, especially in small noisy systems where fluctuations could mask any obvious irreversibility 15 . One such possibility is to observe a violation of the fluctuation-dissipation theorem [16][17][18] ; though this approach requires not just passive observations of a correlation function, but active perturbations in order to measure response properties, which can be challenging in practice. Thus, the development of noninvasive methods to quantitatively measure irreversibility and dissipation are necessary to characterize nonequilibrium phenomena.
Our understanding of the connection between irreversibility and dissipation has deepened in recent years with the formulation of stochastic thermodynamics, which has been verified in numerous experiments on meso-scale systems [19][20][21][22] . Within this framework, it is possible to evaluate quantities as the entropy along single nonequilibrium trajectories 23 . A cornerstone of this approach is the establishment of a quantitative identification of dissipation, or more specifically entropy production rate _ S, as the Kullback-Leibler divergence (KLD) between the probability Pðγ t Þ to observe a trajectory γ t of length t and the probability Pðγ t Þ to observe the time-reversed trajectoryγ t 1,24-29 : where k B is Boltzmann's constant. The KLD between two probability distributions p and q is defined as D½pjjq P x pðxÞ ln ðpðxÞ=qðxÞÞ and is an information-theoretic measure of distinguishability 30 . For the rest of the paper we take k B = 1, so the entropy production rate has units of time −1 . The entropy production _ S in Eq. (1) has a clear physical meaning. It is the usual entropy production defined in irreversible thermodynamics by assuming that the reservoirs surrounding the system are in equilibrium. For instance, in the case of isothermal molecular motors hydrolyzing ATP to ADP+P at temperature T, the entropy production in Eq. (1) is _ S ¼ rΔμ=T À _ W=T, where r is the ATP consumption rate, Δμ = μ ATP − μ ADP − μ P is the difference between the ATP, and the ADP and P chemical potentials, and _ W is the power of the motor 31 . In many experiments, all these quantities can be measured except the rate r. Therefore, the techniques that we develop in this paper can help to estimate the ATP consumption rate, even at stalling conditions.
The equality in Eq. (1) is reached if the trajectory γ t contains all the meso-and microscopic variables out of equilibrium. Hence the relative entropy in Eq. (1) links the statistical time-reversal symmetry breaking in the mesocopic dynamics directly to dissipation. Based on this connection, estimators of the relative entropy between stationary trajectories and their time reverses allow one to determine if a system is out of equilibrium or even bound the amount of energy dissipated to maintain a nonequilibrium state. Such an approach, however, is challenging to implement accurately as it requires large amounts of data, especially when there is no observable current 32 .
Despite the absence of observable average currents, irreversibility can still leave a mark in fluctuations. Consider, for example, a particle hoping on a 1D lattice, as in Fig. 1, where up and down jumps have equal probabilities, but the timing of the jumps have different likelihoods. Although there is no net drift on average, the process is irreversible, since any trajectory can be distinguished from its time reverse due to the asymmetry in jump times. Thus, beyond the sequence of events, the timing of events can reveal statistical irreversibility. Such a concept was used, for example, to determine that the E. Coli flagellar motor operates out of equilibrium based on the motor dwell-time statistics 33 .
In this work, we establish a technique that allows one to identify and quantify irreversibility in fluctuations in the timing of events, by applying Eq. (1) to stochastic jump processes with arbitrary waiting time distributions, that is, semi-Markov processes, also known as continuous time random walks (CTRW) in the context of anomalous diffusion. Such models emerge in a plethora of contexts [34][35][36] ranging from economy and finance 37 to biology, as in the case of kinesin dynamics 38 or in the anomalous diffusion of the Kv2.1 potassium channel 39 . In fact, as we show below and in the Methods section, semi-Markov processes result in experimentally relevant scenarios where one has access only to a limited set of observables of Markov kinetic networks with certain topologies. We begin by reviewing the semi-Markov framework, where we present our main result of the entropy production rate estimator. Next, we apply our approach to general hidden networks, where an observer has access only to a subset of the states, comparing our estimator with previous proposals for partial entropy production that are zero in the absence of currents. Finally, we address a particularly important case of molecular motors, where their translational motion is easily observed, but the biochemical reactions that power their motion are hidden. Remarkably, our technique allows us to even reveal the existence of parasitic mechano-chemical cycles at stalling-where the observed current vanishes or the motor is stationary-simply from the distribution of step times. In addition, our quantitative lower bound on the entropy production rate can be used to shed light on the efficiency of molecular motors operation and on the entropic cost of maintaining their far-from-equilibrium dynamics [40][41][42][43][44] .

Results
Irreversibility in semi-Markov processes. A semi-Markov process is a stochastic renewal process α(t) that takes values in a discrete set of states, α = 1, 2, …. The renewal property implies  Fig. 1 Brownian particle jumping on an one-dimensional lattice. Jumps up and down are equally likely, but with asymmetric jump rates. As a result, the irreversibility of the dynamics is contained solely in the timing fluctuations that the waiting time intervals t α in a given state α are positive, independent, and identically distributed random variables. If the system arrives to state α at t = 0, the probability to jump to a different state β at time [t, t + dt] is ψ βα (t)dt, with ψ βα (t) being the probability density of transition times 45 . These densities are not normalized, with p βα R 1 0 ψ βα ðtÞdt being the probability for the next jump to be α → β given that the walker arrived at α. We assume that the particle eventually leaves any site α, i.e., ψ αα (t) = 0 and P β p βα ¼ 1, so the matrix p βα is a stochastic matrix. Its normalized (right) eigenvector R α with eigenvalue 1, then represents the fraction of visits to each state α.
Consider now a generic semi-Markovian trajectory γ t of length t with n jumps, which is fully described by the sequence of jumps and jump times, occurring with probability Pðγ t Þ ¼ ψ α 2 ;α 1 ðt 1 Þψ α 3 ;α 2 ðt 2 Þ ψ α nþ1 ;α n ðt n Þ. In order to characterize the dissipation of this single trajectory, we must define its time ity is given by Pðγ t Þ ¼ ψ α 0 ;α 1 ðt 1 Þ ψ α nÀ1 ;α n ðt n Þ, see Methods and (1) to this scenario shows that the KLD between the probability distributions of the forward and backward trajectories can be split into two contributions (see Methods): The first term, _ S aff , or affinity entropy production, results entirely from the divergence between the state trajectories, regardless of the jump times, σ ≡ {α 1 , α 2 ,…,α n+1 } and σ fα n ; ; α 1 ; α 0 g, that is, it accounts for the affinity between states: where J ss βα ¼ p βα R α À p αβ R β is the net probability flow per step, or current, from α to β, and the factor T ¼ P α τ α R α is the mean duration of each step, which can be used to transform the units from per-step to per-time 46 . We see that the affinity entropy production vanishes in the absence of currents, as it occurs in arbitrary Markov systems 32,47 .
The contribution due to the waiting times is expressed in terms of the KLD between the waiting time distributions which is the main result of this paper and allows one to detect irreversibility in stationary trajectories with zero current. Notice that R α being the occupancy of state α, p βα R α is the probability to observe the sequence α → β in a stationary forward trajectory, while p μβ p βα R α is the probability to observe the sequence α → β → μ.
Equation (2) is the chain rule of the relative entropy applied to the semi-Markov process and the core of our proposed estimator. In the special case of Poisson jumps, D[ψ(t|β → μ)||ψ(t|β → α)] = 0 since all waiting time distributions for jumps starting at a given site β are equal (see Methods), and we recover the standard expression for the relative entropy of Markov processes _ S ¼ _ S aff . It is worth mentioning that previous attempts to establish the entropy production of semi-Markov processes failed to identify the term S WTD because they assumed that the waiting time distributions were independent of the final state, as occurs in Markov processes [48][49][50] . However, such a strong assumption does not hold in many situations of interest, as in the ones discussed below.
Decimation of Markov chains and second-order semi-Markov processes. Semi-Markov processes appear when sites are decimated from Markov chains of certain topologies. Figure 2 shows representative examples. In Fig. 2a, b, we show two models of a molecular motor that runs along a track with sites {…, i − 1, i, i + 1, …} and has six internal states. If the spatial jumps (red lines) and the transitions between internal states (black lines) are Poissonian jumps, then the motor is described by a Markov process. On the other hand, when the internal states are not accessible to the experimenter, the waiting time distributions corresponding to the spatial jumps i → i ± 1 are no longer exponential and the motion of the motor must be described by a semi-Markov process. Figure 2a shows an example where the decimation of internal states directly yields a semi-Markov process ruling the spatial motion of the motor. The second example, sketched in Fig. 2b, is more involved since the upward and the downward jumps end in different sets of internal states. As a consequence, the waiting time distribution of, say, the jump i → i + 1, depends on the site that the motor visited before site i. Then, the resulting dynamics must be described by a second-order semi-Markov process, that is, one has to consider the states is the current position of the motor and i prev (t) is the previous position, right before the jump.
The same applies to generic kinetic networks, as the one depicted in Fig. 2c. Suppose that the original network is Markovian with states i = 1, …, 5. However, if the experimenter only has access to states 1 and 2, with the rest clumped together into a hidden state H, then the resulting dynamics is also a second-order semi-Markov process with the reduced set i = 1, 2, H.
For second-order semi-Markov processes the affinity entropy production reads is the probability to observe the sequence i→j→k. This entropy is still proportional to the current for one-dimensional processes and therefore vanishes in the absence of flows in the observed dynamics, see Methods. The entropy production contribution due to the irreversibility of the waiting time distributions is Let us emphasize that the calculation of _ S WTD requires collecting statistics on sequences of two consecutive jumps, i.e., i → j → k. We now proceed to apply these results to generic cases of simple kinetic networks and molecular motors.
Hidden networks. We first apply our formalism to estimate the dissipation in kinetic networks with hidden states, which have received increasing attention in recent years owing to their many practical and experimental implications 24,32,47,[51][52][53] .
Consider a network where ω ij is the transition rate from state j to i, with π i the steady-state distribution. The total entropy production rate at steady-state is 54 where the positivity of _ S stems from the positivity of each individual term in the sum 40,52,55 . In order to calculate the total entropy production _ S according to Eq. (7), full knowledge of the steady-state probability distribution {π i } and the transition rates between all the microstates {ω ij } is required. We would like to assign a partial entropy production rate when one only has access to a limited set of states and transitions. To be concrete, we focus on the scenario depicted in Fig. 2c, where only states 1 and 2 can be observed. Previously, two approaches for assigning partial entropy production rate in such a case have been defined in the literature, both of which provide a lower bound on the total entropy production rate 56 : the passive partial entropy production rate due to Shiraishi and Sagawa 52 , and the informed partial entropy production rate due to Polettini and Esposito 53,57 . The passive partial entropy production rate _ S PP for the single observed link is simply given by the corresponding term in Eq. (7) where the observer is assumed to have access to the steady-state populations of the two states, π 1 and π 2 , as well as the transition rates between them. The informed partial entropy production _ S IP for the single link requires additional information: the observer is assumed to have control over the transition rates of the observed link, without affecting any of the hidden transitions, such that they can stall the corresponding current and record the ratio of populations in the two observed states, π stall 1 =π stall 2 . The stalling distribution π stall i produces an effective thermodynamic description of the observed subsystem 53 and an effective affinity with which the informed partial entropy production rate is calculated: Although the informed partial entropy production was proven to produce a better estimation of the total dissipation compared with the passive partial entropy production, i.e., _ S PP _ S IP _ S 56 , both vanish at stalling conditions. Hence, even if the system is in a nonequilibrium steady-state, when the current over the observed link is zero, these estimators cannot give a nontrivial lower bound on the total entropy production. To be fair, we point out that each estimator uses different information.
For the KLD estimator, we assume that the observer can record whether the system is in states 1 or 2, or in the hidden part of the network, H, which is a coarse-grained state representing the unobserved subsystem. In this case, the resulting contracted network has three states, {1, 2, H}. Jumps between states 1 and 2 follow Poissonian statistics, as in a general continuous-time Markov process, with the same rates as in the original network. On the other hand, jumps from H to 1 or 2 are not Poissonian and depend on the state just prior to entering the hidden part. To apply our results for semi-Markov processes, we thus have to consider the states α(t) = [i prev (t) i(t)], where i(t) = 1, 2, H is the current state and i prev (t) = 1, 2, H is the state right before the last jump. To make the equations more compact, we will use the short-hand notation i j ≡ [i j] for the remainder of this section.
Similar to Eq. (2), the semi-Markov entropy production rate for hidden networks, _ S KLD , consists of two contributions: the affinity estimator _ S aff and the WTD estimator _ S WTD . In this case, the affinity estimator, Eq. (5), is given by where J ss 21 is the stationary current per step from 1 to 2, defined as J ss 21 ¼ R ½12 À R ½21 . As expected, this term vanishes when detailed balance holds and the current is zero (see Methods). Applying Eq. (6) to the semi-Markov process results in the following expression for the contribution of the hidden estimator In Methods, we further show that for a network of a single cycle of states the informed partial entropy production _ S IP equals the affinity estimator _ S aff defined in Eq. (5). Summarizing, we have the hierarchy Let us apply the hidden semi-Markov entropy production framework to a specific example of a network with four states, two of which are hidden (Fig. 3a). We have chosen a random 4 × 4 matrix, with non-negative off diagonal entries and zero sum columns, as a generator of a continuous-time Markov jump process over the four states. The rates over the observed link were varied according to ω 12 (F) = ω 12 e βFL and ω 21 (F) = ω 21 e −βFL over a range of values of a force F that included the stalling force F stall , where β = 1/T is the inverse temperature and L is a characteristic length scale. For each value of F, we contracted the dynamics to the three states, 1, 2, and H (Fig. 3b, c), and estimated the waiting time distributions ψ(t| 2 H → H 1) and ψ(t| 1 H → H 2) using a kernel density estimate with a positive support 58,59 (see Methods), depicted in Fig. 3d. From those distributions, we derived the hidden semi-Markov entropy production rate _ S KLD (Fig. 3e). We further calculated both the passive-and informed-partial entropy production rates to compare all the estimators to the total entropy production rate (Fig. 3e). Our results clearly demonstrate the advantage of using the waiting time distributions for bounding the total entropy production rate compared with the two other previous approaches. Our framework can reveal the irreversibility and the underlying dissipation, even when the observed current vanishes, without the need of manipulating the system.
The KLD entropy production rate was also estimated from simulated experimental data, obtained by sampling random trajectories of 10 7 jumps using the Gillespie algorithm 60 . The simulated trajectories (Fig. 3b) were coarse-grained into the set of states of the hidden semi-Markov model (Fig. 3c), and the hidden semi-Markov entropy production rate for the simulated experimental data, _ S Exp KLD , was estimated as above (Fig. 3e, blue crosses). In order to assess the rate of convergence with increasing number of simulated steps, we calculated the _ S Exp KLD for different fractions of the 10 7 steps trajectories, showing <20% error above 10 5 steps at stalling, and <5% error away from stalling for trajectories with as little as 10 4 steps (Fig. 3f, g). Let us stress that the hidden semi-Markov entropy production rate averaged over three simulated experimental trajectories produced a lower bound on the total entropy production rate, which was strictly positive and statistically significant different from zero (p < 0.05, Fig. 3g, inset) for all trajectory lengths tested.

Molecular motors.
A slight modification of the case analyzed in the previous section allows us to study molecular motors with hidden internal states. We are interested in the schemes previously sketched in Fig. 2a, b, where a motor can physically move in space or switch between internal states. The observed motor position is labeled by {..., i − 1, i, i + 1, ...}. All jumps are Poissonian and obey local detailed balance, with an external source of chemical work, Δμ, and an additional mechanical force F that can act only on the spatial transitions.
Analogous to the previous example, the observed dynamics is a second-order semi-Markov process. To make the following equations more intuitive, we use the graphical notation for two consecutive upward jumps (i − 1 → i → i + 1), for a downward jump followed by and upward one, for an upward followed by a downward jump, and for two consecutive downward jumps. Notice that the probabilities are normalized as . Similar to Eq. (2), we have the decomposition of the KLD estimator into a contribution from state affinities given by where the current per step is ) corresponding to the occupancy rate of states moving upward (downward). The contribution due to the relative entropy between waiting time distributions is As in the previous examples, the latter term can produce a lower bound on the total entropy production rate even in the absence of observable currents, in which case _ S aff ¼ 0. Without chemical work (Δμ = 0), however, the waiting time distributions of the and processes become identical and the contribution of _ S WTD vanishes as well.
Let us apply the molecular motor semi-Markov entropy production framework to a specific example. We consider the following two-state molecular motor model of a power stroke engine that works by hydrolizing ATP against an external force F, see Fig. 4a.
The state of the motor is described by its physical position and its internal state, which can be either active, that is, capable of hydrolyzing ATP, or passive. We label the active and passive states as i′ and i, respectively, with i = 0, ±1, ±2, …. Owing to the translational symmetry in the system, all the spatial positions are essentially equivalent. The position of the motor is accessible to an external observer, whereas the two internal states i and i′ are indistinguishable. An example of a trajectory is illustrated in Fig. 4b.
The chemical affinity Δμ, arising from ATP hydrolysis, determines the degree of nonequilibrium in our system and biases the transitions i′↔i + 1, whereas the external force F affects all the spatial transitions, regardless of the internal state. The transition rates between the two internal states are defined as ω i′i = ω ii′ = k s . Transition rates between passive states obey local detailed balance: ω i,i+1 /ω i+1,i = e βFL , where L is the length of a single spatial jump. From the active state, the system can use the ATP to move upward with rates verifying local detailed balance ω i′,i+1 /ω i+1,i′ = e β(FL−Δμ) .
c d e f g Fig. 3 Hidden network. a Four-state network, as seen by an observer, with access only to states 1, 2, H. b, c Illustration of a trajectory over the four possible states (b) where the gray region corresponds to the hidden part. The resulting observed semi-Markov dynamics (c). d Kernel density estimation of the wait time distributions at F = F stall . e Estimated total entropy production rate _ S (solid red line), entropy production for semi-Markov model _ S KLD (dashed blue curve), informed partial entropy production rate _ S IP (dashed-dotted black curve), the passive partial entropy production rate _ S PP (dotted green curve), and the experimental entropy production rate estimated according to the semi-Markov model _ S Exp KLD (blue crosses). f, g Relative error (ratio of experimental entropy production rate to analytical value) for three random trajectories as a function of the number of steps at F = F stall (f) and F = 3β −1 L −1 (g) showing faster convergence away from the stalling force. Inset: p-value for rejecting the null hypothesis that the experimental data was sampled from a zero mean distribution as a function of the number of steps for F = F stall (blue curve), and F = 3β −1 L −1 (red curve), showing that the average is statistically significant different from zero. The numerical simulations were done using the Gillespie algorithm with the following transition rates: The resulting waiting time distributions are shown in Fig. 4c, d, and the estimated entropy production rates as a function of external force are depicted in Fig. 4e, with chemical potential ranging from Δμ = 0 β −1 to 10 β −1 . The total entropy production rate _ S is calculated using Eq. (7). As expected, the dissipation increases with the nonequilibrium driving force, and vanishes when Δμ = FL = 0. Notice that the affinity estimator _ S aff does not provide a lower bound to the total entropy production rate _ S at stalling, as it is not statistically different from zero (Fig. 4f), and thus cannot distinguish between nonequilibrium and equilibrium processes. In contrast, the semi-Markov estimator _ S KLD , which accounts for the asymmetry of the waiting time distributions provides a nontrivial positive bound, even in the absence of observable current.

Discussion
We have analytically derived an estimator of the total entropy production rate using the framework of semi-Markov processes. The novelty of our approach is the utilization of the waiting time distributions, which can be non-Poissonian, allowing us to unravel irreversibility in hidden degrees of freedom arising in any time-series measurement of an arbitrary experimental setup. Our estimator can thus provide a lower bound on the total entropy production rate even in the absence of observable currents. Hence, it can be applied to reveal an underlying nonequilibrium process, even if no net current, flow, or drift, are present. We stress that our method fully quantifies irreversibility. Owing to the direct link between the entropy production rate and the relative entropy between a trajectory and its time reversal, as manifested in Eq. (1), our estimator provides the best possible bound on the dissipation rate utilizing time irreversibility. One can consider utilizing other properties of the waiting time distribution to bound the entropy production, through the thermodynamics uncertainty relations 4,61,62 , for example.
We have illustrated our method with two possible applications: a situation where only a subsystem is accessible to an external observer and a molecular motor whose internal degrees of freedom cannot be resolved. Using these examples, we have demonstrated the advantage of our semi-Markov estimator compared with other entropy production bounds, namely, the passive-and informed-partial entropy production rates, both of which vanish at stalling conditions.
In summary, we have developed an analytic tool that can expose irreversibility otherwise undetectable, and distinguish between equilibrium and nonequilibrium processes. This framework is completely generic and thus opens opportunities in numerous experimental scenarios by providing a new perspective for data analysis.

Methods
Semi-Markov processes, waiting time distributions and steady states. A semi-Markov stochastic process is a renewal process α(t) with a discrete set of states α = 1, 2, …, N. The dynamics is determined by the probability densities of transition times ψ βα (t), which are defined as ψ βα (t)dt being equal to the probability that the system jumps from state α to state β in the time interval [t, t + dt] if it arrived at site α at time t = 0. By definition ψ αα (t) = 0. When the system is a particle jumping between the sites of a lattice, the semi-Markov process is also called a CTRW. For clarity, we will assume this CTRW picture, that is, the system in our discussion will be a particle jumping between sites α.
The probability densities ψ βα (t) are not normalized: is the probability that, given that the particle arrived at site α, the next jump is α → β. We will assume that the particle eventually leaves any site α, i.e., P β p βα ¼ 1. Then is normalized and it is the probability density of the residence time at site α. It is also called the waiting time distribution. Its average S aff offers a lower bound constrained by the statistical uncertainty due to the finite amount of data (green shaded region). Calculations were done using the parameters k s = 1 s −1 , k 0 = 0.01 s −1 , and the trajectories were sampled using the Gillespie algorithm 61 is the mean residence time or mean waiting time. We can also define the waiting time distribution conditioned on a given jump α → β, which is normalized. The function ψ βα (t) is in fact the joint probability distribution of the time t and the jump α → β. The transition probabilities p βα determine a Markov chain given by the visited states α 1 , α 2 , α 3 , ..., regardless of the times when the jumps occur. The transition matrix of this Markov chain is {p βα } and the stationary probability distribution R α verifies i.e., the distribution R α is the right eigenvector of the stochastic matrix {p βα } with eigenvalue 1. Moreover, if the Markov chain is ergodic, then the distribution R α is precisely the fraction of visits the system makes to site α in the stationary regime. Thus, we call R α the distribution of visits. From the distribution of visits one can easily obtain the stationary distribution of the process α(t), since the particle visits the state α a fraction of steps R α and spends an average time τ α in each step. The normalization constant T P α R α τ α is the average time per step.
The stationary current in the Markov chain from state α to β is This is in fact the current per step in the original semi-Markov system since, in an ensemble of very long trajectories, it is the net number of particles that jump from α to β divided by the number of steps. Since the duration of a long stationary trajectory with K steps (K ≫ 1) is KT , the current per unit of time is J ss βα =T . Notice that the average time per step T acts as a conversion factor that allows one to express currents, entropy production, etc. either as per step or as per unit of time.
The Markovian case. If the process α(t) is Markovian, then the jumps are Poissonian and transition time densities are exponential. Let ω βα be the rate of jumps from α to β. The mean waiting time at site α is the inverse of the the total outgoing rate: and the waiting time distributions are ψ βα ðtÞ ¼ ω βα e Àt=τ α ψ α ðtÞ ¼ ψðtjα ! βÞ ¼ e Àt=τ α τ α : ð22Þ with jump probabilities p βα = τ α ω βα . Notice that the waiting time distribution ψ(t|α → β) does not depends on β. The distribution of visits R α verifies and the stationary distribution π α obeys which is the equation for the stationary distribution that one obtains from the master equation Decimation of Markov chains. Semi-Markov processes arise in a natural way when states are removed or decimated from Markov processes with certain topologies. Consider a Markov process where two sites, 1 and 2, are connected through a closed network of states i = 3, 4, … that we want to decimate, as sketched in Fig. 2c. If the observer cannot discern between states i = 3, 4, …, the resulting three-state process with i(t) = 1, 2, H is a second-order semi-Markov chain. We want to calculate the effective transition time distribution ψ decim 21 ðtÞ from state 1 to state 2 in terms of the distributions ψ ij (t) of the initial Markov chain. For this purpose, we have to sum over all possible paths from 1 to 2 through the decimated network.
Consider first the paths with exactly n + 1 jumps, The probability that such a path occurs with an exact duration t is This is a convolution. If one performs the Laplace transform on all timedependent functions, generically denoted by a tilde, which is a sum only over all the decimated sites i, j = 3, 4, … that are connected to sites 1 and 2, respectively. The decimation procedure can be used to derive transition time distributions in a kinetic network when the observer cannot discern among a set of states, say 3, 4, 5,…, that are generically labeled as H for hidden, as in Fig. 2c. For the specific case of the figure, the effective transition time distribution from site 1 to site H, for instance, can be written as where the matrixΨðsÞ is a 3 × 3 matrix corresponding to the Laplace transform of the transition time distributions among sites 3, 4, and 5.
Irreversibility in semi-Markov processes. Here we calculate the relative entropy between a stationary trajectory γ and its time reversalγ in a generic semi-Markov process. A trajectory γ is fully described by the sequence of jumps (see Fig. 5): γ ¼ fðα 1 ! α 2 ; t 1 Þ; ðα 2 ! α 3 ; t 2 Þ; ; ðα nÀ1 ! α n ; t nÀ1 Þ; ðα n ! α nþ1 ; t n Þg ð33Þ and occurs with a probability (conditioned on the initial jump α 0 → α 1 at t = 0) The reverse trajectory is where we assume, for the sake of generality, that states can change under time reversal,α being the time reversal of state α. The probability to observeγ, conditioned on the initial jumpα nþ1 !α n at t = 0, is It is again convenient to consider the forward and backward trajectories without the waiting times, i.e., σ ¼ fα 1 ; α 2 ; α 3 ; ; α n ; α nþ1 g ð 37Þ σ ¼ fα n ;α nÀ1 ; ;α 2 ;α 1 ;α 0 g; ð38Þ and the probability to observe those trajectories are PðσÞ ¼ p α 2 ;α 1 p α 3 ;α 2 p α nþ1 ;α n ð39Þ The initial jumps of γ andγ do not contribute to the entropy production in the stationary regime. Then the relative entropy per jump reads Each time integral can be written as Z 1 0 dt ψ μβ ðtÞ ln ψ μβ ðtÞ where α, β, μ is a substring of the forward trajectory σ (α = α k , β = α k+1 , μ = α k+2 ). Inserting this expression in Eq. (41), Notice that p βα R α is the probability to observe the sequence α, β in the stationary forward trajectory and p μβ p βα R α is the probability to observe the sequence α, β, μ. Finally, we can obtain the expression used in the main text for the entropy production per unit of time dividing by the conversion factor T (average time per step), that is _ S ¼ δS=T . The result is where the entropy production corresponding to the affinity of states reads and the one corresponding to the waiting time distributions is If α ¼α, then the affinity entropy production can be written as which vanishes in the absence of currents.
Second-order semi-Markov processes. A 2nd-order semi-Markov process i(t) also describes the trajectory of a system that jumps among a discrete set of states i = 1, 2, …. However, i(t) is not semi-Markov because the transition time distributions depend on the previous state i prev (t) visited right before the last jump. Hence, the vector α(t) ≡ [i prev (t) i(t)] is indeed a semi-Markov process.
To quantify the irreversibility of a second-order Markov chain, we introduce the time-reversal state of α = [ij], which isα ¼ ½ji. However, this is not enough to reconstruct the backward trajectory, since there is a shift compared with the simple semi-Markov case, as illustrated in Fig. 6. In the forward trajectory, the system spends a time t k in state α k = [i k−1 i k ], with k = 1, …, n, whereas in the backward trajectory it spends the same time t k in stateα kþ1 ¼ ½i kþ1 i k . Consequently, the probabilities of the forward and backward trajectories are, respectively, PðγÞ ¼ ψ α 2 ;α 1 ðt 1 Þψ α 3 ;α 2 ðt 2 Þ ψ α nþ1 ;α n ðt n Þ PðγÞ ¼ ψα 1 ;α 2 ðt 1 Þψα 2 ;α 3 ðt 2 Þ ψα n ;α nþ1 ðt n Þ: Repeating the arguments of the previous section, one obtains The contribution to the entropy production (per step) due to the state affinities now reads ) is the probability to observe the sequence i→j→k in the trajectory and p(ij) = R [ij] is the probability to observe the sequence i→j.
It is interesting to particularize Eq. (51) to a ring with N sites. This is the case of our examples-the hidden network and the molecular motor. In this case, in the stationary regime, pðijkÞ À pðkjiÞ ¼ pðijkÞ þ pðijiÞ À pðijiÞ À pðkjiÞ ¼ pðijÞ À pðjiÞ ¼ J ss ð53Þ since each site has only two neighbors and therefore p(ijk) + p(iji) = p(ij) for any triplet of contiguous sites ijk. Here J ss is the stationary current between any pair of contiguous sites. Hence, we can write the affinity as which is proportional to the current. The argument of the logarithm also vanishes at zero current (see Eq. (57) below); consequently, the affinity entropy tends to zero quadratically when as the force is tuned to the stalling condition. This is the usual behavior in linear irreversible thermodynamics, but recall that for semi-Markov processes the affinity entropy production misses the nonequilibrium signature that is present in the waiting time distributions and is assessed by _ S WTD .
Affinity and informed partial entropy production. Here we show that the informed partial entropy production equals the affinity entropy production for the case of a generic hidden kinetic network proposed in the main text where the observed network forms a single cycle. First, let us generalize the detailed balance condition for a second-order Markov ring with three states, i = 1, 2, H, and zero stationary current. The stationary distribution R [ij] verifies the master Eq. (18): If the current vanishes, R [ij] = R [ji] for all i, j, and these equations reduce to Multiplying the three equations we get the generalized detailed balance condition: In the observable network, the transitions from states 1 and 2 are still Poissonian and independent of the previous state: ...