Abstract
Mathematical models are an essential component of quantitative science. They generate predictions about the future, based on information available in the present. In the spirit of simpler is better; should two models make identical predictions, the one that requires less input is preferred. Yet, for almost all stochastic processes, even the provably optimal classical models waste information. The amount of input information they demand exceeds the amount of predictive information they output. Here we show how to systematically construct quantum models that break this classical bound, and that the system of minimal entropy that simulates such processes must necessarily feature quantum dynamics. This indicates that many observed phenomena could be significantly simpler than classically possible should quantum effects be involved.
Similar content being viewed by others
Introduction
A mathematical model of a system of interest is an algorithmic abstraction of its observable output. Envision that the given system is encased within a black box, such that we observe only its output. Within a second box resides a computer that executes a model of this system with appropriate input. For the model to be accurate, we expect these boxes to be operationally indistinguishable; their output is statistically equivalent, such that no external observer can differentiate which box contains the original system.
There are numerous distinct models for any given system. Consider a system of interest consisting of two binary switches. At each timestep, the system emits a 0 or 1 depending on whether the state of the two switches coincides, and one of the two switches is chosen at random and flipped. The obvious model that simulates this system keeps track of both switches, and thus requires an input of entropy 2. Yet, the output is simply a sequence of alternating 0s and 1s, and can thus be modelled knowing only the value of the previous emission. Intuition suggests that this alternative is more efficient; it demands only an input of entropy 1 (that is, a single bit), when the original model required two. This intuition can be formalized by defining the optimal model of a particular behaviour is the one whose input is of minimal entropy. Indeed, this definition has been already adopted as a principle of computational mechanics^{1,2}.
Efficient mathematical models carry operational consequence. The practical application of a model necessitates its physical realization within a corresponding simulator (Fig. 1). Therefore, should a model demand an input of entropy C, its physical realization must contain the capacity to store that information. The construction of simpler mathematical models for a given process allows potential construction of simulators with reduced informationstorage requirements. Thus, we can directly infer the minimal complexity of an observed process once we know its simplest model. If a process exhibits observed statistics that require an input of entropy C to model, then whatever the underlying mechanics of the observed process, we require a system of entropy C to simulate its future statistics.
These observations motivate maximally efficient models; models that generate desired statistical behaviour, while requiring minimal input information. In this article, we show that even when such behaviour aligns with simple stochastic processes, such models are almost always quantum. For any given stochastic process, we outline its provably simplest classical model. We show that unless improvement over this optimal classical model violates the second law of thermodynamics, our construction and a superior quantum model and its corresponding simulator can always be constructed.
Results
Framework and tools
We can characterize the observable behaviour of any dynamical process by a joint probability distribution , where are random variables that govern the system's observed behaviour, respectively, in the past and the future. Each particular realization of the process has a particular past , with probability . Should there exist a model for this behaviour with an input of entropy C, then we may compress within a system S of entropy C, such that systematic actions on S generates random variables whose statistics obey .
We seek the maximally efficient model, such that C is minimized. As the past contains exactly (the mutual information between past and future) about the future, the model must require an input of entropy at least E (this remains true for quantum systems^{5}). On the other hand, there appears no obvious reason a model should require anything more. We say that the resulting model, where , is ideal. It turns out that for many systems such models do not exist.
Consider a dynamical system observed at discrete times , with possible discrete outcomes dictated by random variables X_{t}. Such a system can be modelled by a stochastic process^{6}, where each realization is specified by a sequence of past outcomes , and exhibits a particular future with probability . Here, , referred to as excess entropy^{7,8}, is a quantity of relevance in diverse disciplines ranging from spin systems^{9} to measures of brain complexity^{10}. How can we construct the simplest simulator of such behaviour, preferably with input entropy of no more than E?
The brute force approach is to create an algorithm that samples from given complete knowledge of . Such a construction accepts directly as input, resulting in the required entropy of , where denotes the Shannon entropy of the complete past. This is wasteful. Consider the output statistics resulting from a sequence of coin flips, such that is the uniform distribution over all binary strings. E equals 0 and yet C is infinite. It should not require infinite memory to mimic a single coin; better approaches exist.
Simplest classical models
ɛmachines are the provably optimal classical solution^{3,4}. They rest on the rationale that to exhibit desired future statistics, a system need not distinguish differing pasts, , if their future statistics coincide. This motivates the equivalence relation, ∼, on the set of all past output histories, such that . To sample from for a particular , a ɛmachine need not store , only which equivalence class, belongs to. Each equivalence class is referred to as a causal state.
For any stochastic process with emission alphabet Σ, we may deduce its causal states that form the state space of its corresponding ɛmachine. At each time step t, the machine operates according to a set of transition probabilities ; the probability that the machine will output , and transition to S_{k} given that it is in state S_{j}. The resulting ɛmachine, when initially set to state , generates a sequence according to probability distribution as it iterates through these transitions. The resulting ɛmachine thus has internal entropy
where S is the random variable that governs and p_{j} is the probability that .
The provable optimality of ɛmachines among all classical models motivates C_{μ} as an intrinsic property of a given stochastic process, rather than just a property of ɛmachines. Referred to in literature as the statistical complexity^{4,11}, its interpretation as the minimal amount of information storage required to simulate such a given process has been applied to quantify selforganization^{12}, the onset of chaos^{3} and complexity of protein configuration space^{13}. Such interpretations, however, implicitly assume that classical models are optimal. Should a quantum simulator be capable of exhibiting the same output statistics with reduced entropy, this fundamental interpretation of C_{μ} may require review.
Classical models are not ideal
There is certainly room for improvement. For many stochastic processes, C_{μ} is strictly greater than E (ref. 11); the ɛmachine that models such processes is fundamentally irreversible. Even if the entire future output of such an ɛmachine was observed, we would still remain uncertain which causal state the machine was initialized in. Some of that information has been erased, and thus, in principle, need never be stored. In this paper, we show that for all such processes, quantum processing helps; for any ɛmachine such that C_{μ}>E, there exists a quantum system, a quantum ɛmachine with entropy C_{q}, such that C_{μ}>C_{q}≥E. Therefore, the corresponding model demands an input with entropy no greater than C_{q}.
The key intuition for our construction lies in identifying the cause of irreversibility within classical ɛmachines, and addressing it within quantum dynamics. An ɛmachine distinguishes two different causal states provided they have differing future statistics, but makes no distinction based on how much these futures differ. Consider two causal states, S_{j} or S_{k}, that both have potential to emit output r at the next time step and transition to some coinciding causal state S_{l}. Should this occur, some of the information required to completely distinguish S_{j} and S_{k} has been irreversibly lost. We say that S_{j} and S_{k} share nondistinct futures. In fact, this is both necessary and sufficient condition for C_{μ}>E (see Methods for proof).
The irreversibility condition
Given a stochastic process with excess entropy E and statistical complexity C_{μ}, let its corresponding ɛmachine have transition probabilities . Then C_{μ}>E iff there exists a nonzero probability that two different causal states, S_{j} and S_{k}, will both make a transition to a coinciding causal state S_{l} on emission of a coinciding output r∈Σ, that is, . We refer to this as the irreversibility condition.
This condition highlights the fundamental limitation of any classical model. To generate desired statistics, any classical model must record each binary property A such that , regardless of how much these distributions overlap. In contrast, quantum models are free of such restriction. A quantum system can store causal states as quantum states that are not mutually orthogonal. The resulting quantum ɛmachine differentiates causal states sufficiently to generate correct statistical behaviour. Essentially, they save memory by 'partially discarding' A, and yet retain enough information to recover statistical differences between .
Improved quantum models
Given an ɛmachine with causal states S_{j} and transition probabilities , we define quantum causal states
where r〉 and k〉 form orthogonal bases on Hilbert spaces of size Σ and S, respectively. A quantum ɛmachine accepts a quantum state S_{j}〉 as input in place of S_{j}. Thus, such a system has an internal entropy of
where . C_{q} is clearly strictly less than C_{μ} provided not all S_{j}〉 are mutually orthogonal^{14}.
This is guaranteed whenever C_{μ}>E. The irreversibility condition implies that there exists two causal states, S_{j} and S_{k}, which will both make a transition to a coinciding causal state S_{l} on emission of a coinciding output r∈Σ, that is, . Consequently , and thus S_{j}〉 is not orthogonal with respect to 〈S_{j}.
A quantum ɛmachine initialized in state S_{j}〉 can synthesise blackbox behaviour that is statistically identical to a classical ɛmachine initialized in state S_{j}. A simple method is to first measure S_{j}〉 in the basis r〉k〉, resulting in measurement values r, k and then to set r as output x_{0} and prepare the quantum state S_{k}〉. Repetition of this process generates a sequence of outputs x_{1},x_{2},... according to the same probability distribution as the original ɛmachine and hence . (We note that while the simplicity of the above method makes it easy to understand and amiable to experimental realization, there is room for improvement. The decoding process prepares S_{k} based on the value of k, and thus still requires C_{μ} bits of memory. However, there exist more sophisticated protocols without such limitation, such that the entropy of the quantum ɛmachine remains at C_{q} at all times. One is detailed in Methods). These observations lead to the central result of our paper.
Theorem: Consider any stochastic process with excess entropy E, whose optimal classical model has input entropy C_{μ}>E. Then, we may construct a quantum system that generates identical statistics, with input entropy C_{q}<C_{μ}. In addition, the entropy of this system never exceeds C_{q} while generating these statistics.
There always exists quantum models of greater efficiency than the optimal classical model, unless the optimal classical model is already ideal.
A concrete example of simulating perturbed coins
We briefly highlight these ideas with a concrete example of a perturbed coin. Consider a process realized by a box that contains a single coin. At each time step, the box is perturbed such that the coin flips with probability 0<p<1, and the state of the coin is then observed. This results in a stochastic process, where each , governed by random variable X_{t}, represents the result of the observation at time t.
For any p≠0.5, this system has two causal states, corresponding to the two possible states of the coin; the set of pasts ending in 0, and the set of pasts ending in 1. We call these S_{0} and S_{1}. The perturbed coin is its own best classical model, requiring exactly a system of entropy C_{μ}=1, namely the coin itself, to generate correct future statistics.
As p→0.5, the future statistics of S_{0} and S_{1} become increasingly similar. The stronger the perturbation, the less it matters what state the coin was in before perturbation. This is reflected by the observation that E→0 (in fact (ref. 9), where is the Shannon entropy of a biased coin that outputs head with probability p (ref. 15)). Thus only of the information stored is useful, which tends to 0 as p→0.5.
Quantum ɛmachines offer dramatic improvement. We encode the quantum causal states within a qubit, which results in entropy . The nonorthogonality of S_{0}〉 and S_{1}〉 ensures that this will always be less than C_{μ} (ref. 16). As p→0.5, a quantum ɛmachine tends to require negligible amount of memory to generate the same statistics compared with its classical counterpart (Fig. 2).
This improvement is readily apparent when we model a lattice of K independent perturbed coins, which output a number that represents the state of the lattice after each perturbation. Any classical model must necessarily differentiate between 2 ^{K} equally likely causal states, and thus require an input of entropy K. A quantum ɛmachine reduces this to KC_{q}. For p>0.2, C_{q}<0.5, the initial condition of two perturbed coins may be encoded within a system of entropy 1. For p>0.4, C_{q}<0.1, a system of coinciding entropy can simulate 10 such coins. This indicates that quantum systems can potentially simulate N such coins on receipt of K≪N qubits, provided appropriate compression (through lossless encodings^{17}) of the relevant past.
Discussion
In this article, we have demonstrated that any stochastic process with no reversible classical model can be further simplified by quantum processing. Such stochastic processes are almost ubiquitous. Even the statistics of perturbed coins can be simulated by a quantum system of reduced entropy. In addition, the quantum reconstruction can be remarkably simple. Quantum operations on a single qubit, for example, allows construction of a quantum epsilon machine that simulates such perturbed coins. This allows potential for experimental validation with present day technology.
This result has significant implications. Stochastic processes have a ubiquitous role in the modelling of dynamical systems that permeate quantitative science, from climate fluctuations to chemical reaction processes. Classically, the statistical complexity C_{μ} is employed as a measure of how much structure a given process exhibits. The rationale is that the optimal simulator of such a process requires at least this much memory. The fact that this memory can be reduced quantum mechanically implies the counterintuitive conclusion that quantizing such simulators can reduce their complexity beyond this classical bound, even if the process they are simulating is purely classical. Many organisms and devices operate based on the ability to predict and thus react to the environment around them. The possibility of exploiting quantum dynamics to make identical predictions with less memory implies that such systems need not be as complex as one had originally thought.
This leads to the open question: is it always possible to find an ideal simulator? Certainly, Fig. 2 shows that our construction, while superior to any classical alternative, is still not wholly reversible. Although this irreversibility may indicate that more efficient quantum models exist, it is also possible that ideal models remain forbidden within quantum theory. Both cases are interesting. The former would indicate that the notion of stochastic processes 'hiding' information from the present^{11} is merely a construct of inefficient classical probabilistic models, whereas the latter hints at a source of temporal asymmetry within the framework of quantum mechanics; that it is fundamentally impossible to simulate certain observable statistics reversibly.
Methods
Proof of the irreversibility condition
Let the aforementioned ɛmachine have causal states and emission alphabet Σ. Consider an instance of the ɛmachine at a particular time step t. Let S_{t} and X_{t} be the random variables that respectively govern its causal state and observed output at time t, such that the transition probabilities that define the ɛmachine can be expressed as
We say an ordered pair is a valid emission configuration iff for some . That is, it is possible for an ɛmachine in state S_{j} to emit r and transit to some S_{k}. Denote the set of all valid emission configurations by Ω_{E}. Similarly, we say an ordered pair is a valid reception configuration iff for some , and denote the set of all valid reception configurations by Ω_{R}.
We define the transition function f: Ω_{E}→Ω_{R}, such that if the ɛmachine set to state S_{j} will transition to state S_{k} on emission of r. We also introduce the shorthand X^{b}_{a} to denote the list of random variables . We now make use of (I) f is onetoone iff there exist no distinct causal states, S_{j} and S_{k}, such that for some S_{l}, (II) iff f is onetoone, (III) , (IV) implies C_{μ}=E, (V) C_{μ}=E implies . Each of these five statements will be proved in the subsequent sections.
Combining (I), (II), (III) and (V), we see that there exists a nonzero probability that two distinct causal states, S_{j} and S_{k} such that for some S_{l} only if . Meanwhile (I), (II) and (V) imply that there exists no two distinct causal states, S_{j} and S_{k} such that for some S_{l} only if C_{μ}=E. The conditions for classical nonoptimality follows.
Proof of statement I
Suppose f is onetoone, then iff S_{j}=S_{k}. Thus, there does not exist two distinct causal states, S_{j} and S_{k} such that for some S_{l}. Conversely, if f is not onetoone, so that . Let S_{l} be the state such that .
Proof of statement II
Suppose f is onetoone. Then for each , there exists a unique (S_{k}, r) such that f(S_{k}, r)=(S_{j}, r). Thus, given , we may uniquely deduce S_{k}. Therefore . Conversely, should , and thus f is onetoone.
Proof of statement III
Note that (i) and (ii) that, since the output of f is unique for a given . (ii) implies that since uncertainty can only decrease with additional knowledge and is bounded below by 0. Substituting this into (i) results in the relation . Thus .
Proof of statement IV
Now assume that , recursive substitutions imply that . In the limit where t→∞, the above equality implies C_{μ}–E=0.
Proof of statement V
Since (i) , and (ii) , it suffices to show that . Now . But, by the Markov property of causal states, , thus , as required.
Constant entropy prediction protocol
Recall that in the simple prediction protocol, the preparation of the next quantum causal state was based on the result of a measurement in basis k〉. Thus, although we can encode the initial conditions of a stochastic process within a system of entropy C_{q}, the decoding process requires an interim system of entropy C_{μ}. Although this protocol establishes that quantum models require less knowledge of the past, quantum systems implementing this specific prediction protocol still need C_{μ} bits of memory at some stage during their evolution.
This limitation is unnecessary. In this section, we present a more sophisticated protocol whose implementation has entropy C_{q} at all points of operation. Consider a quantum ɛmachine initialized in state . We refer the subsystem spanned by r〉 as , and the subsystem spanned by k〉 as .
To generate correct predictive statistics during each iteration, we first perform a general quantum operation on that maps any given S_{j}〉 to is a second Hilbert space of dimension Σ. This operation always exists, because it is defined by Krauss operators that satisfy is then emitted as output. Measurement of in the r〉 basis leads to a classical output r whose statistics coincide with that of its classical counterpart, x_{1}. Finally, the remaining subsystem is retained as the initial condition of the quantum ɛmachine at the next time step.
A circuit representation of the protocol is documented in Fig. 3. The first step does not increase system entropy because entropy is conserved under addition of pure ancilla, while for all j,k. The emission of leaves the εmachine in state , which has entropy C_{q}. Finally, the execution of the protocol does not require knowledge of the measurement result r (in fact, the quantum ɛmachine can thus execute correctly even if all outputs remained unmeasured, and thus are truly ignorant of which causal state they are in). Thus, the physical application of the above protocol generates correct predication statistics without requiring more than memory C_{q}.
Additional information
How to cite this article: Gu, M. et al. Quantum mechanics can reduce the complexity of classical models. Nat. Commun. 3:762 doi: 10.1038/ncomms1761 (2012).
References
Crutchfield, J. P. The calculi of emergence: computation, dynamics and induction. Physica D 75, 11–54 (1994).
Ray, A. Symbolic dynamic analysis of complex systems for anomaly detection. Signal Process. 84, 1115–1130 (2004).
Crutchfield, J. P. & Young, K. Inferring statistical complexity. Phys. Rev. Lett. 63, 105–108 (1989).
Rohilla, S. C. & Crutchfield, J. P. Computational mechanics: pattern and prediction, structure and simplicity. J Stat. Phys. 104, 817–879 (2001).
Holevo, A. S. Statistical problems in quantum physics. In Proceedings of the Second Japan—USSR Symposium on Probability Theory (eds Maruyama, G. and Prokhorov, J. V.) 104–119 (SpringerVerlag, Berlin, 1973) Lecture Notes in Mathematics, vol. 330.
Doob, J. L. Stochastic Processes (Wiley, New York, 1953).
Crutchfield, J. P. & Feldman, D. P. Regularities unseen, randomness observed: levels of entropy convergence. Chaos 13, 25–54 (2003).
Grassberger, P. Toward a quantitative theory of selfgenerated complexity. Int. J. Theor. Phys. 25, 907–938 (1986).
Crutchfield, J. P. & Feldman, D. P. Statistical complexity of simple onedimensional spin systems. Phys. Rev. E 55, R1239–R1242 (1997).
Tononi, G., Sporns, O. & Edelman, G. M. A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc. Natl Acad. Sci. 91, 5033–5037 (1994).
Crutchfield, J. P., Ellison, C. J. & Mahoney, J. R. Time's barbed arrow: irreversibility, crypticity, and stored information. Phys. Rev. Lett. 103, 094101 (2009).
Shalizi, C. R., Shalizi, K. L. & Haslinger, R. Quantifying selforganization with optimal predictors. Phys. Rev. Lett. 93, 118701 (2004).
Li, C. B., Yang, H. & Komatsuzaki, T. Multiscale complex network of protein conformational fluctuations in singlemolecule time series. Proc. Natl Acad. Sci. 105, 536–541 (2008).
Nielsen, M. A. & Chuang, I. L. Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, 2000).
Shannon, C. E. Prediction and entropy of printed English. Bell Syst. Tech. J. 30, 50–64 (1951).
Benenti, G. & Casati, G. S. Principles of Quantum Information and Computation II (World Scientific, 2007).
Bostroem, K. & Felbinger, T. Lossless quantum data compression and variablelength coding. Phys. Rev. A 65, 032313 (2002).
Acknowledgements
M.G. thanks C. Weedbrook, H. Wiseman, M. Hayashi, W. Son, T. Fritz, J. Thompson and K. Modi for helpful discussions. M.G. and E.R. are supported by the National Research Foundation and Ministry of Education, in Singapore. K.W. is funded through EPSRC grant EP/E501214/1. V.V. thank EPSRC, QIP IRC, Royal Society and the Wolfson Foundation, National Research Foundation (Singapore) and the Ministry of Education (Singapore) for financial support.
Author information
Authors and Affiliations
Contributions
M.G., K.W., E.R. and V.V. conceived the idea. M.G. and K.W. derived the technical results. M.G. prepared the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Gu, M., Wiesner, K., Rieper, E. et al. Quantum mechanics can reduce the complexity of classical models. Nat Commun 3, 762 (2012). https://doi.org/10.1038/ncomms1761
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/ncomms1761
This article is cited by

Implementing quantum dimensionality reduction for nonMarkovian stochastic simulation
Nature Communications (2023)

Quantum causal unravelling
npj Quantum Information (2022)

Surveying Structural Complexity in Quantum ManyBody Systems
Journal of Statistical Physics (2022)

Quantumenhanced analysis of discrete stochastic processes
npj Quantum Information (2021)

Optimizing Quantum Models of Classical Channels: The Reverse Holevo Problem
Journal of Statistical Physics (2020)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.