Article | Open

Using quantum theory to simplify input–output processes

  • npj Quantum Information 3, Article number: 6 (2017)
  • doi:10.1038/s41534-016-0001-3
  • Download Citation
Received:
Revised:
Accepted:
Published online:

Abstract

All natural things process and transform information. They receive environmental information as input, and transform it into appropriate output responses. Much of science is dedicated to building models of such systems—algorithmic abstractions of their input–output behavior that allow us to simulate how such systems can behave in the future, conditioned on what has transpired in the past. Here, we show that classical models cannot avoid inefficiency—storing past information that is unnecessary for correct future simulation. We construct quantum models that mitigate this waste, whenever it is physically possible to do so. This suggests that the complexity of general input–output processes depends fundamentally on what sort of information theory we use to describe them.

Introduction

Every experiment involves applying actions to some system, and recording corresponding output responses. Both inputs and outputs are recorded as classical bits of information, and the system’s operational behavior can always be regarded as an input–output process that transforms inputs to outputs. Quantitative science aims to capture such behavior within mathematical models—algorithmic abstractions that can simulate future behavior based on past observations.

There is keen interest in finding the simplest models—models that replicate a system’s future behavior while storing the least past information.1, 2 The motivations are twofold. Firstly, from the rationale of Occam’s razor, we should posit no more causes of natural things than are necessary to explain their appearances. Every piece of past information a model requires represents a potential cause of future events, and thus simpler models better isolate the true indicators of future behavior. The second is practical. As we wish to simulate and engineer systems of increasing complexity, there is always need to find methods that utilize more modest memory requirements.

This motivated systematic methods for constructing such models. The state of the art are ε-transducers, models of input–output processes that are provably optimal—no other means of modeling a given input–output process can use less past information.3 The amount of past information such a transducer requires thus presents a natural measure of the process’s intrinsic complexity. This heralded new ways to understand structure in diverse systems, ranging from evolutionary dynamics to action-perception cycles.47 Yet ε-transducers are classical, their optimality only proven among classical models. Recent research indicates that quantum models can more simply simulate stochastic processes that evolve independently of input.810 Can quantum theory also surpass classical limits in modeling general processes that behave differently on different input?

Here, we present systematic methods to construct quantum transducers—quantum models that can be simpler than their optimal classical counterparts. The resulting constructions exhibit significant generality: they improve upon optimal classical models whenever it is physically possible to do so. Our work indicates that classical models waste information unavoidably and this waste can be mitigated via quantum processing.

Framework

We adopt the framework of computational mechanics.13 An input–output process describes a system that, at each discrete time-step tZ, can be ‘kicked’ in a number of different ways, denoted by some x(t) selected from a set of possible inputs X. In response, the system emits some y(t) among a set of possible outputs Y . For each possible bi-infinite input sequence x = x ( - 1 ) x ( 0 ) x ( 1 ) , the output of the system can be described by a stochastic process, Y = Y ( - 1 ) Y (0) Y (1) , a bi-infinite string of random variables where each Y(t) governs the output y(t). The black-box behavior of any input–output process is characterized by a family of stochastic processes, { Y x } x X . When the input x is governed by some stochastic process X , the input–output process outputs y with probability.

P [ Y = y ] = x P [ Y X = x ] P [ X = x ] .
(1)

Therefore, an input–output process acts as a transducer on stochastic processes—it takes one stochastic process as input, and transforms it into another.1

Computational mechanics typically assumes processes are causal and stationary. Causality implies the future inputs do not retroactively affect past outputs. That is, for all L Z + , we require P Y ( t : t + L ) X =P ( Y ( t : t + L ) X ( t + L + 1 ) ) , where X ( t + L + 1 ) = X ( t + L - 1 ) X ( t + L ) and Y(t:t + L) = Y(t)Y(t + L). This naturally bipartitions each stochastic process Y into two halves, y ( t ) = y ( t - 2 ) y ( t - 1 ) to represent events in the past of t, and y ( t ) = y ( t ) y ( t + 1 ) to describe events in the future, governed, respectively, by Y ( t ) and Y ( t ) . Stationarity implies the process is invariant with respect to time translation, such that P ( Y ( t : t + L ) x ( t ) ) =P ( Y ( 0 : L ) x ( 0 ) ) and P ( Y ( t ) x ( t ) ) =P ( Y (0) x (0) ) , where x ( t ) = x ( t - 1 ) x ( t ) x ( t + 1 ) is governed by X ( t ) . Hence, we can take the present to be t = 0, and omit the superscript (t).

Each instance of an input–output process has some specific past z = ( x , y ) . On future input x , the process will then exhibit a corresponding conditional future governed by P Y X = x , Z = z . A mathematical model of the process should replicate its future black-box behavior when given information about the past. That is, each model records s ( z ) in some physical memory Ξ in place of z , such that upon future input x the model can generate a random variable Y according to P [ Y x , z ] (see Fig. 1).

Fig. 1
Fig. 1

Modeling a general input–output process. Each instance of an input–output process features some specific sequence of past inputs (a) and past outputs (b). A model of such a process describes a systematic method of storing relevant information within a physical system (c), such that for any future input (d), it can replicate the correct statistical output (e)

Simplest classical models

Numerous mathematical models exist for each input–output process. A brute force approach involves storing all past inputs and outputs. This is clearly inefficient. Consider the trivial input–output process that outputs a completely random sequence regardless of input. Storing all past information would take an unbounded amount of memory. Yet, this process can be simulated by flipping an unbiased coin—requiring no information about the past.

A more refined approach reasons that replicating future behavior does not require differentiation of pasts with statistically identical future behavior. Formally, we define the equivalence relation z ~ ε z whenever two pasts, z and z ′, exhibit statistically coinciding future input–output behavior, i.e., whenever P Y X , Z = z =P Y X , Z = z . This partitions the space of all pasts into equivalence classes S= { s i } . Each s j S is known as a causal state, and ε denotes the encoding function that maps each past to its corresponding causal state. In general, a process can have an infinite number of causal states. Classical studies generally concentrate on cases where n=S is finite. In light of this, we have focused our presentation on such cases.

This motivates the ε-transducer, which stores the causal state ε ( z ) in place of z . It then operates according to the transition elements.

T i j y x =P [ S ( t ) = s j , Y ( t ) = y S ( t - 1 ) = s i , X ( t ) = x ] ;
(2)

the probability a transducer in causal state s i S will transition to causal state s j S while emitting yY , conditioned on receiving input xX. Note that this construction is naturally unifiliar—given the state of the transducer at the current time-step, its state at the subsequent time-step can be completely deduced by observation of the next input–output pair.3 Thus iterating through this procedure generates output behavior statistically identical to that of the original input–output process.

For each stationary input process X , causal state s i occurs with probability p X ( i ) =P [ ε ( z ) = s i ] . The ε-transducer will thus exhibit internal entropy.

C X =- i p X ( i ) log p X ( i ) .
(3)

ε-transducers are the provably simplest classical models—any other encoding function s ( z ) that generates correct future statistics, will exhibit greater entropy.3

Complexity theorists regard C X as a quantifier of complexity,3 the rationale being that it characterizes the minimum memory any model must store when simulating { Y x } x X on input X . More precisely, consider the simulation of N such input–output processes, where each instance is driven by X . C x then specifies that in the asymptotic limit (N→∞) we can use the ε-transducer to replicate the future statistics of the ensemble by storing the past within a system of NC X bits.

In the special case where the input–output process is input independent (i.e., Y x is the same for all x ), the causal states are reduced to equivalences classes on the set of past outputs Y . T ij y|x become independent of x and is denoted T ij y. Here, the ε-transducers are known as ε-machines and C X as the statistical complexity. This measure has been applied extensively to quantify the structure of various stochastic processes.1215

For general input–output processes, C X is X -dependent and is known as the input-dependent statistical complexity. In certain pathological cases (e.g., always inputting the same input at every time-step), the transducer may have zero probability of being in a particular causal state, potentially leading to a drastic reduction in C X . Here, we consider non-pathological inputs, such that the transducer has non-zero probability of being in each causal state, i.e., p X (i) > 0 for all i. It is also often useful to quantify the intrinsic structure of input–output processes without referring to a specific input process.3 One proposal is the structural complexity, C ̄ = sup X C X , which measures how much memory is required in the worst case scenario.

Example

We illustrate a simple example of an actively perturbed coin. Consider a box with two buttons, containing a single coin. At each time-step, the box accepts a single bit x{0,1} as input representing which of the two buttons is pressed. In response, it flips the coin with probability p if x = 1, and probability q if x = 0, where 0 < p, q < 1. The box then outputs the new state of the coin, y. The behavior of the box is described by an input–output process.

First note that when p = q = 0.5, the output of the device becomes completely random, all pasts collapse to a single causal state, and the statistical complexity is trivially zero. In all other cases, the past partitions into two causal states, s k = { z : y ( - 1 ) = k } , k = 0,1, corresponding to the two possible outcomes of the previous coin toss. Future statistics can then be generated via appropriate transition elements. For details, see the graphical representation in Fig. 2.

Fig. 2
Fig. 2

ε-transducer for the actively perturbed coin. Here, node 0 (1) is identified with causal state s 0 (s 1) and represents pasts where the last coin toss resulted in tails (heads). Each non-zero transition element T kj y|x is then represented by a directed edge from node k to j labeled by ‘y|x:T kj y|x’. Here, future statistics can be generated by transition elements T 01 1|0 = T 10 0|0 = p, T 00 0|0 = T 11 1|0 = 1−p, T 10 0|1 = T 01 1|1 = q, T 00 0|1 = T 11 1|1 = 1−q

Consider a simple input process X u where x = 1 is chosen with some fixed probability u at each time-step, the symmetry of the transition elements implies s0 and s1 occur with equiprobability. Thus C X u = 1. Furthermore, as this is the maximum entropy a two-state machine can take, the classical structural complexity of the actively perturbed coin, C ̄ , is also 1.

Classical inefficiency

Even though the ε-transducer is classically optimal, it may still store unnecessary information. Consider the example above. The ε-transducer must store the last state of the coin (i.e., whether the past is in s0 or s1). However, irrespective of input x, both s0 and s1 have non-zero probability of transitioning to the same s k while emitting the same output y. Once this happens, some of the information being used to perfectly discriminate between s0 and s1 will be irrevocably lost—i.e., there exists no systematic method to perfectly retrodict whether an ε-transducer was initialized in s0 or s1 from its future behavior, regardless of how we choose future inputs. Thus, the transducer appears to store information that will never be reflected in future observations, and is, therefore, wasted.

We generalize this observation by introducing step-wise inefficiency. Consider an ε-transducer equipped with causal states S and transition elements T ij y|x. Suppose there exists s i , s j S such that irrespective of input x, both s i and s j have non-zero probability of transitioning to some coinciding s k S while emitting a coinciding output yY —i.e., for all xX there exists yY , s k S such that both T ik y|x and T jk y|x are non-zero. This implies that at the subsequent time-step, it is impossible to infer which of the two causal states the transducer was previously in with certainty. Thus, some of the information being stored during the previous time-step is inevitably lost. We refer to any ε-transducer that exhibits this condition as being step-wise inefficient.

Results

Quantum processing can mitigate this inefficiency. Whenever the ε-transducer of a given input–output process is step-wise inefficient, we can construct a quantum transducer that is provably simpler. Our construction assigns each s i a corresponding quantum causal state

s i = x s i x ,with s i x = k y T i k y x y k ,
(4)

where x represents the direct product over all possible inputs xX, while y and k denote some orthonormal basis for spaces of dimension Y and S, respectively. The set of quantum causal states { s i } i form a state space for the quantum transducer. Note that although each state is represented as a vector of dimension ( Y S ) X , in practice they span a Hilbert space of dimension at most n=S, whenever S is finite. In such scenarios, the quantum causal states can always be stored losslessly within a n-dimensional quantum system (see subsequent example and methods).

For an input process X , the quantum transducer thus has input-dependent complexity of

Q X =-Tr [ ρ X log ρ X ] ,
(5)

where ρ X = i p X ( i ) s i s i . In general Q X C X .16 Note that when there is only one possible input, s i = s i 0 . This recovers existing quantum ε-machines that can model autonomously evolving stochastic processes better than their simplest classical counterparts.8 Our transducers generalize this result to input–output processes. We show that they field the following properties:

  1. 1.

    Correctness: For any past z in causal state s i , a quantum transducer initialized in state s i can exhibit correct future statistical behavior. i.e., there exists a systematic protocol that, when given s i generates y according to P [ Y X = x , Z = z ] , for each possible sequence of future inputs x .

  2. 2.

    Reduced Complexity: Q X  < C X for all non-pathologic input processes X , whenever the process has a step-wise inefficient ε-transducer.

  3. 3.

    Generality: Quantum transducers store less memory whenever it is physically possible to do so. Given an input–output process, either Q X  < C X for all non-pathological X , or there exists no physically realizable model that does this.

Correctness guarantees quantum transducers behave statistically identically to their classical counterparts—and are thus operationally indistinguishable from the input–output processes they simulate. The proof is done by explicit construction (see Fig. 3 and details in the methods).

Fig. 3
Fig. 3

The quantum circuit that illustrates how a quantum transducer initialized in|s i 〉 at time t−1, simulates the future behavior, when supplied with an input sequence x (t) x (t + 1)…. Upon receiving x at time-step t, it applies a selection operator S x : s i s i x , followed by the quantum operation A that takes each s i x s i x to some bipartite state y , k T i k y x y s k y s k with bipartitions Σ, spanned by |y〉, and W, spanned by |s k 〉 (This is always possible with suitable ancilla, see methods). Σ is emitted as output, while W is retained as the quantum causal state at the next time-step. Measurement of Σ in the {|y〉} basis by any outside observer yields outcome y (t). Iterating this procedure then generates correct outputs at each future time-step

Reduced complexity implies step-wise inefficiency is sufficient for quantum transducers to be simpler than their classical counterparts. The proof involves showing that if for any potential input x, both s i and s j have non-zero probability of transitioning to some coinciding causal state s k while emitting identical y, then |s i 〉 and |s j 〉 are non-orthogonal (see methods). Thus, provided two such causal states exist (guaranteed by step-wise inefficiency of the transducer), and each occur with some non-zero probability (guaranteed by non-pathology of the input), Q X  < C X .

Generality establishes that step-wise inefficiency is a necessary condition for any physically realizable quantum model to outperform its classical counterpart. Combined with ‘reduced complexity’, they imply that step-wise inefficiency is the sole source of avoidable classical inefficiency and that our particular quantum transducers are general in mitigating this inefficiency. The proof is detailed in the methods, and involves showing that any model which improves upon an ε-transducer that is step-wise efficient allows perfect discrimination of non-orthogonal quantum states.

Together, these results isolate step-wise inefficiency as the necessary and sufficient condition for quantum models to be simpler than their classical counterparts, and furthermore, establish an explicit construction of such a model. It follows that whenever the ε-transducer is step-wise inefficient, the upper-bound,

Q ¯ = sup X Q X ,
(6)

will be strictly less than C ̄ = sup X C X , provided sup X C X is attained for a non-pathological X . Intuitively, this clause appears natural. If an agent wished to drive a transducer to exhibit the greatest entropy, then it would be generally advantageous to ensure the transducer has finite probability of being in each causal state. Nevertheless, as the maximization is highly non-trivial to evaluate, this remains an open conjecture.

Example revisited

We illustrate these results for the aforementioned actively perturbed coin. Recall that the ε-transducer of this process features two causal states, s0 and s1 (see Fig. 2). As this transducer is step-wise inefficient, a more efficient quantum transducer exists. Specifically, set the quantum causal states to

τ 0 = r 0 + 1 - r 1 , τ 1 = 0 ,
(7)

where r = 16pq(1−p)(1−q). Note that while these states do not resemble the standard form in Eq. (4) they are unitarily equivalent. Given { τ i } i , we can initialize the joint four qubit state τ i 000 (with three ancilla), and implement an appropriate 4-qubit unitary U: τ i 000 s i for j = 0,1, where

s 0 = ϕ p 12 ϕ q 34 , s 1 = X ^ 1 X ^ 2 X ^ 3 X ^ 4 s 0 .

are of standard form. Here subscripts 1…4 label each of the four qubits, while ϕ p = 1 - p 00 + p 11 and ϕ q = 1 - q 00 + q 11 , and X ^ i represents the Pauli X operator on the ith qubit. The τ i representation makes it clear that these states can be encoded within a single qubit. Figure 4a then outlines a quantum circuit that generates desired future behavior.

Fig. 4
Fig. 4

The quantum transducer for the perturbed coin can generate appropriate future statistics via the quantum circuit in a. Suppose a transducer, in state |τ i 〉, receives input x at time t. To generate output y, it transforms |τ i 〉 to a 4-qubit quantum state |s i 〉 by application of an appropriate unitary U on |τ i 〉 and three ancilla qubits in state |000〉. The transducer then discards qubits one and two if x = 0, or qubits three and four if x = 1. The two remaining qubits, labeled B 1 and B 2, are subsequently transformed by a unitary V that maps |00〉 to 0 τ 0 and |11〉 to 1 τ 1 (this is always possible as 0 τ 0 ∣1 τ 1 = 0). B 1 is emitted as output while B 2 is retained by the transducer as the causal state for the subsequent time-step. Measurement of B 1 in the computational basis yields y. Iteration of this procedure replicates correct future input–output statistics. The resulting improved efficiency is highlighted in b, which depicts the maximum memory required by a quantum transducer Q ̄ (orange surface) to simulated the actively perturbed coin vs. its classical counterpart, the structural complexity C ̄ (blue surface) for various p and q. While the ε-transducer generally requires 1 bit of memory, the quantum transducer requires less, and becomes increasingly more efficient as p,q→0.5

The resulting quantum transducer is clearly more efficient. τ 0 τ 1 = r > 0, provided 0 < p,q < 1. Thus Q X  < C X for all input processes X . Furthermore, the quantum structural complexity Q ̄ = sup X Q X is attained for any input process where |τ0〉 and |τ1〉 occur with equiprobability, such as any X = X u . The improvement is significant, and can be explicitly evaluated (see Fig. 4b). In particular, lim p , q 0 . 5 Q X C X = 0. Thus the quantum transducer, in limiting cases, can use negligible memory compared to its classical counterpart.

The intuition is that as we approach this limit, the output of the process becomes progressively more random. Thus future black-box behavior of a process whose last output is heads becomes increasingly similar to one whose last output is tails. The equivalence class s0 or s1, then contains progressively less information about future outputs. A classical transducer nevertheless must distinguish these two scenarios, and thus exhibits an entropy of 1 when the scenarios are equally likely. A quantum transducer, however, has the freedom to only partially distinguish the two scenarios to the extent in which they affect future statistics. In particular lim p , q 0 . 5 τ 0 τ 1 1. As the process becomes more random, the quantum transducer saves memory by encoding the two scenarios in progressively less orthogonal quantum states.

Future directions

There is potential in viewing predictive modeling as a communication task, where Alice sends information about a process’s past to Bob, so that he may generate correct future statistical behavior.3, 9 The simpler a model, the less information Alice needs to communicate to Bob. The entropic benefits of harnessing non-orthogonal causal states mirrors similar advantages in exploiting non-orthogonal codewords to perform certain communication tasks.17 Quantum transducers could thus identify a larger class of such tasks, and provide a general strategy to supersede classical limits. Meanwhile, one may also consider generalizations of statistical complexity that use other measures of entropy. The max entropy, for example, captures the minimum dimensionality required to simulate general input–output processes. This may complement existing work in quantum dimensionality testing,1820 pointing to testing the dimensionality of systems by seeing how they transform stochastic processes.

Another interesting question is how quantum transducers relate to quantum advantage in randomness processing.21 In this context, it was shown that quantum sources of randomness (named quoins), in the form p = p 0 + 1 - p 1 can be a much more powerful resource for sampling a coin with a p-dependent bias f(p), than classical coins of bias p. Subsequent experimental implementations have used quoins to sample certain f(p), which are impossible to synthesize when equipped with only p-coins.22 Quantum transducers appear to utilize similar effects. The quantum causal states in Eq. (4) also resemble a quantum superposition of classical measurement outcomes, which can be used to generate desired future output statistics more efficiently than classically possible. Is this resemblance merely superficial? Quantum transducers and the quantum Bernoulli factory certainly also field significant differences—both in how they quantify efficiency and in what they consider as input and output. As such this question remains very much open.

There could also be considerable interest in establishing what resources underpin the performance advantage in quantum transducers. Non-orthogonal quantum causal states, and thus coherence, is clearly necessary. This non-orthogonality then immediately implies that quantum correlations (in the sense of discord23) necessarily exist between the state of the transducer and its past outputs. Could the amount of such resources be related quantitatively to the quantum advantage of a particular transducer? Whether more stringent quantum resources, such as entanglement, are also required at some point to generate correct future statistics, also remains an open question. Certainly, all quantum transducers described here exploit highly entangling operations to generate statistically correct future behavior—and field a significant amount of entanglement during their operation. Is the existence of such entanglement at some stage during the simulation process essential for quantum advantage?

Discussion

In computational mechanics, ε-transducers are the provably simplest models of input–output processes. Their internal entropy is a quantifier of structure—any device capable of replicating the process’s behavior must track at least this much information. Here, we generalize this formalism to the quantum regime. We propose a systematic method to construct quantum transducers that are generally simpler than their simplest classical counterparts; in the sense that quantum transducers store less information whenever it is physically possible to do so. Our work indicates the perceived complexity of input–output processes generally depends on what type of information theory we use to describe them.

A natural continuation is to explore the feasibility of such quantum transducers in real world conditions. A proof of principle demonstration is well within reach of present-day technology. The quantum transducer for the actively perturbed coin can be implemented by a single qubit undergoing one of two different weak measurements at each time-step. To demonstrate a quantum advantage for real world applications would also motivate new theory. For example, noise will certainly degrade the performance of quantum transducers, forcing the use of more distinguishable quantum causal states. The derivation of bounds on the resultant entropic cost would thus help us establish thresholds that guarantee quantum advantage in real-world conditions.

Ultimately, the interface between quantum and computational mechanics motivates the potential for tools of each community to impact the other. Classical transducers, for example, are used to capture emergence of complexity under evolutionary pressures,24 distil structure within cellular automata,25, 26 and characterize the dynamics of sequential learning,27 and it would be interesting to see how these ideas change in the quantum regime. Meanwhile, recent results suggests the inefficiency of classical models may incur unavoidable thermodynamic costs.2830 In reducing this inefficiency, quantum transducers could offer more energetically efficient methods of transforming information. Any such developments could demonstrate the impact of quantum technologies in domains where their use has not been considered before.

Methods

Definitions

Let X(t),Y(t) and S(t) represent, respectively, the random variables governing the input, output and causal state at time t. Let Y(0:t) = Y(0)Y(t) govern the outputs y(0:t) = y(0)y(t) and X(0:t) = X(0)X(t) govern the inputs x(0:t) = x(0)x(t). We introduce ordered pairs Z(t) = (X(t), Y(t)) which take values z(t) = (x(t), y(t))Z, where Z represents the space of potential input–output pairs. In analogy, let z = ( x , y ) represents a particular past, Z be the space of all possible pasts and z(0:t) = z(0)z(t) Z t + 1 the input–output pairs from time-steps 0 to t. Let X denote the size of the input alphabet, Y the size of output alphabet and n=S the number of causal states(if finite). Without loss of generality, we assume the inputs and outputs are labeled numerically, such that X= { x i } 0 X - 1 and Y = { y i } 0 Y - 1 .

Each instance of an input–output process { Y x } exhibits a specific past z = ( x , y ) . When supplied x(0) = x as input, it emits y(0) = y with probability P [ Y (0) = y X (0) = x , Z = z ] . The system’s past then transitions from z to z = ( x x , y y ) = ( x , y ) . This motivates the propagating functions μ x,y on the space of pasts, μ x , y ( z ) = ( x x , y y ) , which characterize how the past updates upon observation of (x,y) at each time-step. Iterating through this process for t + 1 timesteps gives expected output y(0:t) with probability P ( Y ( 0 : t ) = y ( 0 : t ) X ( 0 : t ) = x ( 0 : t ) , Z = z ) upon receipt of input sequence x(0:t). Taking t→∞ gives the expected future input–output behavior P [ Y X = x , Z = z ] upon future input sequence x X .

A quantum model of an input–output process defines an encoding function that maps each past z to some state ( z ) = ρ z within some physical system Ξ. The model is correct, if there exists a family of operations M= { M x } x X such that application of M x on Ξ replicates the behavior of inputting x. That is, M x acting on ρ z should (1) generate output y with probability P [ Y (0) X (0) = x , Z = z ] and (2) transition Ξ into state [ μ x , y ( z ) ] = ρ μ x , y ( z ) . (1) ensures the model outputs statistically correct y(0), while (2) ensures the model’s internal memory is updated to record the event z(0) = (x(0),y(0)), allowing correct prediction upon receipt of future inputs. Sequential application of M x = M x (0) , M x (1) , then generates output y with probability P [ Y X = x , Z = z ] . Let Ω= { ρ z } z Z be the image of . We now define quantum models as follows.

Definition 1

A general quantum model of an input–output process is a tripleQ= ( , Ω , M ) , where, Ω andMsatisfy the conditions above.

Each stationary input process X induces a probability distribution p X ( z ) over the set of pasts, and thus a steady state of the machine ρ X . The resulting entropy of Ξ,

Q X [ Q ] =-Tr ( ρ X log ρ X ) ,
(8)

then defines the model’s input-dependent complexity. For the quantum transducers in the main body, Ω= { s i s i } i corresponds to the set of quantum causal states, and the encoding function maps each past z to s i s i , whenever z s i . Specifically, if we define the classical encoding function ε: Z S that maps pasts onto causal states such that ε ( z ) = s i iff z s i , then s i = ε ( z ) .

We also introduce input strategies. Suppose Bob receives a model of some known input–output process, initialized in some (possibly unknown) state ρ z Ω. Bob now wants to drive the model to exhibit some particular future behavior by using an input strategy—a specific algorithm for deciding what input x(t) he will feed the model at each specific t ≥ 0, purely from the model’s black-box behavior.

Definition 2

An input strategy is a family of functionsF= { f ( t ) t Z + } , where f ( t ) : Z t Xis a map from the space of pre-existing inputs–outputs onto the input at time t, such that x(t) = f(t)(z(0:t−1)). We denote a sequence of future inputs which is determined using x(t) = f(t)(z(0:t−1)), as x F .

In subsequent proofs, we will invoke input strategies on classical ε-transducers. Here, we denote

P s i , F [ y ] = P [ Y = y , X = x F , S ( - 1 ) = s i ] ,
(9)

as the probability distribution that governs future outputs Y , when an input strategy F is used to select the future inputs x F to an ε-transducer initialized in some causal state s i S.

We also make use of the trace distance, D [ P , Q ] =12 y Y P [ y ] -Q [ y ] between two probability distributions P [ y ] and Q [ y ] . Similarly, any two quantum states τ and σ have trace distance D[τ,σ] = 1/2(Tr|τ−σ|), where A A A .

Proof of correctness

Here, we prove a quantum transducer can generate correct future statistical behavior when supplied with a quantum system Ξ initialized in state s i , encoding information about z s i . That is, there exists a family of quantum processes M= { M x } whose action on Ξ produces an output y sampled from P [ Y (0) = y X (0) = x , S ( - 1 ) = s i ] , while transforming the state of Ξ to ε ( x x , y y ) .

Proof

Recall that s i = x s i x where s i x = k =0 n - 1 y Y T i k y x y k . Let Σ be the Y dimensional Hilbert space spanned by { y } y Y and K be the n-dimensional Hilbert space spanned by { k } k = 0 n - 1 . Then each s i x lies in ω=ΣK, and each causal state |s i 〉 lies within W= ω X . The set { s i } i = 0 n - 1 spans some subspace of W of dimension at most n. This implies that the causal states can be stored within K without loss, i.e., there exists quantum states { τ i } i = 0 n - 1 in K and a unitary process U such that U: τ i s i for all i (Note, this assumes appending suitable ancillary subspaces to each |τ i 〉). An explicit form for |τ i 〉 can be systematically constructed through Gram-Schmidt decomposition.31 We refer to |τ i 〉 as compressed causal states, and U as the decompression operator.

We define the selection operator S x :Wω such that S x : ( x ϕ x ) ϕ x . Physically, if W represents a state space of X qudits each with state space ω labeled from 0 to X-1, S x represents discarding (or tracing out) all except the xth qudit. Meanwhile, let B be a quantum operation on ω such that B: y k y k y τ k y τ k . This operation always exists as it can be implemented by Kraus operators E y k = y τ k y k . (Fig. 5).

Fig. 5
Fig. 5

a The process by which the quantum transducer generates future statistics as outlined in Fig. 3. b A more detailed breakdown of. Upon input x at time t, the transducer first applies the selection operator S x . The subsequent operator A can be decomposed into two operators, a linear mapping B: s i x s i x y , k T i k y x y τ k y τ k , and a decompression operator U that rotates each τ k into |s k 〉 (always possible when suitable ancillary systems in states |0〉 are supplied). Σ is emitted as output, while W is retained as the subsequent causal state at time t. This circuit makes it clear that τ k also make perfectly valid causal states

The quantum transducer operates as follows. Upon input x at time t, it applies S x on Ξ, followed by execution of B. This transforms the state of the system from s i x to k = 0 n - 1 y Y T i k y x y τ k y τ k . Application of the decompression operator then gives k = 0 n - 1 y Y T i k y x y s k y s k on state space ΣW. The machine then emits Σ as the output. Measurement of Σ by an external observer in basis { y } y Y gives the output y at time t, while inducing W to transition to the subsequent quantum causal state. W is then retained in Ξ.

The above procedure establishes a family of quantum operations { M x } x X that maps each quantum causal state |s i 〉 to |s j 〉 while emitting output y with probability T i j y x . Thus, the quantum transducer operates statistically indistinguishably from its classical counterpart, and must be correct. To simulate the future behavior of the input–output process upon future input x , the quantum transducer iterates through the above process by sequential application of M x (0) , M x (1) , etc. This establishes the transducer is a valid quantum model.

Proof of reduced complexity and generality

Given an arbitrary input–output process, whose ε-transducer and quantum transducer has respective input dependent complexities of C X and Q X , we show that

  • Reduced Complexity: Q X  < C X for all non-pathological input processes X , whenever the ε-transducer is step-wise inefficient.

  • Generality: Given an input–output process, either Q X  < C X for all non-pathological input processes X or there exists no physically realizable model that does this.

Our strategy makes use of the following theorem.

Theorem 1. For any input–output process the following statements are equivalent:

(I) s i s j = 0for any s i , s j { s i } i where i≠j.

(II) For any pair ( s i , s j ) S×S where i≠j, xXsuch thatyY , s k S, the product T i k y x T j k y x = 0.

(III) For any pair ( s i , s j ) S×Swhere i≠j, there exists an input strategy F such thatD [ P s i , F , P s j , F ] =1.

(IV) For any input process X , any physically realizable (quantum) model must have complexity at least C X .

To prove this theorem, we show A. (I) is equivalent to (II). B. (II) implies (III), C. (III) implies (IV), and D. (IV) implies (I).

Proof of A. We prove this by showing (I) is false iff (II) is false. First, assume (I) is false, such that there exists s i , s j { s i } i with s i s j = Π x X ( y , k T i k y x T j k y x ) > 0. This implies that for all xX there exists s k S,yY such that T i k y x T j k y x 0. Thus (II) is false. Meanwhile assume (II) is false, i.e., there exists s i and s j such that x we can find y x Y , s k x S for which T i k x y x x T j k x y x x > 0. It follows that s i s j > Π x ( T i k x y x x x T j k x y x x ) >0 and (I) is false.

Proof of B. To prove this, we introduce the update function g:S×X×Y S, such that g(s i ,x,y) = s k iff T i k y x 0. Note that g is always a function by joint unifiliarity of the ε-transducer.3 That is, the triple (s(t−1),x(t),y(t)) uniquely determines s(t).

We also use the following game to elucidate the proof. At time t = −1, Alice initializes an ε-transducer in either s i or s j and seals it inside a black box. Alice gives this box to Bob and challenges him to infer whether S(−1) = s i or S(−1) = s j , based purely on the transducer’s future black-box behavior. We first prove that if (II) is true, then for each pair (s i ,s j ) there exists an input strategy F ij that allows Bob to discriminate S(−1) = s i from S(−1) = s j to arbitrary accuracy purely from the transducer’s output behavior.

Specifically (II) implies that for all pairs ( s i , s j ) S×S, there exists some x ij such that yY , s k S, T i k y x i j T j k y x i j = 0. At t = 0, Bob inputs X(0) = x ij . Let y = Y(0) be the corresponding output. Note that because we have observed y = Y(0), there must have been some non-zero probability of observing y on input x ij , i.e., at least one of (i) P[Y(0) = y|X(0) = x ij ,S−1 = s i ]≠0 or (ii) P[Y(0) = y|X(0) = x ij ,S−1 = s j ]≠0 must be true. This presents two different possible scenarios:

  • Only one of (i) and (ii) is true. That is, one of s i or s j never outputs y upon input x ij —such that either P[Y(0) = y|X(0) = x ij , S−1 = s i ] = 0 or P[Y(0) = y|X(0) = x ij , S−1 = s j ] = 0.

  • Both (i) and (ii) are true, implying g(s i , y, x ij )≠g(s j , y, x ij ).

If (a) occurs, then Bob can immediately determine whether S(−1) = s i or S(−1) = s j and we are done. If (b) occurs, let s i ′ = g(s i , y, x ij ) be the new causal state if S(−1) = s i . Let s j ′ = g(s j , y, x ij ) be the new causal state if S(−1) = s j . Due to joint unifiliarity of the ε-transducer, Bob is able to uniquely determine s j′ and s i′ upon observation of z(0) = (x ij , y). Given s i′ and s j′ , (II) implies Bob can find x ij ′ such that yY , s k S the product T i k y x i j T j k y x i j =0. Thus, we can repeat the steps above choosing x(1) = x ij ′. Iterating this procedure defines an input strategy F ij , which determines each input x(t) as a function of observed inputs and outputs. At each point in time t, Bob will be able to identify some s i (t) which is the current causal state if S(−1) = s i , and some s j (t) which is the current causal state if S(−1) = s j .

Eventually, either scenario (a) will occur allowing Bob to perfectly rule out S(−1) = s i . Alternatively, in the limit of an infinite number of time steps, Bob can synchronize the ε-transducer based on the observed inputs and outputs (that is, the causal state at time t is entirely determined by observation of the past in limit of large t1, 3). Thus, Bob can determine s(t) in the limit as t→∞, allowing inference of whether S(−1) = s i or S(−1) = s j . This constitutes an explicit input strategy F ij that allows Bob to discriminate between S(−1) = s i and S(−1) = s j to any arbitrary accuracy. Accordingly D [ P s i F i j , P s j , F i j ] =1.

Proof of C. We prove this via its contrapositive. That is, suppose (IV) is false, such that there exists a quantum model Q with identical input–output relations to the process’s ε-transducer, which stores Q X [ Q ] < C X for some X . We show that if (III) is true then the data processing inequality is violated.16

We first make use of the following observation: If a model Q = , Ω , M satisfies ( z ) ( z ) for some z ~ ε z , then we can always construct an alternative model Q= ( , Ω , M ) such that Q X [ Q ] Q X [ Q ] for all input processes X , and ( z ) = ( z ) iff ε ( z ) =ε ( z ) for all z , z Z . (This is a consequence of the concavity of entropy, see methods in ref. 32). That is, for any model Q with quantum states Ω′ not in 1-1 correspondence with classical causal states, there always exists a simpler model Q whose quantum states are in 1-1 correspondence with the causal states.

Thus, falsehood of (IV) implies there must exist some quantum model Q= ( , Ω , M ) such that (i) ( z ) = ( z ) if and only if z ~ ε z and (ii) Q X [ Q ] < C X for some X . Now by virtue of (ii), there must exist two states ρ i , ρ j Ω such that the trace distance

D [ ρ i , ρ j ] <1.
(10)

The data processing inequality, therefore, implies that any quantum operation M x :Ξ Y that generates future output statistics must satisfy D [ M x ( ρ i ) , M x ( ρ j ) ] D [ ρ i , ρ j ] <1.

However, all models of the same input–output process have identical black-box behavior. In particular the ε-transducer of the input–output process that Q models, must behave identically to Q. As such, there exists two causal states of the classical ε-transducer, s i , s j S such that D [ P s i , F , P s j , F ] =D [ M x F ( ρ i ) , M x F ( ρ j ) ] <1 for all possible input strategies F. This implies (III) is false. Thus we have used proof by contrapositive to show (III) implies (IV).

Proof of D. The quantum transducer is a physically realizable model. Thus (IV) implies that Q X  ≥ C X for all X . However, we note from our construction Q X  ≤ C X for all X . Therefore, Q X  = C X . Since the causal states of the quantum transducer are all pure, all |s i 〉 are mutually orthogonal.16

Proof of Main Result. Reduced complexity and generality are consequences of the above theorem. Specifically given a particular input–output process, falsehood of (II) implies that its transducer is step-wise inefficient. Meanwhile, falsehood of (I) implies Q X  < C X for all non-pathological X . Thus reduced complexity is implied by equivalence of (I) and (II). Generality is proven by contradiction. Assume that for some non-pathological X , quantum transducers yield no improvement (i.e., Q X  = C X ) but some other physically realizable model has complexity less than C X . The former implies (I) is true, the latter implies (IV) is false, which violates the theorem. Thus, both reduced complexity and generality must hold.

Notes

  1. 1.

    We also note that there are some subtleties in defining probability distributions over infinite strings, as in Eq. (1). These issues have been thoroughly discussed in complexity theory literature, for more details see, refs 3,11.

References

  1. 1.

    Shalizi, C. R. & Crutchfield, J. P. Computational mechanics: Pattern and prediction, structure and simplicity. J. Stat. Phys. 104, 817–879 (2001).

  2. 2.

    Crutchfield, J. P. & Young, K. Inferring statistical complexity. Phys. Rev. Lett. 63, 105 (1989).

  3. 3.

    Barnett, N. & Crutchfield, J. P. Computational mechanics of input—output processes: Structured transformations and the epsilon-transducer. J. Stat. Phys. 161, 404–451 (2015).

  4. 4.

    Meddis, R. Simulation of auditory--neural transduction: Further studies. J. Acoust. Soc. Am. 83, 1056–1063 (1988).

  5. 5.

    Rieke, F. Spikes: Exploring the Neural Code (MIT press, 1999).

  6. 6.

    Tishby, N. & Polani, D. In Perception-action Cycle, 601–636 (Springer, 2011).

  7. 7.

    Gordon, G. et al. Toward an integrated approach to perception and action: conference report and future directions. Front. System Neurosci. 5, 20 (2011).

  8. 8.

    Gu, M., Wiesner, K., Rieper, E. & Vedral, V. Quantum mechanics can reduce the complexity of classical models. Nat. Commun. 3, 762 (2012).

  9. 9.

    Mahoney, J. R., Aghamohammadi, C. & Crutchfield, J. P.Occam’ss quantum strop: Synchronizing and compressing classical cryptic processes via a quantum channel. Sci. Rep. 6, 20495 (2016).

  10. 10.

    Tan, R., Terno, D. R., Thompson, J., Vedral, V. & Gu, M. Towards quantifying complexity with quantum mechanics. Eur. Phys. J. Plus. 129, 1–12 (2014).

  11. 11.

    Kallenberg, O. Foundations of Modern Probability (Springer Science & Business Media, 2006).

  12. 12.

    Tiňo, P. & Köteles, M. Extracting finite-state representations from recurrent neural networks trained on chaotic symbolic sequences. IEEE T Neural Networ 10, 284–302 (1999).

  13. 13.

    Larrondo, H., González, C., Martin, M., Plastino, A. & Rosso, O. Intensive statistical complexity measure of pseudorandom number generators. Phys. A. 356, 133–138 (2005).

  14. 14.

    Gonçalves, W., Pinto, R., Sartorelli, J. & de Oliveira, M. Inferring statistical complexity in the dripping faucet experiment. Phys. A. 257, 385–389 (1998).

  15. 15.

    Park, J. B., Lee, J. W., Yang, J.-S., Jo, H.-H. & Moon, H.-T. Complexity analysis of the stock market. Phys. A. 379, 179–187 (2007).

  16. 16.

    Nielsen, M. A. & Chuang, I. L. Quantum Computation and Quantum Information (Cambridge University Press, 2010).

  17. 17.

    Perry, C., Jain, R. & Oppenheim, J. Communication tasks with infinite quantum-classical separation. Phys. Rev. Lett. 115, 030504 (2015).

  18. 18.

    Ahrens, J., Badzig, P., Cabello, A. & Bourennane, M. Experimental device-independent tests of classical and quantum dimensions. Nat. Phys. 8, 592–595 (2012).

  19. 19.

    Gallego, R., Brunner, N., Hadley, C. & Acn, A. Device-independent tests of classical and quantum dimensions. Phys. Rev. Lett. 105, 230501 (2010).

  20. 20.

    Kleinmann, M., Gühne, O., Portillo, J. R., Larsson, J.-Å. & Cabello, A. Memory cost of quantum contextuality. New. J. Phys. 13, 113011 (2011).

  21. 21.

    Dale, H., Jennings, D. & Rudolph, T. Provable quantum advantage in randomness processing. Nat. Commun. 6, 8203 (2015).

  22. 22.

    Yuan, X. et al. Experimental quantum randomness processing using superconducting qubits. Phys. Rev. Lett. 11, 010502 (2016).

  23. 23.

    Modi, K., Brodutch, A., Cable, H., Paterek, T. & Vedral, V. The classical-quantum boundary for correlations: discord and related measures. Rev. Mod. Phys. 84, 1655 (2012).

  24. 24.

    Crutchfield, J. P. & Görnerup, O. Objects that make objects: the population dynamics of structural complexity. J. R. Soc. Interface 3, 345–349 (2006).

  25. 25.

    Hanson, J. E. & Crutchfield, J. P. The attractorbasin portrait of a cellular automaton. J. Stat. Phys. 66, 1415–1462 (1992).

  26. 26.

    Shalizi, C. R. Causal Architecture, Complexity and Self-Organization in the Time Series and Cellular Automata. Ph.D. thesis, Univ. Wisconsin-Madison (2001).

  27. 27.

    Crutchfield, J. P. & Whalen, S. Structural drift: The population dynamics of sequential learning. PLoS Comput. Biol. 8, e1002510 (2012).

  28. 28.

    Wiesner, K., Gu, M., Rieper, E. & Vedral, V. Information-theoretic lower bound on energy cost of stochastic computation. In Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, vol. 468, 4058–4066 (The Royal Society, 2012).

  29. 29.

    Garner, A. J. P., Thompson, J., Vedral, V. & Gu, M. When is simpler thermodynamically better? [arXiv:1510.00010] (2015).

  30. 30.

    Cabello, A., Gu, M., Gühne, O., Larsson, J.-Å. & Wiesner, K. Thermodynamical cost of some interpretations of quantum theory. Phys. Rev. A 94, 052127 (2016).

  31. 31.

    Noble, B. & Daniel, J. W. Applied Linear Algebra, Vol. 3 (Prentice-Hall, New Jersey, UAS, 1988).

  32. 32.

    Suen, W. Y., Thompson, J., Garner, A. J. P., Vedral, V. & Gu, M. The classical-quantum divergence of complexity in the ising spin chain. [arXiv:1511.05738] (2015).

Download references

Acknowledgements

We thank Suen Whei Yeap, Blake Pollard, Liu Qing, and Yang Chengran for their input and helpful discussions. This work was funded by the John Templeton Foundation Grant 53914 ‘Occam’s Quantum Mechanical Razor: Can Quantum theory admit the Simplest Understanding of Reality?’; the Oxford Martin School; the Ministry of Education in Singapore, the Academic Research Fund Tier 3 MOE2012-T3-1-009; the Foundational Questions Institute Grant Observer-dependent complexity: The quantum-classical divergence over ‘what is complex?’ the National Research Foundation of Singapore and in particular NRF Award No. NRF–NRFF2016–02.

Author contributions

All authors conceptualized the project and developed the examples. J.T. worked through the detailed calculations. J.T. and M.G wrote the manuscript. M.G. led the project.

Competing interests

The authors declare that they have no competing interests.

Author information

Affiliations

  1. Centre for Quantum Technologies, National University of Singapore, 3 Science Drive 2, Singapore, 117543, Singapore

    • Jayne Thompson
    • , Andrew J. P. Garner
    • , Vlatko Vedral
    •  & Mile Gu
  2. Atomic and Laser Physics, University of Oxford, Clarendon Laboratory, Parks Road, Oxford, OX1 3PU, UK

    • Vlatko Vedral
  3. Department of Physics, National University of Singapore, 3 Science Drive 2, Singapore, 117543, Singapore

    • Vlatko Vedral
  4. School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore, 639673, Singapore

    • Mile Gu
  5. Complexity Institute, Nanyang Technological University, Singapore, 639673, Singapore

    • Mile Gu

Authors

  1. Search for Jayne Thompson in:

  2. Search for Andrew J. P. Garner in:

  3. Search for Vlatko Vedral in:

  4. Search for Mile Gu in:

Corresponding authors

Correspondence to Jayne Thompson or Mile Gu.

Creative Commons BY

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/