Most physicists now take the view that quantum physics is irreducibly non-deterministic and that nature is fundamentally ruled by chance. Those who still hope otherwise — inspired by the earlier efforts of figures such as Albert Einstein, David Bohm or Roger Penrose — now occupy the fringe of physics belief. They haven't given up, however, and continue to churn out some extremely creative ideas about how quantum randomness might actually be the result of a fully deterministic underlying process.

One of the most provocative recent efforts along these lines comes from Oxford physicist Tim Palmer, who began his research career thirty years ago in general relativity, studying under Stephen Hawking. He then made an abrupt shift to climate modelling. Over several decades, Palmer made seminal contributions to climate science, especially by helping to establish methods emphasizing the unavoidable uncertainty in climate projections that arises from inherently chaotic dynamics.

Ironically, Palmer now believes that the theory of dynamical chaos and 'strange attractors' — the geometrical structures signifying chaos in dissipative systems — might be the key concepts required to tackle a number of fundamental physics issues, among them building a sensible theory of quantum gravity. Moreover, he suggests that a return to determinism might follow. Incredible? Well, it takes a few steps to see how all that might come about.

You might wonder how chaos could possibly be relevant. After all, the Schrödinger equation of non-relativistic quantum theory is linear, and so cannot produce chaos or any strange attractor. As Palmer notes, however, the flow of an ensemble of trajectories for a chaotic system — the famous Lorenz equations, for example — also follows a linear equation, the so-called Liouville equation of classical mechanics. This essentially describes the conservation of probability within the system, and its linearity is fully consistent with strong nonlinearity in the underlying system dynamics.

Physics itself might be non-computational in a way few physicists have considered.

Hence, there's nothing at all logically problematic about the linear Schrödinger equation possibly emerging as a probability-level description of deeper, underlying nonlinear dynamics. What might those dynamics be? Palmer doesn't try to work out them out directly, but instead takes a geometric approach.

In dynamical systems theory, the key object in the description of the long-term dynamics of a dissipative system is its attractor — an invariant set that almost all system trajectories approach asymptotically. In a chaotic system, this is a strange attractor with fractal or multi-fractal geometry. In the famous Lorenz system, for example, the attractor looks crudely like two intersecting surfaces that resemble a butterfly. On closer inspection it turns out to be an infinitely intricate set of nested surfaces — a fractal set of non-integer dimension.

Palmer makes a conjecture — but a natural one — that the dynamics of the stuff of our Universe may be similarly described as approaching some invariant attractor. He refers to this as the 'invariant set postulate' — that the Universe is evolving causally and deterministically on (or very close to) some measure-zero, fractal invariant set. This is simply an assumption, although he points to some basic aspects of the dynamics of gravity (especially the loss of information in black holes) that could provide a mechanism for the progressive loss of phase-space volume. This would lead to an attractor.

It's here that things become really interesting. If you suppose that the trajectory of the Universe is flowing over some crazy fractal invariant set, does this idea actually lead us to any of the weird stuff we know from the quantum world — things like uncertainty, the impossibility of certain variables taking simultaneous definite values or the spooky and seemingly non-local Bell-type correlations? Palmer suggests that much of this falls out almost immediately.

For example, take the Schrödinger equation itself. Despite its loose similarity to the classical Liouville equation, it also differs in a number of extremely important ways. It contains a factor i = √(−1) and also Planck's constant. Moreover, whereas the focus of the Liouville equation is a normal function defined over a phase space, the Schrödinger equation involves a wave function defined in an abstract and complex-valued Hilbert space. Where might these differences come from? Palmer argues that all three tumble out of the invariant-set perspective once one makes an explicit mathematical representation of the dynamics on this set.

Out of these symbolic dynamics also emerges another profound idea — that physics itself might be non-computational in a way few physicists have considered. That is, that the dynamical evolution on the invariant set may involve processes that cannot be computed even though they remain completely deterministic. This idea has been proposed before by Penrose, who suggested that non-computational processes might arise from a proper treatment of gravity. Palmer's proposal is consistent with this idea, but places the origin in the fractal structure of the attractor. This raises the appealing idea that quantum mechanics might look strange not because it's non-deterministic, but because it is not computable.

By this point, I'm sure readers are wondering: but what about the Bell inequalities? Don't the experiments showing their violation prove that a local deterministic theory of this kind is impossible? Well, Palmer suggests the answer to that is 'no'. The fractal geometry of the attractor implies that some of the necessary preconditions for Bell-type inequalities to be derived can never be fully met. A key assumption is measurement independence — the idea that experimenters in distant separated regions can choose their experimental settings completely independently. Palmer suggests that the invariant set provides a 'non-conspiratorial' mechanism by which this postulate can be partially violated. The fractal nature of the attractor implies that certain counter-factual states involved in deriving Bell inequalities simply do not exist as possibilities.

Overall, Palmer comes at these issues from an entirely new direction, and brings deep and highly counter-intuitive ideas from dynamical systems theory into a somewhat foreign setting. The world of deterministic dynamics, he suggests, already has enough weirdness in it to account for everything in quantum physics — but only if we really take deterministic chaos seriously at the universal scale.