I have to admit to a sneaking hope that some new physics will emerge that makes our world locally realistic after all.

If quantum theory is correct, then the world is fundamentally different from what most of us suppose. That's the gist of John Bell's famous 'theorem' of forty-one years ago, which suggests that quantum theory is inconsistent with so-called local realism — the notion that the world exists 'out there' with properties of its own, and that physical influences always act locally, without any 'spooky' action at a distance.

Quantum theory could be wrong, of course, but many experiments back it up — and I don't know a single physicist who finds this surprising. Measure the properties of pairs of entangled particles — such as the polarization of photons sent to very distant locations — and you find correlations stronger than any locally realistic picture can explain. Every few months another experiment gives similar confirmation, with better statistics, and you have to wonder: do further experiments have any point? It is tempting to answer no, but the issues involved could be more subtle than many physicists suppose.

It is widely acknowledged that no experiment has yet been truly 'loophole free'. In particular, photon detectors only detect a fraction of the photons in any experiment. If these pairs give an unbiased sample of all pairs, this is no problem. But what if the physics that determines which photons get detected is somehow linked to the physics that underlies the correlations?

That may seem unlikely, and closing this detection loophole may be no more than a mopping-up operation. “It is hard for me to believe,” Bell himself commented, “that quantum mechanics works so nicely for inefficient practical setups and is yet going to fail badly when sufficient refinements have made.” Then again, we cannot know the results of an experiment in advance.

In 1933, sensible physicists thought Otto Stern was wasting his time measuring the magnetic moment of the proton, as this particle would obviously behave much as the electron, following Dirac's theory. But Stern found a moment two-and-a-half times larger than expected — prefiguring the discovery of internal nucleon structure. It makes sense to close loopholes, whatever our expectations.

Besides, there's another possibility that Bell, apparently, did not consider — that local realism might still prevail, even if quantum theory doesn't fail. Ian Percival, among others, has argued that even if quantum theory is strictly correct, it could be that some quantum states — in particular those that allow 'weak non-locality' of the kind that violates local realism — might be impossible to realize in practice. The analogy would be with the second law of thermodynamics, which rules out many processes (for example, heat flowing in the wrong direction) that otherwise respect the laws of physics.

In principle, quantum theory could be correct, but constrained, and in such a way that no loophole-free experiment is in fact possible. It's certainly a fascinating possibility, worth taking seriously, and several groups are doing just that, by pursuing loophole-free experiments. Time will tell if they can realize them in practice.

Personally, I have to admit to a sneaking hope that they cannot — and that some new physics will emerge that makes our world locally realistic after all, as Einstein insisted it should be, even though quantum theory remains correct. Maybe, just maybe, Einstein and Bohr could both have been right.