Tim’s adventures in quantum mechanics

Sabine Hossenfelder

Munich Center for Mathematical Philosophy, Germany

The 2022 Nobel Prize in Physics was awarded to Alain Aspect, John Clauser, and Anton Zeilinger for experimental tests of Bell’s inequality. Countless global headlines declared that the Prize had been awarded for proving that “spooky action at a distance” is real. I can only imagine how Albert Einstein, who coined the phrase, would have felt, but unfortunately (or fortunately, depending on perspective), he is dead. I don’t have to imagine how Tim Palmer would have felt because I know. He was, understandably, rather frustrated.

Einstein’s Spooky Action

Albert Einstein coined the phrase “spooky action at a distance” (“spukhafte Fernwirkung”) in a 1947 letter to Max Born (Einstein, 1947). He was referring to the key mystery of quantum mechanics: the measurement process. In quantum mechanics, everything is described by a wave-function. The wave-function, however, is not itself observable, and it does not predict the outcome of a measurement. From it, one can merely calculate probabilities for measurement outcomes. This is why the question whether the wave-function itself is “real” has remained a headache for philosophers.

That the wave-function only predicts probabilities of measurement outcomes means that it must be updated when one makes a measurement. For if one has made a measurement, one knows the outcome with 100% chance, even if previously the probability of obtaining this outcome from the wave-function was only, say, 50%. This update of the wave-function (sometimes also called the “collapse” or “reduction” of the wave-function) happens instantaneously, everywhere in space. It is therefore non-local: The act of measurement in one place seems to immediately affect another place.

An example that came from Einstein himself in 1927 is that the wave-function of a single electron, when left undisturbed, will spread radially into all directions of space. And yet, when one measures the position of the electron, it is suddenly in one place, meaning the probability that it is anywhere else jumped to zero.

In 1927, Einstein used the phrase “an entirely peculiar mechanism of action at a distance” for this behaviour (Bacciagaluppi and Valentini, 2009). I am not sure why he later referred to it as “spooky” rather than “peculiar,” but it might have reflected the opinion which had by then set in the physics community, which is that the update of the wave-function might be instantaneous – violating the speed-of-light limit – but it is also unobservable, so no violations of Einstein’s theories is ever measurable.

The term “spooky action” is nowadays often mistakenly used to refer to entanglement. But entanglement is merely a type of correlation. It can be non-local, but there is no action in it. Most importantly, if one has a pair of entangled particles, spatially separates them, and performs an action on one of them that is not a measurement, then the other particle will not be affected.

Bell’s Theorem

The obvious interpretation for the update of the wave-function is that the wave-function isn’t real itself, it is merely a way of keeping track of probabilities, but there is an underlying reality in which a particle either went one way or it didn’t. The wave-function, then, describes an ensemble of states in an underlying theory, in much the same way that we use ensembles in statistical mechanics.

When one measures a particle, one merely updates information about where it is. There is nothing “spooky” about it because no interaction happened.

In this case, the true state of the particle was determined by additional information that quantum mechanics does not encode. This missing information is often referred to as “hidden variables” and it seems to be the explanation of the measurement update that Albert Einstein favoured. This isn’t so surprising because it’s what is required if one wants to return to a local theory that agrees with Einstein’s theories of Special and General Relativity.

The particle physicist John Bell, however, formulated a very influential theorem in 1964 (Bell, 1964). He assumed that the hidden variables cannot depend on the measurement settings, and then showed that any local explanation using these hidden variables must obey a bound on the correlations between entangled particles. What Aspect, Clauser, and Zeilinger showed is that this bound is violated in experiments.

This demonstrates first and foremost that the predictions of quantum mechanics are correct. And their experiments opened the way to a vast number of applications of quantum mechanics, from quantum teleportation to quantum cryptography, quantum metrology, and quantum computing. The Nobel Prize was without doubt well deserved.

But the experiments that the Prize was awarded for didn’t show that spooky action at a distance must be real. The reason is that Bell, for his proof to work, had to use the rather odd assumption that the measurement settings are not correlated with the hidden variables. This assumption is often referred to as “measurement independence”. Violations of measurement dependence do not require non-locality or “spooky action”, in Einstein’s words. As a consequence, a local theory with hidden variables can very well violate Bell’s equality, it just has to violate measurement independence.

Indeed, seeing that locality is a founding principle of Einstein’s General Relativity, one would have thought that the conclusion which physicists would draw from Bell’s theorem would have been that measurement independence is violated. This, however, is not how things developed.

Instead, physicists at the time (John Clauser among them) associated the assumption of “measurement independence” with the “free will” of the experimenter to choose the measurement settings. It was also sometimes referred to as “free choice”. A violation of this assumption then required giving up the free will of the experimentalist, which sounds hugely worrisome.

I believe the reason for this is a misinterpretation of the mathematics. It is mistaking a correlation for a causation, and in addition positing that the hidden variables are properties of particles, while they could instead by entirely contextual. If one, in that way, falsely thinks that violating measurement independence requires that the particle which one wants to measure causes a specific measurement setting, then it is easy to see why this possibility was discarded swiftly, and dismissed as a “conspiracy theory”. John Bell gave it the name “superdeterminism”.

That a superdeterministic hidden variables theory (ie, one that violates measurement independence) could both be local and explain observations (in particular violations of Bell’s inequality) is widely accepted in the quantum foundations community, but scarcely known outside. Even physicists in other disciplines typically don’t know this. This is despite several toy-models having been put forward, starting in the 1980s, to mathematically prove the point. As the 2022 Nobel Prize headlines demonstrate, few science writers have ever heard of it.

Maybe what was missing was an underlying model in which those violations would arise. This is what Tim Palmer proposed with his approach that he dubbed “Invariant Set Theory” (Palmer, 2011).

Palmer’s Invariant Set Theory

Quantum mechanics is a linear theory, and this linearity is the origin of its problems. If a dead cat is a solution of the theory and an alive cat is, too, then a sum of a dead and an alive cat is also a solution of the theory. It is this absurdity that Erwin Schrödinger highlighted with his thought-experiment.

Any hidden variables theory that does away with the instantaneous update of the wave-function is therefore necessarily non-linear. It isn’t strictly speaking necessary, but it is a reasonable guess, that such a theory would also be chaotic. This is because if the outcome of a quantum measurement didn’t sensitively depend on the initial conditions, then chances are we would already have noticed that measurement outcomes are deterministic after all.

Tim now proposed that violations of measurement independence could come about from a specific property that systems with chaotic dynamics can have. They can have strange attractors that are fractals. Tim posited that the state-space of the hidden variables theory is really such a fractal, and not, as we normally assume, a continuum. Because it is a fractal, this state-space has gaps. And because it has gaps, certain combinations of prepared states and measurement settings just do not exist – they are not elements of the state-space of theory.

However, if one does not know about this fractal structure of the state-space, and instead assumes it is continuous, then one finds that the prepared state and the measurement settings are correlated. That is, the assumption of measurement independence to Bell’s theorem is violated.

Invariant Set Theory hence provides an explanation for why a hidden variables theory would not fulfil the assumptions of Bell’s theorem, and yet be local. In a series of papers, Tim has demonstrated that this simple idea reproduces the observed outcomes of quantum experiments, entirely without spooky action at a distance.

The Physics Nobel Prize in 2042?

It is too early to judge whether Tim’s proposal, in its present form, is the correct description of nature. But I believe that he is on the right path. It is a lonely path, for the only other people I know who work on superdeterminism at the moment are Gerard ‘t Hooft and myself. Of the three of us, Tim was the first to take this path. By now he is so well ahead of his time, that few physicists (not to mention the wider public) even understand what he is on about. But eventually, I am sure, they will come to agree that spooky action has no place in physics and that, therefore, superdeterminism must be correct. And who knows, maybe the 2042 Nobel Prize in physics will go to Tim Palmer and his Invariant Set Theory.