Will we ever understand quantum theory?

And finally for now… a piece for BBC Future's "Will We Ever...?" column on the conundrums of quantum theory (more to come on this, I think).

___________________________________________________________

Quantum mechanics must be one of the most successful theories in science. Developed at the start of the twentieth century, it has been used to calculate with incredible precision how light and matter behave – how electrical currents pass through silicon transistors in computer circuits, say, or the shapes of molecules and how they absorb light. Much of today’s information technology relies on quantum theory, as do some aspects of chemical processing, molecular biology, the discovery of new materials, and much more.

Yet the weird thing is that no one actually understands quantum theory. The quote popularly attributed to physicist Richard Feynman is probably apocryphal, but still true: if you think you understand quantum mechanics, then you don’t. That point was proved by a poll among 33 leading thinkers at a conference on the foundations of quantum theory in Austria in 2011. This group of physicists, mathematicians and philosophers was given 16 multiple-choice questions about the meaning of the theory, and their answers displayed little consensus. For example, about half believe that all the properties of quantum objects are (at least sometimes) fixed before we try to measure them, whereas half felt that these properties are crystallized by the measurement itself.

That’s just the sort of strange question that quantum theory poses. We’re used to thinking that the world already exists in a definite state, and that we can discover what that state is by making measurements and observations. But quantum theory (‘quantum mechanics’ is often regarded as a synonym, although strictly that refers to the mathematical methods developed to study quantum objects) suggests that, at least for tiny objects such as atoms and electrons, there may be no unique state before an observation is made: the object exists simultaneously in several states, called a superposition. Only during the measurement is a ‘choice’ made about which of these possible states the object will possess: in quantum-speak, the superposition is ‘collapsed by measurement’. Before measurement, all we can say is that there is a certain probability that the object is in state A, or B, or so on. It’s not that, before measuring, we don’t know which of these options is true – the fact is that the choice has not yet been made.

This is probably the most unsettling of all the conundrums posed by quantum theory. It disturbed Albert Einstein so much that he refused to accept it all his life. Einstein was one of the first scientists to embrace the quantum world: in 1905 he proposed that light is not a continuous wave but comes in ‘packets’, or quanta, of energy, called photons, which are in effect ‘particles of light’. Yet as his contemporaries, such as Niels Bohr, Werner Heisenberg and Erwin Schrödinger, devised a mathematical description of the quantum world in which certainties were replaced by probabilities, Einstein protested that the world could not really be so fuzzy. As he famously put it, “God does not play dice.” (Bohr’s response is less famous, but deserves to be better known: “Einstein, stop telling God what to do.”)

Schrödinger figured out an equation that, he said, expressed all we can know about a quantum system. This knowledge is encapsulated in a so-called wavefunction, a mathematical expression from which we can deduce, for example, the chances of a quantum particle being here or there, or being in this or that state. Measurement ‘collapses’ the wavefunction so as to give a definite result. But Heisenberg showed that was can’t answer every question about a quantum system exactly. There are some pairs of properties for which an increasingly precise measurement of one of them renders the other ever fuzzier. This is Heisenberg’s uncertainty principle. What’s more, no one really knows what a wavefunction is. It was long considered to be just a mathematical convenience, but now some researchers believe it is a real, physical thing. Some think that collapse of the wavefunction during measurement is also a real process, like the bursting of a bubble; others see it as just a mathematical device put into the theory “by hand” – a kind of trick. The Austrian poll showed that these questions about whether or not the act of measurement introduces some fundamental change to a quantum system still cause deep divisions among quantum thinkers, with opinions split quite evenly in several ways.

Bohr, Heisenberg and their collaborators put together an interpretation of quantum mechanics in the 1920s that is now named after their workplace: the Copenhagen interpretation. This argued that all we can know about quantum systems is what we can measure, and this is all the theory prescribes – that it is meaningless to look for any ‘deeper’ level of reality. Einstein rejected that, but nearly two-thirds of those polled in Austria were prepared to say that Einstein was definitely wrong. However, only 21 percent felt that Bohr was right, with 30 percent saying we’ll have to wait and see.

Nonetheless, their responses revealed the Copenhagen interpretation as still the favourite (42%). But there are other contenders, one of the strongest being the Many Worlds interpretation formulated by Hugh Everett in the 1950s. This proposes that every possibility expressed in a quantum wavefunction corresponds to a physical reality: a particular universe. So with every quantum event – two particles interacting, say – the universe splits into alternative realities, in each of which a different possible outcome is observed. That’s certainly one way to interpret the maths, although it strikes some researchers as obscenely profligate.

One important point to note is that these debates over the meaning of quantum theory aren’t quite the same as popular ideas about why it is weird. Many outsiders figure that they don’t understand quantum theory because they can’t see how an object can be in two places at once, or how a particle can also be a wave. But these things are hardly disputed among quantum theorists. It’s been rightly said that, as a physicist, you don’t ever come to understand them in any intuitive sense; you just get used to accepting them. After all, there’s no reason at all to expect the quantum world to obey our everyday expectations. Once you accept this alleged weirdness, quantum theory becomes a fantastically useful tool, and many scientists just use it as such, like a computer whose inner workings we take for granted. That’s why most scientists who use quantum theory never fret about its meaning – in the words of physicist David Mermin, they “shut up and calculate” [Physics Today, April 1989, p9], which is what he felt the Copenhagen interpretation was recommending.

So will we ever get to the bottom of these questions? Some researchers feel that at least some of them are not really scientific questions that can be decided by experiment, but philosophical ones that may come down to personal preference. One of the most telling questions in the Austrian poll was whether there will still be conferences about the meaning of quantum theory in 50 years time. Forty-eight percent said “Probably yes”, only 15 percent “probably no”. Twelve percent said “I’ll organize one no matter what”, but that’s academics for you.