Albert Einstein famously devoted the later part of his life to seeking a theory that would offer, at least in principle, a comprehensive description of the laws of nature. This ‘unified field theory’, Einstein believed, would endow all of nature's laws with the beauty of general relativity. Ultimately, Einstein left us with plenty of inspiration, but not many ideas about how to proceed.

In fact, there are ample reasons why one might doubt whether Einstein's vision is achievable, or at least achievable in the foreseeable future. Crucial clues may be hopelessly out of reach. When looking back at Einstein's own work, most physicists would say that many of the most important clues for a unified field theory — involving strong and weak nuclear interactions, the role of gauge theory and the world of elementary particles — were simply not known in Einstein's day.

Moreover, even if we could somehow find the unified field theory, it is not at all clear whether we could determine that it is right. From a simple combination of Planck's constant, the speed of light, and Newton's gravitational constant, one can construct a natural unit of length — the Planck length. First defined by Max Planck a century ago, this length is so fantastically small that if it, or something close to it, is fundamental in physics, then some of the most important phenomena may be permanently beyond our experimental reach.

I remember vividly how impressed I was in 1981 when the distinguished experimentalist Norman Ramsey (who later won the Nobel prize for his work on magnetic resonance) forecast that within 50 years there would be a clear outline of a unified field theory, “with all the forces fitting in together, even if not perfectly”. I certainly did not see any useful way to work on such a thing, and doubted that I would see it.

Meanwhile, I was only dimly aware of the work that was being done by Michael Green, John Schwarz, Lars Brink — and very few others — to revive string theory. Originally a candidate ‘theory of the nuclear force’, string theory had been developed and discarded a decade earlier. Its revival was motivated by the hope that it would give the basis for a unified field theory.

Credit: J. KAPUSTA/IMAGES.COM/CORBIS

By 1982 or 1983, I began to notice this work, which had advanced to the point that it was possible to formulate fairly convincing theories of quantum gravity unified with matter. There was, however, what seemed like an obvious flaw. String theory seemed incompatible with ‘parity violation’ in weak interactions (such as nuclear beta decay). Parity violation — the fact that the laws of nature are not invariant under a mirror reflection — is one of the most important findings ever made about elementary particles.

In 1984, this flaw was abruptly overcome when Green and Schwarz discovered an elegant new mechanism of ‘anomaly cancellation’. Not only could the weak interactions violate parity but, especially after the invention of the heterotic string, it soon became possible to derive semi-realistic models of elementary particles with all their known forces, including gravity. At this point, it really did seem reasonable to work on unified field theory.

I suppose that there are three basic reasons why string theory has attracted so much interest in the past 20 years. One is that it is there. String theory is the only known generalization of relativistic quantum field theory that makes sense. The framework of special relativity plus quantum mechanics is so rigid that it practically forces quantum field theory upon us. The tightness of the modern framework is one of the main reasons why physicists were able to discover what has become the standard model of elementary particles. A big idea like a consistent generalization of quantum field theory comes along only every now and then. So we are duty-bound to take it seriously.

A second reason has to do with what physicists have learned in developing string theory. String theory forces general relativity upon us, whereas standard quantum field theory apparently makes it impossible to incorporate general relativity. And string theory leads in a remarkably simple way to a reasonable rough draft of particle physics unified with gravity.

And finally, string theory has proved to be remarkably rich, more so than even the enthusiasts tend to realize. It has led to penetrating insights on topics from quark confinement to quantum mechanics of black holes, to numerous problems in pure geometry. All this suggests that string theory is on the right track; otherwise, why would it generate so many unexpected ideas? And where critics have had good ideas, they have tended to be absorbed as part of string theory, whether it was black-hole entropy, the holographic principle of quantum gravity, noncommutative geometry, or twistor theory.

But what is string theory? It may well be the only way to reconcile gravity and quantum mechanics, but what is the core idea behind it? Einstein understood the central concepts of general relativity years before he developed the detailed equations. By contrast, string theory has been discovered in bits and pieces — over a period that has stretched for nearly four decades — without anyone really understanding what is behind it. As a result, every bit that is unearthed comes as a surprise. We still don't know where all these ideas are coming from — or heading to.

One day we may understand what string theory really is. But even if we do, and the theory is on the right track, will we be able to learn how it works in nature? I certainly hope so. Realistically, it all depends on many unknowns, including the nature of the answer, how clever we will be, and the clues we can get from experiment.

FURTHER READING

Ramsey, N. F. Phys. Today 34, 26–34 (November 1981).

Zwiebach, B. A First Course in String Theory (Cambridge Univ. Press, 2004).

Greene, B. The Elegant Universe (W. W. Norton, 1999).