In science, the victorious theories get all the accolades. We're intimately familiar with Einstein's theory of relativity, but have all but forgotten Irish physicist George Fitzgerald's ideas on the ether-induced contraction of moving bodies. We celebrate Maxwell's theory of electrodynamics, and see as mere footnotes to history a host of contemporary theories postulating invisible effluvia as explanatory mechanisms. It probably can't be otherwise; who, after all, can afford the luxury of exploring the nearly infinite world of ideas that just didn't work out?

Yet this filtering of history may blind us to the way that failed theories drive science forwards, in part by exploring false paths and making the painful mistakes required for learning, and also by yielding partial insights that, however incomplete and unsatisfying at the time, often turn out to be crucial elements of later success.

This notion certainly resonates with an exploration of the history of theories of (ordinary) superconductivity by Jörg Schmalian (http://arxiv/abs/1008.0447). A passing acquaintance with superconductivity brings to mind the successful theory of Bardeen, Cooper and Schrieffer, of course, and the more phenomenological work of Ginzburg and Landau, as well as of brothers Fritz and Heinz London. Yet I hadn't realized that this problem had attracted (and largely defeated) great physicists including Albert Einstein, Niels Bohr, Werner Heisenberg, Max Born and Richard Feynman.

As Schmalian shows, the history of their failed efforts reflects a fascinating process of gradual learning that largely laid the ground for the ultimate success of BCS. It also illustrates how these physicists, in struggling with this problem, were driven to create ideas that have influenced physics for the past half century and more.

Dutch physicist Kamerlingh Onnes first discovered superconductivity in mercury in 1911. Ironically, he was motivated in part by his doubt of Kelvin's theory that the resistivity of all substances should become infinite at sufficiently low temperatures. Another failed theory; another profound puzzle for physics.

The filtering of history may blind us to the way that failed theories drive science forwards.

At the time, Einstein, Bohr, Sommerfeld and others were struggling to build the early quantum theory. As the quantum hypothesis had explained black-body radiation, the photoelectric effect and myriad details of atomic spectroscopy, it isn't surprising that Einstein as early as 1922 suggested that superconductivity might also be a quantum phenomenon, speculating about “molecular conduction chains where electrons undergo continuous cyclic exchanges”, although he offered no specific theory.

More surprising — and of lasting historical importance — is Lev Landau's effort of 1933. All physics students today learn about Landau's theory of phase transitions, normally in the context of ferromagnetism. Landau, they learn, made a bold attempt to derive general insights about phase transitions by making only minimal assumptions about how the free energy of a substance might depend on temperature and other manifested properties such as magnetization.

Yet it turns out that Landau first proposed these ideas in the context of superconductivity, thinking not of magnetization but of electrical current. He expanded the free energy F around the state of zero current, j = 0, and argued that as the direction of the current shouldn't affect F, the odd terms should vanish. This gives an equation of the form F(j) = F(0) + aj2 + bj4. Assuming b > 0 and that a passes through zero at a critical temperature Tc, he showed that there could be an abrupt transition from zero to non-zero current below Tc.

This early theory conflicted with observations — it erroneously predicted j ∼ (Tc − T)1/2 just below the critical temperature — and Landau went back to the drawing board. Yet here already were the seeds of the later Ginzburg–Landau theory of phase transitions. And Landau's introduction of the notion of an 'order parameter' as a convenient handle on order and how it changes has influenced physics ever since, even if it did appear in a failed theory.

Equally imaginative, Schmalian relates, was Ralph Kronig, who had the crazy idea that the electrons in a metal might, if their kinetic energy were low enough, actually crystallize into a rigid electron solid, which would then be able to slide coherently through the ionic lattice. The rigidity of the electron crystal would suppress the scattering by individual electrons and account for the lack of resistance. Moreover, he suggested, superconductivity would naturally vanish above the temperature at which the electron crystal would melt.

This idea strikes me as so beautiful and simple that it seems a shame it didn't turn out to be true. Even so, Kronig hit on the profound idea that the coherence of a macroscopic state could strongly suppress the scattering of individual particles. He also introduced the notion of an electron crystal organized by Coulomb interactions, and did so two years before Eugene Wigner's famous work on the topic.

But it was other failed theories, striking closer to the core of the problem, that set the stage for its eventual solution. In particular, Schmalian notes that Fritz London in 1948 argued that it might be possible to derive the macroscopic London equations from first principles by considering coherent quantum states. In vague terms, he even suggested that such states might emerge naturally if exchange interactions could lead to attractive forces between electrons in momentum space.

It took nearly ten years for other physicists to put together the first complete microscopic theory along these lines. Experiments in the early 1950s confirmed the isotope effect — dependence of the superconducting transition temperature on ion mass — showing that lattice vibrations played a key role. Various theorists, including John Bardeen, showed that lattice vibrations could indeed cause an attraction between electrons. Most of the details fell into place when Leon Cooper demonstrated that any attraction would lead to bound pairs of electrons and behaviour closely associated with superconductivity.

But it may be that Landau, Kronig and all the others responsible for failures along the way deserve as much credit for this ultimate success as the crowned theory, even if that's not the way it seems in the textbooks.