In a July 1991 Nature paper1, astronomers Andrew Lyne, Matthew Bailes and S. L. Shemar made an electrifying announcement: the discovery of the first planet outside our Solar System. To everyone's surprise, it was not orbiting a Sun-like star but a pulsar — the dense, spinning neutron-star offspring of a supernova explosion. The putative planet gave itself away by altering the period of radio-frequency flashes given off by the pulsar.

Credit: PETE ELLIS/DRAWGOOD.COM

Unfortunately, Lyne and Bailes had to retract this result a few months later after uncovering an error, which they reported2 in Nature in January 1992. The astronomers courageously announced that they had not corrected adequately for Earth's motion around the Sun. Lyne's revelation of the blunder at a meeting of the American Astronomical Society that month won him a standing ovation. But the story had a happier ending.

Immediately after Lyne's presentation, astronomer Aleksander Wolszczan announced that he and his colleague Dale Frail had discovered two planets orbiting another pulsar using the same technique. These turned out to indeed be the first discoveries of extrasolar planets. Wolszczan told me that Lyne's original paper had acted as a “confidence booster”, convincing him that the signals in his data were real. By the time Lyne withdrew his result, Wolszczan had performed enough tests to be certain.

Blunders are an essential part of the scientific process. Research is not a linear march to the truth but a zigzag path, involving trial and error. Mistakes are not the exclusive province of sloppy or inexperienced scientists. Even the brightest luminaries — including Charles Darwin and Albert Einstein — made serious blunders.

Truly innovative ideas require a willingness to embrace risks, and acceptance of the fact that errors can be portals to progress. Although this is well known in some private companies engaged in research and development, academics today are slow in recognizing the necessity of blunders.

Chemist Linus Pauling knew it. His former postdoc, Jack Dunitz, recalls being told: “Mistakes do no harm in science because there are lots of smart people out there who will immediately spot a mistake and correct it. You can only make a fool of yourself and that does no harm, except to your pride. If it happens to be a good idea, however, and you don't publish it, science may suffer a loss.”

Knotty problem

Preposterous ideas can lead to important insights. In 1867, the eminent physicist William Thomson (Lord Kelvin) proposed3 that atoms were not point-like but 'knotted vortex tubes of the ether'. Ether was the supposed fluid that pervaded space, providing a medium for electricity and magnetism.

Inspired by work on vortices in fluids by the nineteenth-century German physicist Hermann von Helmholtz, Kelvin identified three characteristics of knotted vortex tubes that made them attractive models for atoms.

First, vortices in fluids were astonishingly stable — mirroring to Kelvin the “unalterable distinguishing qualities” of atoms — and each knot could be classed according to its geometrical properties. Second, the variety of chemical elements could reflect the “endless variety” of knots. Finally, just as smoke rings vibrate, the oscillations of ether vortex tubes might produce atomic spectral lines.

To explain the periodic table, Kelvin needed to classify knots according to their forms, discarding any that could be manipulated from one to another. In Kelvin's theory, the circular 'unknot' represented the hydrogen atom; the triple-looped 'trefoil', carbon.

Kelvin's theory of vortex atoms is obviously wrong. The ether does not even exist. But these failures did not deter everyone. Whereas physicists lost interest for a while, knots began to intrigue mathematicians, becoming an active area of study for decades.

In the 1980s, knot theory reconnected with physics. Mathematician Vaughan Jones discovered an algebraic expression that is unique for every knot. Physicist Edward Witten linked it to quantum field theory, the branch of physics that describes fields and the subatomic world. In classical physics, the path of a particle travelling from point A to point B is determined by Newton's laws of motion. In the quantum regime, one has to consider all the possible paths connecting A to B, including winding and knotted ways.

Subsequent work linked knots, quantum field theory and string theory, which by describing particles as vibrations of strings, harks back to Kelvin's idea. Today, knots are used in chemistry and biology to analyse the actions of enzymes on DNA molecules. In a process known as site-specific recombination, enzymes align segments of the genetic sequence, cut the two strands of DNA open and recombine the four ends in various ways, which can be described using knot theory.

Extraordinary claims

Blunders are sometimes hard to correct. Modern experiments can be so intricate and require such big investments in time and funds that replicating them becomes prohibitive. When a result is widely assumed to be wrong, few scientists are motivated to repeat the work.

But there can be rewards for doing so. The sensational claim4 in Science by geomicrobiologist Felisa Wolfe-Simon and her colleagues to have discovered a bacterium that substitutes arsenic for phosphorus to sustain its growth brought a wave of criticism.

A few critics checked the experiment, including microbiologist Rosemary Redfield at the University of British Columbia in Vancouver, Canada, who blogged the process (see go.nature.com/bmb62d). The effort proved fruitful, showing that the bacterium goes to great lengths to dodge arsenic. Redfield and her colleagues detected no arsenic in the bacterium's DNA to much lower limits than in the original paper. Molecular biologist Dan Tawfik and his team at the Weizmann Institute of Science in Rehovot, Israel, identified the mechanism by which some of the proteins of this and related bacteria bind to phosphate and not to arsenate.

Although one lesson is obvious — extraordinary claims require extraordinary evidence — the original paper still had some scientific value. It stimulated discussion and inspired curiosity about different possibilities for life.

In the nineteenth century, Scottish author Samuel Smiles wrote: “We often discover what will do, by finding out what will not do; and probably he who never made a mistake never made a discovery.” His statement should not be taken as advocacy for slapdash science but as an encouragement to think originally and take calculated risks.

Can research failure be accommodated in today's fast-paced, funding-starved, impact-driven atmosphere? I believe it must. We should make space for risky scientific proposals in grant and evaluation processes.

Until a decade ago, the committees that allocated observing time on the Hubble Space Telescope were encouraged to give up to 10% of the time to proposals with a low probability of success but potentially high return. A similar philosophy could be adopted more widely.

One problem is that committees tend not to approve risky programmes. Efforts to reach consensus converge to a mean. Such obstacles can be overcome if decisions are left to one person. In the case of Hubble, a pool of 'director's discretionary time' on the telescope is available, for which anyone can apply. From it came the Hubble Deep Field, one of the most detailed images of the Universe ever made.

Today, telescopes including Hubble are turned towards addressing the profound outcome of another 'blunder'. Einstein regretted his attempt to model a static cosmos using a repulsive-gravity force (see 'Did Einstein ever say “biggest blunder”?'). But since it was revealed by supernovae observations in 1998 that our Universe is accelerating, understanding the nature of that repulsive force is one of the biggest challenges that physics faces today.

Researchers must embrace blunders that come from thinking outside the box. Evaluation processes should allow for originality, even at the risk of false starts and blind alleys.