The clamour in a Panamanian rainforest is deafening to human ears: bugs shriek, birds sing and bats screech throughout the humid night. To avoid attracting predators, male katydids (Tettigoniidae) trill out short, infrequent mating calls less than a second long.

Ecology is one of a few fields moving towards the multiple-working-hypotheses method of investigation. Credit: Philip Lee Harvey/Cultura RM Exclusive/Getty

Postdoc Laurel Symes, who studies sensory perception and decision-making at Dartmouth College in Hanover, New Hampshire, wants to understand how female katydids find their mates. She first thought they must have highly sensitive hearing. But she juggles other ideas at the same time: maybe katydids always meet up on a certain type of host plant, have neural mechanisms that filter out background noise or use another trick entirely.

These aren't just idle musings: Symes's collection of hypotheses is an integral part of her research. The approach helps her to home in on answers and avoid investment in a sole idea — a common tendency in science that can lead to trouble. History contains numerous examples of scientists who missed important clues because they clung too tightly to a favourite hypothesis. One way to avoid this fate is to consider many potential hypotheses.

Proponents of the multiple-working-hypotheses method say that it prevents scientists from developing 'tunnel vision', and enables them to embrace the possibility that several hypotheses might be true at once. Practising the approach takes discipline: researchers must brainstorm possible explanations for a scientific phenomenon before collecting or analysing data, and use techniques such as scrambling the order of samples and blinding data to help to counteract favouritism. It also demands that scientists remain open-minded during the entire research process, and continually refine their hypotheses.

A long history

The method of multiple working hypotheses was formally articulated1 in 1890 by geologist Thomas Chrowder Chamberlin, then president of the University of Wisconsin–Madison. Building on the ideas of fellow geologist Grove Karl Gilbert, Chamberlin warned that when scientists come up with an original idea, they tend to develop affection for it, which can cloud their ability to do objective work. He argued that the solution was to generate and explore a family of hypotheses. By coming up with alternatives, he suggested, scientists would not be inclined to favour one idea.

If the hypotheses are mutually exclusive, you are going to learn something.

Although the concept has faced criticism, aimed mainly at the impossibility of conceiving — let alone testing — all possibilities, many scientists say that it is as relevant today as ever. The pressure to publish in high-profile journals, win grants and build a reputation can prompt researchers — consciously or not — to seek support for pet ideas. One study posted to the preprint server arXiv2 in June found that when programmers introduced these kinds of incentives into a model, simulated research groups succumbed to pressures to show support for original ideas, often erroneously.

Ecologist Barry Brook of the University of Tasmania in Australia thinks that resurrecting Chamberlin's ideas could help. In 2007, he co-authored a paper on the merits of using multiple working hypotheses for twenty-first-century science3. In many cases, he argues, the method produces more insightful results than testing null hypotheses, which reveals only whether a specific factor has a discernible effect. Multiple hypotheses, by contrast, can help scientists to work out whether that effect is important, and whether several factors might be at play.

Brook, for example, wanted to know why small mammals such as brown bandicoots (Isoodon macrourus) were disappearing from Northern Australia's Kakadu National Park. Many scientists had pointed in the past to introduced predators, such as cats, which seemed plausible. But when he considered other hypotheses and looked at historical population data, he found that cats had a negligible role, and that intense wildfires bore most blame4. “You can be surprised at how little support most of your well-crafted hypotheses can have,” he says.

It might seem simpler to consider just one possible explanation, but ignoring other models can be dangerous. “That's not only dishonest, but it will also lead you down bad inferential pathways,” Brook says.

Resist temptation

It can be challenging to put the method into practice because researchers must battle their own natural enthusiasm for an alluring idea. The first step is to set aside time to articulate other hypotheses before one starts to gain traction. If not, a favoured hypothesis might skew the process of data collection or analysis when one heads out into the field, starts an experiment or dives into a data set. “If you have a hypothesis or you're looking for a pattern, sometimes you won't actually honour what pattern is there,” says Kathleen Nicoll, a geographer at the University of Utah in Salt Lake City.

When coming up with a collection of hypotheses, it can be helpful to have patience and consult labmates — and to include a seemingly outrageous hypothesis. This idea was first advocated in 1926 by William Morris Davis, a retired geologist from Harvard University in Cambridge, Massachusetts, as a way to break out of conventional thinking. Many notable scientific advances fall into this category, including Alfred Wegener's then-scandalous claim in 1912 that continents migrate across Earth's surface5 (they do), and the heretical proposal, developed in the 1920s by geologist J Harlen Bretz, that a catastrophic flood scoured out the heavily channeled landscapes of Washington (in fact, many violent floods swept through the region).

Symes finds that using multiple hypotheses yields the best results if researchers generate ideas that rely on different processes and make distinct predictions. In her research, a host-plant preference might lead to katydids having the same food in their guts, whereas using sound might imply that female katydids in Panama have more sensitive ears than species in forests without predatory bats. By identifying possible outcomes, she can design her experiments in ways that help to distinguish these ideas. “If the hypotheses are mutually exclusive or different in their mechanism, then you are going to learn something,” she says.

Consideration of multiple working hypotheses continues during data processing and analysis, when scientists must take other steps to protect their objectivity (see 'Don't play favourites').

For Lydia Tackett, who studies marine fossils at North Dakota State University in Fargo, the solution is as simple as analysing samples out of order. Working chronologically through a geological sequence led her to identify trends prematurely and anticipate what she would find in subsequent layers. “Now, I collect the bulk samples I need and randomize the order,” she says. She codes them so that she doesn't know exactly which layer each sample came from.

Others rely on statistical tools. Instead of using P values to reject individual models one at a time, Trevor Branch, a fisheries scientist at the University of Washington in Seattle, embraces a model-selection technique called Akaike's information criterion (AIC). This statistical method determines which of a set of models best explains data collected about an often-complex system. Branch says that it's a mathematical way of implementing Chamberlin's method of multiple working hypotheses.

Brook uses the AIC as well as the similar Bayesian information criterion, which is useful for distinguishing between a few simple models. When several models seem to be true, these methods help to weight their relative importance, so that their combined effects can be explored through something called multimodel inference. That involves merging several different models and considering them simultaneously to explain as much as possible.

Physicists and astronomers often take extreme measures to prevent researcher bias from creeping into their analyses. Saul Perlmutter, an astrophysicist at the University of California, Berkeley, relies on software or colleagues to hide potentially telling clues in the data before he sees them, a technique called blind analysis. This might include adding randomly generated numbers to data values, shifting them by random amounts or hiding the axes on a graph. The goal is to make sure that the researchers don't see anything that could prime their minds towards a particular interpretation, such as a preliminary trend or hint of a discovery.

Before unblinding data, scientists on Perlmutter's team must circulate a memo explaining their hypotheses and how they plan to test and differentiate between them. “Everybody can decide ahead of time whether that feels fair — that they haven't treated any of the alternatives differently than the others,” he says. Last year, Perlmutter and psychologist Robert MacCoun of Stanford University in California argued in a Nature Comment6 that this approach could reduce researcher bias in many fields.

Of course, there are situations in which multiple hypotheses aren't helpful — or even feasible. If researchers stumble on a mysterious finding, they might struggle to come up with even a single plausible explanation. And even if they can cobble together a few, there is no guarantee that the correct hypothesis is among them. This is why hypotheses must remain 'working', so that they can be refined in light of new information.

Other situations present the opposite challenge: too many hypotheses. Freya Blekman, an experimental physicist at the Dutch-speaking Free University of Brussels, searches for elementary particles at facilities such as the Large Hadron Collider at CERN, Europe's particle-physics lab near Geneva, Switerland. In her field, theorists have already posited countless possibilities, and her task is to work out which ones the evidence supports.

Because these models are often mutually exclusive, she typically evaluates them one at a time using P values — albeit held to an exceptionally high standard of significance. In fields such as psychology and medicine, there is a growing movement to abandon this technique because it can tempt researchers to seek out analytical approaches that produce significant results. But Blekman says that the physics community has largely eliminated this problem through blinding and by creating a culture so steeped in the ethos of multiple hypotheses that finding nothing is as important as finding something. “In our field, a null result is a valuable result,” she says.

Indeed, the method of multiple hypotheses doesn't always have to be practised at the individual level, and can take place across entire fields. Different groups can advance various hypotheses, as long as they remain open-minded, and the peer-review process can also help to promote the practice. “I think we have a duty as editors and reviewers to bring up alternatives,” says Branch, “and to require authors that come up with a new hypothesis to also include alternatives when they bring it up the first time around”.

Regardless of how they apply the method, many researchers say that they stumbled across the idea of multiple hypotheses by accident, as graduate students or later. Branch had never heard of the concept until a few years ago, but was so struck by it that he wrote an article last year arguing that researchers should not seek a single, universal explanation for how fisheries affect marine food webs, but should consider how different models might apply in various parts of the world7.

A few researchers say that their advisers encouraged them to read classic philosophy-of-science texts, such as Thomas Kuhn's Structure of Scientific Revolutions (Univ. Chicago Press, 1962), or fostered discussions on the practical side of the scientific method at lab meetings. But many scientists can make it through their entire careers without any formal training in how to develop hypotheses.

That's too bad, because learning and applying the multiple-hypothesis method can improve the calibre of scientists' work and empower scientists themselves, says Symes, who published a guide last year on teaching the research process8. “It always pains me to see students who define success and failure as whether they support a particular hypothesis,” she says. “Failing is not collecting the data you need. Succeeding is being able to differentiate the possibilities.”