Scientific contests are not new; X-Prize challenges have been running for years, as have various critical assessment projects where structural biologists or genomicists test their modeling mettle. But last year's DREAM7 challenge added an interesting twist. The use of Synapse—a computational platform that allows users to share data, as well as have access to programming codes and analytical tools—enabled competitors to riff on the best strategies based on a real-time 'leaderboard' that ranked the performance of each model. “It was great because we were able to use the other models that were in that ecosystem,” says Andrew Su, a DREAM7 participant and associate professor at the Scripps Research Institute in La Jolla, California. According to DREAM co-founder Gustavo Stolovitzky, of the IBM Computational Biology Center in Yorktown Heights, New York, the 'open-source' nature of the contest allowed researchers to quickly assess the performance of their approach relative to those of other competitors and thereby improve their methodology. He also notes that the whole of the pool in such competitions can be more than the sum of its parts. “There is wisdom in the crowds,” says Stolovitzky, “and if you aggregate the community predictions, it is very often better than the best model.”
The Sage-DREAM model primarily draws upon scientists with at least some biological expertise, but other groups are casting an even broader net. “There's a whole community of people out there that has skills [which] are useful to you that you are either unaware of or not tapped into,” says Eva Guinan of the Dana-Farber Cancer Institute in Boston. She and colleagues at Harvard abstracted a biological problem—analyzing gene structure in a repertoire of recombined antibody sequences—and posted it in computational format on TopCoder, an online programmers' community. The outcome was remarkable; within two weeks, they received 122 code submissions from programmers around the world, none of whom professed any biology expertise (Nat. Biotechnol. 31, 108–111, 2013). Out of these, 16 considerably outperformed existing best-in-class solutions in terms of processing speed and accuracy.
This is a preview of subscription content, access via your institution