Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • News & Views
  • Published:

DREAMing of benchmarks

Nature Biotechnology volume 33, pages 49–50 (2015)Cite this article

Subjects

The concept of carrying out a community challenge to solve a problem in biomedical research was popularized by an initiative called DREAM (Dialogue for Reverse Engineering Assessments and Methods). DREAM was launched by IBM's (Armonk, NY, USA) Gustavo Stolovitzky and collaborators, who in 2006 began brainstorming about computational platforms that could facilitate research in systems biology. Out of those conversations (in DREAM 1.0) came the first set of challenges (DREAM 2.0).

DREAM challenges focus on problems amenable to solution by a community-wide effort. Data, algorithms and software associated with the challenge are hosted on a technology platform (called Synapse) that is provided by the nonprofit Sage Bionetworks (Seattle), headed by Stephen Friend. Each year, at the DREAM annual meeting, which takes place jointly with the Annual RECOMB/ISCB Conference on Regulatory and Systems Genomics (in San Diego in 2014) and is open to anyone, participants and organizers discuss problems that would be well-suited to the challenge concept. Once the question is framed, “we do a dry run in which we analyze the data set using off-the-shelf methods to see if there is a signal,” says Stolovitzky. “We need to know if it is possible to make a prediction with the available data before issuing a challenge,” he explains. The submissions are scored to identify top-performing algorithms, and the results—and the code of the top performers—are made publicly available.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Boutros, P.C. et al. Nat. Genet. 46, 318–319 (2014).

    Article  CAS  Google Scholar 

  2. Costello, J.C. et al. Nat. Biotechnol. 32, 1202–1212 (2014).

    Article  CAS  Google Scholar 

  3. Bansal, M. et al. Nat. Biotechnol. 32, 1213–1222 (2014).

    Article  CAS  Google Scholar 

  4. Küffner, R. et al. Nat. Biotechnol. 33, 51–57 (2015)

    Article  Google Scholar 

  5. Bilal, E. et al. PLoS Comput. Biol. 9, e1003047 (2013).

    Article  CAS  Google Scholar 

  6. Margolin, A.A. et al. Sci. Transl. Med. 5, 81re1 (2013).

    Google Scholar 

  7. Plenge, R.M. et al. Nat. Genet. 45, 468–469 (2013).

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Associate Editor,

    Irene Jarchum

  2. Senior Editor,

    Susan Jones

Authors
  1. Irene Jarchum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susan Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jarchum, I., Jones, S. DREAMing of benchmarks. Nat Biotechnol 33, 49–50 (2015). https://doi.org/10.1038/nbt.3115

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt.3115

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics