The process of scientific discovery can be a messy one, and groundbreaking work that opens up new areas of research may not get everything right. Nonetheless, independent validation of the published data is crucial for building on prior work. In the last few years, scientists at Bayer and Amgen have reported their inability to reproduce published results from preclinical research in a large majority of cases (Nat. Rev. Drug Discov. 10, 712, 2011; Nature 483, 531, 2012). Although this dismal state of affairs is often decried in both the scientific and popular press, realistic solutions have been few and far between.

The Reproducibility Initiative, launched in mid-August by Science Exchange, a Palo Alto, California–based startup, aims to tackle this problem by engaging independent laboratories to repeat experiments and validate results.

The core business of the company is to match scientists seeking to outsource preclinical research with laboratories and contract research organizations that can perform whatever services are needed—everything from DNA sequencing and bioinformatics to tissue histology and metabolomic profiling. Studies to validate research findings would fit in as another service to be outsourced.

The scheme would work as follows: authors of a published study would submit methodologies and results, either published or not, to Science Exchange. The Initiative would then decide which experiments are most crucial to the conclusions of the study and play matchmaker by finding a third party to repeat the results—all for a fee paid by the original authors, of which Science Exchange takes a 5% cut. On the basis of the results of the validation experiments, the Initiative then decides whether or not the work has been successfully reproduced—if it has, they provide the authors with a certificate saying so. The authors of the original study can publish the results of the validation study at a special portal hosted by PLoS ONE and share the primary data in the open-access repository figshare (which is supported by a technology division of Macmillan Publishers, of which the Nature Publishing Group is also a part).

But how would this work in practice? Would researchers be willing to pay for a third party, not of their own choosing and with unclear expertise, to try to replicate their results? How would contract labs deal with work requiring more specialized equipment or skills, such as behavioral assays in animals? If the study is reproduced using starting materials provided by the authors—cell lines or mice, for example—does that count as full reproducibility, or does it perhaps merit an asterisk on the certificate? It's also easy to envisage cases in which some but not all findings of a study are replicated, so it may not be such a simple matter to decide whether the study overall has been reproduced. Lastly, as it's up to the scientists responsible for the original data to publish the findings of the replication study, failure to reproduce the results is not likely to be publicized.

Another set of issues has to do with how the Initiative is set up. As Science Exchange is a private, for-profit company, the operations and decisions of the Initiative will undoubtedly be subject to second guessing. Although the members of the Initiative's advisory board have impressive credentials and many of their studies on reproducibility have been valuable for this field, precisely how these individuals were picked for the board isn't clear. It's also not clear who at the Initiative would be dealing with the complexities of analyzing the results of the experiments and deciding whether a study has been reproduced.

The Initiative is currently seeking financial support from the US National Institutes of Health (NIH) and other funding agencies so that scientists don't have to dip into their existing grants to pay for the service. But if the NIH underwrites the costs, it's fair to wonder whether the middleman is superfluous. If the NIH is serious about getting into the business of therapeutics development and translational medicine, ensuring that the science it funds yields trustworthy results should become a larger priority.

To succeed, systematic validation of published research needs to be fully transparent and might best be orchestrated by the world's leading funders of biomedical science. The NIH's fledgling National Center for Advancing Translational Sciences (NCATS) could take up the reproducibility challenge itself and fund a network of centers committed to replicating promising findings. That would go a long way toward helping fulfill NCATS's mission of decreasing the risks associated with drug development and might serve as a model for funding agencies in other countries.