Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

No guarantees in translation

A drug company executive thinks academic institutions should give money back for research that turns out to be 'irreproducible'.

Like many others in industry, Michael Rosenblatt is fed up with the lack of reproducibility of academic research. According to the few studies that have attempted to quantify the problem, more than half of academic research findings are irreproducible. That is a headache for companies that build businesses from research results like Merck (where Rosenblatt is employed as chief medical officer). As a solution, Rosenblatt proposes “incentive-based” agreements between academia and industry involving “money-back guarantees.” Unfortunately, his idea is almost guaranteed never to be reproduced in the real world.

Current practice in research collaborations is that industry pays for promising academic work and provides funding for confirmatory studies. If it turns out the original data cannot be reproduced, the partnership comes to an end. End of story.

Writing in Science Translational Medicine (8, 336ed5, 2016), Rosenblatt suggests that, instead, academic research establishments should “stand behind” their data. They should do this by offering a form of “money-back guarantee” if results cannot be repeated. This, he argues, would mean that companies could proceed more rapidly with more projects. Although not written explicitly, this also means that industry would have to spend less effort and resources on confirmatory studies.

Offering a money-back guarantee would allow an institution with high confidence in its data not to replicate before partnering, he writes. But, equally, there would be a financial incentive for academic researchers to replicate before collaborating with industry. According to Rosenblatt, rather than validating work with an industrial partner, it could be done in the original laboratory, or in a colleague's laboratory or in a contract laboratory.

All well and good. Except Rosenblatt ignores the competitive market for academic assets; he assumes penny-pinched academic institutions will/can pay for confirmatory studies in the first place and will/can pay back for irreproducible studies; and he neglects the focus of academic science on exploration, rather than pharmaceutical R&D.

Imagine Merck's business development team (or another pharma company) asking a university technology transfer office or its regent to sign up to a “money-back” deal. What would be the circumstances under which the university would sign on to such terms? In no particular order: 1) every single other pharmaceutical company seeks deals on similar terms; 2) no other company is interested in the project; 3) the university is so confident in its asset that success is a certainty; or 4) the university wants to project confidence in a duff project and thereby attain some kind of deal rather than no deal at all. So, in theory, Merck might get the deals it wants if there were industry-wide collusion, or if no one else wanted the asset or if the assets were probably worthless.

The risk of non-reproducibility would undoubtedly be diminished if academic institutes would repeat more studies. But the premiums offered by a company for such work would have to be sufficient to overcome academia's systemic financial constraints and cultural inertia.

According to the National Science Foundation, federal funding for academic R&D has been in its longest multiyear decline since FY 1972. This is not a good environment to suddenly find new money to do industry validation studies. In terms of culture, why would grant-funded PhDs and post-docs choose to repeat research rather than moving onto the next intriguing problem? Why spend one to two years using scant grant money to redo experiments that might undermine original published findings and damage career prospects? Why encourage others to challenge original results—getting through peer review is hard enough already. Would anyone in the lab next door agree to 'redo' their colleague's research?

As if these pragmatic considerations were not enough, a more fundamental obstacle to Rosenblatt's proposal is that academic science and pharmaceutical R&D are almost wholly incompatible. So profoundly different are they that the biggest surprise of all is that there are crossover opportunities at all.

Academic research is about frontiership—establishing rights (intellectual property or bragging) to previously undiscovered territory. It consists of resource-poor teams attempting to test the plausibility of hypotheses and to stake out new intellectual ground. Within the constraints of grant ceiling and resource inflexibility, the essential skills involve the identification of avenues of research that lead to interesting places, the definition of plausibility tests that can reach far enough to result in a publishable unit, and the subsequent establishment of a reputation that attracts more funding and cheap labor.

The other endeavor—pharmaceutical R&D—is radically different, building a commercial edifice on existing rights. It consists of converting flimsy virgin scientific territory into a solid foundation than can power human testing through regulatory hurdles. It is about resolving questions and closing loopholes, providing data for structured decision-making that can ramp up a company's resource allocation to a project, all in a timescale telescoped by the ticking patent clock and the need to outstrip competition.

Of course, Merck and other pharmaceutical companies would like to shave the costs of acquiring academic research. As Rosenblatt points out, “On average, it takes approximately two to six scientific personnel 1 to 2 years of work in an industry laboratory to try to reproduce the original investigations; cost estimates average $500,000 to $2 million.” But academic institutions will rightly ask why do they now need to bankroll such validation studies? Why should they pay industry for industry's work, particularly when big pharma has been slashing its internal R&D?

Translating bench research to the bedside is already fraught with enough challenges. Paying a market premium for well-validated academic research makes sense. Shifting the financial burden to academia via a money-back guarantee does not. Merck should not be looking for guarantees; it should be looking for opportunities.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

No guarantees in translation. Nat Biotechnol 34, 573 (2016). https://doi.org/10.1038/nbt.3619

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt.3619

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing