Nature | News

Tracker flags up failures to report clinical trials

Computerized search of trial registry lists worst offenders.

Article tools

An automated tool has trawled through thousands of records on the world’s leading clinical-trials database to reveal which drug firms and academic institutions are failing to publish the results of their trials (see 'Unreported clinical trials').

The failure is already well documented: multiple studies have variously reported that 25–50% of clinical-trial results remain unpublished years after the trials are completed. And in September, the US Department of Health and Human Services announced tougher rules to push the researchers that it funds to publish clinical-trial designs and results.

But software, such as the tool described in a paper published online at F1000Research1 on 3 November, allows for a more comprehensive search than was previously possible, says the paper's corresponding author Ben Goldacre, a clinical-research fellow at the University of Oxford, UK. (The publication has yet to be peer reviewed.)

Automating the process also means that results can be updated regularly, which keeps the pressure on trial sponsors who fail to report — and enables them to take action to improve their scores.

Unreported clinical trials

An automated tool has ranked the sponsors with the most unreported trials registered on ClinicalTrials.gov.

Name of sponsor Trials missing Trials eligible* Percent missing
Ranbaxy Laboratories Limited 35 35 100%
Nanjing Medical University 32 35 91.4%
Rambam Health Care Campus 27 30 90%
Isfahan University of Medical Sciences 44 49 89.8%
City of Hope Medical Center 39 44 88.6%
University Hospital, Caen 34 39 87.2%
National Institute on Drug Abuse (NIDA) 33 38 86.8%
Centre Hospitalier Universitaire de Saint Etienne 31 36 86.1%
University Hospital, Rouen 36 42 85.7%
University Hospital, Ghent 51 60 85%

*Reported as 'complete' on ClinicalTrials.Gov between Jan 2006 and Oct 2014.

Data from Trials Tracker, as of 3 November.

“If anyone wants to improve their score or improve their ranking, all they have to do is publish their results,” says Goldacre.

Computerized check

Goldacre and his Oxford-based co-author Anna Powell-Smith developed the tool to search the ClinicalTrials.gov database for trials that were completed at least two years ago. The computer then attempted to match those trials with results published in that database or in the research repository PubMed. 

Of nearly 26,000 trials evaluated, 45.2% had no published results. The team also built a website that enables users to view clinical-trials sponsors in order of who is the best — or worst — at publishing their results. The lists contain a mix of academic and industry sponsors from around the world.

Automated, rather than manual, analyses are increasingly the norm for studies that scan for clinical-trial transparency, says Jennifer Miller, a medical ethicist at New York University’s Langone Medical Center. She points to the Good Pharma Scorecard initiative, founded by Bioethics International, a charity that Miller founded.

The initiative ranks new drugs and companies on clinical-trial transparency, on the basis of automatic analyses and machine learning. But it is careful to check its work manually and to confirm its findings with clinical-trial sponsors, she says. The Scorecard, she says, searches other clinical trials registries and research databases, including Google Scholar, which could capture a bigger pool of trials and papers than does Goldacre's tool.

Automating the search can lead to a sacrifice in precision, Goldacre acknowledges. For example, the search might miss published results if they are not tagged with a number assigned by the ClinicalTrials.gov database, or if the journal in which they are published is not listed in the research repository PubMed.

But although Goldacre says that his team did find some discrepancies in how individual studies were scored, overall trends from his tracker are similar to those previously published by manual surveys on smaller subsets of data. And he hopes that the ability to regularly update results will incentize trial sponsors to improve their score.

“This is such a serious business,” he says. “We need to maintain the pressure.”

Journal name:
Nature
Volume:
539,
Pages:
153
Date published:
()
DOI:
doi:10.1038/nature.2016.20930

References

  1. Powell-Smith, A. & Goldacre, B. F1000Research http://dx.doi.org/10.12688/f1000research.10010.1 (2016).

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments for this thread are now closed.

Comments

1 comment Subscribe to comments

  1. Avatar for Sophie Petit-Zeman
    Sophie Petit-Zeman
    Patients who give their time to design and join trials will be heartened to see this TrialsTracker up and running. When our host, Oxford University Hospitals NHS Foundation Trust, signed AllTrials - http://oxfordbrc.nihr.ac.uk/blog/a-commitment-to-transparency/ - our patients wanted us to commit not only to its ideals, that all trials are registered and their results reported, but to associated actions. Eighteen months later, BMJ Open published our audit assessing our own record in these areas - http://bmjopen.bmj.com/content/6/3/e009285.full - a paper which also included recommendations to improve the transparency regarding which clinical trials receive support, to encourage their timely publication, and committing to improving performance. One of our six trials units has now assessed the ease of being, or becoming, compliant with these recommendations - http://oxfordbrc.staging.wpengine.com/wp-content/uploads/2015/10/revisedSynopsis-of-how-OCTRU-has-enacted-the-BRC-recommendations_Jun2016SPZ-3.pdf - and its five sister units have agreed to enshrine any necessary changes within their Standard Operating Procedures. In parallel, we are working with patients to ensure lay information about trials meets their needs, and with researchers to ensure they place such information on the two main trial registers (clinicaltrials.gov and ISRCTN) to allow correct drawdown onto the NHS UK Clinical Trials Gateway - https://www.ukctg.nihr.ac.uk/. Our method for this can be found in a PDF here - https://www.admin.ox.ac.uk/researchsupport/ctrg/resources/ - under “links” in a document called “Guidance on providing lay summaries for all researchers.” Much of this work was funded by our patient involvement working group, reflecting their desire for openness and transparency of trial data. We hope that our audit methods and the recent AllTrials Roadmap - http://www.alltrials.net/wp-content/uploads/2016/10/AllTrials-Roadmap.pdf, which heralds our work as a world first, may be the carrots that enable others to take similar steps; the TrialsTracker may be the stick that provides further encouragement. There is an urgent need to improve practice in the areas where we have, at the behest of our patients, devoted time and effort. To still be in an era when millions of people join thousands of trials that are never published betrays their trust and leaves evidence-based medicine without the tools it needs to be delivered. Dr Sophie Petit-Zeman; Director of Patient Involvement; National Institute for Health Research (NIHR) Oxford Biomedical Research Centre and Unit. sophie.petit-zeman@ouh.nhs.uk
sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.