Clinical science has a compatibility problem. Although there are set protocols to test medicines and to treat patients, no such standards exist to compare clinical-trial data.
The problem arises because each research group has a preferred method of collecting and categorizing results. Differences can be as great as omitting or including the gender and ethnicity of patients enrolled, or as mundane as the vocabulary used in medical records. For example, a study published in The BMJ this week challenged the ‘weekend effect’ — the idea that people in the United Kingdom admitted to National Health Service hospitals at weekends are more likely to die compared with those admitted on weekdays — by saying that stroke patients’ conditions are frequently miscoded by UK hospital staff (L. Li et al. Br. Med. J. 353, i2648; 2016). If proved, such a finding could cast doubt on research studies that use medical records as a data source.
The lack of a single place to report descriptive metadata compounds the problem. Data sharing is most challenging for developing countries, which often lack the resources for large-scale statistical analysis (L. Merson et al. N. Engl. J. Med. http://doi.org/bhmb; 2016).
The latest attempt to address this problem is Vivli, described last week in The New England Journal of Medicine as a universal platform to “link existing data-sharing platforms and communities, while hosting data from investigators who aspire to share data but lack the resources to do so” (B. E. Bierer et al. N. Engl. J. Med. http://doi.org/bhmc; 2016).
The platform, to be run by the Multi-Regional Clinical Trials Center of Brigham and Women’s Hospital and Harvard in Cambridge, Massachusetts, will initially employ curators to convert the clinical-trial data they receive from researchers into a standard, anonymized format and post it on the platform for other researchers to access on request. Eventually, the system’s creators hope to switch to automated curation.
Vivli’s creators acknowledge its limitations. It will no doubt be useful for researchers to verify findings, avoid duplicating trials and prioritize next steps. Similar efforts by scientific societies and researchers on specific diseases have made progress in these directions. But data anonymization and standardization is time-consuming, and so only limited amounts of medical data will at first be available.
Yet the best database in the world cannot address the underlying and pernicious deficiency in clinical-trial data sharing. Unless government regulation is rapidly and substantially changed, Vivli will lack data from the majority of clinical trials — leaving it far from the comprehensive database that its creators envisaged. Despite US laws requiring trial data to be publicly disclosed in the ClinicalTrials.gov database, lax enforcement, loopholes and ambiguous interpretation have allowed pharmaceutical companies to selectively report positive results, alter endpoints and avoid publishing data on time.
Most trials disappear altogether. Some studies estimate that only one-third of trials for drugs approved by the US Food and Drug Administration (FDA) are ever published (J. S. Ross et al. PLoS Med. 6, e1000144; 2009). One assessment found that only two of the ten large pharmaceutical companies complied with FDA regulations on data reporting for drugs approved in 2012 (J. E. Miller et al. BMJ Open 5, e009758; 2015). Gilead and Sanofi Pharmaceuticals were the worst offenders according to author Jennifer Miller of Bioethics International. And private industry is not the only offender — about one-third of federally funded trials remain unpublished four years after completion (J. S. Ross et al. Br. Med. J. 344, d7292; 2012).
Hidden clinical-trial data have long been an open secret in the field,but researchers are now starting to assess it more quantitatively. On 17 May, the Laura and John Arnold Foundation awarded Bioethics International a US$3.6-million grant, which will allow the organization to assess trial-data publication, or lack of it, for all new FDA-approved drugs and biologics.
Ideally, knowledge of such poor compliance will shock the public, pushing the FDA and other regulators into creating stronger incentives for companies to publish complete data, such as serious fines and penalties. Without such intervention, well-intentioned efforts such as Vivli are doomed to remain tragically incomplete, robbing researchers of the opportunity to truly link their work.
- Journal name:
- Date published: