University officials admit data withheld from review panel before misconduct charges arose.
It was a weekend that Michael Cuffe, vice-president for medical affairs at Duke University in Durham, North Carolina, says he will never forget. It began on 16 July 2010, when Cuffe learned of a damning revelation in The Cancer Letter, a Washington DC-based publication with a reputation for probing controversial topics in cancer research. A story in that day's issue alleged that Anil Potti, a cancer geneticist whose data had been used to design three clinical trials then under way at Duke, had lied on multiple federal grant applications, falsely claiming, among other things, to have been a Rhodes scholar.
That was bad news, but for Cuffe and for Sally Kornbluth, Duke's vice-president for research, that was not the worst of it. Months earlier, the same trials had been suspended after critics raised questions about the underlying science. Yet Cuffe and Kornbluth had decided to restart them when a review panel seemed to validate Potti's method. The allegations that Potti, who worked at Duke's Institute for Genome Sciences and Policy (ISGP), had padded his CV changed everything. "When it comes to light that someone may have been less than honest in one aspect of their professional life, one begins to wonder whether they have been less than honest in another aspect," says Kornbluth. That weekend, she and Cuffe suspended the trials once again, and initiated a misconduct investigation that is still ongoing. Potti, who could not be reached for comment, resigned from the ISGP in November and took full responsibility for irregularities in his data.
Now, in response to information obtained by Nature under the US Freedom of Information Act, Kornbluth and Cuffe have offered their account of the mistakes that led the trials to be restarted even after they learned of potential flaws in the underlying data. The affair will have an impact beyond Duke, as the Institute of Medicine, part of the US National Academies in Washington DC, begins to examine research on genome-based patient testing. Originally commissioned to investigate Duke's controversial trials, the institute's US$687,000 study is now expected to focus on providing broader recommendations for the design of clinical trials that similarly use genomic data from individual patients to tailor therapy.
Starting in 2006, Potti and his colleagues at Duke had filed patent applications and published papers1,2 describing genomic predictors — computer algorithms that take gene-expression data from a cancer cell and predict whether the cancer will be sensitive to a particular therapy (see 'How events unfolded').
Duke began three clinical trials based on the technology, ultimately enrolling around 110 patients with lung or breast cancer. But Potti's work was under fire. Keith Baggerly and Kevin Coombes, biostatisticians at the University of Texas M. D. Anderson Cancer Center in Houston, published a technical comment3 and a paper4 saying that they had been unable to replicate Potti's findings. As a result, officials at the National Cancer Institute (NCI) in Bethesda, Maryland — which had received a proposal from Duke to start a fourth clinical trial based on Potti's work — contacted the university to request a review. The trials were suspended in September 2009 and a panel of anonymous reviewers, external to Duke, began its work.
While the review was under way, Baggerly and Coombes analysed data that Potti (and his co-authors, none of whom have been implicated in any misconduct) had posted online. The biostatisticians wrote to Kornbluth and Cuffe, pointing out that the data did not match the raw data in the public database from which they were supposedly sourced. "They had numbers with labels that the Duke group said applied, but the labels were wrong," says Baggerly. Yet in December 2009, Duke's review panel concluded that Potti's work was valid, and the trials were restarted.
In an NCI report obtained by Nature, Duke's external reviewers say that they can replicate the results using data provided by Potti, but seem unaware of any doubts about the data. Kornbluth and Cuffe admit that, in consultation with John Harrelson, who was acting as chairman of Duke's Institutional Review Board, they decided not to forward the latest communication from Baggerly and Coombes to the rest of the board or the external reviewers.
Kornbluth denies trying to protect Potti. "Our motivation was not to protect [him], it was to give [him] complete fairness," she says. They judged that the panel could be improperly biased if it received Baggerly and Coombes's letter, says Cuffe. He and Kornbluth emphasize that, in the early stages of the case, before misconduct allegations were made, they believed that Potti was providing them with "full and truthful answers". But Cuffe acknowledges that the possibility of misconduct may have crossed their minds. "It's not like we had never considered this," he says. If the situation arose again, he says that he would forward "every shred" of evidence to a review panel.
Baggerly regrets that the document wasn't forwarded. "A lot of the back and forth of the past year could have been avoided," he says.
Even before Potti's CV padding came to light, doubts about his work were re-emerging. Last April, officials at the NCI determined that they were part-funding the clinical trials through a grant to Potti. NCI documents show that officials weren't reassured by the review panel's report, having found themselves unable to replicate Potti's results using data from the public database. The institute asked Duke to provide it with the information needed for replication, but that process was overtaken by the allegations from The Cancer Letter.
Kornbluth and Cuffe say that Duke is doing its best to learn from the case, and has set up a committee to decide what checks should be in place on basic research before starting clinical trials. Meanwhile, the Institute of Medicine has begun its study, which is expected to last 15 months, and will cover a variety of 'omics-based' tests, including genomics, proteomics and metabolomics.
David Brooks, chief medical officer of Generation Health, a pharmacogenetics testing company in Upper Saddle River, New Jersey, estimates that more than half of cancer clinical trials involve some kind of genomic testing, so the institute's review may have widespread relevance. Richard Simon, a biostatistician at the NCI, says that one consideration in planning trials is whether genomics data should be used to allocate patients to one therapy or another. An alternative study design is to allocate therapy randomly and have the genomics studied at the same time to see how the two correlate. "It avoids the ethical issues," says Simon.
Back at Duke, other researchers are under scrutiny. Steven McKinney, a statistician at the British Columbia Cancer Research Centre in Vancouver, Canada, has written to the Institute of Medicine to raise the concern, not about misconduct, but that statistical methods used in other Duke studies overlap with a method used by Potti. McKinney told Nature that he is troubled by one study co-authored by Geoffrey Ginsburg, a physician at the ISGP. In the study5, approved by Duke's Institutional Review Board and review boards at four further institutions, 57 people were infected with cold or influenza viruses. Gene-expression data were then obtained to see whether it was possible to retrospectively 'predict' which virus they had been infected with — information that could aid the prompt detection of a pandemic. McKinney is concerned that the statistical method being used does not seem to have been proven.
Ginsburg says that "in contrast to the concerns around the cancer work we have gone to great lengths to replicate the results using a variety of statistical methologies". He and his co-authors will look in detail at McKinney's concerns. Kornbluth echoes this sentiment. "This is something we're going to take seriously and look at very closely," she says.
Potti, A. et al. Nature Med. 12, 1294-1300 (2006).
Hsu, D. S. et al. J. Clin. Oncol. 25, 4350-4357 (2007).
Coombes, K. R., Wang, J. & Baggerly, K. A. Nature Med. 13, 1276-1277 (2007).
Baggerly, K. A. & Coombes, K. R. Ann. Appl. Stat. 3, 1309-1334 (2009).
Zaas, A. K. et al. Cell Host Microbe 6, 207-217 (2009).
Related external links
About this article
Cite this article
Samuel Reich, E. Cancer trial errors revealed. Nature 469, 139–140 (2011). https://doi.org/10.1038/469139a
Applied Microbiology and Biotechnology (2015)
The failure of protein cancer biomarkers to reach the clinic: why, and what can be done to address the problem?
BMC Medicine (2012)
BMC Research Notes (2012)