Biostatisticians Keith Baggerly and Kevin Coombes, like many, were intrigued by claims of personalized chemotherapy treatments by geneticist Anil Potti in 2006. Then at Duke University in Durham, North Carolina, Potti published results indicating that gene expression signatures could identify which chemotherapy drug could best treat lung or breast cancer — results that led to the setup of three clinical trials. But Baggerly and Coombes quickly found something amiss in the data. What began as concerns over apparent errors, including mislabelled samples and mismatched gene names, eventually snowballed into one of the most notorious cases of scientific misconduct in the United States in recent years.

During some 1,500 hours of work over four years, Baggerly and Coombes, both of the University of Texas MD Anderson Cancer Center in Houston, repeatedly showed that Potti's findings did not match the raw data. They analysed the data, had conversations with Potti and his supervisor, alerted the US National Cancer Institute to the likely mistakes and contacted the editors of the journal publishing Potti's work.

Credit: B. MELLOR

Repeated enquiries and complaints by Baggerly and Coombes led senior officials at the University of Texas to advise them to drop what was starting to look like a vendetta. “We were focused on the fact that the data used to justify clinical trials were wrong; we thought that should be enough,” says Baggerly. “How the data got in this shape was not our immediate concern.”

Their objections were finally proved valid. Six years on, ten papers by Potti have been retracted, and the clinical trials were halted eventually. Baggerly and Coombes say that their persistence was down to their obligation to scientific ethics and the consequences of a clinical trial based on incorrect data.

Gauging the amount of misconduct in science is very difficult, but last year there were 381 journal retraction notices — up from 22 in 2001 — according to the Thomson Reuters database Web of Knowledge. Indeed, 2011 was dubbed the “year of the retraction” by the blog Retraction Watch. Last year also saw 13 misconduct rulings by the US Office of Research Integrity (ORI) in Rockville, Maryland, which oversees misconduct investigations and publishes the findings on its website.

Some reports suggest cases of misconduct may be more prevalent than previously suspected. In January, a survey1 in the British Medical Journal found that of the 2,782 doctors and academics that responded, 13% had first-hand knowledge of misconduct. These findings mirror a 2009 meta-analysis of misconduct surveys conducted by Daniele Fanelli from the University of Edinburgh, UK, which found that almost 2% of scientists admitted to having fabricated, falsified or modified data or results at least once, and 14% knew of fabrication or falsification by colleagues2. “Yet consistently, almost no one reports misconduct to the proper authorities,” says Fanelli. According to his analysis, five studies asked survey respondents if they had taken any action to correct or prevent misconduct. Only about half of the alleged cases that were reported resulted in any action — and even then, this amounted mainly to informal confrontations or discussions with colleagues.

According to a 2010 survey3 on the response of researchers to wrongdoing, 63% of the 2,193 respondents said that they had intervened but that most action was informal — discussing concerns with a supervisor or questioning the suspect behaviour, rather than lodging a formal complaint.

Would-be whistleblowers face tough decisions about whether their concerns are the result of misconduct, scientific disagreement or simply an honest error. Knowing where or how to lodge complaints can also be difficult, given that misconduct investigations are pursued confidentially. From identifying the proper place to report allegations to understanding how an investigation will unfold, there are a number of considerations, policies and procedures that would-be whistleblowers need to know.

To allege or not?

The first step towards determining whether to air integrity concerns is to understand what is and what is not scientific misconduct. Misconduct is not simply bad behaviour; it is the falsification, fabrication or plagiarism of results. Honest errors, differences in the interpretation of results, authorship disputes, sexual harassment or threatening language are issues of concern, but are not misconduct. At the core of misconduct is intent. “There is not a finding of misconduct unless it can be proven that the person acted with intention or was seriously reckless,” says Mark Barnes, chief research compliance officer at Harvard University in Cambridge, Massachusetts.

Keith Baggerly (left) and Kevin Coombes felt they had an obligation to uphold scientific ethics. Credit: B. SMITH/UNI. OF TEXAS

Plagiarism cases have become fairly easy to identify — in part because of software tools such as Turnitin and iThenticate. Increasingly common are allegations of image manipulations (for example, falsely labelling the protein bands on a gel), which can also be identified quickly with software.

Even with a clear definition of misconduct, grey areas can still arise. Cherry-picking statistical methods to obtain desired results or omitting outlier points on a graph can fall under the rubric of the honest difference in the interpretation of results. Evolutionary biologist Marc Hauser resigned from Harvard last year after he was found solely responsible for eight instances of scientific misconduct, most notably in a paper showing that cotton-top tamarin monkeys could learn simple rule-like patterns. The paper was retracted in 2010. A former trainee of Hauser's, who prefers to remain anonymous, did not agree with the final findings of the investigation. “I can see data getting overlooked or favourably interpreted, but I would be surprised if data had been intentionally changed,” says the source. Several of the experiments in question have since been replicated by Hauser and colleagues and the results published, highlighting the fine line between a lack of data integrity and misconduct.

Researchers who are concerned about the conduct of a colleague should make sure that they understand the nature of the research, says David Resnik, a bioethicist and chair of the Institutional Review Board for the National Institute of Environmental Health Sciences in Research Triangle Park, North Carolina. If someone has concerns about whether research methods are appropriate, Resnik says that they should consult a colleague who is familiar with the accepted scientific norms of that discipline.

If the concerns persist, the next step is to decide how to lodge the complaint — either anonymously using telephone hotlines found in many institutes, or in person. The decision can be difficult. Hotline calls may protect the whistleblower, but may not lead to a misconduct investigation. “Anonymous reports may not allow the university to gather enough credible information to proceed with an investigation,” says Jan Allen, associate dean for PhD programmes at Columbia University in New York.

In the United States, legislation exists to protect whistleblowers from retaliation, and universities do their best to maintain confidentiality throughout an investigation. “Protections are in place to keep complainants anonymous, but confidentiality can't be absolutely guaranteed — for example, if university policy is being violated, we have to report it,” says Allen. In some cases, the university can move complainants to a different lab. “But it is hard to deal with lesser acts of retaliation, such as the refusal to write a letter of recommendation,” admits David Wright, a former research integrity officer (RIO) at Michigan State University in East Lansing and the director of the ORI. Those contemplating blowing the whistle should consider the university's policies on misconduct.

Seeking counsel

In the United States, a university's RIO is hired by that institution to handle allegations of research misconduct. But faculty and staff are often not aware that they exist and thus do not seek their help. Furthermore, not all researchers will receive the same assistance, and not all countries have these measures (see 'International standards'). A study that interviewed 79 RIOs4 found that they were “not uniformly well prepared” to handle allegations of scientific misconduct, and that 54% of RIOs had never called ORI to report misconduct. Wright says that cultures vary greatly by campus, which is one reason why the ORI boot camp, launched in 2007, offers ongoing formal training to RIOs. Still, there is some indication that if whistleblowers are dissatisfied with their institution's response, they may have to contact the ORI themselves. “When the ORI receives allegations that are substantial, we request that the institution move immediately to an inquiry,” says John Dahlberg, director of the division of investigative oversight at the ORI. UK universities may also have designated RIOs, if funders adhere to calls made in February, following a research misconduct meeting organized by the British Medical Journal and the Committee on Publication Ethics.

Specificity is a key component of any evidence used to substantiate an allegation, says Dahlberg. Resnik says “anytime you go forward, you need documentation to back up what you say so the allegations are not tossed out.” Whistleblowers should never file a formal complaint on the basis of a rumour or information gained from a third party, says Gerald Koocher, associate provost at Simmons College in Boston, Massachusetts. “If you don't have a smoking gun, at least have a gun,” he says. Ideally, whistleblowers will be able to describe the nature and whereabouts of any additional evidence that may support the allegation, says Wright.

Once allegations are made, there is a danger that data sets could become adulterated or vanish, says Barnes. The ORI requires institutions that have received credible allegations to seize the computer, e-mails or data that may be used as evidence, to prevent this happening.

However, tipping off the perpetrator is a valid concern in a lab where people work closely together. Colleagues are likely to discuss their suspicions before making a formal allegation, creating an opportunity for the perpetrator to tamper with evidence. If researchers really believe there may be misconduct, they should either make a copy of the raw data before suspicions are aired or go straight to the authorities with their suspicions, says Barnes. The worst thing complainants can do is convince themselves that they are the prosecutor who needs to build a case against the suspect, says Wright. Complainants are wise to simply give any evidence to an impartial investigator; otherwise, their motives could be called into question.

Investigation unfolds

If an allegation is deemed to have merit, a university committee starts an inquiry to review the evidence supporting the allegation and to decide whether a formal investigation is necessary. Inquiries found to have sufficient evidence will often then lead to the formation of a new committee to undertake the investigation.

In Wright's experience, at a research-intensive institution it is not uncommon to have ten significant allegations made in a year. Of those, only about six will go to an inquiry; two may evolve into investigations, and only one or none at all will result in findings of misconduct.

Complainants should also realize that investigations can go on for a year or more. During an investigation, Barnes suggests carrying on as normal. For example, during the Hauser investigation, a former colleague says that although senior members of the lab were aware of the investigation, it was hardly ever discussed. But Koocher adds that complainants are wise to document everything they witness during the course of the investigation.

Preventing fraud

High-profile misconduct cases such as those of Hauser and Potti mean that data undergo increasing scrutiny by university administrators. In the wake of the Potti case, Duke University is planning a 'data lockbox', essentially an electronic means to track who has handled data and files, and the changes they have made. The university also plans to embed biostatisticians within clinical research groups to help prevent against inadvertent errors in data analysis.

“Part of the problem with complex data sets inherent to today's science is that you can't pick them up and know instantly that something is fishy,” says Sally Kornbluth, vice dean for basic sciences at Duke's School of Medicine. Resnik notes, for example, that genome-wide association studies have been controversial because of the evolving statistical methods that people are using. Baggerly advocates more open sharing of data-analysis methods. “The main things that we were stymied by was simply trying to get the raw data and the code used to perform Potti's analyses,” he says.

Being vigilant in cases of apparent fraud or misconduct not only corrects the record, but saves others from wasting time, effort and money. For graduate students or postdocs, deciding whether to publicly question the practices of their colleagues can be tough. But Koocher reminds junior scientists that they are often the first to take the fall if something fails or proves unreliable in the lab. “In cases where misconduct is suspected, it's way better, and smarter, to take action that is self-protective,” he says, “rather than risk getting any of the blame.”