Two papers published by protein engineer Homme Hellinga's lab at Duke University Medical Center in Durham, North Carolina, have been challenged.

Last year, Hellinga retracted papers in Science1 and the Journal of Molecular Biology2 after John Richard, a physical chemist at the State University of New York at Buffalo, found that enzymes designed by Hellinga's lab did not work as reported3. Now, questions have been raised about a 2003 paper in Nature4 and a 2004 paper in the Proceedings of the National Academy of Sciences5. In both, Hellinga's group used a computer program called Dezymer to design proteins that could bind new molecules, or ligands.

The work was considered a milestone for showing that it was possible to use computer algorithms to design proteins that bind tightly to small molecules.

Clearly they can't both be right. ,

Birte Höcker, a former postdoctoral fellow of Hellinga's, and her team at the Max Planck Institute for Developmental Biology in Tübingen, Germany, assembled and analysed five of the designed proteins that seemed to work best6. She found that all five were very unstable, and one was too unstable to analyse further. The group then examined the structure of one of the proteins using crystallography and found that its binding pocket was similar to that predicted by Dezymer — but that it did not bind its intended ligand.

And using three methods to detect the changes in stability, heat and shape that normally occur when proteins bind their ligands, the team found no evidence that the designed proteins were binding their intended ligands.

Hellinga says that Höcker used higher concentrations of proteins in her tests than his group did in its original paper, and that this could have affected her results. He wrote in a statement that his lab will study the designs she reanalysed. "If we fail to observe binding in the studies outlined above," he says, "then we will draw the same conclusions" as Höcker.

Höcker speculates that she obtained different results because she used different methods to test binding, including direct measurements. Hellinga used an indirect method: he designed the proteins so that they included a fluorophore that emitted a signal when the proteins changed shape, which his team interpreted as a sign that the proteins had bound their ligands.

Other scientists call the results puzzling. David Baker, of the University of Washington in Seattle, says that Höcker did not test every protein reported in the original papers, so it is possible that some of the proteins do work. But for the proteins Höcker did characterize biochemically, he says, "it's a little hard to see how they could have worked as designed".

"There is a qualitative difference in the results from Hellinga's lab and the German lab," adds Jack Kirsch, a chemist and biologist at the University of California, Berkeley. "Clearly they can't both be right."

Jeff Smith, who collaborated on the 2003 Nature paper while a postdoctoral fellow in Hellinga's lab, says that he performed follow-up studies on three of the proteins after the paper was published. Two of them consistently did not work, but a third — which Höcker did not analyse — did, says Smith, now chief science officer of a biotechnology company in Durham. This, he says, convinced him that the third protein did work: "I would stand by it," he says.

Hellinga says the proteins' instability was mentioned in his papers.

In 2007, Hellinga charged Mary Dwyer, a co-author of the Nature paper, with misconduct in connection with the retractions. Dwyer denied the charges. Duke University School of Medicine convened an enquiry and cleared Dwyer in February 2008. Hellinga wrote to Nature in July 2008 indicating that Duke had, at his request, opened an enquiry into his own actions involving the retractions7. Duke declined to answer questions about the status of the enquiry.