Published online 12 October 2009 | Nature | doi:10.1038/news.2009.998


Protein-design papers challenged

Reanalysis does not find same results as key 2003 study.

Two papers published by protein engineer Homme Hellinga's lab at Duke University Medical Center in Durham, North Carolina, have been challenged in a new analysis of the data.

Last year, Hellinga retracted papers in Science1 and the Journal of Molecular Biology2 after John Richard, a physical chemist at the State University of New York at Buffalo, found that enzymes designed by Hellinga's lab did not work as reported (see 'Designer debacle').

Homme Hellinga.

Now, questions have been raised about a 2003 paper in Nature3 and a 2004 paper in the Proceedings of the National Academy of Sciences (PNAS)4. In both papers, Hellinga's group used a computer program called Dezymer to design proteins that could bind new molecules, called ligands, for the neurotransmitter serotonin, the explosive trinitrotoluene (TNT), the compound lactate and an indicator of the presence of the nerve gas soman.

The work was considered a milestone for showing that it was possible to use computer algorithms to design proteins that bound very tightly to small molecules. In September 2004, Hellinga earned one of the inaugural Director's Pioneer Awards from the US National Institutes of Health, worth more than $500,000 a year for five years.

Protein puzzle

Now Birte Höcker, a former postdoctoral fellow of Hellinga's, and her team at the Max Planck Institute for Developmental Biology in Tübingen, Germany, have assembled and analysed five of the designed proteins that seemed to work best5. She found that all five were very unstable, and one was too unstable to analyse further.

The group then examined the structure of one of the proteins using crystallography and found that its binding pocket was similar to that predicted by Dezymer — but that it did not bind its intended ligand, which was serotonin. And using three methods to detect the changes in stability, heat and shape that normally occur when proteins bind their ligands, the team found no evidence that the designed proteins were binding their intended ligands.

"All together, our combined analysis of the binding properties of the designs indicates that no specific binding of the target ligands to the respective designs occurs," Höcker's team reports.

Hellinga says that Höcker used higher concentrations of proteins in her tests than his group did in its original paper, and that this could have affected her results. He wrote in a statement that his lab will study the designs she re-analysed using "techniques that can determine the free energies of protein stability and ligand binding using very low concentrations of protein....If we fail to observe binding in the studies outlined above, then we will draw the same conclusions" as Höcker.

Höcker speculates that she obtained different results from Hellinga because she used different methods to test binding. Her methods included direct measurements of binding, such as isothermal titration calorimetry and nuclear magnetic resonance spectroscopy. Hellinga used an indirect method: he designed the proteins so that they included a fluorophore that emitted a signal when the proteins changed shape, which his team interpreted as an indicator that the proteins had bound their ligands. But Höcker says the assay might have given a false-positive signal if the ligand or the solvent in which it was dissolved interacted with the fluorophore.

Hellinga says if his studies of low concentrations of proteins find that they do not bind their ligands, then he will accept Höcker's explanation: "If these studies also show that our original interpretations are in error, we deeply regret that our reports of these designed receptors do not live up to closer scrutiny," he wrote.

Conflicting results

Other scientists call the results puzzling. David Baker, of the University of Washington in Seattle, says that Höcker did not test every protein reported in the original papers, so it is possible that some of the proteins do work. "If they do work, [Hellinga's Nature] paper is still a very important paper," Baker says. But for the proteins Höcker did biochemically characterize, he says, "it's a little hard to see how they could have worked as designed".

"There is a qualitative difference in the results from Hellinga's lab and the German lab," adds Jack Kirsch, a chemist and biologist at the University of California, Berkeley, who helped Richard to reanalyse Hellinga's papers. "Clearly they can't both be right."

Steve Mayo, a chemist at California Institute of Technology in Pasadena, says that Höcker's results will give the chemistry community a lot to think about. "The original Nature paper was a staggering result, and that's why the Höcker results are so sobering," Mayo says. He cautions that Höcker did not analyze the same exact proteins as Hellinga reported, since her proteins did not contain a fluorophore, which might have affected the proteins' binding properties. "It really does raise the anticipation for a critical analysis of the actual molecules that were used in the 2003 study," he says.

Follow-up studies

Jeff Smith, who collaborated on the 2003 Nature paper3 while a postdoctoral fellow in Hellinga's lab, says that he had performed follow-up studies on three of the proteins after the paper was published. All were intended to bind TNT. Two of them consistently did not work, and a third — which Höcker did not analyse — did, says Smith, now chief science officer of a biotechnology company in Durham. This, he says, convinced him that the third protein did work: "I would stand by it," he says.

Loren Looger, who now runs a lab at Howard Hughes Medical Institute's Janelia Farm Research Campus in Ashburn, Virginia, worked on both papers while a graduate student with Hellinga. "I am not terribly surprised by [Höcker's] results," Looger wrote to Nature, adding that he had begun to think "that the designed proteins did not work very well, due to several lines of data" collected in Hellinga's and other labs.

For instance, while collecting the data for the 2003 Nature paper, a positive readout on the fluorescence assay would "not infrequently drop off over the course of days or weeks", Looger says. Also, "there were a number of instances in which a given designed protein did not seem to bind its target ligand; these were not reported in the paper," he adds. "Non-publication of negative results is extraordinarily common in science; this did not seem inappropriate."

Hellinga says the instability of the proteins was mentioned in the main text of both the Nature and PNAS papers. "As for reproducibility," Hellinga wrote in an email, "we found that it was necessary to use freshly prepared protein for the assays."

In retrospect, Looger says in his opinion that "more attention [should] have been drawn to the aggregation and instability of the proteins, and how that might give rise to artefacts."

Another group has analysed the structure of a different set of engineered proteins described by Hellinga in a 2001 PNAS paper6, and found that the proteins behaved differently than Hellinga predicted they would7. The designed proteins did bind their intended ligand, a zinc ion, but did not adopt the 'closed' state normally associated with binding.

Hellinga says that he has seen a similar phenomenon in unpublished studies on a new set of proteins designed to bind the anti-inflammatory drug ibuprofen. He says that a series of experiments similar to those Höcker performed showed "interactions between ibuprofen and the mutant proteins", but that crystallography of the designed proteins "reveals that the designed [proteins] do not adopt the closed form".

"Our interpretation is that i) observation of ibuprofen binding in the designs is corroborated by very different techniques, ii) the binding mode is not as predicted," Hellinga wrote.

Next steps

In 2007, Hellinga charged Mary Dwyer, who coauthored the 2003 Nature paper with Smith, Looger and Hellinga, with misconduct in connection with the retractions. Dwyer denied the charges. Duke University School of Medicine convened an enquiry and cleared Dwyer of the charges in February 2008. Hellinga wrote to Nature in July 2008 indicating that Duke had, at his request, opened an enquiry into his own actions in connection with the events surrounding last year's retractions8. Duke declined to answer questions about the status of the enquiry.

Other scientists said that the new developments should spur Duke to complete its analysis of both the previous retractions and the current developments. "I think it should be brought to a conclusion fairly quickly because the scientific community is perplexed by the contradictory results both from this and Richard's analysis," says Kirsch.


Höcker says that she had a "good relationship" with Hellinga, and had discussed crystallizing the designed proteins with Hellinga while she was still at Duke. "Since I never heard of the outcome [of the crystallization trials], and the program Dezymer, which I myself was using, was under question, it became more and more important for me to know what it was good at and what not," she says.

Höcker says her results also have broader importance for the protein-design field. "I think that we need to focus again on binding in order to improve receptor design," she says, "as well as enzyme design". 

  • References

    1. Dwyer, M. A., Looger, L. L. & Hellinga, H. W. Science 304, 1967-1971 (2004). | Article | PubMed | ISI | ChemPort |
    2. Allert, M., Dwyer, M. A. & Hellinga, H. W. J. Mol. Biol. 366, 945-953 (2007). | Article | PubMed
    3. Looger, L. L., Dwyer, M. A., Smith, J. L. & Hellinga, H. W. Nature 423, 185-190 (2003). | Article | PubMed | ISI | ChemPort |
    4. Allert, M., Rizk, S. S., Looger, L. L. & Hellinga, H. W. Proc. Natl Acad. Sci. USA 101, 7907-7912 (2004). | Article | PubMed | ChemPort |
    5. Schreier, B., Stumpp, C., Wiesner, S. & Höcker, B. Proc. Natl Acad. Sci. USA advance online publication doi:10.1073/pnas.0907950106 (2009).
    6. Marvin, J. S. & Hellinga, H. W. Proc. Natl Acad. Sci. USA 98, 4955-4960 (2001). | Article | PubMed | ChemPort |
    7. Telmer, P. G. & Shilton, B. H. J. Mol. Biol. 354, 829-840 (2005). | Article | PubMed | ChemPort |
    8. Hellinga, H. W. Nature 454, 397 (2008). | Article | PubMed | ChemPort |
Commenting is now closed.