Researchers have found certain stem-cell studies notoriously difficult to replicate. Erika Check finds out why, and whether it is slowing down the field.
Eight years ago, a team of scientists based in Canada and Italy published an astonishing paper in Science. They started with mouse neural stem cells — cells destined to give rise to brain tissue. At the time, scientific dogma held that cells such as these, which are already moving towards one destiny, cannot switch paths and generate other types of cell.
The Science paper challenged that view. The team transplanted the stem cells into mice whose bone marrow had been wiped out. Lo and behold, a miraculous transformation seemed to occur: the stem cells changed their fate and gave rise to the haematopoietic cells that normally reside in bone marrow and generate blood and immune cells. The paper was provocatively titled 'Turning brain into blood'1. It was part of a wave of studies reporting a fate-switching phenomenon that some termed 'transdifferentiation', and that offered a possible way to grow replacement tissues without destroying embryos for stem cells.
It took a separate group of scientists more than two and a half years of painstaking work to show that the Science paper wasn't as exciting as it had first seemed. This group tried to replicate the findings by transplanting neural stem cells into each of 128 mice whose bone marrow had also been killed. But after exhaustive analysis, they never saw these stem cells generate haematopoietic cells. They concluded that the cells may rarely switch fates, but that this occurs only because of genetic changes that accumulate after long periods growing in a Petri dish2. The authors of the Science paper disputed this conclusion, and there is still no consensus on whether this phenomenon can ever be useful in medical practice.
Reputation at stake
The brain-to-blood debate highlights a more pervasive problem in the adult stem-cell field: that many teams struggle to reproduce others' seemingly promising results. Replication is a crucial and arduous part of the scientific process3, and it is one that has been especially problematic in this area because of restrictions surrounding the use of stem cells, inexperienced researchers and technical difficulties. Many scientists feel that these problems have slowed progress and harmed the field's reputation. “Overall damage to the field has been enormous,” says stem-cell biologist Naohiro Terada of the University of Florida in Gainesville. “If we keep producing hype, no one will trust stem-cell researchers.”
The issue of replication was most recently thrust into the spotlight because of renewed attention on a 2002 Nature paper4 by Catherine Verfaillie of the University of Minnesota, Minneapolis, and her colleagues. After culturing mouse bone-marrow cells for months, the researchers reported that they could extract an exceptional population of cells they termed multipotent adult progenitor cells, or MAPCs.
When they injected these cells into developing mouse embryos, and studied the chimaeric baby mice that were born from them, the MAPCs seemed to contribute to all the major cell types of the body, including brain, heart, bone marrow, skin, blood and lung. This and other experiments raised the possibility that MAPCs could be used therapeutically in organ regeneration and repair. The finding created a buzz, because no other adult stem cell had been found to generate such a variety of different cell types — a range almost equivalent to that produced by embryonic stem cells.
Verfaillie's work has since proven exceedingly difficult to replicate, although some groups have reproduced certain parts. Earlier this year, after spending several years learning to work with MAPCs, Scott Dylla, a former postdoctoral fellow in the lab of Stanford University biologist Irving Weissman, was able to use MAPCs to make blood-forming haematopoietic stem cells in mice5. But although many groups have tried, none has managed to repeat the key aspect of Verfaillie's paper — injecting MAPCs into an embryo to create all the major cell types of the body. “I have not seen any convincing data showing that anyone has repeated the chimaera experiment, so I don't think this part of it is true,” says Rudolf Jaenisch of the Whitehead Institute in Cambridge, Massachusetts, whose lab tried and failed to reproduce the work.
Stuart Orkin of Harvard Medical School in Boston, Massachusetts, requested MAPCs from Verfaillie's lab a few months after her paper was published. He found clauses in the material transfer agreement — the contract that governs use of shared research material — saying that users could not disclose information about the cells to others not working on the project. “We couldn't accept that,” says Orkin. More broadly, researchers sometimes have difficulty laying their hands on stem-cell lines and methods because of competition between labs and because these cells could prove commercially valuable and so access to them can be restricted.
“Methodology is everything, and the devil is in the details. Paul Simmons”
Those labs that could negotiate licences for MAPCs ran up against another problem: the cells themselves are very tricky to work with. Two labs told Nature that they couldn't keep the MAPCs alive long enough to study them, and so abandoned attempts to replicate Verfaillie's work. Both labs followed Verfaillie's published and unpublished instructions closely.
A large part of the problem is that methodology on how to handle the cells keeps evolving, says Paul Simmons at the University of Texas Health Science Center in Houston and one of those who tried in vain to repeat the work. “There's been a series of moving goal-posts as far as the conditions for replication are concerned,” he says. For instance, it has now become clear from Verfaillie's later work that MAPCs grow better in low-oxygen conditions, and that they are best taken from very young mice.
“Methodology is everything, and the devil is in the details,” says Simmons, who is also president of the International Society for Stem Cell Research. “There's a real need to have methodologies out there that include all the protocol details.” But scientists say they feel forced to skimp on their methods sections so they can cram as much data as possible into high-profile journals with severe page limits. And Verfaillie says that the improvements in culture conditions were discussed at meetings and published.
The circumstances surrounding Verfaillie's paper became even murkier last month when questions about duplicated figures used in her Nature paper and another published in Experimental Hematology6 prompted the University of Minnesota to convene an inquiry into her study. The inquiry found that some of her procedures were flawed.
Verfaillie has said that the duplicated figures were an honest mistake. And other scientists have not suggested that she committed fraud — on the contrary, Verfaillie, now at the Catholic University of Leuven, Belgium, enjoys a reputation as a meticulous researcher. Verfaillie says that the Nature paper discussed several possible explanations for the chimaera results and that its conclusions still stand. But she admits that the duplication errors in her published work may have contributed to confusion in the field: “We have made mistakes for which I take responsibility, and we have done everything possible to alert the scientific community regarding these mistakes,” Verfaillie wrote in an e-mail to Nature. The confusion looks set to continue: last week, new questions were raised about duplicated images in other work from Verfaillie's lab.
Lack of experience
Technical problems have scuppered many attempts to replicate stem-cell experiments. The best way to isolate and handle fiddly stem-cell lines is only just being worked out, and by their very nature, stem cells constantly divide and change, making it demanding to extract or maintain precisely the same population of cells in culture.
In 2001, a team led by biologist Diane Krause of Yale University in New Haven, Connecticut, published a high-profile paper in Cell claiming that a single bone-marrow cell could give rise to multiple cell types, from gut to lung to skin7. But the following year, a second group disagreed, saying they could not obtain such a range of cell types from a bone-marrow cell8.
Krause says that although some groups have replicated aspects of her work, others have not, and that it is important to her to understand why. Probably the main reason is that different labs have used different experimental techniques, she says, so they may have isolated slightly different starting populations of bone-marrow stem cells.
The same year, two groups showed that results such as Krause's might be explained by cell fusion, in which stem cells that seem to adopt new fates were actually merging with other cells9,10. The fusion experiments helped to discredit the idea of transdifferentiation, which has now become a taboo word among most serious stem-cell researchers. Krause says she is performing experiments to clarify how much cell fusion contributed to her results.
Inexperience is another problem in the stem-cell field, which attracts new researchers with its white-hot reputation. This has been a particular issue with the transdifferentiation studies, says cell biologist Sean Morrison of the University of Michigan, Ann Arbor. Some may make honest mistakes because they are new to the studies. Others may be sloppy. “In some cases people have a get-rich-quick mentality and are more interested in publishing high-profile papers than in getting the answers right,” he says.
Does any of this really matter? Similar problems crop up in many young fields, in which scientists can struggle to reproduce initially exciting data. Inevitably, some results fall by the wayside if they cannot be repeated.
But the adult stem-cell field is under particular scrutiny because of its medical promise and its potential to bypass embryonic stem-cell research. It takes tremendous time and resources to see whether an experiment can be repeated. The difficulties with replication might partly explain why there is no consensus on the properties of most adult stem-cell lines, or which lines are most medically promising.
“If your research depends critically on the veracity of an observation, it's incumbent upon you to reproduce it — if it's not a solid observation, it becomes a roadblock,” Simmons says. Add up enough of these roadblocks, and it is easy to see how a field can get bogged down chasing spurious leads, instead of forging ahead in new directions.
Bjornson, C. R. R. et al. Science 283, 534–537 (1999).
Morshead, C. M. et al. Nature Med. 8, 268–273 (2002).
Giles, J. Nature 442, 344–347 (2006).
Jiang, Y. et al. Nature 418, 41–49 (2002).
Serafini, M. et al. J. Exp. Med. 204, 129–139 (2007).
Jiang, Y. et al. Exp. Hematol. 30, 896–904 (2002).
Krause, D. S. et al. Cell 105, 369–377 (2001).
Wagers, A. J. et al. Science 297, 2256–2259 (2002).
Terada, N. et al. Nature 416, 542–545 (2002).
Ying, Q.-L. et al. Nature 416, 545–548 (2002).
About this article
β-Cell regeneration through the transdifferentiation of pancreatic cells: Pancreatic progenitor cells in the pancreas
Journal of Diabetes Investigation (2016)
Stem Cell Reports (2013)
Very Small Embryonic-Like Stem Cells Purified from Umbilical Cord Blood Lack Stem Cell Characteristics
PLoS ONE (2012)
Nature Medicine (2012)
Materialwissenschaft und Werkstofftechnik (2010)