The publication record of health-care start-up companies doesn’t seem to matter to investors, according to an analysis of nearly 50 biomedical ‘unicorns’ — venture-capital-backed companies valued at more than US$1 billion.
The analysis1, led by health-policy researcher John Ioannidis at Stanford University in California, finds no correlation between a company’s market valuation and its publication record — defined as the number of peer-reviewed papers authored directly by a firm. That’s a cause for concern, the authors say.
Among the 47 unicorns analysed, nearly half had not published any highly cited papers, defined as having at least 50 citations.
Eight companies had produced no publications at all (see ‘Unicorns and peer review’). And half of the papers published on the 47 companies’ work were authored by just 2 of the firms: the personal-genomics company 23andMe in Mountain View, California, and the Adaptive Biotechnologies, headquartered in Seattle, Washington, which makes treatments for immune-system disorders.
“I wasn’t expecting us to find such a scarcity or such a dearth of publications,” says Ioana Cristea, a psychologist at Babeş-Bolyai University in Cluj-Napoca, Romania, and co-author of the study.
The analysis looked at companies around the world that develop biomedical products ranging from RNA treatments to prosthetic knees.
Publishing pays off
The findings mean that investors and the public might be building their expectations about a company’s products on evidence that is less closely scrutinized — such as information in patents, press releases and self-published reports — than that provided by the peer-reviewed literature, suggests the analysis.
“Healthcare products not subjected to peer-review but rather based on internal data generation alone may be problematic and non-trustworthy,” the authors conclude.
The authors suggest that the effort that goes into publishing can pay off in the long term. Ultimately, lack of evidence can mean that health-care products fail to gain approval from regulators such as the US Food and Drug Administration; lead to lawsuits for faulty products; and even cause harm. The analysis gives several examples of such outcomes for companies that did not have many peer reviewed papers published.
Cristea acknowledges that the peer-review process isn’t perfect. “Peer review misses a lot of things and it has a lot of issues and failures,” she says. “But also the alternative of nothing is, I think, even more problematic.”
Potential investors might instead look at the evidence presented in a firm’s patents — but patents don’t always reveal holes in a company’s data. “If you see a company that really has nothing published to back their claims, that might be a signal to heed,” Cristea says.