How the world is adapting to preprints

During a string of recent scholarly publishing conferences, I came to count on two things: that the event would be remote and that someone would be talking about preprints.

A certain trend emerged from the preprint discussions. Praise for preprints and their virtues was reliably bracketed by an acknowledgement that the medium was a bit green and a bit untamed: the Wild West of scientific publishing, where anything can happen.

Similar analogies from the publishing community have been abundant. At a talk organized by the Society for Scholarly Publishing, Shirley Decker-Lucke, Content Director at SSRN, likened preprints to raw oysters: They’re generally safe, but sometimes you get a bad one. On the same panel, Lyle Ostrow, Assistant Professor of Neurology at Johns Hopkins, compared the adoption of preprints to the shift from horse-drawn buggies to automobiles: They’re faster, they’re better, but they will require education to use safely. The following week, a Scholarly Kitchen article about preprints drew a parallel with unruly teenagers: “tremendous promise, but in need of more adult supervision to achieve their potential”.

I don’t disagree with any of these views. We must–and we will–learn to live in a world where most research is available to read before it undergoes peer review. Some of it may never get that scrutiny, and we’ll learn to live with that, too. But for all the challenges this new medium presents, it brings many more opportunities. It is a substrate for innovation. A preprint-first world holds potential for radical transparency, a realignment of incentives in scholarly publishing, an escape from the burden of impact factors, and a shift toward recognition of merit at the individual article level. Collectively, these changes stand to address problems that have plagued science for decades.

The new kid in town

While preprints aren’t actually new, they surged in popularity during the pandemic, quickly becoming the most visible source of information about the novel coronavirus. Everyone had to learn and adjust. Preprint platforms had to recalibrate to a new magnitude of submissions, scaling technology and operations in an effort to keep up. The character of incoming papers changed, too; many espoused poorly supported prophalaxes and treatments for COVID-19. A new burden of potential consequence meant preprint screeners had to reconsider their protocols while maintaining the low barriers to entry that authors have come to expect. Some platforms adopted new policies to correct for a preponderance of “noise”, and nearly all of the major servers unveiled new disclaimers to help calibrate readers’ interpretations of the content. Nonetheless, journalists often reported on these studies without acknowledging that they hadn’t been peer reviewed. And armchair epidemiologists, virologists and immunologists ran amok with their own hot takes over social media. Preprints on Twitter: the Wild West squared.

But over the subsequent months, some course correction ensued. Journalists and tweeters alike became more careful with their citations, and people started to take better notice of the disclaimers and to correct each other online. Perhaps more importantly, several high-profile retractions of COVID-19-related research in respected journals awakened people to a reality that has always existed: peer review doesn’t protect us against bad information, particularly at a time when scientists and editors are under enormous pressure to supply the world with useful, actionable data.

The literature is awash with bad science, and the small collection of retracted articles are dwarfed by reams of articles that could have been retracted were they not simply ignored. Journal articles based on bad (poorly conducted, irreproducible, unsupported) science typically fade into obscurity; so too, shall bad preprints.

Taming the Wild West

We will learn to live in a world with preprints as we learn to see them for what they are–part of a process that is, de facto, impressively messy. In fact, this is how we should view all scientific outputs, no matter how polished and well-formatted they appear. At a webinar hosted by Scholastica in September, my co-panelist Peter Coles, Editor in Chief of The Open Journal of Astrophysics, said, “Perhaps it's time for the public to understand what science is really like.” Preprints, by their imperfect, mutable nature are a reflection of the scientific process, which is, as Ed Yong put it best, “a stumble toward ever less uncertainty”.

So, how will the Wild West be tamed? As it is, most preprint servers have not been agnostic about the papers they post. Generally, they restrict passage of content that is clearly unscientific, unethical, potentially harmful or not representative of a novel, empirically derived finding. That is, they already impose some editorial standards.

Preprint platforms offer authors a legitimate place to host their work with unprecedented speed, for free. In time, they could be in a position to enforce, or at least strongly incentivize, standards that are widely acknowledged to support research integrity, like data and code availability, details of randomization and blinding, study limitations and lay summaries for findings that are consequential to human health.

Community peer review of preprints, a practice that has already seen some uptake, will eventually become fully normalized. There will be an emergence of unprecedented transparency in research reporting: studies with links to their comprehensive data and analysis posted alongside their criticisms and plaudits. Researchers who look forward to this future can hasten it by uploading preprints and by commenting substantively or conducting more structured reviews on preprints in their field via mechanisms like PREreview and Peer Community In.

The West did not stay wild. The preprint model, likewise, will be tamed by the communities that have shaped and strengthened it through the digital age, and it will become the basis of a new stage in the evolution of scholarly communication.