Three reasons to share your research failures

There’s a new journal for that.

22 September 2020

Paul Bradbury/Getty Images

Science is often a one step forward, two steps back process, but most journals and researchers are reluctant to air the failures and drawbacks that precede success.

That’s where the new Journal of Trial and Error (JOTE) comes in. Launched in July 2020, its remit is to publish and discuss non-significant findings, technical and methodological flaws, rejected grant applications, and failed experiments.

Founded by a team of early-career researchers in the Netherlands, the journal is open access and multidisciplinary, and aims to “close the gap between what is published and what is researched”.

JOTE published its first paper in July, a psychology study that attempted to replicate previous research on the link between viewing images of alcohol and aggressive thoughts.

Led by Julie Leboeuf, a psychology researcher at Bishop’s University in Canada, the paper describes how it failed to replicate previous findings due to “poor design considerations”, such as a small sample size.

While JOTE accepts contributions from researchers at all levels, authors in early-career stages, including late masters students, PhD candidates, and post-docs, are particularly encouraged to submit papers.

The journal uses a double-blind peer review system where author and reviewer identities are withheld from each other prior to publication. Full peer reviews are published alongside the manuscript.

Max Bautista

Although sharing mistakes so openly can be daunting for early-career researchers, it’s an important step towards making science more open and transparent, says Max Bautista, co-founder and creative director of JOTE.

“I think many researchers are less confident in their results than what appears in their publications,” says Bautista, who studies the history and philosophy of science at Utrecht University in the Netherlands. “We launched the journal in response to that.”

Below, Bautista elaborates on three reasons to publish research failures and null results:

1. Keeping it real

Publishing mistakes and setbacks presents a more realistic view of the scientific process, which is often fraught with challenges.

“You get the message across that things are neither a complete success nor a complete failure,” says Bautista. “It just means that the story of your research is not finished yet.”

It also ensures a more balanced and reliable scientific record. The bias of journals and researchers towards significant results can lead to inaccurate meta-analyses, for example.

“It’s about reporting in a more accurate, transparent way,” says Bautista. “Framing everything as a success only leads to bias in the literature.”

2. Publications have value

Despite the pressure to share only statistically significant results, publishing a paper reporting null findings or a failed experiment is more valuable than letting a manuscript sit in a drawer, says Bautista.

As counterintuitive as it might seem, publishing experimental mistakes and errors can tap into the scientific reward system, which counts papers and citations as a measure of success, he says.

“If the error turns out to be about the system or tool you are working with, it will likely be cited a lot,” says Bautista.

3. Sharing knowledge

Whether it’s a dodgy line of code or a poor-quality cell line, sharing what works and what doesn’t can prevent other researchers from repeating your mistakes.

While some researchers may discuss their findings with their lab colleagues or at conferences, formally publishing them can make this “tacit knowledge” or flawed methods or inadequate tools more accessible to those who cannot attend meetings, says Bautista.

“Chatting in informal settings can exclude a large number of scientists and reinforce elite groups,” says Bautista. “Publishing the trial-and-error process formalizes this knowledge.”