WORLD VIEW

A toast to the error detectors

Let 2020 be the year in which we value those who ensure that science is self-correcting.
Simine Vazire is a professor of psychology at the University of California, Davis.
Contact

Search for this author in:

Last month, I got a private Twitter message from a postdoc bruised by the clash between science as it is and how it should be. He had published a commentary in which he pointed out errors in a famous researcher’s paper. The critique was accurate, important and measured — a service to his field. But it caused him problems: his adviser told him that publishing such criticism had crossed a line, and he should never do it again.

Scientists are very quick to say that science is self-correcting, but those who do the work behind this correction often get accused of damaging their field, or worse. My impression is that many error detectors are early-career researchers who stumble on mistakes made by eminent scientists, and naively think that they are helping by pointing out those problems — but, after doing so, are treated badly by the community.

Stories of scientists showing unwarranted hostility to error detectors are all too common. Yes, criticism, like science, should be done carefully, with due diligence and a sharp awareness of personal fallibility. Error detectors need to keep conversations focused on concrete facts, and should be open to benign explanations for apparent problems.

Even when criticism is done well, error detectors are often subjected to personal attacks. Junior scientists are accused of bullying their seniors. In one case, early-career researchers who showed that a famous scientist had engaged in extensive self-citation and recycled his own publications were accused of being vigilantes and mounting a witch hunt. Scientists who found flaws in high-profile nutrition research that required retractions were accused of cyberbullying and, bizarrely, of holding a grudge against school-lunch programmes. And those are just a few incidents that became public.

Researchers are often warned against pointing out errors — and sometimes kindness is used as justification. They are told to focus on improving their own research, or to state only the positive aspects of research done by others. If you don’t have anything nice to say, don’t say anything at all.

There are several problems with these arguments. First, we scientists present ourselves as a community of individuals committed to scrutinizing each other. Historian of science Naomi Oreskes, in urging non-scientists to trust science, argues that “scientists have a kind of culture of collective distrust”. We cannot tell people to trust us because we monitor each other, and then appeal to kindness to halt that scrutiny.

Second, when we suggest that those working on error detection and correction are being unkind, we are the ones being unkind. Imagine that you are a trainee. You feel that science values self-correction, and that it’s not about any one person’s ego, but the collective motivation to find new knowledge, to check everything thrice or more, to discard false hypotheses and so to move ever closer to truth. Thus, when you find an error, you trust that it’s okay to point it out. And then you find yourself accused of being a destructive, sanctimonious second-stringer — all for applying the ‘scientific values’ that you’d been taught.

Yes, error detectors can make research less comfortable — but that discomfort is healthy. We should feel responsible for minimizing errors in our work, and worried that we might have missed some.

Scientific criticism must not be conflated with bullying. It’s not fair to victims of actual bullying to use the term so loosely and inappropriately. Instead, we need mechanisms to protect those who engage in scientific criticism. These mechanisms would make science fairer and more inclusive. Advisers can get away with awful behaviour — bullying, harassment and other abuses of power — because their trainees are so dependent on them for funding, recommendations and other opportunities. Universities need to hold themselves and senior faculty members accountable for preventing abuse, including intimidation and bullying of error detectors.

We should do more to make criticism an established part of science. Universities need policies that assess inappropriate responses to criticism. Responsible research training should include sessions on how to assess whether apparent anomalies could be substantive problems, how to communicate concerns and how to respond when issues arise. Funders and research-evaluation committees should find ways to support and recognize all the work that error detection requires.

Furthermore, journals need to make clearer and firmer commitments to self-correction. In my opinion, they have a responsibility to share replication attempts for the work that they originally publish — which includes creating explicit criteria to enable publication of high-quality replications. Consider the Social Science Replication Project (C. F. Camerer et al. Nature Hum. Behav. 2, 637–644; 2018), which focused on systematically repeating 21 experiments published in Science and Nature. It was an author, not either journal, who said that both journals had rejected the submission and shared the reasons given for doing so. As a former editor-in-chief of Social Psychological and Personality Science, I was shocked at how easy it would be to reject or hide criticism of the editorial process. There should be greater transparency and other measures of accountability over editors, senior authors and reviewers.

It’s time to be kinder to those doing the criticizing, and to demand more accountability and humility from those in power. Instead of punishing people who flag errors, we should scramble to hire them, give them prizes and award them grants so they can keep improving science. The least we can do is provide a space for fact-based criticism that is safe from intimidation and retaliation. It’s only thanks to error detectors that we can proclaim that science is self-correcting.

Nature 577, 9 (2020)

doi: 10.1038/d41586-019-03909-2

Nature Briefing

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.