Nicolas Gisin. Credit: UNIGE

A heated debate about arXiv’s screening policies has flared up after a high-profile physicist said that moderators at the popular preprint server rejected reasonable papers without justification.

The site – where physicists, mathematicians and other researchers routinely post their articles before peer review — has previously been accused of bias for filtering out some of the wilder ideas it receives. But in a December blogpost that is now provoking debate, Nicolas Gisin, a quantum physicist at the University of Geneva, Switzerland, suggests that arXiv moderators wrongly blacklisted two of his students from posting their work.

Gisin notes that being unable to post to arXiv has a detrimental effect on young researchers’ careers, because it is so influential — the preprint server holds more than 1.1 million papers and receives well over 9,000 submissions each month. He is concerned that the site’s volunteer moderators have too much power. In reply to Gisin’s complaint, arXiv moderators say that they do not blacklist people and cannot comment on this particular case. But other physicists say that the preprint server needs to be more transparent about its screening process.

Preprints rebuffed

For Gisin, the trouble began when two of his graduate students, Thiago Guerreiro and Fernando Monteiro, fell foul of arXiv’s filter. The two had written a paper that — startlingly — calculated that it is impossible to fall into a black hole because it would evaporate before a person came near enough to be swallowed. Gisin, who admits that he is no expert on black holes, felt that it must be wrong, but neither he nor his colleagues could find a mistake, so the result was submitted to arXiv in December 2014. It was placed on hold by the site’s moderators, and rejected three months later. A modified version of the paper later passed peer review and was published last October in the journal Physics Letters A1.

Gisin asked for an explanation, and got sent the site’s publicly stated rules for moderation — which include papers with inappropriate formatting or topic, papers in need of “significant review and revision”, or concerns over copyright, duplicated submissions or multiple postings in quick succession. Those policies are sensible and rarely trouble seasoned users, adds Gisin (who has posted more than 300 papers under the site’s quantum physics category), but the moderators wouldn’t say which rule the students’ paper had violated.

Sabine Hossenfelder, an expert on quantum gravity at the Frankfurt Institute for Advanced Studies in Germany, says that the black-hole paper is scientific and clearly argued, but is wrong because it uses an equation in a slightly different regime to that in which it should be applied. She calls this a “common and understandable mistake”. Hossenfelder says that if she had been approached by a journal to review the paper, she would have recommended rejection. But despite that, she still believes the version which was published was suitable for posting on the arXiv.

The students’ next paper, on black-hole radiation, was less controversial because it used a simplified technique to reproduce an accepted result. However, after its submission in May 2015, it was rejected a week later by arXiv — after which Gisin aired his thoughts online. Gisin adds that he appreciates that arXiv’s volunteer moderators may well be working with “goodwill”, but it should not be their responsibility to vet papers, he says. (Guerreiro and Monteiro said they preferred not to comment.)

Quality control

Asked by Nature for a response, Daniel Gottesman, an arXiv moderator and chair of its physics advisory committee, said that moderators are unable to comment on specific cases. He added, however, that “there is no arXiv blacklist” — besides measures to ban users who flagrantly attempt to bypass an appeals procedure by resubmitting rejected papers.

“It is often the case that people who submit one paper that does not meet arXiv standards will submit more with similar problems, so future submissions by the same person are likely to get more scrutiny,” says Gottesman, who is based at the Perimeter Institute in Waterloo, Ontario.

The grounds for rejection listed on arXiv should not be taken as comprehensive, Gottesman adds. Moderators are also asked to check whether the papers meet a certain minimum quality standard, based on their expert judgement. The standards imposed by arXiv are less stringent than those of a high-quality peer-reviewed journal, he says. “If a paper is rejected by arXiv and accepted by a journal, that does not mean that arXiv is the one that made a mistake.”

ArXiv.org founder Paul Ginsparg, a quantum physicist at Cornell University, in Ithaca, New York, who was not involved in the moderation in this case, adds that arXiv moderators are also now sensitive to the fact that the site is checked daily by the news media. This gives moderators another reason to avoid posting “manifestly outlandish” claims that might confuse the public, he says.

Hossenfelder thinks that the arXiv moderators may have misfired on this occasion, but has sympathy for them, given the volume of papers they sift through. “I hope that this episode will shake up the community and result in more funding for the arXiv, which in turn will help them to provide a better and more transparent moderation,” she says.

Ian Durham, a quantum physicist at Saint Anselm College in New Hampshire, agrees that the black-hole paper is wrong but should have been accepted by arXiv, so that the students could be exposed to community feedback. “I do not approve of the way in which the arXiv handled this,” he says. “I understand that they have to protect against crackpots posting truly crazy papers. But in that case the process should be more transparent and the moderation must be more meaningful with some feedback provided.”

Previous complaints over arXiv’s moderation policy motivated independent physicist Philip Gibbs to set up the filter-free repository viXra in 2009. Although viXra has a reputation for hosting speculative papers from non-academics, Gibbs argues that problems with arXiv have led to a rise in the number and quality of viXra submissions. “We now seem to be reaching a breakout point where more people from within academia can see the point of what we are doing too,” he says.