London

A mathematics journal has withdrawn a paper that claimed to crack one of the discipline's great mysteries after reviewing and accepting the work and publishing it online.

On 18 November, Nonlinear Analysis published a paper by Elin Oxenhielm — a postgraduate student in mathematics at the University of Stockholm, Sweden — which presented itself as a solution to the second part of Hilbert's sixteenth problem, one of a set of challenges laid out by German mathematician David Hilbert in 1900.

If a solution were validated, mathematicians agree, it would be a significant step towards a complete solution to the problem. Oxenhielm predicts just that: “We could find one in a year or so, if we're lucky,” she says.

The work was described in a 24 November press release from Oxenhielm and covered in several media outlets including the BBC. But the paper immediately came under fire from mathematicians. “It's completely inadequate — I can't imagine who would have thought it was a proof,” says John Mather of Princeton University, New Jersey.

Critics include Oxenhielm's supervisor, Yishao Zhou, who put a statement on her website saying: “The paper is incomplete and contains serious mistakes.”

Hilbert specified 23 problems that he said should drive mathematical research. Solving any one of them is almost guaranteed to make a mathematician's name, and by 2000 all but three had been solved.

The sixteenth, the problem of the topology of algebraic curves and surfaces, deals with the territory where geometry meets algebra. Its second part involves showing that the number of periodic solutions to a differential equation is finite.

Such periodic solutions are also known as limit cycles — stable, oscillating trajectories to which a system will return if perturbed. Limit cycles are common in nature, and a proof of the second part could lead to a better understanding of heartbeats, animal movements and the kind of runaway vibrations that can shake a structure to bits.

Oxenhielm formulated her proof using 'describing functions' — which can predict roughly the presence of limit cycles in nonlinear equations.

A few minutes' scrutiny is enough to show that her reasoning is false, says mathematician Grigori Rozenblioum of Chalmers University of Technology in Gothenburg, Sweden. The approximate solutions studied by Oxenhielm cannot provide the exact answers demanded in a proof, he says, and some of her equations contain exact terms where approximate ones should be used.

The work should never have been published, Rozenblioum says: “It's impossible to understand the behaviour of the journal, which is one of the leaders in its field.”

Nonlinear Analysis pulled the paper on 4 December. “Publication has been halted until a thorough investigation into the matter has been handled,” says editor-in-chief V. Lakshmikantham, a mathematician at the Florida Institute of Technology in Melbourne.

Originally approved by one reviewer, the paper has now been sent to two more mathematicians for further round of review, along with a defence by Oxenhielm, who says that the critics do not understand her methods.

She refuses to comment further. “Nonlinear Analysis' editors have evaluated the paper, they accepted it for publication and they have the copyright of its contents — and thus they are responsible for its correctness,” she told Norwegian newspaper Aftenposten.