Errors reported in this issue by authors of a Nature paper pose a dilemma about trust.
The practice of science — and the publication of science in particular — is often lauded for its capacity for self-correction, and, to a large extent, deservedly so. During the peer-review process, basic mistakes and errors of judgement are frequently identified, minimizing the number that make it through into often much-improved final publications. Erroneous results that do slip through the peer-review net may be promptly identified; if sufficiently serious, they are corrected or even withdrawn.
The reality of science publication does not always accord with such idealistic expectations, however, and the correction published this week (on page 235) is a case in point.
When an important error has been made in a published piece of work, yet the central claim or result still stands, the publication of a Corrigendum is the most sensible way forward; others are thereby alerted to any inaccuracies in the paper, which may have an impact on their own research. If, on the other hand, the errors that have been made undermine the principal message of the paper, then a retraction is in order — the paper may still contain valid scientific information, but the original publication has now lost its raison d'être.
There is a grey area in between, exemplified by the events — past and present — that have now culminated in the aforementioned Corrigendum. The original paper (Nature 366, 143–146; 1993) is viewed by many as a landmark in its field: an experimental 'first', in which compositional analysis at atomic resolution had been achieved with an electron microscope. But the paper did not have an easy time with referees, with one reviewer in particular maintaining that there were “disquieting questions” over both the provenance of key data and inconsistencies in the manner in which they had been subsequently reprocessed (in response to concerns raised earlier during the review process) in order to better substantiate the central claim. The authors, however, had provided Nature with a robust defence of their work in response to these remaining criticisms — firm assurances that at the time we accepted at face value. Combined with a positive endorsement of the work that was offered by a second referee, we felt it appropriate to proceed with publication of the paper.
Seen with the benefit of hindsight, we made an error of judgement in taking those assurances on trust — perhaps influenced by our awareness of intense competition in the field at the time. For it has belatedly transpired that many of the more critical referee's “disquieting questions” did indeed have a sound technical basis, as key data were misrepresented by the authors, both during the review process and in the final published version of the paper. This has now been fully acknowledged by the authors, who say that they are mystified as to why they offered the assurances they did when originally challenged on these points.
So what does this mean for the paper? It could be argued, as we have done during extensive deliberations, that confidence in the validity of the work has been severely compromised as a consequence of these errors, and that the paper should be formally retracted. But we have concluded for several reasons that the authors' Corrigendum should be accepted. Thirteen years have elapsed since publication, and the authors no longer have available the source data to reanalyse and resubmit to peer review — and so are not in a position to mount a thorough defence of their published results against the remaining criticisms. Furthermore, an investigation commissioned by Oak Ridge National Laboratory in Tennessee has robustly cleared the authors of any intent to deceive; we have received assurances that the original data, if consistently analysed as intended, would still have supported the central thesis of the paper. And most importantly, the authors have both acknowledged and proposed corrections for their earlier mistakes.
In the end, it comes down to an issue that is at the very heart of the practice and communication of science: the question of trust. After all, if researchers and editors cannot safely assume, even as a starting point, that scientific results are essentially true as reported, then the advancement of science is in serious trouble.
Without doubt, there has been in this case a severe breach of the trust on which the publication of science is based. But the reasons underlying it, and the hypothetical outcome for the work had these concerns been tackled more robustly when first raised, can now only be speculated about. Other researchers will have their own take on the situation (see page 129).
There is some consolation in the fact that the experimental capability first reported in the flawed Nature paper was soon exceeded, as reported by these authors and others, in a range of different contexts. Thus we can at least be relieved that the progress of science was not impeded by this particular episode.
About this article
Investigación Bibliotecológica: archivonomía, bibliotecología e información (2019)
The EMBO Journal (2015)
New knowledge versus consensus – a critical note on their relationship based on the debate concerning the use of barrel-vaults in Macedonian tombs
European Journal of Archaeology (2007)
EMBO reports (2007)