Jana Christopher discusses her job as a data integrity analyst at EMBO Press.
Jana Christopher has one of the rarest jobs in science publishing. Working for the publishers EMBO Press, in Heidelberg, Germany, she checks all images in manuscripts accepted by the publisher’s four scientific journals in an effort to spot aberrations before the papers are published. Nature spoke to her at the sidelines of the 4th World Conference on Research Integrity in Rio de Janeiro, Brazil, in early June.
How did you get into image forensics?
My background is not in science. I used to be a theatre make-up artist, at the English National Opera in London’s West End, and a translator. Then I ended up as a journals assistant and came to EMBO — the European Molecular Biology Organization — in Heidelberg. I’ve been assessing scientific manuscripts for image aberrations and plagiarism since 2011.
What does the job involve?
I check all image-based scientific data in manuscripts that are about to be accepted across the four journals we publish (The EMBO Journal, EMBO reports , EMBO Molecular Medicine and Molecular Systems Biology), as well as dealing with any issues that arise on papers already published. Normally I’d spend a minimum of 10 minutes on a manuscript, but if something comes up I can spend hours or even days on it. Last year I checked around 2,000 figures from around 350 accepted manuscripts.
Because I don’t have a scientific background, I don’t really understand the content of the papers. And I don’t need to for the work that I do. I check to see if micrographs, photographs and data we typically publish, such as western blots and gels, are duplicated or illicitly manipulated — flipped, rotated, inserted, cropped, duplicated, spliced or otherwise digitally altered. I use tools provided by the US Office of Research Integrity that work with Photoshop: they allow you in a semi-automated procedure to adjust the contrasts and settings to highlight flaws.
It’s a simple process, but you need to know where to look. For example, it is much harder to spot when the blank background on a gel has been cloned — it takes an experienced eye to spot those patterns. I’m very aware of the limitations. If someone really tries to cheat and they cheat well, it’s unlikely that we would see that. But most of what we find are genuine mistakes, which we prevent from entering the scientific literature.
How many manuscripts have flaws?
Around 20% of the papers that we check just before acceptance have issues that need to be rectified. Most of those are cases of naive ‘beautification’, in which images are altered to look prettier or simply because of an accident when the paper was put together. Around 1% of those amount to manipulation that is so serious that we don't publish the manuscript as a result. So that's pretty low, but still significant.
What do you do when you find a problem?
I always go back to discuss the matter with a scientific editor and sometimes the chief editor. It’s not just based on my judgement. We have clear guidelines about how to react when we find these things. For example, when we go back to the authors, we generally don’t specifically say what the problem is. We ask them to provide the raw data for a series of figures and may hint that they should ensure it is carefully checked. If they spot the mistake themselves, we interpret that as a good sign.
We have a three-tier system to classify the severity of the problems we find and the response — for example, at what point we should contact the research institutions.
Has your work led to lower rates of image manipulation?
It’s pretty stable, surprisingly. The Journal of Cell Biology, which has been checking images for more than a decade, sees no change either in the proportion of accepted publications it revokes for image manipulation [about 1%]. You would think that if people know you’re a journal that checks images, they’d be more careful.
Shouldn't peer reviewers spot the errors?
Obviously, all the problems I see have got past peer review and no one noticed them. Sometimes they are things that are hard to spot, and it’s not the referee’s job, I think, to do image forensics. But sometimes you really do wonder how they got past. Last year our chief editor, Bernd Pulverer, passed me the [now-retracted] Nature papers on ‘STAP’ stem cells to check — without telling me their background — and I spotted three errors that had got past the journal and peer reviewers. Ideally, data checking would happen upstream in the labs, way before a paper gets to journals. It’s surprising that some researchers seem to regard figures more as illustrations than representations of scientific data.
Do you look at websites such as PubPeer, on which images in published papers are regularly flagged?
When a paper from EMBO journals is flagged on PubPeer I will check it immediately. They are usually older papers that were published before 2010, which means I may only have a low-resolution image to look at and no original data, and then the issue is more sensitive. We will not make a conclusion unless we can confidently say there is a problem, and then of course, the author is innocent until proven guilty — but PubPeer comments have led to retractions and corrections.
Do you enjoy the work?
I really enjoy it. It’s a small contribution I make, but a significant one. If you look at the number of mistakes we manage to correct before publication, I think it is worth it. It’s especially nice to get replies from authors who are really grateful when we spot errors. We are protecting the published record and their reputation.
Few other journals or publishers have data integrity analysts. Should they?
Of course I think my work is very valuable, so I think it’s something that everybody should be doing. You don’t have to be a scientist; anyone with a good eye and good training could do it.
Other journals that employ image checkers who inspect all figures at acceptance stage include The Journal of Cell Biology and five journals published by the UK publisher The Company of Biologists. In response to a query from Nature’s news team after this interview, a Nature spokesperson said: “Nature Publishing Group utilises tools to do spot-check analysis of images and requests full scans of key western blot data, but we currently do not have the resources to undertake detailed image analysis on all of our papers. However, we continue to review our procedures regarding image checking and have increased the number of routine spot checks that we undertake on Nature journal papers to approximately two papers per week (two per issue). This is as much as current resources allow, but we will be doing further work to assess the value of more extensive checking.”