Access
This article is part of Nature's premium content.
Published online 15 June 2009 | Nature | doi:10.1038/news.2009.571
News
Editor will quit over hoax paper
Computer-generated manuscript accepted for publication in open-access journal.
The editor-in-chief of a journal is to resign after claiming that the publisher, Bentham Science Publishing, accepted a hoax article for publication without his knowledge.
The fake, computer-generated manuscript was submitted to The Open Information Science Journal by Philip Davis, a graduate student in communication sciences at Cornell University in Ithaca, New York, and Kent Anderson, executive director of international business and product development at The New England Journal of Medicine.
To read this story in full you will need to login or make a payment (see right).
Comments
Reader comments are usually moderated after posting. If you find something offensive or inappropriate, you can speed this process by clicking 'Report this comment' (or, if that doesn't work for you, email webadmin@nature.com). For more controversial topics, we reserve the right to moderate before comments are published.
Please see a description of the experiment on The Scholarly Kitchen: http://scholarlykitchen.sspnet.org/2009/06/10/nonsense-for-dollars/ --Phil Davis
And an editorial on The Scholarly Kitchen: http://scholarlykitchen.wordpress.com/2009/06/10/tip-of-an-iceberg/
I like how the director of publications says that the whole thing was a trick to find out the researcher's true identity after their first failed bogus submission. Hard to know if that's true, and it sounds like a made up story. This is some pretty terrible PR for Bentham.
It's unclear how this may relate to open access journals or the pays-for-publication model, as two year old prior work using the same program SCIGen resulted in acceptance in the Elsevier journal "Applied Mathematics and Computation" according to the SCIGen authors (1). This journal is neither open access (2) nor pays-for-publication (3). 1. http://pdos.csail.mit.edu/scigen/blog/index.php?entry=entry070626-110103 2. http://www.elsevier.com/wps/find/journaldescription.cws_home/522482/bibliographic 3. http://www.elsevier.com/wps/find/journaldescription.cws_home/522482/authorinstructions
What sucks is that the anti-science crowd (ie., creationists) will cite this as evidence that scientific journals and scientists are nonsensical too...just watch.
Reviewing a paper is serious business! I have'nt reviewed that many papers to make me an "expert" on reviews, but the ones that I reviewed, I tried to do my best. Being a "victim" of sloppy, occasionally even malintentioned reviewers, I didn't want to inflict the same pain on the authors of the manuscripts sent to me to be reviewed. But a thorough review takes time, lots of it--at least for me. Occasionally, if I'm lucky, it's all too easy. You paste a sentence from the manuscript in Google's search window, and voila, there it is. Entire paragraphs in the manuscript appear right in your browser: Reject! Other times I go through the calculations that the authors must have done, and again if I am lucky, catch an error after 2 hours. I recheck my calculations for another half hour, and: Reject. All of that takes time and commitment. And what do I get in return? Peace of mind. I'm probably not alone in fooling myself that this is for everyone's benefit, that I should not betray the trust that has been invested in me by the editor and the author. But in return, I'd like to, at the very least, receive the same care for my manuscripts in return---which may not always be the case. The truth of the matter is that, all too often, reviewer gets practically nothing tangible in return. I'm increasingly disillusioned as to the fairness of this transaction; I believe a reviewer should get something more tangible in return, more precisely a certain amount of compensation, no matter how symbolic. It could be a souvenir from the publisher, or enough to buy beer for a few friends, or have lunch at a half-decen restaurant maybe. This way, the editor can demand a certain care in the review process, and the reviewer has some, albeit tiny, reason to spend a certain amount of time reviewing the manuscript. After all, as everyone knows, time is money, and there is no free lunch!
Interesting. I've had six requests to review papers from a Bentham Science journal over the last year. The first was for a paper in my field, but I refused as I was very busy at the time. The other five have been for papers in fields I know nothing about - I would have been an utterly inappropriate reviewer had I accepted the requests (I didn't). Each request had the full paper attached to the email, rather than just an abstract. That seems like a very unusual review process to me.
Submitting a paper under a false name with fraudulent data... this sure looks a lot like garden variety academic dishonesty, no matter how virtuous the intentions or amusing the results. I hope the submitter consulted with the appropriate review boards at their institution before embarking on this exercise.
Phil, keep up the good work! I see that you've referenced the Sokal Affair -- a classic which is highly recommended to all readers, and upon which this effort clearly builds: http://en.wikipedia.org/wiki/Sokal_affair I confess to being lightly astonished at Douglas Ridgway's, et al., assertion that this could be in any way compared to "garden variety academic dishonesty." IMO, the fundamental principals of science should encourage and support this kind of testing. Bogus papers : real, peer-reviewed journal :: spam : legitimate email. If you can't tell the difference between the two, practically autonomically, you're going to either waste a lot of time, or fall for a lot of ridiculous nonsense. As Sokal neatly demonstrated, intellectual rigor / moral authority / credibility / knowledge hygiene / whatever you care to call it, comes first & foremost from policing ONE'S OWN DISCIPLINE. I think Phil's work extends this nicely, to include peer reviewed journals where financial interest is at stake.
Great experiment and I agree totally with the sentiments of Sami Sozuer while offering a somewhat more radical solution. I spent the last weekend diligently reviewing a manuscript, produced a two page critique to support my rejection of the work and including many contructive comments to help the authors. As a result I have my integrity, professional pride, societal contribution and peace of mind.Clearly there are many other scientists and publishers who share my values but the problem is that there are now too many who do not. Commercialism and science are not natural bedfellows and unfortunately science is increasingly threatened by unethical behaviour. Bankers we all know about but there is a more general malaise. Science is the big one; the final frontier for take over by liberal left thinking. The solution ? Peer review must remain central to publishing of science but formal accreditation is, unfortunately, now a necessity. This will only be achieved via establishment of independent licensing bodies, for both journals and reviewers. Such bodies should also incorporate the means for redress by authors against irresponsible, illogical, malicious and anonymous reviewers.
Again, I agree with the comments from Sami Sozuer and Kevin Iee. Yes, reviewing is a hard job, or better to say, very time-consuming job. For me personally, it`s always even more than two hours, it`s usually several days with two hours a day. Yes, I also feel that "science is increasingly threatened by unethical behaviour" (citation of Kevin), just modern technologies alowed unethical behaviour easier to get through. I believe, however, it will be always like this and there is only one way out - just continue to do your job as good as you can. It is the same with democracy - it`not given to people for granted once and forever, you are able to enjoy it only when you strive for it every day, and the experiment of Davis & Anderson is excellent! At the end I preserve my integrity and professional pride, combined with the hope that I helped the authors to understand better their results or to get more out of their experiments. Was the work worth it? When honestly evaluating, I believe, it was. N.B. Well, the idea of free beer from the publisher(for me, please, a cup of really nice coffee), I support anyway. Posted by: Lyudmyla M-O
As a teacher, Each year I give my students a legitimate looking bogus journal article, or two, for them to abstract/summarize/present in order to build up their crap detectors so that they learn to ask for the data/evidence supporting a common sense nonsensical statement (one I have used: The Endochronic Properties of Resublimated Thiotimoline (a fictional piece by Isaac Asimov). It's a sad time when Authors have to divulge their funding sources in their Journal articles and now we have to question the peer review process as well. Mahmood Alam, director of publications at Bentham Science Publishing, told Nature in an e-mail that "submission of fake manuscripts is a totally unethical activity and must be condemned." Well shouldn't acceptance for paid publication of fake manuscripts also be condemned as unethical? Something good comes from everything. I now have a new place to mine for articles to give to my students to build their discrimination abilities; Bentham Publications.
There is no such thing as a 'fake manuscript' ... if it exists, it's real, no matter what was the intention of the writers. It is entirely up to the editors and reviewers to decide whether any manuscript, 'fake' or not, is worth publishing. That's called objectivity!
I commend Bambang Parmanto's decision to resign over this. And I am quite surprised that no one seems to be calling for the resignation of Mahmood Alam, director of publications at Bentham Science Publishing. Not only is the integrity of the journal and its publisher his responsibility, but he offers no apology or (believable) explanation. This incident and Alam's response leave me embarrassed that I have published in a Bentham journal myself. I will never submit another manuscripy to a Bentham journal.
I see nothing unethical about testing whether an outfit will publish anything for money, and I could not disagree more with Douglas Ridgway. I too have received numerous solicitations from this publisher for articles that I never responded to. I am relieved that I did not respond.
I think we're being too hard on Bentham publications here and there is no point in rubbing it in. Besides, the editor showed the good sense and quit his job. Even the most prestigious journals such as Nature, Science and Physical Review, have been fooled, many many times, a total of 17 times to be exact, by just one fellow by the name of Jan Hendrik Schön, remember? I could be wrong, but I don't remember any editors resigning then! And how was that discovered? Schön was simply and incredibly overconfident! He realized it had become so easy to fool people, even in the most prestigious journals that he didn't even bother to generate separate data for different papers. He simply used the same computer generated data in different papers. If he hadn't been that relaxed, there's a good chance he might be a department head in some prestigious Ivy League school, or the director of Bell Labs or something. Let's face it. We're all human, sometimes too much so. When we see names of prestigious institutions, many of us drop all our guards. The converse is also true in many cases: When the name of the institution is obscure, we tend to become negative, condescending, and often reject good work. Just look at the names of the author institutions in high-profile journals and you'll know what I mean. More on Scön can be found at http://en.wikipedia.org/wiki/Jan_Hendrik_Sch%C3%B6n
I am not familiar with Bentham and their interesting publishing practices. I do not know the story that maybe preceded the submission but apparently something triggered this "experiment". The authors do not have a history of vile behavior and violent attacks on publishers in attempt to waste everyone's time and undermine the standing of journals. I do not remember the journal and who did it but some time in the 60s an attempt of nonsensical submission has been made to one of the respected journals - Physical Review Letters, or similar. The editor-in-chief caught it immediately and came up with three different reviews he wrote himself mimicking the styles of different reviewers. Of course, turned down. That was some work. Now, Mr. Douglas and Mr. Tom - congratulations. Don't you see that you are completely off the point? What fraudulent data? There was no data, there was an obvious nonsense (as far as I understand it). "Objectivity" - hm-m-m, go ahead call it that, unless you can find a better word - "greed". The publisher went for the money and he has got the nerve to talk about unethical behavior. Instead of telling stories about the well organized review process he could present the review that has been completed by "one of the board members". Yes, I agree with Mr. or Ms. Sami Sozuer and respect the dedication. The peer review has many problems but it is like democracy - far from being perfect but there is nothing better that we could use. Again, it is a different story. The publisher had not his finest moment this time. Should the editor-in-chief resign? I am not sure but if as he claims he has not had an acess to the paper, I certainly feel his emotions and would perhaps resign if I was in his shoes.
Please pardon my self-advertisement, but about 3.5 years ago when the first papers by the folks from the MIT group got accepted to a fake conference we developed a tool to detect such papers (not perfect, but worked great for SCIgen) and published at SIAM Data Mining 2006. Obviously, Bentham folks didn't know about that :) www.inauthentic.org
Perhaps it might help if the names of the reviewers were published along with the paper. The readers would get more information, the reviewers would receive at least token value, and it might give a lazy reviewer second thoughts.