Nature | Toolbox

Rate that journal

Consumer-oriented websites allow researchers to compare the merits of scientific journals and review their publishing experiences.

Article tools

Illustration by the Project Twins

What if scientific journals were like hotels, restaurants and holiday operators — easy to compare online and reviewed by those who use them? That thought occurred to conservation biologist Neal Haddaway two years ago: frustrated by a bad experience publishing his work with a journal he prefers not to name, he decided to launch Journalysis.org, a journal-review site that he likens to TripAdvisor. “I wanted to basically reward the journals that were doing a good job and, within reason, name and shame the ones that weren't doing so well,” he says.

Haddaway, now a project manager at the Mistra Council for Evidence-based Environmental Management in Stockholm, was not alone in his thinking. His site is one of a handful of comparison websites that have sprung up in the past few years. Those developing these tools say that, although the practice of rating journals online has been slow to take hold, the sites help authors to become discriminating consumers of publishing services, choosing the journals that suit them and dodging questionable operators.

Journal-comparison tools allow authors to search or filter journals by various dimensions of performance, from prestige to publishing speed. Many of these tools are free to use, created by consultancy firms that make their money from related services for researchers, such as English-language editing and advice on publishing. Among those websites are Journal Selector, created by the London-based firm Cofactor, collating several hundred journals; JournalGuide, from Research Square in Durham, North Carolina, covering more than 46,000 journals; and the Edanz Journal Selector, a site designed by the Edanz Group in Fukuoka, Japan, compiling some 28,000 titles.

These comparison websites might seem unnecessary: researchers tend to identify the best destinations for their work by checking where studies they admire have been published, or by asking colleagues for advice. But Keith Collier, chief operating officer at Research Square, says that his company sees a market in researchers who may be unfamiliar with English-language journals, especially those located outside the United States and Western Europe.

Even Western researchers might feel overwhelmed by the rapid growth in the scientific literature, finding it hard to keep track of the number of journals sprouting up. Online comparison tools could help them to select the best journal for interdisciplinary work — or steer them away from predatory publishers that take researchers' money, but offer little in return (see 'The right one for me').

The right one for me

How to pick a journal that fits your research

With tens of thousands of journals to choose from, some researchers might have trouble working out which journal best suits their article — perhaps their cross-disciplinary research falls outside conventional venues. Some tools allow researchers to identify suitable publications on the basis of the title or abstract. The earliest example of this is JANE, the Journal/Author Name Estimator, which is built on a database of millions of Medline records. Given an article title or abstract, JANE suggests related journals, authors or articles.

Created in 2007, JANE now fields about 20,000 queries a month, says Martijn Schuemie, who designed the tool and is now an associate director at Johnson & Johnson, working in Hong Kong. He uses JANE himself, he says, especially to pinpoint journals closely aligned with his research. Even journal editors use it, notes PeerJ web developer Alf Eaton, to identify potential peer reviewers. Other sites that suggest suitable journals on the basis of the title include Journal Selector from the Edanz Group in Fukuoka, Japan, and JournalGuide from Research Square in Durham, North Carolina.

Inexperienced users might also have trouble recognizing predatory or counterfeit outfits that do no legitimate peer review and little, if any, editing for the money they charge (see Nature 495, 433–435; 2013). Publishing in such journals not only wastes authors' money; it can even damage a career, says Jeffrey Beall, scholarly communications librarian at the Auraria Library at the University of Colorado Denver. He has compiled on his website (http://scholarlyoa.com) blacklists of publishers and journals that he considers predatory. Others, such as the Directory of Open Access Journals (http://doaj.org/), have compiled 'whitelists' of trustworthy publishers (see Nature 512, 17; 2014). JournalGuide has a built-in whitelist that highlights reputable (or 'verified') journals.

Most sites provide an indicator of prestige, such as a score that denotes how many citations on average an article in that journal accrues. But there is much more to picking journals than this one measure, says James Maclaurin, a philosopher at the University of Otago in Dunedin, New Zealand. He has developed a mobile app called HelpMePublish (available on Apple's iOS operating system, with an Android version in development), which indexes more than 6,000 journals.

The consumer psyche

Some researchers want only open-access journals; others care more about the acceptance rate, fees and time to publication. For Maclaurin, a killer detail is whether a journal allows 'double-blind' peer review, in which the authors' and peer reviewers' identities are withheld from each other to prevent prejudice.

Often, sites are simply aggregating data from individual journals' webpages. But the relevant information is not always available: Maclaurin randomly selected 300 journals from his database, and found that nearly half of their websites made no mention of peer-review policies and only one specified the journal's acceptance rate. To get information for his database, he surveys journal editors, giving him access to details not available online.

Maclaurin's app allows people to search for free, but charges for access to certain data such as a journal's acceptance rate: individual subscriptions are US$4.99 a year, and institutional ones $1,250. He says that the app has been downloaded by “thousands of people” from 28 countries. JournalGuide shows similar success, hosting, at present, 13,500 users per month, according to product-management director Laura Stemmle.

Visit the Toolbox hub for more articles

The emergence of such tools reflects a shift in the dynamic between publisher and researcher, argues Peter Binfield, co-founder and publisher of the open-access journal PeerJ. As long as an outlet is of reasonable quality, he says, researchers are starting to recognize that the content of their article matters more than esteem garnered from the reputation of the journal. “It's not where you publish; it's what you publish,” he says. As such, authors are shifting towards a more transactional, consumer-like attitude to publishing, Binfield thinks — they are looking for the best deals on fees and time to publication (even though many still also hanker after prestige).

If that is true, consumer-oriented researchers might relish the chance to read reviews and leave ratings of their own — “a Yelp restaurant review for journals”, as Binfield puts it. HelpMePublish restricts users to numerical ratings — on a scale of 1 to 5 — on topics such as refereeing practice and communication. But some websites enhance the experience even further. Journalysis and SciRev — both of which are free — provide space for free-form comments. Created by economists Jeroen Smits of Radboud University in Nijmegen, the Netherlands, and Janine Huisman of the Centre for International Development Issues Nijmegen, SciRev boasts some 14,000 journals in its database and has received more than 1,000 user reviews in its one year of operation.

In theory, a lot of bad reviews might push publishers to change their procedures. But scientists have been slow to embrace the feature. Inexplicably, some publishers have received many reviews on SciRev, whereas others have received few. For example, the Open Access Macedonian Journal of Medical Sciences has 38 reviews (all accompanied by positive ratings), yet Science has 6 and Nature only 2, none of which includes ratings.

JournalGuide initially accepted user reviews, but dropped them due to a poor response rate. Journalysis is faring little better; few researchers have left ratings. “That's where we're all falling down really — users aren't submitting enough data,” Haddaway says. This is despite the fact that reviewing sites allow user anonymity. Typically, these tools require registration only with a validated academic e-mail address.

“I think there is just a reluctance to say something about a journal you may need to go back and try to submit something later on,” says Collier. If that is true, perhaps it is not surprising that review features have yet to achieve critical mass. Even so, the lack of engagement so far has not jolted Haddaway's conviction that the tools are needed. “I think there needs to be more transparency,” he says.

Journal name:
Nature
Volume:
520,
Pages:
119–120
Date published:
()
DOI:
doi:10.1038/520119a

Author information

Affiliations

  1. Jeffrey M. Perkel is a writer based in Pocatello, Idaho.

Author details

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments for this thread are now closed.

Comments

1 comment Subscribe to comments

  1. Avatar for Jeroen Smits
    Jeroen Smits
    Thanks Jeffrey for your overview. Two additional points. First, that researchers are not prepared to share their experiences with the review process is not what we observe at SciRev. The over 1000 reviews available at www.scirev.sc/reviews -- all provided by registered researchers -- suggest the opposite. In particular young researchers for whom a speedy publication process is essential for their career startup -- and who are generally more experienced web participants -- take the effort to provide this information. Second, the role of the scientific journals is missing in the article. These journals can play a more active role in creating transparency than they are doing now. They may put a link to an independent review site at their homepage and ask their authors at the end of the review process to share their experience. Journals with a quick and high quality review process can only gain from transparency (and the less efficient ones may be persuaded to organize things better). The reason that the Open Access Macedonian Journal of Medical Sciences (mentioned in the article) has received so much reviews might have to do with the fact that they put a link to SciRev on their website (www.mjms.mk).
sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.