Lars Bjørnshauge set up the Directory of Open Access Journals in 2003. Credit: Marina Martínez-López

When Lars Bjørnshauge founded a website to index open-access journals in 2003, just 300 titles made the list. But over the next decade, the open-access publishing market exploded, and Bjørnshauge’s Directory of Open Access Journals (DOAJ) along with it. Today the DOAJ comprises almost 10,000 journals — and its main problem is not finding new publications to include, but keeping the dodgy operators out.

Scientist killed on MH17 flight fought for access to HIV therapy Quantum gravity suggests black holes will turn into 'white holes' Understanding the effects of weight-loss surgery

Now, following criticism of its quality-control checks, the website is asking all of the journals in its directory to reapply on the basis of stricter criteria. It hopes the move will weed out ‘predatory journals’: those that profess to publish research openly, often charging fees, but that are either outright scams or do not provide the services a scientist would expect, such as a minimal standard of peer review or permanent archiving. “We all know there has been a lot of fuss about questionable publishers,” says Bjørnshauge.

The reapplication process will also create one of the largest ‘whitelists’ of acceptable open-access journals, helping the DOAJ to become a more useful tool for funders, librarians and researchers who want to look up information on a publication or import its metadata into their catalogues. Those journals meeting the highest criteria — expected to be about 10–15% of the total — will also be given a ‘seal’ of best practice.

The DOAJ, which receives around 600,000 page views a month, according to Bjørnshauge, is already supposed to be filtered for quality. But a study by Walt Crawford, a retired library systems analyst in Livermore, California, last month (see go.nature.com/z524co) found that the DOAJ currently includes some 900 titles that are mentioned in a blacklist of 9,200 potential predatory journals compiled by librarian Jeffrey Beall at the University of Colorado Denver (see Nature 495, 433–435; 2013). In addition, journalist John Bohannon last year proved that at least 73 journals in the DOAJ were suspect; in a sting operation, he sent them an obviously flawed paper which they then accepted for publication (J. Bohannon Science 342, 60–65; 2013). The DOAJ removed the journals from its index (see ‘Stunted growth’).

The DOAJ had the idea of introducing stricter standards a few years ago, says Alma Swan, co-founder of the non-profit company IS4OA, which now operates the DOAJ (previously it was hosted by Lund University in Sweden). “We need to show which journals come up to a minimum standard of quality,” she says.

Since May, would-be new members have had to fill in a tougher entry form containing more than 50 questions, which will now form the basis of the reapplication criteria. They include requests for information on a journal’s digital archiving policy, its editorial board and its content licensing. “I suspect about 10% of journals on the list will not be able to pass the reapplication,” says Bjørnshauge.

Paul Peters, the chief strategy officer at open-access publishers Hindawi, headquartered in Cairo, believes that the new criteria will be “incredibly important”. “Scholarly researchers need a way to determine whether a given journal is adhering to best practice, and I believe that the DOAJ can provide a trusted and scalable mechanism for doing so,” he says.

It is not clear whether the DOAJ’s whitelist will become the pre-emiment index of trustworthy open-access journals. Beall says that the directory’s credibility has already been hurt and that its new approach is “too little, too late”. He is also not sure how the DOAJ will spot when a publisher is lying about its services. Moreover, Beall points out, many researchers and universities will instead judge a journal’s quality by whether it is indexed in major citation databases, such as Elsevier’s Scopus index, rather than looking at the DOAJ’s list.

Bjørnshauge says that a small cohort of some 30 voluntary associate editors — mainly librarians and PhD students — will check the information submitted in reapplications with the publishers, and there will be a second layer of checks from managing editors. He also finds it “extremely questionable to run blacklists of open-access publishers”, as Beall has done. (Crawford’s study found that Beall’s apparently voluminous list includes many journals that are empty, dormant or publish fewer than 20 articles each year, suggesting that the problem is not as bad as Beall says.)

But will any kind of whitelist help vulnerable researchers to avoid publishing in substandard journals? Beall doesn’t think so. “There’s no evidence that the whitelist approach has been helpful in encouraging researchers not to become victims of scams,” he says. “Bad open-access publishers are still growing like crazy.”