It's a rare scientist these days who does not know his or her citation index, most commonly in the form of the h-index introduced in 2005 by physicist Jorge Hirsch1. Proposed as a measure of the cumulative impact of one's published works, such citation metrics are increasingly being used to evaluate researchers' careers.

Do online journals narrow your horizons? Credit: Punchstock

But does a citation count provide an honest measure of a paper's worth? A study published today in Science by sociologist James Evans of the University of Chicago adds a new ingredient to this debate2. He has shown that the increasing availability of papers and journals online, including decades of back issues, is paradoxically leading to a narrowing of the number and range of papers cited. Evans suggests that this is the result of the way browsing of print journals is being replaced by focused online searches, which tend to identify more recent papers and quickly converge on a smaller subset of them.

Evans's argument is that as fewer people flick through the print version of a journal, there is less chance that readers will just happen across a paper that could influence their work. He has compiled citation data for 34 million articles from a wide range of scientific disciplines, some dating back as far as 1945, and studied how citation patterns changed as many of the journals became available online.

He found that, on average, a hypothetical journal would see the number of its papers that were cited plummet from 600 to 200 simply by making five years of its issues available online. That sounds like a bad business model, but there are some important caveats.

It doesn't necessarily mean that the journal itself gets cited less when it goes online, only that the sum total of citations get focused on fewer articles. And all these changes are set against an ever-growing body of published work, which means that more and more papers are getting cited overall. Still, the trend for online access is clearly making citation patterns narrower than they would be otherwise, fixating on fewer papers and fewer journals.

Straight to the point

In some ways, the narrowing is not a bad thing. Online searching can deliver you more quickly to just those papers that are most immediately relevant to your own work, without having to wade through more peripheral material.

Online access should also make it much easier for researchers to actually read all the work they cite — one study has suggested that 70-90% of literature citations have simply been copied from other reference lists, rather than being directly consulted3,4.

And with online resources it may even be possible to reach scientific consensus more quickly. This might mean that less attention, time and effort get wasted down dead ends. But it also means there is more chance of missing something important. "It pushes science in the direction of a press release," says Evans. "Unless they are picked up immediately, things will be forgotten more quickly."

Another worry is that this trend exacerbates the already lamented narrowing of researchers’ horizons. It is by scanning through the contents pages of journals that you find out what others outside your field are doing. If scientists are reading only the papers that are directly relevant to their immediate research, science as a whole will suffer because its tightly drawn disciplines will cease to be fertilized by ideas from outside.

There's also a possibility of collective amnesia, with the past mattering less in the desperate scramble to keep abreast of the present. Older scientists have probably been complaining that youngsters no longer ‘read the old literature’ ever since science journals existed, but it seems that neglecting the history of your field is made more likely with online tools. Indeed, Evans's data indicate that online access is driving citations to become 'younger', and reducing an article's shelf-life. This must surely increase the danger of reinventing the wheel.

Save the journal club

A similar trend in the news media has been much debated, as the possibilities for personalization of content seem to lead to a solipsistic outlook in which individuals hear only the things they want to hear. We may be approaching the point where that also applies to young scientists, particularly if it is the model they have become accustomed to as children.

It is no response – or at least a futile one – to say that we must bring back the days when scientists would have to visit the library each week and pick up the journals. The efficiency of online searching and the availability of archives are both to be welcomed. But a laissez-faire attitude to this 'literature market' could have some unwelcome consequences, in particular the risk of reduced meritocracy, loss of valuable research and increased parochialism. The paper journal may be on the way out, but we’d better make sure that the journal club doesn’t go the same way.