Citation analysis reveals the game changers

A study identifies papers that stand the test of time.

29 May 2018

Piriya Photography/Getty

Fewer than two out of every 10,000 scientific papers remain influential in their field decades after publication, finds an analysis of five million articles published between 1980 and 1990.

Among these seminal papers is the first tool for sequencing the genome, a longitudinal study linking the hepatitis B virus to higher rates of liver cancer among Chinese men, and the use of hypnosis to investigate the role of emotions in memory.

These types of papers represent “the shoulders on which the rest of research stands” says Lutz Bornmann, a sociologist of science at the Max Planck Society in Germany, who co-authored the study in the Journal of Documentation along with colleagues in China. While most papers see a gradual drop in citations two to three years after publication, the papers identified by Bornmann continue to gather a high number of citations decades after being published. This suggests that they establish the foundation for many branches of science.

To identify these classic papers, the researchers analysed the citation impact of scientific articles published between 1980 and 1990 over a period of 25 years. They categorised papers as fairly, remarkably, or outstandingly cited, based on how much higher their citation impact was from the mean in their field.

Of the five million papers, only 1,013 were outstandingly cited, indicating that they maintained a high number of citations for decades after publication. A subset of 40 papers stood out as being exceptionally cited, many of which were in the fields of chemistry, internal medicine, biochemistry, and molecular biology. Some 80% of these standout papers were authored by researchers based in the United States.

Bornmann says the ‘landmark’ paper metric is a simple way to identify research that makes an important leap in a field.

Scientific progress, he says, is largely shaped by discoveries that prove successful in the long-run, but research evaluations often rely on indicators that span shorter timeframes. The Journal Impact Factor, for example, tracks the average number of citations up to two years after a paper is published.

However, Michael Hazoglou at the University of California, San Diego, says that Bornmann’s approach might overlook some papers that have had a lasting impact on a field.

Hazoglou analysed 151,082 papers from Physical Review to describe how citation counts can change over time. He found that some papers are slow to gather citations in the first few years after publication, but gather momentum in later years. Other papers are quick to rise but fall flat shortly after they peak.

Hazoglou also points out that some papers quickly become standard knowledge and may lose out on gaining citations because they end up being cited from textbooks.

Below are some of the top landmark papers identified in Bornmann’s analysis.

George Church and Walter Gilbert, Proceedings of the National Academy of Sciences, 1984

In the late 1970s, geneticist George Church teamed up with Nobel Prize winning biochemist Walter Gilbert, which led to a paper that would change the face of genomics.

Using mouse DNA, the team at Harvard University developed the first method for directly sequencing the genome. The technique was cheap, fast and could identify specific sequences of DNA with greater precision than prior methods, without the need for cloning.

A decade after its development, the method was used to sequence the whole genome of a pathogenic bacteria. “I hoped that it would be successful, but expected it to be replaced quickly,” says Church.

The method was used with minor changes until 1997 and laid the foundation for next-generation sequencing, which allows researchers to sequence whole genomes in a single experiment. The findings also paved the way for the Human Genome Project.

Chris Hunter and Jeremy Sanders, Journal of the American Chemical Society, 1990

From the helical shape of DNA to crystalline materials, a wide variety of structures are controlled by interactions between circular molecules called aromatic rings. Known as pi-pi interactions, these phenomena are fundamental to the design of drugs and semiconductors.

Despite their importance, scientists in the field of supramolecular chemistry were confused about the nature of these pi-pi interactions. In 1990, chemists Chris Hunter and Jeremy Sanders published findings revealing that the interactions are governed by local electrostatics.

“It was the first simple explanation for the interactions between aromatic rings,” says Hunter, who currently heads the Hunter Group at Cambridge University. He says that the findings have been used to develop efficient methods for assessing drug–receptor interactions.

Alan Guth, Physical Review D, 1981

The Big Bang Theory covers a lot of ground when it comes to understanding the beginnings of the Universe, including the origin of matter and the laws of physics. But there was a problem in the theory.

A tiny value, called the critical density, seemingly fine-tunes the amount of matter in the Universe to prevent it from expanding too quickly or collapsing in on itself. Physicists were perplexed about how such a specific value could give rise to the Universe of today.

In 1981, Alan Guth at the Massachusetts Institute of Technology published a paper that offered a solution to the problem. He proposed that the Universe underwent rapid expansion shortly after the Big Bang and continued to expand at a slower rate, an idea that is now widely accepted in physics. It explains why the Universe seems to have a ‘flat’ geometry and why it appears the same in all directions.