Analysis suggests an increasing tendency to exaggerate and polarize results.
Scientists have become more upbeat in describing their research, an analysis of papers in the PubMed database suggests.
Researchers at the University Medical Center Utrecht in the Netherlands say that the frequency of positive-sounding words such as ‘novel’, ‘amazing’, ‘innovative’ and ‘unprecedented’ has increased almost nine-fold in the titles and abstracts of papers published between 1974 and 2014. There has also been a smaller — yet still statistically significant — rise in the frequency of negative words, such as ‘disappointing’ and ‘pessimistic’.
Psychiatrist Christiaan Vinkers and his colleagues searched papers on PubMed for 25 'positive' words and 25 'negative' words (which the authors selected by manually analysing papers and consulting thesaurus listings). The number of papers containing any of the positive words in their title or abstract rose from an average of 2% in 1974–80 to 17.5% in 2014. Use of the 25 negative words rose from 1.3% to 2.4% over the same period, according to the study, published in the British Medical Journal on 14 December1.
The change seems to be a trend particular to research papers, the authors say: an analysis of the same words in published books (using the Google Books Ngram viewer) showed little change. Nor did the researchers see significant changes in how often research papers used a list of ‘neutral’ words, and another list of 100 randomly chosen common nouns and adjectives.
The most obvious interpretation of the results is that they reflect an increase in hype and exaggeration, rather than a real improvement in the incidence or quality of discoveries, says Vinkers. The findings “fit our own observations that in order to get published, you need to emphasize what is special and unique about your study," he says. Researchers may be tempted to make their findings stand out from thousands of others — a tendency that might also explain the more modest rise in usage of negative words.
The word ‘novel’ now appears in more than 7% of PubMed paper titles and abstracts, and the researchers jokingly extrapolate that, on the basis of its past rise, it is set to appear in every paper by the year 2123.
Vinkers’ team also broke down the figures according to journal impact factors and the geographical affiliations of the authors. They found that the increase in positive words was smaller in recent years for 20 high-impact journals, compared to the average for all journals. It was also smaller over the past decade or so for authors affiliated in English-speaking countries, relative to authors outside these countries.
Computer scientist Peter Sheridan Dodds at the University of Vermont in Burlington, a specialist in text analysis, cautions that the findings are preliminary, because the work is based on just a few handpicked words. “The overall trend may well hold up with a full analysis, but we have to look at how the whole word ecology is changing,” he says. It may be that equally positive words were once common in the literature but have fallen out of favour.
But Vinkers and his colleagues think that the trend highlights a problem. “If everything is ‘robust’ and ‘novel’”, says Vinkers, then there is no distinction between the qualities of findings. “In that case, words used to describe scientific results are no longer driven by the content but by marketability.”
Whether the same trends exist in other disciplines isn’t yet clear, but Vinkers thinks that the PubMed corpus might be especially prone to these effects. “We expect that the publication pressure which may underlie these results is particularly present in biomedical, social and psychological sciences”, he says.
Vinkers, C. H., Tijdink, J. K. & Otte, W. M. Br. Med. J. 351, h6467 (2015).
Related links in Nature Research
Related external links
About this article
Cite this article
Ball, P. ‘Novel, amazing, innovative’: positive words on the rise in science papers. Nature (2015). https://doi.org/10.1038/nature.2015.19024