Main

The body of scientific knowledge has been expanding rapidly. Beginning about the middle of the twentieth century, review articles became popular reading for overwhelmed scientists in biology and medicine hoping to stay abreast of the most significant recent findings.1 Not surprisingly, as the number of original research articles has increased, the number of review articles has increased as well. The question must therefore be asked: have we reached a point where there is now an unmanageable quantity of review articles? If so, is the review literature serving its intended purpose? Moreover, what are the reasons for authors to write and journal editors to publish so many review articles?

This Editorial Perspective analyzes the growth of the review literature in the life sciences with a focus on pathology. We examine the role of review articles in the current scientific climate, and offer some thoughts on how to publish quality reviews that will meet the needs of today's scientists.

The Growth of Biological and Medical Knowledge

In 1978, Durack2 quite literally determined the ‘weight of medical knowledge’ by determining the weight of the Index Medicus® for each year from 1879 through 1977. He found that there had been an exponential increase in the mass of the publication starting in 1946 and continuing through 1977. A 1989 update by Madlon-Kay3 showed that the rate of increase had since become linear, but nonetheless, a remarkable amount of new data was accumulated each year.

In order to update this information for 2007, hoisting bound volumes to the scales would not be the method of choice, since the last print issue of Index Medicus® was published in December 2004. More significantly, the key MEDLINE indicators are now freely available on the National Library of Medicine web site (see Materials and Methods). Internet access permits ready appreciation of the fact that for year 2006 there were 5020 journals, from which 623 089 articles were indexed for MEDLINE, up from 4538 journals containing 502 056 articles in 2002. Remarkably, there were 896 million MEDLINE/PubMed searches performed in 2006, up from 380 million in 2002. Clearly, in the Digital Age, it is easier than ever to retrieve web-based information from the biomedical archive. But despite these great advances in cataloging, sorting and accessing the literature, it takes no less time to read and understand a paper today, and there are just too many papers to read.

Are Too Many Review Articles Published?

As the original scientific literature has increased dramatically, the review literature seems to have expanded as well. Ostensibly, review articles are instruments that enable discerning readers to ‘keep up’ with the literature. However, publication of more review articles than any reasonable scientist can sort through efficiently makes the review literature almost as difficult to peruse as the original literature. Thus, redundancies in the review literature and an abundance of poor-quality reviews would belie the very purpose of review articles.

In approaching this topic, the first question asked was whether review articles have indeed increased commensurate with the overall biomedical literature. Using two topics as introductory illustrations, Box 1 presents some compelling preliminary evidence that the review literature may be proliferating at a rate that could cause some concern. In these two examples (‘hepatitis’ and ‘hepatic stellate cells’), the review literature is growing at more than twice the rate of the literature pertaining to original research. One can also marvel at the publication of one new review for every seven new original research papers! Certainly, the original literature will expand as either a field matures and/or becomes more complex, but are these increases in the review literature justified?

Box 1 Evidence for the expansion of the review literature
Full size table

Definitions of a Review Article

We might consider the review article easy to define, but do we know one when we see it? Categorizing paper as a review seems simple enough, because a salient feature of a review article is that it does not present original data.4 In fact, presentation of original data in a review article subverts the process of peer review of original work. Rather, review articles ‘collect, select, order and interpret the huge outpouring of scientific reports, putting relevant findings and generalizations in a form useful to researchers outside the immediate group working on the problem’.4 In more pragmatic terms, ISI Web of KnowledgeSM Science Citation Index (see Materials and Methods) categorizes a published paper as a review: (a) if it has more than 100 references; or (b) if it appears in a review journal or the review section of a research journal; or (c) if the paper states in the abstract that it is a review. By these definitions, a review article might simply be an uncritical compendium of published facts, with abundant references.

Those who professionally catalog articles do not always agree as to what constitutes a review. According to ISI Web of KnowledgeSM Science Citation Index, in 2006 the 6164 journals in the Journal Citation Reports® Science Edition included 46 094 reviews. However, according to PubMed, the 5020 journals they covered in 2006 contained 79 844 review articles. This appears to be a wide discrepancy, even though the two sources do cover somewhat different journals and neither covers every scientific journal.

To add to the confusion, sometimes journal editors do not agree with ISI or PubMed as to which of their journal papers are review articles. To use our own experience, the Executive Editors of Lab Invest (Laboratory Investigation) believe that the journal published 10 review articles in 2006, but ISI recognizes only seven as reviews, and PubMed claims nine. The discrepancy in this journal's case arises from inconsistent categorization of the five provocative ‘Pathobiology in Focus’ (PIF) papers published that year. This series provides critical appraisals of developing areas of biomedical science,5 so are distinct from ‘minireviews’ and are written with little concern as to how ISI or PubMed categorize the works.

The Value of Review Articles to Authors

An editorial invitation to write a review article for a prestigious journal is an exhilarating event for a young or mid-career biomedical scientist. The review gives opportunity for the younger scientist to provide her/his unique appraisal of knowledge in her/his area of expertise. Reality hits some months later with a deadline looming, when the ‘weight of medical knowledge’ becomes a heavy burden to the writing author, not to mention the challenge of succinctly summarizing some 5–10 years of personal scientific work! Regardless, the ultimate publication of a review is an important addition to a Curriculum Vitae, and gives evidence that the author has achieved a noteworthy level of recognition from her/his scientific peers. Such evidence is considered at the time of review for Promotion and Tenure: a Curriculum Vitae with ‘only’ original research articles is not quite as impressive as one with a selection of reviews in key journals (it can be overdone). For a more senior author, reviews may or may not have career value. On the one hand, the senior author carries a key responsibility for providing perspective on burgeoning scientific knowledge and hence an obligation to accept the editorial invitation to write a review. On the other hand, and as this perspective might suggest, invitations to write reviews can become unceasing and quite burdensome. A cycle is then generated: the senior scientist invites a junior colleague to author the review, thereby enhancing the career of the junior author. Indeed, the writing of a review article might be a valuable educational experience for a junior person, since it requires comprehensive reading of the scientific literature—often well beyond the regions of one's own expertise.

The points above pertain to invited reviews. Unsolicited reviews have a less clear outcome for authors. In our editorial experience, receipt of a completed review from an author usually leads to a ‘reject’, since it is the editors who decide what thematic scope reviews should fulfill for the journal. Receipt of an unsolicited suggestion for a review from a potential author has a higher likelihood of success, since the editors can then work with the author during the crafting of the review so as to optimally meet journal needs and specifications. In this latter instance, an author can use this mechanism for self-promotion to successfully advance her/his career this interests through the peer recognition afforded by a published review.

The Value of Review Articles to a Journal

Why do journal editors recruit review articles? Ideally, editors strive to publish reviews that fit within the scope of the journal and serve the readership. The editors of society journals must also consider the needs of the members and the mission of the sponsoring society.6 However, editors are especially likely to recruit review articles that will increase the readership of the journal.7 Because the number of journals increases annually (Thomson ISI Web of Science® Journal Citation Reports covered 5550 journals in 1999 and 6164 in 2006) and library budgets for journal subscriptions (electronic or hardcopy) seem to shrink with time, wider visibility is important to editors and publishers. In our case, although the editors of Lab Invest publish reviews that focus on the mechanisms of disease for a primary audience of investigative and practicing pathologists, we are keenly aware that mechanistic studies are important to scientists in many fields. Hence, a broad readership would presumably benefit from this journal's reviews. In turn, if such readers are impressed with a review article, they might read other papers in this journal, and perhaps submit their own work to Lab Invest. Fortunately, when such editorial policies lead to an increased submission rate and more web views of published articles, results are apparent fairly quickly, sometimes within a matter of months.

Quite frankly, review articles are viewed by many editors as a mechanism used by competing editors to ‘drive the impact factor’.6, 8, 9 However, to engage in this practice, one must be patient, because it takes a lot of time before the outcome is apparent. For example, reviews that were recruited in 2003 were most likely not published until 2004 and therefore were only reflected in the impact factors for 2005 and 2006, which were released in mid-2006 and 2007, respectively. This is a lag time of almost 4 years! Nevertheless, the lure of review articles is revealed by a look at the Journal Citation Reports: the impact factors of review journals are inordinately high, to the envy of editors of original research journals. In 2006, six of the top 10 journals (by impact factor) were review journals. In the top 1% of journals, almost half (29 out of 62) were review journals. Interestingly, although it has long been known that review articles are cited more often than research articles,10 there are no documented reasons for this phenomenon.11 One can posit that it is simply easier to cite a review than to read, understand, judge and sort original data.10 However, the best review articles may be cited frequently precisely because they ‘are authoritative summaries of research that distinguish between the good and the bad, providing guidance for further research’.10

To our knowledge, there has been no systematic study on the effect of review articles on the impact factors of journals that primarily publish original research articles. Although it is beyond the scope of this Perspective to assess the quality of review articles in the existing literature, this study uses the pathology literature to examine the quantity of research and review articles, the citation rate of review articles and the impact of review articles upon pathology journals. It is our hope that the ideas presented in this study will enable authors, editors and referees to look with fresh eyes at the review manuscripts under their consideration.

MATERIALS AND METHODS

Quantity of Scientific Literature Listed in PubMed

PubMed searches (www.pubmed.gov) were performed on 12 July 2007, for the each of the years 1994–2006, using the ‘limits’ tool set to ‘year published’. No topic was chosen and the total numbers of articles were revealed in the ‘All’ tab and review article counts were shown on the ‘Review’ tab.

NIH Funding and NIH-Supported Publications

The search strategy for articles that were supported by NIH funding can be found at http://www.nlm.nih.gov/bsd/pubmed_subsets/nih_support_strategy.html, and was performed for the years 1994–2005 by using the limits tool set to ‘year published’. NIH awards by fiscal year and funding mechanism for years 1994–2005 were found at http://grants.nih.gov/grants/award/trends/fund9405.html. Data reported are for extramural research projects, not the total NIH budget.

Thomson ISI Searches

Thomson ISI Web of Science Science Citation Index and Journal Citation Reports (http://portal.isiknowledge.com/portal.cgi) were employed to generate data on article and review count, citations of articles and reviews, impact factors of journals and countries of origin of authors. This material is only available to users by subscription.

Calculation of Journal and Review Journal Quantities

The total number of journals in the Science Citation Index was determined by examination of the Journal Citation Reports for each year from 1999 to 2006. The number of review journals in the top 10 and the top 1% of journals was determined by counting the number of journals that have the words ‘review’ or ‘trend’ in the title.

Pathology, Physiology and Cardiology Reviews

The total number of pathology research articles from 1991 to 2006 was determined by a general search of the Science Citation Index, restricted to the document type ‘article’ with the topic ‘patholog*’, which found articles with the term in the title, abstract or as a key word. Likewise, searches were for ‘physiolog*’ and ‘cardio*’. The symbol * denotes ‘wild card’. The searches were performed on March 8, 2007.

Articles and Reviews in Specialty Journals

To determine the numbers of total pathology, cardiology and physiology articles in specialty journals dedicated to those fields, respectively, the searches described above were then restricted to subject categories. For example, the subject category ‘pathology’ in the Science Citation Index corresponds to the ‘pathology’ journals in the Journal Citation Reports.

Impact Factors without Journal Reviews or Self-Citations

Impact factor calculation is described in detail in the Journal Citation Reports. Briefly, the 2006 impact factor for a journal is the number of citations in 2006 to articles published in 2004 and 2005, divided by the number of citable articles published in the same 2 years. Impact factors for each of twelve top pathology journals with and without review articles were calculated, as were the impact factors of the review articles alone. Published journal impact factors were obtained directly from Journal Citation Reports, and were identical to our calculated total journal impact factors using their contained data, thereby validating the calculations performed for this study.

The Journal Citation Reports web site also lists self-cites under ‘cited journal data’ and ‘citing journal data’ for each journal. Thus, journal self-cites can be subtracted from the total citations to calculate the modified impact factor. In order to calculate the journals' impact factor without review articles, a search of the Science Citation Index was performed for the relevant years to determine the number of review articles and the number of citations, and the data were used to recalculate the impact factors.

Statistics

Selected comparisons of journal impact factors calculated with and without reviews were made using the two sample one-tailed t-test, assuming equal variance.

Review Articles in Pathology Journals

To determine the total number of review articles in the 2005 pathology literature, a general search of the Science Citation Index was performed for review articles published in 2005 and sorted by times cited. No topic was chosen. The search was refined by subject categories and pathology was chosen. A further analysis was also performed to determine the countries of origin of the corresponding authors. Whether the corresponding authors were NIH-funded was determined by a search of ‘PI name’ on CRISP http://crisp.cit.nih.gov/.

RESULTS

Expansion of the Scientific Literature

Figure 1a shows the expansion of the scientific literature available on PubMed from 1994–2006. The overall number of articles grew from 423 665 in 1994 to 727 469 in 2006, a 1.7-fold increase in just 12 years. Review articles grew at the same rate (from 47 158 to 79 884, also 1.7-fold), comprising 11–13% of the literature any given year.

Figure 1
figure 1

The growth of the scientific literature, 1994–2006. The numbers of articles and reviews were retrieved from PubMed and NIH expenditures were obtained from http://grants.nih.gov/grants/award/awardtr.htm#c; for search strategies see the Materials and Methods. (a) ▪, Total number of scientific articles; •, total number of review articles (note that the scale for review articles is one-tenth that of scientific articles); (b) ▪, NIH expenditures on research grants (fiscal year), •, NIH-funded research papers (calendar year); (c) ▪, NIH expenditures on research grants (fiscal year); •, NIH-funded review articles (calendar year).

Full size image

How much of this literature has been funded by the NIH, and do NIH-funded articles and reviews reflect the recent budget increases? The NIH budget doubled between 1998 and 2003, from $13.7 billion to $27.1 billion (a twofold increase), and extramural research grant funding increased accordingly, from $9.8 billion in 1998 to $18.5 billion in 2003 (a 1.9-fold increase). Based on the premise that there is a lag time between the performance of funded research and its publication, the number of NIH-funded articles published from 2000 through 2005 was examined. Figure 1b shows that there was a 1.5-fold increase in the number of NIH-funded articles published from 2000 through 2005, with the largest increase between 2002 and 2005. The number of NIH-funded review articles increased as well (1.6-fold; Figure 1c), but stayed at a steady 10% of the funded literature. This increase was not quite as great as that for NIH funding. The NIH-funded review articles represent 8% of the world literature of reviews (Figure 1a vs c), similar to the 10% of the world's literature represented by total NIH-funded articles.

Number of Reviews in the Science Citation Index

Thomson ISI Web of Science Science Citation Index and Journal Citation Reports contain citation and impact factor data that cannot be found in PubMed, although the journal coverage is different. The total number of review articles in the Science Citation Index database grew from 14 815 in 1991 to 45 829 in 2005, about a threefold increase in 15 years. In what types of journals were these articles published? Using what data were available, the number of review journals in the Science Citation Index database increased from 163 journals with 19 189 reviews in 1999, to 198 journals with 25 691 reviews in 2006 (Figure 2a and b). However, review articles in review journals made up 73% of the review literature in 1999, and only 56% of the review literature in 2005 (Figure 2b). Thus, the greatest growth in the review literature, especially since 2002, occurred in ‘non-review’ research journals. In other words, the original research journals filled more of their pages with review articles.

Figure 2
figure 2

Review journals, number of review articles published in review journals vs research journals, 1999–2006. The numbers of articles and reviews were obtained from the ISI Web of KnowledgeSM Journal Citation Reports (JCR) as described in the Materials and Methods. (a) ▪, Total number of journals indexed in JCR; •, review journals indexed in JCR (note that the scale of ‘Journals in JCR’ is approximately 30 times that of ‘Review Journals in JCR’); (b) ▪, Total number of review articles in all journals; •, review articles only in review journals.

Full size image

Expansion of the Pathology Literature

One specialty field, pathology, was examined in detail. From 1991 to 2006, the total number of pathology research articles per year increased 2.3-fold, from 5916 to 13 870 (Figure 3a). Over the same time frame, the number of review articles in pathology grew from 384 to 2152, a 5.6-fold increase (Figure 3a). Where were these articles published? The vast majority of pathology research articles (about 90%; Figure 3b) and reviews (also about 90%, Figure 3c) were not published in pathology journals (note the 1/10 scale for the right hand y-axes). The percentage of ‘pathology’ articles published in pathology journals dropped slowly over time, from 10.4% in 1991 to 8.5% in 2006 (data not shown).

Figure 3
figure 3

Sources of the pathology literature, 1991–2006. The numbers or pathology articles and reviews were determined by the search strategy described in the Materials and Methods. (a) ▪, Pathology articles in all journal types, •; pathology reviews in all journal types; (b) ▪, pathology articles in all journal types; □, pathology articles in pathology journals; (c) •, pathology reviews in all journal types; ○, pathology reviews in pathology journals.

Full size image

To determine whether the trends seen for the pathology literature are similar in other specialties, the same analysis was performed for the cardiology and physiology literature; two other fields of comparable size. Figure 4a compares the research articles in these three specialties published in any type of journal. There are modestly more papers published in the field of physiology than pathology, but the upward trends are similar over time. The cardiology literature has grown more than the other two fields, especially since 1998. An analysis of the number of review articles in each of the specialties, published in all types of journals, is shown in Figure 4b. Although the numbers vary, all three fields showed even greater increases in the numbers of review papers published, the most dramatic changes occurring from 2003 forward. The 2006 ratio of reviews to original research articles for pathology was approximately one review published for every six original articles! For physiology and cardiology, these numbers are 1-in-6 and 1-in-7, respectively.

Figure 4
figure 4

Distribution of pathology, physiology and cardiology research articles and reviews by journal type, 1991–2006. ▪, Pathology; •, physiology; and ▴, cardiology articles or reviews, as indicated. The search strategy is described in the Materials and Methods. (a) Total number of articles in each of the three fields, in all journal types; (b) total number of reviews in each of the three fields, in all journal types; (c) total number of articles in each of the three fields, in specialty journals and (d) total number of reviews in each of the three fields, in specialty journals. (Note that the scale for number of review articles is one-tenth that of total articles.)

Full size image

Figure 4c shows the numbers of total pathology, cardiology and physiology articles in specialty journals dedicated to pathology, cardiology and physiology, respectively. The number of articles published in their respective specialty journals nearly doubled for pathology and physiology, but for cardiology the value almost tripled. Unlike pathology and physiology (both under 10%), in 2006 26% of all cardiology articles were published in cardiology journals (compare Figure 4a–c). Figure 4d shows the increase in reviews in these specialty journals from 1991 to 2006. During that time frame, physiology reviews in physiology journals and cardiology reviews in cardiology journals expanded sixfold, while pathology reviews in pathology journals increased almost fivefold.

Effect of Reviews on Journal Impact Factors

Figure 5a shows the 2006 and Figure 5b the 2005 impact factors for each of 12 top pathology journals, with and without review articles included in the calculation, along with the impact factors of the reviews alone. Although the calculated 2006 impact factors of review articles alone were relatively high for three journals (Am J Pathol (The American Journal of Pathology), 12.917; J Pathol (Journal of Pathology), 10.969; Lab Invest, 15.429), they had only modest influence on the reported impact factors for two of these journals (Am J Pathol: 5.917 with reviews, 5.802 without reviews; Lab Invest: 4.453 with reviews, 4.179 without reviews). Only one of these journals garnered an impact factor benefit more than 0.5 units (J Pathol: 5.759 with reviews, 5.246 without reviews).

Figure 5
figure 5

Impact factors of top pathology journals, 2005 and 2006. The impact factors ((a), 2006 and (b), 2005) were obtained from ISI Web of Knowledge Journal Citation Reports; and the impact factors without review articles and for review articles alone were calculated as described in the Materials and Methods. For each year, the data are reported in the order of the official impact factor. Solid black bars, impact factors as reported; open bars, impact factors with review articles excluded and gray bars, impact factors of reviews only.

Full size image

Recognizing that an article entitled ‘…case report and review of the literature’ would have been included in this analysis, it is worth noting that the journals which publish case reports (Am J Surg Pathol, Mod Pathol, Human Pathol) showed negligible effect of ‘review’ articles on their impact factors. Overall, of the 538 reviews publishing in pathology journals in 2005 (one of the two years contributing to the 2006 impact factor), only 27 reviews were also indexed by as Thomson ISI Web of Science Science Citation Index as ‘case reports’ (5.0%). Noting the discrepancy between Thomson ISI Web of Science Science Citation Index and PubMed, the former classifies 441 of the total 46 010 reviews published in 2005 as ‘case reports’ (1.0%), and the latter classifies 1147 of the total 81 961 reviews published in 2005 as ‘case reports’ (1.4%). Thus, pathology seems to have proportionally more ‘case reports with review…’. Nevertheless, the contribution of these publications to our analysis (6%) seems to be minimal.

However, it is necessary to consider the quantity of review articles in each journal, not just the apparent quality as measured by number of citations per article. Accordingly, J Pathol, with 32 reviews published in 2004 and 2005 (the years used to calculate the 2006 impact factor), experienced the biggest jump in impact factor of any of the journals examined. In contrast, during that 2-year period, Am J Pathol published 12 reviews and Lab Invest seven reviews, so it is understandable that the impact of review articles would be less. In 2005, J Pathol surpassed Am J Pathol as the ‘number one journal in Pathology’, when judged by reported 2005 impact factor alone. However, if only the original articles in both journals are counted, Am J Pathol was on top for that year (Am J Pathol: 2005 impact factor 5.796 with reviews, 5.751 without reviews; J Pathol: impact factor 6.213 with reviews, 5.473 without reviews).

The 2005 and 2006 impact factors of these same 12 pathology journals with and without journal self-citations were also calculated, since these values have a dramatic effect on impact factor in some instances (data not shown). Two journals had self-citation rates exceeding 20%—Int J Immunopath Ph and Cell Oncol (78% in 2005 and 50% in 2006; 64% in 2005 and 24% in 2006, respectively). The remaining 10 journals' self-citation rate ranged from 1 to 10% in 2006 and 2 to 11% in 2005, deemed well with in the industry norm.

Figure 6 tracks the impact factors of Am J Pathol, J Pathol and Lab Invest from 1998 through 2006, with and without review articles. While the review articles have had only a small effect on the impact factors of Am J Pathol and Lab Invest, the effect on J Pathol is more pronounced. Starting in about 2000, the impact factor of the journal increased by more than 0.5 each year due to the review articles, but the lines are close to parallel and the trajectory trends upward, regardless of whether the reviews are counted. For the entire 2000–2006 interval, the review article-related increase in impact factor for J Pathol was 0.610±0.153 units (±s.d.), which was significantly greater than those for Am J Pathol (0.109±0.086, P<0.00001) and Lab Invest (0.147±0.088, P<0.00002). Other than the one instance in 2005, discussed above, the relative ranks of the three journals did not change whether or not reviews were considered. It is worth noting that for this time period, the average number of review articles per year for Am J Pathol was 5; J Pathol, 17 and Lab Invest, 5. The average total number of research articles per journal per year was as follows: Am J Pathol, 405; J Pathol, 184 and Lab Invest, 162. Interestingly enough, this translates into ratios of reviews:original articles of 1:81, 1:11 and 1:32, respectively.

Figure 6
figure 6

Impact factors, 1998–2006, with and without reviews, for Am J Pathol, J Pathol and Lab Invest. Impact factors were obtained from ISI Web of Knowledge Journal Citation Reports, and impact factors without review articles were calculated as described in the Materials and Methods. ▴: Am J Pathol with reviews, Δ: Am J Pathol without reviews; •: J Pathol with reviews, ○: J Pathol without reviews; ▪: Lab Invest with reviews, □: Lab Invest without reviews.

Full size image

Citation Rate of all 2005 Pathology Journal Review Articles

A search of the Science Citation Index shows that there were 538 review articles published in pathology journals in 2005. The total number of citations for each article was determined from the date of publication to the publication deadline for this article (17 September 2007). Figure 7 shows a plot of the number of citations per review article, simply ranked as a descending order. It is apparent that very few of the articles are highly cited. To date, 76 of the papers (14%) have not been cited and 201 (37%), have been cited twice or less. Only 115 of the reviews (21%) have been cited more than ten times.

Figure 7
figure 7

Total citations of review articles published in pathology journals, 2005. The total number of citations for journals categorized as ‘pathology’, from publication through 8 August 2007, was obtained as described in Materials and Methods.

Full size image

Table 1 shows the top 20 reviews ranked by total citation number. Note that there are a great variety of subject areas and organ systems covered. Interestingly, whereas 43% of the pathology reviews have US authors, only 30% (a total of six) of the 20 most-cited have corresponding authors with a US address. Of these six US authors, four are NIH-funded (20%). Six of the reviews, including the paper ranked 1, were from J Pathol, which published 14 review articles in 2005. Two of the reviews, ranked 2 and 3, were from in Lab Invest. Significantly, these two papers were the only reviews that the latter journal published in 2005. The review ranked fourteenth was from Am J Pathol, which published six reviews that year. The highest-ranked ‘case report with review of the literature’ was 186/538 with six citations, and it went down from there (data not shown). Of the 27 ‘case report with review of the literature’, 10 had zero citations.

Table 1 The 20 most-cited review articles in pathology journals, 2005
Full size table

DISCUSSION

The purpose of this study was to determine the growth of the review literature relative to the marked growth of the overall biomedical literature. As scientific knowledge expands, there are many challenges in our knowledge base that have been well described: the changing roles of journals and databases in the age of genomics;12 derivation of technologies for ‘mining the bibliome’13 and the controversies surrounding the current methods for assessing article and journal quality, such as impact factor.14, 15 To our knowledge, the problems that potentially arise from the sheer growth of the review literature have not been addressed.

These results demonstrate the growth of the review literature and its impact (or not) on the journals publishing these reviews. Using the pathology literature, focus was placed on the quantity of review articles in general, and the impact of review articles in particular. We found that the quantity of review literature has expanded at a faster rate than research articles, and that review journals are not responsible for the majority of the increase. Rather, the growth in review articles has occurred primarily in journals that otherwise publish original research articles. Furthermore, the recent increases in reviews in at least three scientific fields (pathology, physiology, cardiology) are not due to publications in specialty journals in those fields; rather, they are published elsewhere. However, as is also true for original research articles, only a small number of review articles in pathology are highly cited. It seems to be widely believed that publishing review articles is an easy way for a journal to increase visibility and citations to the journal. However, this study shows that unless a journal consistently publishes a relatively large proportion of high-quality review articles, the impact factor will not substantially increase. Put differently, these data document that publication of high-quality original research articles remains the primary mechanism for elevation of journal impact factor.

Furthermore, we believe that the editors who recruited the most highly cited pathology reviews would say that it is not simple and requires a lot of time, thought and effort. It would take far less effort to enhance a journal's impact factor through self-citations,6, 8, 9 although the deliberate attempt to do so is unethical and can result in the removal of the journal from the Journal Citation Reports database. The Web of Science web site states that a journal self-citation rate of 20% or less is the norm; 80% of journals analyzed by ISI in the Journal Citation Reports Science Edition fall into this category. Only two of the top pathology journals in 2005 and 2006 have impact factors that benefited from unusually high self-citations.

The lurking question throughout this exercise is: are there now too many review articles, so that it is impossible to sort through them? More importantly, are the review articles providing the services that they are supposed to provide? The citation rates for reviews in pathology journals, at least, suggest that only a few of the reviews are earning their keep. One could argue that a reader does not have to read all the review articles, only the ‘good ones’. Considering that a reader would not necessarily know which were the ‘good ones’ from a literature search (other than the obvious ‘wow’ factor of a major senior author), we venture that the discerning reader would rely on journal editorial teams to deliver a high quality product. Hence, the reader would preferentially seek out reviews in reputable journals, which may explain the extraordinary success of selected ‘review’ journals. It is for the rank-and-file journal editors to establish that they, too, can publish high-quality review articles.

What then can be done to improve the state of the review literature? It is not our goal to provide a detailed list of criteria that can be used by authors and referees of review articles; this kind of information can be found in the instructions to authors and referees for many journals. Such a list, and a pointed discussion of the topic, can be found in reference 16. In our opinion, these editors well defined the most important criteria for a quality review, including objective assessment of the cited articles, discussion of the limitations of the included data, critical analysis of variation in the findings and exclusion of unpublished observations.

Furthermore, we believe that before commissioning, writing, or refereeing a review, editors, authors, and referees must first ask a simple question (verbatim from 8): does the world's scientific literature really need another review article on this topic? Additionally, an editorial search of the present literature must be performed in order to support the decision to recruit a review.

We also feel strongly that authors have to read the papers that they cite. ‘The busy researcher may be tempted to simply read the abstract rather than find the original paper…(and) carefully read its whole text in order to assess its value. This tendency…can only devalue and slow down the advancement of science, and it also slows the death of poor papers’.9 Nowhere is this more true than for review articles. A recent letter to Science describes a disturbing propagation of error in review articles.17 Several published reviews discuss as fact a ‘scientific breakthrough’ that cannot be found upon examination of the references cited, nor does it exist anywhere in the scientific literature. The fallacy is repeated verbatim, or with small variations, in these reviews.18 Surely this is the worst outcome of repetition without verification, but how useful is any review that pilfers material from previous reviews, even if the information is correct? How can an author offer insights on the significance of a paper that has not been read critically?

What then are the characteristics of review articles with lasting influence? A review article should not provide just a re-hash of the literature. Rather, and to quote liberally from Meyers and Sinding ‘…the writer of a review shapes the literature of a field into a story in order to enlist the support of readers to continue that story. At any moment in the development of a field, the past has a canonical shape, recorded in the historical introductions of textbooks, in citations of ‘classic’ articles, in eponymous terms. But the present is still a scattering of articles reporting various results with various methods aimed at various immediate problems….The review selects from these (research) papers, juxtaposes them, and puts them in a narrative that holds them together, a narrative with actors and events but still without an ending. It draws the reader into the writer's point of view of what has happened, and by ordering the recent past, suggests what can be done next’.4 Furthermore, ‘speculation is not some imaginative flight tolerated in the last sentences of a research report; it is a crucial part of many reviews, for it focuses on the ending, the future work that will give the story its shape’.4 Clearly the best reviews are not only concerned with what was done in the past, but also present a means to sculpt the future. ‘…Reviews in modern biology do not merely provide information on what is known and accepted at a particular time; they can also allow the construction or reconstruction of knowledge claims’.1

In ‘The Paradox of Choice: When More Is Less’, the author describes ‘our experience of choice as a burden rather than a privilege’.19 Although this book concerns modern consumerism, the concept is well applied to the surfeit of scientific review literature. However, we as scientists are not only the readers who must slog through this ‘weight of medical knowledge’—we are the authors, the referees and the editors of the material. It is therefore up to us to optimize the quantity and quality of our published works. It is our hope that this critical analysis of reviews in the biomedical literature will help empower journal editors, referees and authors of such reviews to provide a high quality published product to an appreciative reading public.