Introduction

Systematic reviews (SRs) involve the analysis of all available evidence to answer a research question in a precise and unbiased way.1 They differ from narrative reviews by adhering to a strict predefined methodology governing factors such as inclusion/exclusion criteria and search strategies, allowing for the reproducibility of findings. Meta-analyses (MAs) quantitatively integrate the results of studies included in SRs using specific statistical techniques (http://www.cochrane.org/glossary). Some researchers consider MAs to be the source of the highest grade of evidence available regarding the efficacy of an intervention.2

Several sources of bias in studies included in SRs3 may, however, reduce our confidence in the final meta-analytic estimates, even when those reviews are correctly planned and conducted according to known guidelines, such as preferred reporting items for systematic reviews and meta-analysis (PRISMA) and meta-analysis of observational studies in epidemiology (MOOSE).4,5 Threats to the validity of an SR, such as publication bias and variability in the quality of studies pooled,2 should be taken into account when the results are critically assessed. Potential sponsorship bias should also be considered. Evidence suggests that the outcomes of industry-funded trials may be more inflated and better than those of trials supported by other sources.6 Readers of SRs should be aware that a body of evidence containing a large number of industry-funded primary trials may provide biased meta-analytic results. Authors of SRs must therefore report information on the funding of trials included in these studies.

The objective of this paper was to assess whether authors of SRs published in periodontology and implant dentistry adequately report the funding sources of primary studies included in those reviews.

Materials and methods

Literature search

The full literature search strategy for this manuscript has been described elsewhere.7 Briefly, the PubMed and Cochrane Database of Systematic Reviews databases were searched to identify abstracts of SRs on the efficacy of interventions published in the fields of periodontology and implant dentistry, with the last search conducted on 11 November 2012. Eligible SRs contained MAs. The previous study7 focused on the quality of reporting in abstracts of SRs. To increase the sensitivity of the literature search, the records of ten high-impact journals in periodontology and implant dentistry were also searched manually to identify additional SRs published between November 1990 and November 2012. The reference lists of included papers were also scrutinised to identify potentially eligible SRs. The literature search was performed independently and in duplicate by two authors. Any disagreement regarding the selection and inclusion of papers was resolved by discussion and consensus between at least two reviewers. Any persisting disagreement was resolved by a third reviewer (MA).

Criteria for assessment

Two authors (CMF and DGZ) retrieved the following information from each SR independently and in duplicate: (a) reporting of funding sources of primary studies included, (b) discussion of potential sponsorship bias, (c) statistical or graphical (that is, funnel plot) assessment of publication bias, and (d) reporting of the SR funding source. These data were entered directly into a table in dichotomous (yes/no) format to indicate whether each criterion was met. Any discrepancy in extracted information was resolved by discussion until consensus was achieved. The third reviewer (MA) made final decisions in cases of non-consensus between the first and last authors.

Results

One hundred and forty-six SRs were included in the assessment. The authors of 45 (31%) SRs reported the funding sources of primary studies included. The reporting of sponsorship in SRs seems to have improved in recent years, but the number of publications has also increased (Table 1).

Table 1 Numbers of systematic reviews published in periodontology and implant dentistry reporting and discussing primary studies' sources of financial support
Full size table

The authors of 14 (10%) SRs discussed the possible influence of sponsorship on the meta-analytic findings (Table 2). This potential influence was assessed objectively (for example, sensitivity analysis) in four SRs, whereas the remaining SRs reported the presence of potential sponsorship bias without carrying out any analysis. Three SRs found no difference among sources of funding,8,9,10 whereas one SR11 observed that industry-funded trials produced more favourable results.

Table 2 Methods of assessing primary studies' funding sources in systematic reviews
Full size table

Thirty (21%) SRs attempted to investigate publication bias, 98 (67%) SRs contained no description of such assessment, and 18 (12%) SRs reported that the investigation of publication bias was not possible due to the small number of primary studies included.

Industry funding was reported in 2% of SRs. Forty (27%) SRs reported self-funding or no receipt of external funding, and were thus considered to have received non-industry support. However, most (n = 103, 71%) SRs provided no information about funding.

Discussion

The omission of information about funding sources in the majority of SRs assessed in this study is alarming, as readers require comprehensive information of this type to adequately understand the potential influence of sponsorship bias on the results of these reviews. Accumulating evidence overwhelmingly indicates that industry-funded trials generate more favourable results than studies with other sources of support.6,12,13,14,15,16,17,18 For example, annual dental implant failure rates reported in industry-associated trials are significantly lower than those reported in trials without industry support.19 Dental practitioners' underestimation of the impact of sponsorship bias may influence clinical decision making. In addition, industry sponsorship may influence the ways in which clinical trials are planned, conducted, and reported.6,20 For example, industry-funded trials may include unbalanced comparators (that is, test and control groups), which promote positively skewed assessment of the product or technique of interest.21 Moreover, such trials frequently use placebo-control groups, rather than active or positive controls,22,23 which prevents determination of whether a new therapy is more effective (or does less harm) than proven standard therapies — so-called gold standards.24 Such methodological issues concerning the comparison of treatment groups may raise ethical concerns about the conduction of the clinical trial.25

Few SRs included in this analysis assessed the potential influence of sponsorship bias in primary studies on meta-analytic estimates. This omission may be the result of the lack of detailed information in primary studies about financial support and conflicts of interest (COIs), such as authors' employment by supporting companies, stock ownership, and receipt of grants and research materials. Furthermore, SRs in the present sample assessed a wide variety of products and/or techniques. In addition, only some of the appraised outcomes in each MA were relevant to the conclusions. Thus, it can be unreliable to compare the potential influence of sponsorship bias on meta-analytic estimates in SRs that report primary funding source to those that fail to report it.

Selective outcome reporting (partial publication of findings, depending on COIs) is a concern, especially in reports of industry-supported randomised controlled trials (RCTs).26 Inadequate reporting of outcomes may interfere with the critical interpretation of evidence from these trials, and the suppression of non-significant findings could lead to the implementation of harmful therapies.27 Moreover, industry support seems to contribute to publication bias characterised by the greater likelihood of reporting on trials with significant (versus nonsignificant) results or delayed publication of trial findings because of potential COIs.28 Similarly, the strengthening of publication policies in scientific journals may result in bias by creating potential barriers to publication.29 Only 21% of SRs in the present sample used statistical or graphical methods to investigate publication bias.

Despite concern about the potential bias presented by industry sponsorship, one must note that many studies included in SRs may not have been performed without industry funding. Grants from non-profit organisations (for example, governmental organisations, foundations) and universities are usually limited and cannot solely support the large number of studies being conducted on diverse topics. Implant dentistry, in particular, benefits from associations between industry and researchers because the costs involved in purchasing dental implants and surgical kits for large-scale research can be prohibitive. The donation of materials by industry bodies enables researchers to use grants to cover other costs, such as laboratory materials and equipment, consumables, and supporting personnel. However, although cooperation between industry and academia seems to be relevant in the current context, SRs should provide detailed and explicit information about the nature of this cooperation.

Clinicians and patients often depend on the findings of SRs to support decision-making. With the continual increase in the number of SRs published, and given the advantages of such studies over small clinical trials,1 the reporting of any threat of bias in the assessment of treatment effects must be as transparent as possible. Moreover, clinicians may have difficulty understanding and interpreting the findings of SRs,11 which could be alleviated by SR authors' provision of a critical discussion of potential sponsorship bias.

Recommendations

Authors should take the following measures to improve the assessment and reporting of sponsorship bias in SRs:

  • Obtain the original protocols of primary studies (that is, RCTs), which are normally recorded in official registries (for example, clinicaltrials.gov), to assess potential selective outcome reporting

  • Conduct sensitivity analyses when they suspect that funding sources and/or disclosed COIs have influenced meta-analytic estimates8

  • Report in detail the funding sources of primary studies included in SRs30,31 and the authors' potential financial ties to industry. Similarly, they should report their own potential COIs, for example by describing the sources of financial support for SRs and the authors' industry associations.