Introduction

Delay and failure to publish research material contributes to publication bias. Publication bias has been defined as the tendency on the part of investigators to submit, or reviewers and editors, to accept manuscripts based on the direction or strength of the study findings.1 Although this definition implies that studies with the strongest and most positive results are most likely to be published, a broader definition of publication bias is that it is any influence that reduces the amount of good science appearing in the literature.2

The importance of publication bias is that by failing to commit a study to an accessible public forum, only the published results are likely to be reviewed in the evidence base of the average clinician or researcher. As the most positive findings have a greater likelihood of publication this can skew the evidence that is available to the readers. This effect can be also be compounded by delaying publication such that results may become superseded by newer evidence or scientific techniques.

An accepted method of investigating publication bias is to review the publication rate of research originally presented at speciality conferences and meetings. Assessments of the publication rate for abstracts presented as summary reports at conferences are consistent at around 20–55%, with publication usually within a three to four year time period from presentation as an abstract with a median time to publication of 17 months.3 However, the majority of these studies have been conducted within medical specialities and to date there has been little research into publication bias within the dental sciences.4

The aims of this study were to assess the publication rate of abstracts presented at three prestigious dental conferences and to record the time taken to achieve full publication. In addition an assessment was made of the impact of the mode of presentation of the abstract (oral or poster) on the publication rate.

Materials and methods

The 1993 conference proceedings from International Association for Dental Research (IADR), European Organisation for Caries Research (ORCA) and European Orthodontic Society (EOS) were identified. They were published as supplements to the Journal of Dental Research (IADR) and Caries Research (ORCA) or within the substance of a regular issue of the European Journal of Orthodontics (EOS), respectively.

All abstracts presented at the EOS and ORCA 1993 conferences were examined and a 10% random sample of the abstracts from IADR was assessed. The randomisation was selected from a random number table and prepared blind to the principle investigator.

Assessment of abstracts

All abstracts were assessed to ascertain the following details:

Was the abstract published as a full paper?

An electronic database search system, “Medline” was used to identify full reports of the studies reported in each conference abstract under investigation. Searches were made using a cross-referenced search of the abstract title and names of all contributing authors. The conference abstracts were classified as “published” if Medline identified a full report related to the conference abstract. All searches were performed using the SilverPlatter version of Medline (Ovid New York, NY, USA) throughout the study. The search was censored from one year prior to and five years after presentation at the conference.

Time taken to publication as a full report

The date of abstract publication was recorded as the month and year that the conference took place and recorded on the data sheets as TIME 1.

The date of publication was recorded as the month and year that the full report was published, as identified by Medline, this was noted as TIME 2. The time taken to publish was calculated as the number of months between TIME 1 and TIME 2.

The mode of presentation

The mode of presentation was categorised as oral or poster from details supplied in the conference proceedings or by direct contact with editors of the journal.

Reliability studies

To verify the reliability of the Medline search and to identify fully published reports, a reliability study was performed on a 10% random sample of all the abstracts. The random sample of abstracts was reassessed a minimum of three months after initial assessment with data recorded blind to the results of the original searches. A kappa score was used to assess the reliability of the Medline searches.

Statistical methods

Individual conferences

All data were transferred to Microsoft® Excel(Microsoft, Corporation Redmond, WA, USA) and to Minitab® (Minitab Inc. State College, PA, USA) for analysis. Data cleansing was performed to detect and correct any data transfer inaccuracies. Descriptive statistics were used to assess the data.

Comparisons between conferences

χ2 test was used to compare whether publication rates between conferences were compatible with chance.

Oral vs poster

Relative Risk ratios and 95% confidence intervals were used to calculate the relative differences in publication rates between oral and poster presentations.

Results

Number of abstracts identified

In 1993, 2539 abstracts were published for the conference proceedings of IADR. The random 10% sample of IADR abstracts provided 256 IADR abstracts for the study together with 175 for EOS and 115 for ORCA, making a total of 546.

Reliability studies

Searches

A total of 61 abstracts were reassessed for the reliability study. This represented a random sample of 11% of the total number of abstracts investigated. The kappa score for agreement for the Medline searches was 0.97 showing a very good level of agreement.

Number and proportion of abstracts that proceed to full publication

Over the censored 6-year period (1992–1998), 252 abstracts were found as fully published reports giving an overall publication rate of 46.1% (252/546). The differences in publication rates between the conferences were not statistically significant (χ2 test=1.097 df=2 NS P>0.2) (see Table 1).

Table 1 The number and time taken for full publication of research from conference abstracts in 1993
Full size table

Time to publication

The median time to publication was 18 months (IQR 9, 30 months). The results for the individual conferences are given in Table 1. The overlap of the inter-quartile ranges suggests that there was no statistically significant difference between the median time to publication of abstracts presented at each conference. Publication rate peaked at 12–18 months post conference presentation, with a gradual decline up to the time of search censorship (Figure 1).

Figure 1
figure 1

The time taken to achieve full publication of abstracts presented at EOS, IADR and ORCA conferences in 1993.

Full size image

Mode of presentation

The relative risk ratios for publication of oral vs poster presentations for the pooled results for all three conferences favoured oral presentations.

Relative Risks Oral vs Poster=1.37; 95% CI 1.19, 1.15

Relative Risks Poster vs Oral=0.73; 95% CI 0.55, 0.91

DISCUSSION

Selection of abstracts

The three conferences investigated in this study were chosen on the basis that they are established annual meetings with large numbers of published abstracts. They included a broad range of dental research covering all dental specialities and therefore offered a representative sample of the dental research.

Use of Medline

There are a number of different computer databases that can be used for rapid searching of the international scientific literature. Medline searching favours those papers written in English and published predominantly in US and European journals, this is also the database included in the free search engine PubMed from the National Library of Medicine and as such represents a “real life” search for the practising dentist in the UK. However, it is important to recognise that more papers potentially could have been found if we had included searches in other databases such as Embase. Embase favours European journals and although it contains a similar number of citations as Medline there is only a 56% crossover of journals. This means that by searching both databases there is a potential for increasing search yield.

The success of any database search can vary between users, search strategies and the type of software applied. For these reasons they have been criticised for their failure to locate all of the available and relevant references,5 but used correctly Medline has also proven one of the most sensitive electronic databases when compared to alternatives such as Embase.6 However, it is inevitable that a small number of the studies, originally presented at these three dental conferences were missed, for example, if they had been published in local journals, letters, foreign language journals or audit publications that are not listed by Medline.

Publication rate of abstracts

In total 252 (46.1%) of the abstracts investigated were found as fully published reports. ORCA showed the highest publication rate of 50%, closely followed by IADR, 45.3% and EOS 44.6% but these differences were not statistically significant. These publication rates compare very favourably with investigations of conference abstracts in other health care disciplines, which range from 10 to 78%.3 It is also a substantial improvement from the investigation by Corry in 1990 of a 10% random sample of IADR conference abstracts presented in 1983 and 1984 when the publication rates were only 21.6 and 24.2%, respectively.7

Previously reported factors associated with failure to publish include, poor quality of research design, small sample size and negative findings. Surprisingly though it is more likely that studies remains unpublished because of failure of researchers to submit work rather than because of its rejection from a journal. This failure to submit manuscripts has been reported to be due to a number of factors including unimportant results, incomplete analysis, and investigators being too busy or having lost interest.1, 8, 9 The specific causes of failure to publish were beyond the remit of this study and would require direct contact with the abstract authors.

Time to publication

Although this study is based on research presented 10 years ago sufficient time was required to elapse following the conference for reports to be written up and submitted. It offers baseline data with which prospective studies can compare future publication rates. However, this study is one of the first of its kind to also censor the search for full publication at one year preceding the conference. This study did demonstrate that a small proportion of reports (7.5%) were published prior to presentation at a conference so a pre-conference censor time should also be included in future studies.

Peak incidence of publication for all three conferences occurred between 6 and 12 months with a median time to publication of 18 months. This timing of 12–18 months would seem appropriate to allow a sufficient period to elapse for writing up the research, submitting the paper, undergoing peer review and corrections before then joining the waiting list for publication.

This time lag has also been reported in other health care specialities but varies substantially for example from 26% of Cystic Fibrosis abstracts published in 2 years10 to 90% within 2 years for Neuroradiology.11

In this study the percentage of papers published up to 24 months post conference was 65%, rising to 85% within 3 years. But what should be considered as being published too late? From these results it would seem that an acceptable limit of 30 months should be aimed for but any literature published more than five years after presentation at a conference could be considered to be “out of date” and of less significant scientific value. However, not all areas of research will experience the same susceptibility to time lag bias while some “late publications” have the potential to be obsolete because of being superseded by more recent reports, nevertheless they may contain the one key result which has the potential to change the current understanding or consensus on a subject. Assessment of the usefulness of “late research” in this way must therefore be left up to the editors and referees of journals.

Effect of mode of presentation on publication

In total, 99 of the oral presentations (57%) and 153 of the poster presentations (41%) were fully published. This trend for a greater proportion of oral abstracts to be published is visible across all three conferences and is significant on the pooled data for all three conferences. The Relative Risk value of 1.37 confirms that oral presentations are 1.37 times more likely to be published than posters in this study. These results are similar to the work of Evers12 who also found that oral presentations are more frequently published than posters in field of human reproduction. This could be the result of prioritisation by conference organisers of those studies with the most powerful and positive results to be presented orally as these potentially mean a more interesting programme for delegates. This may in turn mean that this group of presentations are likely to be more acceptable for publication13 and conversely rejected researchers may well lose interest in publication or feel more pessimistic about the chances of publication.14

Reducing failure to publish

Researchers need greater encouragement to, at least, attempt to publish their work. This encouragement could come directly from senior research colleagues and the professions and institutions, where this work was carried out. Additional pressure can be brought to bear by ethics committees where studies, particularly those involving animals and humans, could be approved on the basis of the protocol, including an attempt to publish.15, 16 A commitment to attempt publication could also be instigated as part of the acceptance of research theses submitted for postgraduate qualifications.

Providing additional journal space for this new influx of research material could prove an important stumbling block as journal space is finite. However, the development of online journals and better indexing may improve accessibility.

Promotion of the use structured abstracts

With a significant amount of research presented at conferences remaining unpublished it is crucial that the information published in the conference proceedings is more comprehensive as this may be the only public place of publication. The use of structured abstracts would provide a uniform construction of research information at this level, which would facilitate a more accurate appraisal of the strength and significance of the results.17, 18

Conclusions

  • More than half of the research presented at EOS, IADR and ORCA in 1993 remained unpublished 5 years after presentation at the conferences. This is similar to the publication rates from other health care specialities.

  • Abstracts presented orally were published more frequently than posters.

  • When assessing the evidence base it is important to recognise that current literature may be missing important unpublished data.

  • Researchers should be encouraged to publish their research findings, at the earliest opportunity, regardless of the strength or direction of the findings.