Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017

To gain insight into changes of scholarly journals’ recommendations, we conducted a systematic review of studies that analysed journals’ Instructions to Authors (ItAs). We summarised results of 153 studies, and meta-analysed how often ItAs addressed: 1) authorship, 2) conflicts of interest, 3) data sharing, 4) ethics approval, 5) funding disclosure, and 6) International Committee of Medical Journal Editors’ Uniform Requirements for Manuscripts. For each topic we found large between-study heterogeneity. Here, we show six factors that explained most of that heterogeneity: 1) time (addressing of topics generally increased over time), 2) country (large differences found between countries), 3) database indexation (large differences found between databases), 4) impact factor (topics were more often addressed in highest than in lowest impact factor journals), 5) discipline (topics were more often addressed in Health Sciences than in other disciplines), and 6) sub-discipline (topics were more often addressed in general than in sub-disciplinary journals).

To determine the reporting or citation of topics in manuscripts and addressing of topics in ItAs 51 (33) To recommend standards for topics 11 (7) To determine factors associated with differences in ItAs 9 (6) To raise awareness of a topic(s) and its mention in ItAs 8 (5) To determine changes in ItAs over time 7 (4) To determine addressing of a topic(s) in ItAs and policy documents 5 (3) To improve the quality of journals within a country 4 (3) To determine if ItAs can be a proxy for journal quality and library subscription decisions 2 (1) To determine if there is a standard way of addressing a topic 1 (0) To determine how authors deal with differences in ItAs between journals 1 (0) To provide guidance on writing and updating ItAs 1 (0)

Section 2. Series of Meta Analyses
For each topic we meta-analysed, we first present overall findings and possible differences between different disciplines, and then effects found only in studies that analysed Health Sciences journals, as most studies of Instruction to Authors (ItAs) were conducted on Health Sciences journals. In order to preserve the same reference order as in the paper, references 1-20 represent those used in the introduction and methods, while references  refer to the 153 studies that were included in our systematic review and meta-analyses. As time trends were estimated using regression models, related reported percentages may somewhat differ from the percentages reported in individual studies. All data is available at: 10.17632/53cskwwpdn.4.

Results Per Topic
Authorship

Association between citation metrics and addressing of authorship in Health Sciences journals
Two studies reported APs for journals grouped according to their impact factor (IF).
One analysed journals from plastic surgery, oral-craniomaxillofacial/facial plastic surgery, and otorhinolaryngology/head and neck surgery in Journal Citation Reports (JCR). 81

Conflicts of interest
Thirty-six publications analysed if conflicts of interest (COI) were addressed in ItAs. 22 These studies reported 112 COI percentages (CPs, defined as a number of journals whose ItAs addressed conflicts of interest divided by a total number of journals analysed in a study). Only three studies, however, reported CPs across disciplines. The first one, in 1996, analysed top 1396 journals ranked by IF or the number of times journals were cited, and reported a CP of 16% (95% CI 14-18%). 34 The second, in 1998, analysed 41 journals (mostly Multidisciplinary, Health, and Life Sciences journals which had previously published retractions or corrections due to misconduct cases), and reported a CP of 41% (95% CI 28-57%). 40 Finally, the third study, analysed top five Health and Life Sciences journals in 2010, and reported a CP of 100% (95% CI 38-100%). 88 We observed a rise in CPs with time, with the increase of 26% (95% CI 12-41%) between 1997 and 1998, and of 59% (95% CI 13-72%) between 1998 and 2010 (Note: this observation may be influenced by the fact that the latter two studies had much smaller sample sizes than the first, 41 and 5, vs 1396 of the first study, and they had a higher prevalence of health journals, which we show below have much higher percentages of CPs).
Two studies by the same author of open access journals in Croatia compared differences between disciplines, but had slight differences in journal selection and text analysis. 143

Conflicts of interest in Health Sciences Journals
Thirty-three publications reported CPs in Health Sciences journals. 22   (Note: as both the sub-discipline and country effects were demonstrated in the paragraphs above, 6 studies that reported country specific sub-discipline percentages were not included in pooled analyses 54,55,109,140,145,151 ). Five studies reported Health sub-disciplinary journals CPs (Supplementary Figure 3). 80,81,91,103,137 Of these, the CPs reported in one study were based on a sample of 3 journals per IF category so we excluded it from the integrated analysis. 91 This study, which analysed journals One additional study (which did not report data in a way that allowed for meta-analysis)

Association of citation metrics and addressing of conflicts of interest in Health Sciences
reported no association between IF of the journal and the addressing of conflicts of interest. 165 Data Sharing Ten publications analysed if data sharing was addressed in ItAs and reported 22 data sharing percentages (DPs, defined as a number of journals whose ItAs addressed data sharing divided by a total number of journals analysed in a study). 24,36,40,65,83,88,117,143,144,164 Five studies which covered a time span of 1992-2010 reported on DPs of predominantly top journals across disciplines. 24,36,40,83,88 While the heterogeneity of these percentages was considerable (I 2 =97%, Q-value=138.8, df=4, p<0.001), time almost entirely explained the observed betweenstudy variability of DPs. Namely, an annual increase in odds of addressing data sharing was 23% (pM-RC <0.001, pseudo-R 2 =100%; Supplementary Figures 4A and 4B) with the DP of 15% in 1992 to 88% in 2010. 88 The time effect, however, was not observed for subdiscipline journals from different disciplines (molecular biology and biochemistry, 65 Figure 4A). Additionally, one of these studies, (which did not report data in a way that would allow for meta-analysis) also Two studies by the same authors compared differences between disciplines, specifically of open access journals in Croatia, analysing almost the same set of journals, but applying a different coding and extraction methods, and journal classification. 143

Association between citation metrics and addressing of data sharing in journals
Only one study investigating journals publishing papers on substance abuse reported DPs for different IF quartiles, 117 with one step increase in quartile category increasing the odds of addressing data sharing by 127%, corresponding to 22% percent increase in DP per quartile (pM-RC=0.035, pseudo-R 2 =100%). The top quartile had significantly higher DP than the lowest one (89% vs 38%, mean difference 51%, 95% CI 6-77%). Two additional studies (which did not report data in a way that would allow for meta-analysis) indicated that a stronger data policy was associated with higher impact factor values. 65,83 Ethics approval Thirty-one publications analysed if requiring ethics or institutional review board approval (IRB) was addressed in ItAs, and reported 123 ethics approval percentages (EPs, defined as a number of journals whose ItAs addressed ethics approval divided by a total number of journals analysed in a study). 26, 27, 30, 40, 45, 58, 60, 67, 69, 72, 74, 75, 77, 86, 89, 91-93, 97, 99, 108, 114, 115, 119, 137, 143, 145, 149, 151, 165, 166 Only three studies reported EPs not exclusively based on Health Sciences journals. Two studies reported country-specific EPs across disciplines: a study of Cameroonian journals in 2009 which showed an EP of 22% (95% CI 6-58%), 72 and a study of Croatian open access journals in 2014, which reported EP 8% (95% CI 5-13%). 143 No significant difference was found between the countries, likely due to high uncertainty of the Cameroon estimate.
Finally, the study of Spanish and Latin-American journals indexed in JCR Social Sciences in 2014 reported a EP of 13% (95% CI of 8-21%). 149 The Croatian study additionally compared non-Health Sciences (EP of 5%, 95% CI 3-10%) with that of Health Sciences (21%, 95% CI 11-37%) and found non-Health EP to be significantly lower.

Ethics approval in Health Sciences Journals
Five studies reported 6 EPs of predominantly Health Sciences journals, 26 Figure 5). We, therefore, performed meta-regression using time and journal categories (AIM or top) as covariates and found that both significantly affected EPs (pM-RC for both ≤0.0002, pseudo-R 2 =100%). Annual increase in odds of addressing EP was 16%; whereas AIMs indexation increased the odds by 164% relative to top journals.
Supplementary Figure 5. Percentage of instructions to authors mentioning ethics approval in Health Sciences journals. Percentages with 95% CIs are shown in blue, with the blue rectangle representing point estimate that was reported in a study and size of the rectangle representing weight used in a meta-analysis model. Summary effects including 95% CIs are shown in orange. For groups of studies where we found significant effect of time, marked with orange brackets, we did not pool the data. References are shown as superscript numbers.
After excluding EPs calculated on sample sizes of ≤3 journals, seven studies remained which reported EPs in Health Sciences subdisciplines between 1995 and 2015. 26,58,74,75,91,93,137 These studies showed moderate heterogeneity (I 2 =47%, Q-value=45.5, df=24, p=0.005), which was partly due to low precision of estimates, as more than half of reported EPs (13 out of 25, 52%) were derived from analyses based on ten or fewer journals. Nevertheless, we found  (Note: as the sub-discipline, time and country effects were demonstrated in the paragraphs above, a study in India that reported on country specific sub-discipline percentages was not included in the pooled analyses. 151 )

Association between citation metrics and addressing of ethics approval in journals
Two studies reported EPs in relation to IF values of journals. 91,114 Of these, the EPs in one study were based on a sample of 3 journals per IF category so we excluded it from summary analysis. 91  Webqualis website by 21% (EPs of 88% vs 67%, 95% CI for difference from -3% to 42%, p=0.089significant at 0.1 level). 114 Four studies (which did not report data in a way that would allow for meta-analysis) reported on associations between IF and ethics approval, one demonstrating that higher impact factors journals were more likely to require ethics approvals, 75 while three others found no statistically significant associations with IF. 99,137,165 Funding disclosure Eleven publications analysed if funding or grants was addressed in ItAs, and reported 35 funding percentages (FPs, defined as a number of journals whose ItAs addressed funding disclosure divided by a total number of journals analysed in a study). 22,23,40,54,91,115,119,138,143,149,150  ). 90 One additional study (which did not report data in a way that would allow for metaanalysis) also reported no association of medical association membership, publication language, and country of publication with Ups in 2010. 101 Fourteen studies reported country or region-specific UPs. 26 One study (which did not report data in a way that would allow for meta-analysis) reported no significant association between IF and UP. 165