Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Do trials reduce uncertainty? Assessing impact through cumulative meta-analysis of neonatal RCTs

Journal of Perinatology volume 37, pages 1215–1219 (2017)Cite this article

Subjects

Abstract

Objective:

To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.

Study Design:

We selected meta-analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.

Results:

For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ⩾20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.

Conclusion:

Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1
Figure 2
Figure 3

Similar content being viewed by others

References

  1. McGuire W, Fowlie PW, Soll RF . What has the Cochrane collaboration ever done for newborn infants? Arch Dis Child Fetal Neonatal Ed 2010; 95 (1): F2–F6.

    Article  Google Scholar 

  2. Guyatt G, Rennie D, Meade M, Cook D . Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. McGraw-Hill Education, American Medical Association: New York, NY, 2015.

    Google Scholar 

  3. The Cochrane Library, Issue 7. [Internet]. Wiley: Chichester, 2013.

  4. Review Manager (RevMan) [Computer program]. 5.3 ed. The Nordic Cochrane Centre, The Cochrane Collaboration: Copenhagen, 2014.

  5. Sinclair J, Bracken MB . Effective Care of the Newborn Infant. Oxford University Press: Oxford, 1992.

    Google Scholar 

  6. Sinclair JC, Haughton DE, Bracken MB, Horbar JD, Soll RF . Cochrane neonatal systematic reviews: a survey of the evidence for neonatal therapies. Clin Perinatol 2003; 30 (2): 285–304.

    Article  Google Scholar 

  7. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 2013; 14 (5): 365–376.

    Article  CAS  Google Scholar 

  8. DeMauro SB, Giaccone A, Kirpalani H, Schmidt B . Quality of reporting of neonatal and infant trials in high-impact journals. Pediatrics 2011; 128 (3): e639–e644.

    PubMed  Google Scholar 

  9. Sinclair JC . Meta-analysis of randomized controlled trials of antenatal corticosteroid for the prevention of respiratory distress syndrome: discussion. Am J Obstet Gynecol 1995; 173 (1): 335–344.

    Article  CAS  Google Scholar 

  10. Ioannidis JP . Why most published research findings are false. PLoS Med 2005; 2 (8): e124.

    Article  Google Scholar 

  11. Clarke M, Brice A, Chalmers I . Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS ONE 2014; 9 (7): e102670.

    Article  Google Scholar 

  12. Eppstein MJ, Horbar JD, Buzas JS, Kauffman SA . Searching the clinical fitness landscape. PLoS ONE 2012; 7 (11): e44901.

    Article  Google Scholar 

  13. Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS . Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013; 369 (20): 1926–1934.

    Article  CAS  Google Scholar 

  14. Dechartres A, Altman DG, Trinquart L, Boutron I, Ravaud P . Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 2014; 312 (6): 623–630.

    Article  CAS  Google Scholar 

  15. Boadi K . Erosion of funding for the National Institutes of Health threatens U.S. leadership in biomedical research. 2014 Available from https://www.americanprogress.org/issues/economy/reports/2014/03/25/86369/erosion-of-funding-for-the-national-institutes-of-health-threatens-u-s-leadership-in-biomedical-research/.

  16. Wetterslev J, Thorlund K, Brok J, Gluud C . Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol 2008; 61 (1): 64–75.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, MA, USA

    S C Hay, W-Y Mao & J A F Zupancic

  2. Division of Newborn Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, MA, USA

    S C Hay & J A F Zupancic

  3. Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA, USA

    H Kirpalani & S B DeMauro

  4. Department of Obstetrics and Gynecology, New York University, New York, NY, USA

    C Viner

  5. Division of Neonatal Perinatal Medicine, University of Vermont College of Medicine, Burlington, VT, USA

    R Soll

  6. Division of Neonatology, Oregon Health and Science University, Portland, OR, USA

    D Dukhovny

  7. Division of Neonatology, Stanford University, Stanford, CA, USA

    J Profit

Authors
  1. S C Hay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H Kirpalani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C Viner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R Soll
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. D Dukhovny
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. W-Y Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. J Profit
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. S B DeMauro
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. J A F Zupancic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J A F Zupancic.

Additional information

Author contributions

SCH: contributed to the study design, acquisition of the data, analysis and interpretation of data, drafting of the manuscript and revision based on the comments of the co-authors; HK: guided the study conception and design, and critical revision of the manuscript; CV: contributed to the acquisition of the data; RS: contributed to critical revision of the manuscript; DD: contributed to the study conception and design, analysis and interpretation of the data and critical revision of the manuscript; W-YM: contributed to the analysis and interpretation of data; JP and SBD: contributed to critical revision of the manuscript; JAFZ: guided the methodology, study conception and design, data checking and interpretation, drafting and critical revision of the manuscript. SCH and JAFZ had full access to all of the data in the study and are guarantors of the work. All authors approved the final manuscript as submitted.

The authors declare no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hay, S., Kirpalani, H., Viner, C. et al. Do trials reduce uncertainty? Assessing impact through cumulative meta-analysis of neonatal RCTs. J Perinatol 37, 1215–1219 (2017). https://doi.org/10.1038/jp.2017.126

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/jp.2017.126

This article is cited by

Search

Advanced search

Quick links