Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

How to make replication the norm

Access options

Rent or buy this article

Get just this article for as long as you need it


Prices may be subject to local taxes which are calculated during checkout

Nature 554, 417-419 (2018)


This work was funded by the Berkeley Initiative for Transparency in the Social Sciences and by the Laura and John Arnold Foundation.

Updates & Corrections

  • Correction 22 March 2018: An earlier version of this Comment stated that only authors of non-replicated articles from the 3ie project described antagonism. It also gave the wrong number of replications. The data set is now available as supplementary information.


  1. Nosek, B. A. et al. Science 348, 1422–1425 (2015).

    Article  PubMed  CAS  Google Scholar 

  2. Christensen, G. & Miguel, E. Preprint at (2017).

  3. Berry, J., Coffman, L. C., Hanley, D., Gihleb, R. & Wilson, A. J. Am. Econ. Rev. 107, 27–31 (2017).

    Article  Google Scholar 

  4. International Initiative for Impact Evaluation. 3ie Replication Programme (3ie, 2017); available at

    Google Scholar 

  5. Davey, C., Aiken, A. M., Hayes, R. J. & Hargreaves, J. R. Int. J. Epidemiol. 44, 1581–1592 (2015).

    Article  PubMed  Google Scholar 

  6. Miguel, E. & Kremer, M. Econometrica 72, 159–217 (2004).

    Article  Google Scholar 

  7. McCullough, B. D, McGeary, K. A. & Harrison, T. D. J. Money Credit Bank. 38, 1093–1107 (2006).

    Article  Google Scholar 

  8. Chang, A. C. & Li, P. Finance Econ. Disc. Ser. (2015).

    Article  Google Scholar 

  9. Gherghina, S. & Katsanidou, A. Eur. Polit. Sci. 12, 333–349 (2013)

    Article  Google Scholar 

  10. Key, E. M. Polit. Sci. Politics 49, 268–272 (2016).

    Article  Google Scholar 

Download references

Supplementary Information

  1. Gertler et al. Comment supplementary information
  2. Summary_3ie full data set


Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing


Quick links