Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

How to make replication the norm

The publishing system builds in resistance to replication. Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed economics journals to find out how to fix it.
  1. Paul Gertler

    1. Paul Gertler is a professor of economics in the Haas School of Business at the University of California, Berkeley, USA.

    View author publications

    You can also search for this author in PubMed  Google Scholar

  2. Sebastian Galiani

    1. Sebastian Galiani is a professor of economics at the University of Maryland, College Park, USA.

    View author publications

    You can also search for this author in PubMed  Google Scholar

  3. Mauricio Romero

    1. Mauricio Romero is a graduate student in economics at the University of California, San Diego, USA.

    View author publications

    You can also search for this author in PubMed  Google Scholar

Replication is essential for building confidence in research studies1, yet it is still the exception rather than the rule2,3. That is not necessarily because funding is unavailable — it is because the current system makes original authors and replicators antagonists. Focusing on the fields of economics, political science, sociology and psychology, in which ready access to raw data and software code are crucial to replication efforts, we survey deficiencies in the current system.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 554, 417-419 (2018)

doi: https://doi.org/10.1038/d41586-018-02108-9

This work was funded by the Berkeley Initiative for Transparency in the Social Sciences and by the Laura and John Arnold Foundation.

Updates & Corrections

  • Correction 22 March 2018: An earlier version of this Comment stated that only authors of non-replicated articles from the 3ie project described antagonism. It also gave the wrong number of replications. The data set is now available as supplementary information.

References

  1. Nosek, B. A. et al. Science 348, 1422–1425 (2015).

    Article  PubMed  CAS  Google Scholar 

  2. Christensen, G. & Miguel, E. Preprint at http://dx.doi.org/10.17605/OSF.IO/9A3RW (2017).

  3. Berry, J., Coffman, L. C., Hanley, D., Gihleb, R. & Wilson, A. J. Am. Econ. Rev. 107, 27–31 (2017).

    Article  Google Scholar 

  4. International Initiative for Impact Evaluation. 3ie Replication Programme (3ie, 2017); available at http://go.nature.com/2eoe7o5

    Google Scholar 

  5. Davey, C., Aiken, A. M., Hayes, R. J. & Hargreaves, J. R. Int. J. Epidemiol. 44, 1581–1592 (2015).

    Article  PubMed  Google Scholar 

  6. Miguel, E. & Kremer, M. Econometrica 72, 159–217 (2004).

    Article  Google Scholar 

  7. McCullough, B. D, McGeary, K. A. & Harrison, T. D. J. Money Credit Bank. 38, 1093–1107 (2006).

    Article  Google Scholar 

  8. Chang, A. C. & Li, P. Finance Econ. Disc. Ser. http://dx.doi.org/10.17016/FEDS.2015.083 (2015).

    Article  Google Scholar 

  9. Gherghina, S. & Katsanidou, A. Eur. Polit. Sci. 12, 333–349 (2013)

    Article  Google Scholar 

  10. Key, E. M. Polit. Sci. Politics 49, 268–272 (2016).

    Article  Google Scholar 

Download references

Supplementary Information

  1. Gertler et al. Comment supplementary information
  2. Summary_3ie full data set

Subjects

Latest on:

Nature Careers

Jobs

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links