Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Justify your alpha

In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The proportion of studies3 replicated at α = 0.05 (with a bin width of 0.005).


  1. Benjamin, D. J. et al. Nat. Hum. Behav. 2, 6–10 (2018).

    Article  Google Scholar 

  2. Wacholder, S., Chanock, S., Garcia-Closas, M., El Ghormli, L. & Rothman, N. J. Natl Cancer Inst. 96, 434–442 (2004).

    Article  PubMed  Google Scholar 

  3. Open Science Collaboration Science 349, aac4716 (2015).

  4. Senn, S. Statistical Issues in Drug Development 2nd edn (John Wiley & Sons, Chichester, 2007).

  5. Mayo, D. Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars (Cambridge Univ. Press, Cambridge, 2018).

  6. Johnson, V. E., Payne, R. D., Wang, T., Asher, A. & Mandal, S. J. Am. Stat. Assoc. 112, 1–10 (2017).

    Article  CAS  Google Scholar 

  7. Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P. & Willmes, K. Cortex 66, A1–A2 (2015).

    Article  PubMed  Google Scholar 

  8. Lyons, L. Preprint at (2013).

  9. Field, S. A., Tyre, A. J., Jonzen, N., Rhodes, J. R. & Possingham, H. P. Ecol. Lett. 7, 669–675 (2004).

    Article  Google Scholar 

  10. Grieve, A. P. Pharm. Stat. 14, 139–150 (2015).

    Article  PubMed  Google Scholar 

  11. Mudge, J. F., Baker, L. F., Edge, C. B. & Houlahan, J. E. PLoS ONE 7, e32734 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Skipper, J. K., Guenther, A. L. & Nass, G. Am. Sociol. 2, 16–18 (1967).

    Google Scholar 

  13. Neyman, J. & Pearson, E. S. Phil. Trans. R. Soc. Lond. Ser. A 231, 694–706 (1933).

    Article  Google Scholar 

  14. Fisher R. A. Statistical Methods and Scientific Inferences (Hafner, Oxford, 1956).

  15. Casadevall, A. & Fang, F. C. mBio 7, e01902–16 (2016).

    PubMed  PubMed Central  Google Scholar 

Download references


We thank D. Barr, F. Cheung, D. Colquhoun, H. IJzerman, H. Motulsky and R. Morey for helpful discussions while drafting this Comment. D.L. was supported by Nederlandse Organisatie voor Wetenschappelijk Onderzoek 452-17-013. F.G.A. was supported by CONICET. M.A.J.A. was funded by a Biotechnology and Biological Sciences Research Council AFL Fellowship (BB/M013596/1). G.S.C. was supported by the National Institute for Health Research Biomedical Research Centre, Oxford. Z.C. was supported by the Economic and Social Research Council (grant number C106891X). E.S.C. was supported by the European Research Council (ERC-2015-StG-677270). L.D. is supported by the European Research Council (ERC-2014-CoG-647910 KINSHIP). A.-L.v.H. is funded by a Royal Society Dorothy Hodgkin Fellowship (DH150176). M.R.H. was supported by the National Science Foundation under grant SBE SPRF-FR 1714446. J.Lao was supported by the Swiss National Science Foundation grant 100014_156490/1. C.L.d.O. was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico. A.E.M. was supported by the Economic and Social Research Council of the United Kingdom (grant number ES/K009095/1). J.-J.O.d.X. is supported by an internal grant from the KU Leuven (STG/14/054) and by the Fonds voor Wetenschappelijk Onderzoek (1519916N). S.P. was supported by the European Research Council (FP7/2007–2013; ERC grant agreement no. 324176). G.M.A.L. was funded by Nederlandse Organisatie voor Wetenschappelijk Onderzoek 453-14-016. S.G.S. is supported by a Cancer Research UK Fellowship (C42785/A17965). V.S. was supported by the National Institute for Neurological Disorders and Stroke Intramural Research Program (IRP). M.A.V. was supported by grant 2016-T1/SOC-1395 from Comunidad de Madrid. T.Y. was supported by National Institutes of Health award R01MH109682.

Author information

Authors and Affiliations



D.L., N.W.F., M.G.-M., J.A.G., N.P.H., A.A.K., S.R.M., V.S., S.D.B. and C.N.S. participated in brainstorming, drafting the Comment and data analysis. C.J.A., S.E.A., T.B., E.M.B., B.V.C., Z.C., G.S.C., S.D., D.J.D., B.D.E., J.D.F., J.G.F., A.-L.v.H., M.I., P.M.I., H.K.I., J.Lao, G.M.A.L., D.M., A.E.M., K.M., A.Q.X.N., G.N., C.L.d.O., J.-J.O.d.X., G.P, K.A.Q., I.K.S., Z.S., S.G.S., J.R.S., M.A.L.M.V.A., M.N.W., D.R.W., T.Y. and R.A.Z. participated in brainstorming and drafting the Comment. F.G.A., R.B.B., M.I.F., F.H. and S.P. participated in drafting the Comment and data analysis. M.A.J.A., D.E.B., S.-C.C., B.C., L.J.C., H.D., L.D., M.R.H., E.G.L., R.J.M., J.J.S., S.A.S., T.S., N.S., W.Ś. and M.A.V. participated in brainstorming. F.A., A.R.C., R.C., E.S.C., A.F, C.G., A.P.G., R.G., J.G., K.D.H, C.J., K.J, D.A.K., B.K., J.Lukavský, C.R.M., D.G.M., C.M., M.S., S.E.W. I.Z. did not participate in drafting the Comment because the points that they would have raised had already been incorporated, or they endorse a sufficiently large part of the contents as if participation had occurred. Except for the first author, authorship order is alphabetical.

Corresponding author

Correspondence to Daniel Lakens.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lakens, D., Adolfi, F.G., Albers, C.J. et al. Justify your alpha. Nat Hum Behav 2, 168–171 (2018).

Download citation

  • Published:

  • Issue Date:

  • DOI:

This article is cited by


Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics