In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
Testing practical relevance of treatment effects
Statistical Papers Open Access 17 April 2024
-
Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions
Translational Psychiatry Open Access 03 February 2024
-
On the use of receiver operating characteristic curve analysis to determine the most appropriate p value significance threshold
Journal of Translational Medicine Open Access 04 January 2024
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
References
Benjamin, D. J. et al. Nat. Hum. Behav. 2, 6–10 (2018).
Wacholder, S., Chanock, S., Garcia-Closas, M., El Ghormli, L. & Rothman, N. J. Natl Cancer Inst. 96, 434–442 (2004).
Open Science Collaboration Science 349, aac4716 (2015).
Senn, S. Statistical Issues in Drug Development 2nd edn (John Wiley & Sons, Chichester, 2007).
Mayo, D. Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars (Cambridge Univ. Press, Cambridge, 2018).
Johnson, V. E., Payne, R. D., Wang, T., Asher, A. & Mandal, S. J. Am. Stat. Assoc. 112, 1–10 (2017).
Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P. & Willmes, K. Cortex 66, A1–A2 (2015).
Lyons, L. Preprint at http://arxiv.org/abs/1310.1284 (2013).
Field, S. A., Tyre, A. J., Jonzen, N., Rhodes, J. R. & Possingham, H. P. Ecol. Lett. 7, 669–675 (2004).
Grieve, A. P. Pharm. Stat. 14, 139–150 (2015).
Mudge, J. F., Baker, L. F., Edge, C. B. & Houlahan, J. E. PLoS ONE 7, e32734 (2012).
Skipper, J. K., Guenther, A. L. & Nass, G. Am. Sociol. 2, 16–18 (1967).
Neyman, J. & Pearson, E. S. Phil. Trans. R. Soc. Lond. Ser. A 231, 694–706 (1933).
Fisher R. A. Statistical Methods and Scientific Inferences (Hafner, Oxford, 1956).
Casadevall, A. & Fang, F. C. mBio 7, e01902–16 (2016).
Acknowledgements
We thank D. Barr, F. Cheung, D. Colquhoun, H. IJzerman, H. Motulsky and R. Morey for helpful discussions while drafting this Comment. D.L. was supported by Nederlandse Organisatie voor Wetenschappelijk Onderzoek 452-17-013. F.G.A. was supported by CONICET. M.A.J.A. was funded by a Biotechnology and Biological Sciences Research Council AFL Fellowship (BB/M013596/1). G.S.C. was supported by the National Institute for Health Research Biomedical Research Centre, Oxford. Z.C. was supported by the Economic and Social Research Council (grant number C106891X). E.S.C. was supported by the European Research Council (ERC-2015-StG-677270). L.D. is supported by the European Research Council (ERC-2014-CoG-647910 KINSHIP). A.-L.v.H. is funded by a Royal Society Dorothy Hodgkin Fellowship (DH150176). M.R.H. was supported by the National Science Foundation under grant SBE SPRF-FR 1714446. J.Lao was supported by the Swiss National Science Foundation grant 100014_156490/1. C.L.d.O. was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico. A.E.M. was supported by the Economic and Social Research Council of the United Kingdom (grant number ES/K009095/1). J.-J.O.d.X. is supported by an internal grant from the KU Leuven (STG/14/054) and by the Fonds voor Wetenschappelijk Onderzoek (1519916N). S.P. was supported by the European Research Council (FP7/2007–2013; ERC grant agreement no. 324176). G.M.A.L. was funded by Nederlandse Organisatie voor Wetenschappelijk Onderzoek 453-14-016. S.G.S. is supported by a Cancer Research UK Fellowship (C42785/A17965). V.S. was supported by the National Institute for Neurological Disorders and Stroke Intramural Research Program (IRP). M.A.V. was supported by grant 2016-T1/SOC-1395 from Comunidad de Madrid. T.Y. was supported by National Institutes of Health award R01MH109682.
Author information
Authors and Affiliations
Contributions
D.L., N.W.F., M.G.-M., J.A.G., N.P.H., A.A.K., S.R.M., V.S., S.D.B. and C.N.S. participated in brainstorming, drafting the Comment and data analysis. C.J.A., S.E.A., T.B., E.M.B., B.V.C., Z.C., G.S.C., S.D., D.J.D., B.D.E., J.D.F., J.G.F., A.-L.v.H., M.I., P.M.I., H.K.I., J.Lao, G.M.A.L., D.M., A.E.M., K.M., A.Q.X.N., G.N., C.L.d.O., J.-J.O.d.X., G.P, K.A.Q., I.K.S., Z.S., S.G.S., J.R.S., M.A.L.M.V.A., M.N.W., D.R.W., T.Y. and R.A.Z. participated in brainstorming and drafting the Comment. F.G.A., R.B.B., M.I.F., F.H. and S.P. participated in drafting the Comment and data analysis. M.A.J.A., D.E.B., S.-C.C., B.C., L.J.C., H.D., L.D., M.R.H., E.G.L., R.J.M., J.J.S., S.A.S., T.S., N.S., W.Ś. and M.A.V. participated in brainstorming. F.A., A.R.C., R.C., E.S.C., A.F, C.G., A.P.G., R.G., J.G., K.D.H, C.J., K.J, D.A.K., B.K., J.Lukavský, C.R.M., D.G.M., C.M., M.S., S.E.W. I.Z. did not participate in drafting the Comment because the points that they would have raised had already been incorporated, or they endorse a sufficiently large part of the contents as if participation had occurred. Except for the first author, authorship order is alphabetical.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Rights and permissions
About this article
Cite this article
Lakens, D., Adolfi, F.G., Albers, C.J. et al. Justify your alpha. Nat Hum Behav 2, 168–171 (2018). https://doi.org/10.1038/s41562-018-0311-x
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-018-0311-x
This article is cited by
-
On the use of receiver operating characteristic curve analysis to determine the most appropriate p value significance threshold
Journal of Translational Medicine (2024)
-
Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions
Translational Psychiatry (2024)
-
Narcissism Moderates the Association Between Autonomy-Supportive Parenting and Adolescents’ Prosocial Behavior
Journal of Youth and Adolescence (2024)
-
Testing practical relevance of treatment effects
Statistical Papers (2024)
-
How transparent are quantitative studies in contemporary technology education research? Instrument development and analysis
International Journal of Technology and Design Education (2024)