Stats: is this therapy useful?

Northeast Area Statistics Group, US Department of Agriculture, Beltsville, Maryland, USA.

Search for this author in:

I disagree that the current misuse of P values in biomedical science could be solved by ‘retiring’ statistical significance (V. Amrhein et al. Nature 567, 305–307; 2019).

Like it or not, some of the blame for current practices lies in researchers’ infatuation with simply disproving the null hypothesis. They often see this as a more ‘objective’ way of doing science: collect data and let the decision about its importance be made by statistics.

The real question is whether a treatment effect is important, not whether it differs ‘significantly’ from a control. To answer this, the researcher should justify beforehand how large the effect size needs to be. Then, if a 10% improvement over the control is required, the probability that this has been attained can be calculated from the data using familiar statistical tools for hypothesis testing and sample-size determination.

Nature 569, 192 (2019)

doi: 10.1038/d41586-019-01452-8
Nature Briefing

Sign up for the daily Nature Briefing email newsletter

Stay up to date with what matters in science and why, handpicked from Nature and other publications worldwide.

Sign Up