Points of significance: Power and sample size

Journal name:
Nature Methods
Volume:
10,
Pages:
1139–1140
Year published:
DOI:
doi:10.1038/nmeth.2738
Published online
Corrected online
Corrected online

The ability to detect experimental effects is undermined in studies that lack power.

At a glance

Figures

  1. When unlikely hypotheses are tested, most positive results of underpowered studies can be wrong.
    Figure 1: When unlikely hypotheses are tested, most positive results of underpowered studies can be wrong.

    (a) Two sets of experiments in which 50% and 10% of hypotheses correspond to a real effect (blue), with the rest being null (green). (b) Proportion of each inference type within the null and effect groups encoded by areas of colored regions, assuming 5% of nulls are rejected as false positives. The fraction of positive results that are correct is the positive predictive value, PPV, which decreases with a lower effect chance.

  2. Inference errors and statistical power.
    Figure 2: Inference errors and statistical power.

    (a) Observations are assumed to be from the null distribution (H0) with mean μ0. We reject H0 for values larger than x* with an error rate α (red area). (b) The alternative hypothesis (HA) is the competing scenario with a different mean μA. Values sampled from HA smaller than x* do not trigger rejection of H0 and occur at a rate β. Power (sensitivity) is 1 − β (blue area). (c) Relationship of inference errors to x*. The color key is same as in Figure 1.

  3. Decreasing specificity increases power.
    Figure 3: Decreasing specificity increases power.

    H0 and HA are assumed normal with σ = 1. (a) Lowering specificity decreases the H0 rejection cutoff x*, capturing a greater fraction of HA beyond x*, and increases the power from 0.64 to 0.80. (b) The relationship between specificity and power as a function of x*. The open circles correspond to the scenarios in a.

  4. Impact of sample (n) and effect size (d) on power.
    Figure 4: Impact of sample (n) and effect size (d) on power.

    H0 and HA are assumed normal with σ = 1. (a) Increasing n decreases the spread of the distribution of sample averages in proportion to 1/√n. Shown are scenarios at n = 1, 3 and 7 for d = 1 and α = 0.05. Right, power as function of n at four different α values for d = 1. The circles correspond to the three scenarios. (b) Power increases with d, making it easier to detect larger effects. The distributions show effect sizes d = 1, 1.5 and 2 for n = 3 and α = 0.05. Right, power as function of d at four different a values for n = 3.

Change history

Corrected online 26 November 2013
In the print version of this article initially published, the symbol μ0 was represented incorrectly in the equation for effect size, d = (μAμ0)/σ. The error has been corrected in the HTML and PDF versions of the article.
Corrected online 03 August 2015
In the version of this article initially published, the terms "sensitivity" and "specificity" and the related descriptors "sensitive" and "specific" were mistakenly switched in three instances. The errors have been corrected in the HTML and PDF versions of the article.

References

  1. Button, K.S. et al. Nat. Rev. Neurosci. 14, 365376 (2013).
  2. Moher, D., Dulberg, C.S. & Wells, G.A. J. Am. Med. Assoc. 272, 122124 (1994).
  3. Breau, R.H., Carnat, T.A. & Gaboury, I. J. Urol. 176, 263266 (2006).
  4. Krzywinski, M.I. & Altman, N. Nat. Methods 10, 809810 (2013).

Download references

Author information

Affiliations

  1. Martin Krzywinski is a staff scientist at Canada's Michael Smith Genome Sciences Centre.

  2. Naomi Altman is a Professor of Statistics at The Pennsylvania State University.

Competing financial interests

The authors declare no competing financial interests.

Author details

Supplementary information

Other

  1. Supplementary Table 1 (606 KB)

    Worksheets demonstrating power and effect size. Please note that the workbook requires that macros be enabled.

Additional data