Zwanzger et al (2013) recent article entitled ‘Acute shift in glutamate concentrations following experimentally induced panic with cholecystokinin tetrapeptide—a 3T-MRS study in healthy subjects’ demonstrates a creative experimental design for studying the neurochemical processes underlying panic attacks. In 18 healthy volunteers, they acquired a series of 1H-MRS measurements in the anterior cingulate cortex to measure the Glx signal (arising primarily from glutamate) before and after an intravenous injection of a panic-inducing dose of CCK-4. They concluded that ‘CCK-4-induced panic was accompanied by a rapid and significant increase of Glx concentrations in the ACC.’ Unfortunately, they used a statistically invalid approach to test the likelihood that their observations could have occurred by chance.

Their experimental design included six 5-minute MRS acquisitions, each yielding a measurement of Glx. The first measurement was made before the CCK-4 injection and was considered the baseline measurement. The five subsequent Glx measurements were considered post CCK-4 measurements. To test their hypothesis that CCK-4 causes an increase in ACC Glx, the authors used a repeated measures analysis of variance to compare the baseline Glx value to the maximum Glx value observed in any of the five post CCK-4 measurements. They reported a highly significant result: F(1,17)=15.94, P=0.001. However, this statistical approach implicitly assumes that the null hypothesis is zero, or ‘no difference’ between the baseline and the maximum subsequent Glx value. This is clearly incorrect. In a series of six random numbers, the first number will be lower than the maximum of the subsequent five numbers 83% of the time. To illustrate the seriousness of this statistical illusion, I used a random number generator to create three data sets similar to that reported by Zwanzger et al (2013). Each data set included 18 ‘subjects’ and each subject was represented by a series of six random numbers. The random numbers were generated to have a mean and standard deviation identical to those inferred from Figure 2 in the Zwanzger et al (2013 article. Running a repeated measures ANOVA comparing ‘baseline’ to ‘Maximum post baseline’ gave the following results for the three randomly generated data sets: F(1,17)=34.67, F(1,17)=21.85, and F(1,17)=17.55—all ‘highly significant’ if one assumes the null hypothesis is zero difference. Yet, the F ratio reported by Zwanzger et al (2013) was lower than any of the F ratios I obtained using random numbers as ‘data.’ The same statistical illusion invalidates their analysis of the heart rate change following CCK-4.

The article by Zwanzger et al tests an interesting hypothesis and includes a scholarly and valuable discussion of the potential role of glutamatergic mechanisms in panic disorder. Unfortunately, the empirical data presented do not support the conclusion that CCK-4 induces a rise in ACC Glx. In fact, the pattern of changes in Glx they observed following CCK-4 resembles what one would expect to see by chance.

FUNDING AND DISCLOSURE

We declare that, except for income received from our primary employer, no financial support or compensation has been received from any individual or corporate entity over the past 3 years for research or professional service and there are no personal financial holdings that could be perceived as constituting a potential conflict of interest. The authors declare no conflict of interest.