We wish to comment on the Chida et al.1 Review that investigated the contribution of stress-related variables to cancer incidence, survival time and mortality using a series of meta-analyses. Chida et al.1 reported significant associations for incidence of cancer (P = 0.005), survival time (P <0.001), and cancer mortality (P <0.001). The Review has attracted attention despite major flaws that we discuss below. Although the authors call for caution in interpreting these results due to evidence of a publication bias, we propose that the quality of the literature they review and their application of meta-analysis were inadequate and pose serious challenges to the validity of their conclusions.

The Review included results from a vast number of studies, most of which reported a null effect. The meta-analyses for incidence of cancer (142 studies; average sample size 87,062), survival time (157 studies; average sample size 418), and cancer mortality (50 studies; average sample size 93,059) involved large aggregate samples. Therefore, hazard ratios (HRs) and confidence intervals (CIs) are more informative than statistical significance because trivial effect sizes can nonetheless be highly significant with large sample sizes.2 Indeed, the CIs reported by Chida et al.1 for cancer incidence (HR 1.06, 95% CI 1.02–1.11) and survival (HR 1.03, 95% CI 1.02–1.04) barely excluded 1.0 and were thus close to non-significance. Such trivial effect sizes do not typically generate much interest in the larger epidemiological community, especially when authors acknowledge statistical indications of a publication bias, as was the case in this Review.

Almost all CIs of HRs for studies of cancer mortality (overall HR 1.29, 95% CI 1.16-1.44) either included or barely excluded 1.0, with the glaring exception of five extreme outlier HRs ranging from 23.8 to 74.2, all from the work of Grossarth-Maticek and colleagues.3,4 This work has been discredited because of strong suspicions that it relies on invalid data.5,6,7 Anyone familiar with the scandal surrounding this work would be surprised that it was included in a meta-analysis.

Apart from the inclusion of discredited work, the meta-analyses were also conducted incorrectly. A number of samples were counted multiple times, effectively treating each as independent cohorts, rather than more appropriately entering one effect size per cohort.8 For instance, 29 effect sizes were entered for one cohort,9,10 in which the effects of the death of a spouse, death of a child, and divorce on cancer incidence were each considered independently with respect to different cancer sites. Entering multiple effect sizes from the same sample in this way undermines any claims to the validity of these meta-analyses.

The stress-related variables that Chida et al.1 considered to be equivalent for the purposes of meta-analysis (such as fighting spirit, shift work, death of child, neuroticism, and Minnesota Multiphasic Personality Inventory (MMPI) Lie scores) were so heterogeneous as to defy any integrative theoretical interpretation of the omnibus effect size produced by a meta-analysis. An analogous situation would be to meta-analyze all surgical interventions to treat all cancers and then interpret the outcome as a meaningful indicator of whether surgery is effective in treating cancer. Furthermore, a number of these variables (for example, the MMPI Lie Scale) are not stress-related and others are of dubious validity, such as those developed by Grossarth-Maticek,2,3 and used by others.11

Many of the cited studies lack adequate statistical controls; indeed, 28% of the incidence studies, 1% of the survival studies, and 40% of the mortality studies included no controls. Thus, lack of control over known prognostic variables remains an alternative explanation of positive results. For instance, measures of distress obtained from cancer patients are likely to be strongly confounded with overall disease burden and the patient's knowledge of their prognosis. It would seem wise to conduct sensitivity analyses or simply exclude such studies from meta-analysis because they do not provide unconfounded estimates of the association between these 'stress-related' variables and mortality. In addition, many of the highest estimates of an association between stress-related variables and cancer came from underpowered studies with inadequate statistical controls. The adequacy of a sample for calculating a HR is based on the number of events being explained (deaths or incident cases of cancer). Despite large overall samples, a number of the studies12,13 had too few events to warrant the multivariate analyses on which the reported HRs were based. One study had 111,974 person-years of observation of day versus fixed night versus rotating shift work, but only 31 cases of prostate cancer to explain, making inclusion of 13 covariates inappropriate.12

Claims of an association between stress and cancer incidence, survival time, and mortality have great appeal among laypersons and professionals alike. However, much of the literature reviewed by Chida et al.1 is of poor quality and some studies have dubious validity, and the quality of data going into a meta-analysis will be reflected in the quality of the results. But regardless, the authors did not conduct an appropriate meta-analysis of this literature and few, if any, meaningful conclusions can be drawn from their work.