We share many of the concerns expressed by Coyne and colleagues about the standard of scientific research in this field. As we emphasized in our Review,1 many published studies are of poor quality, due to an insufficient number of cases (resulting in low statistical power) and limited controls, and we agree that publication biases are often likely to be a factor. Nonetheless, we take issue with several aspects of this critique.

Coyne et al. appear to have misunderstood the nature of systematic reviews and meta-analyses. This technique of aggregating data is based on specified procedures and pre-defined criteria for identifying published articles.2 All studies that fulfill these criteria must be included. Coyne et al. suggest that some published studies should have been excluded from the meta-analyses, but it is improper to pick and choose studies for inclusion in relation to the investigators' preconceptions. We took methodological quality into account by defining quality criteria, rating each study, and doing sub-analyses of only high-quality studies.1 The combined hazard ratios for the high-quality studies were similar to those of the complete dataset (Figure 2 and Figure 3), and did not show signs of publication bias. This makes it difficult to conclude, as Coyne et al. do, that inclusion of poorer quality studies exaggerated the strength of associations between stress-related factors and cancer incidence.

We are aware of the concerns about the studies performed by Grossarth-Maticek and colleagues. These studies were included in only one of the three sets of meta-analyses (Figure 41). Furthermore, because their methodological quality was poor, they were excluded from sub-analyses of high-quality studies. The hazard ratio for high-quality studies (1.19, 95% CI 1.07–1.33), compared with that for the complete set (1.29, 95% CI 1.16–1.44), indicates that this work had limited impact on the overall findings.

Coyne et al. also take issue with the inclusion of several effect sizes from a single study. However, this is a common procedure and is accepted by statistical reviewers for a variety of authoritative journals.3,4,5 There are two reasons why the inclusion of effect sizes in this way is valuable. First, longitudinal prospective studies in this field are costly and difficult to conduct, so investigators rightly maximise their use of the data. If a study investigated not only major life events but also assessed psychological factors such as Type A personality and neuroticism (such as two studies by Lillberg et al.6,7), it would be inappropriate to combine them in an arbitrary metric. Second, we cannot assume that all cancers share the same associations with psychosocial factors. The study by Kvikstad et al.8 highlighted by Coyne and colleagues is a case in point. It showed positive associations between the death of a child and parental incidence of cervical, lung and hematopoietic cancers, and negative associations between the death of a child and parental incidence of breast, colorectal, thyroid and brain cancers, although none of the associations were significant. It would be meaningless to generate a single effect size from such diverse outcomes.

It is disingenuous of Coyne et al. to state that we treated factors as diverse as neuroticism and the death of a child as equivalent. On the contrary, these factors were clearly categorized into the subgroups of stress-prone personality and stress exposure, respectively. Due to the nature of this field of research, any particular classification methodology is open to dispute and, therefore, we detailed our categorization in the supplementary tables so that other investigators could scrutinize our choices.1 Nevertheless, we are of the opinion that many of the reservations outlined by Coyne and colleagues are unwarranted.