Sir,

We thank Mei et al (2018) for their interest in our article (Samarendra et al, 2017) and insightful comments, and respond to these comments as follows.

Literature searching concerns. Dr Zu-Bing Mei et al (2018) identify two studies that they claim are not included in our analysis despite meeting the inclusion criteria. Furthermore, they indicate that had these articles been included, the results of our meta-analysis may have been different.

In order to clarify, we would like to point out that one of the cited articles (Saigusa et al, 2010) was identified by our search strategy; however, the authors did not include sufficient data for inclusion in our meta-analysis. The authors were contacted by email but did not respond to our request for further data.

The second study identified by Mei et al (Yu et al, 2016) was not identified by our search strategy. This is likely because it was published in the same month as we ran our search strategy and therefore may not have been indexed at that time.

On review of (Yu et al, 2016), the article reports on 54 osteosarcoma patients, demonstrating a significant association between elevated CXCL12 expression and overall survival (HR 2.64 (1.12–6.20)). This is a small study and the resultant effect estimate is associated with a reasonably wide confidence interval; it is unlikely to have significantly altered our overall survival estimates.

Methodological concerns. The respondents indicate that meta-regression should have been performed in order to investigate the effects of certain study-level factors on outcome estimates, and thereby explain potential sources of inter-study heterogeneity. In meta-regression approaches the study is the unit of analysis and patient-level data is analysed as the aggregate figure presented in the original study. In this way relationships can be drawn between summary statistics of patient-level factors and outcome effects.

While we recognise the use of meta-regression as a method for investigating the underlying causes of between-study heterogeneity, it has limitations of relevance to our meta-analysis. Most significantly, meta-regression is dependent upon reasonable reporting of patient-level summary statistics (Schmid et al, 2004). As discussed at length in our review (see penultimate paragraph of discussion in particular), the reporting of such statistics was poor in the included studies and furthermore is significantly limited by their retrospective nature. As a result, the respondent’s assertion that because greater than 10 studies are included in our analysis of overall and recurrence-free survival, meta-regression could be used, is inaccurate. Indeed, meta-regression would require that at least 10 included studies accurately report summary statistics of patient-level factors. Although more than 10 studies reported basic demographic factors including average age and sex, factors of relevance to cancer biology including mutation status or characterisation of the immune infiltrate were not routinely reported and certainly were not reported in >10 of the included studies. Hence meta-regression would not have been possible in this instance.

As an alternative to meta-regression, we reasoned that a significant degree of such heterogeneity in our analyses of overall and recurrence-free survival (RFS) for all included studies resulted from the inclusion of cancer types from different organs and differences in their underlying biology. Such differences in the biology of cancers from different organs are evidenced by their differing genetic profiles, propensity to pro- or anti-tumourigenic immune responses and clinical behaviour among numerous other factors. This was the basis on which we performed sub-group analysis on cancer type. The fact that this significantly reduced between study heterogeneity (see Figures 5 and 6 in the original article) in all but colorectal cancer indicates that this is a valid approach, and that a large proportion of the heterogeneity seen in analyses of all studies meeting our inclusion criteria resulted from differences between cancer types.

Concerns about specific points. The respondents claim that our meta-analysis selects improper study endpoints and indicate that we should have used recurrence-free survival as the primary outcome measure rather than overall survival (OS).

This contention is somewhat confusing as we have presented meta-analytical data for both overall and recurrence-free survival and have not indicated whether we consider OS or RFS as the primary end-point. The data we present for RFS supports our findings for OS as stated in our original article.

We recognise the shortcomings of over-all survival data in the analysis of potential prognostic biomarkers; however, as we have mentioned in our discussion, OS data are more readily available. This is again because the majority of included studies are retrospective and so ascertaining cause of death may have been problematic.