We would like to thank you for the opportunity to respond to the issues raised in the letter by Dr Zhou and colleagues. We are also grateful to Dr Zhou and colleagues for their interest in our work.
In their letter to the editor, Dr Zhou and colleagues raise concerns about the choice of databases in our literature search. We limited our search to Medline and Embase, and additionally searched the Cochrane Library and reference lists of reviews on related topics. Indeed, a well-conducted systematic review requires a broad coverage of the literature to identify as many relevant studies as possible. According to the Cochrane Handbook, The Cochrane Central Register of Controlled Trials (CENTRAL), Medline and Embase are considered the most important bibliographic sources for inclusion in systematic reviews on intervention studies or studies on the accuracy of diagnostic tests.1 Prognostic marker studies are usually explorative, retrospective studies and not clinical intervention studies,2 which is why we think that searching The Cochrane CENTRAL and ClinicalTrials.gov, as suggested by Zhou and colleagues, is likely to be of limited benefit. Searching Web of Science in addition to Medline and Embase in combination with Google Scholar, was recommended in a recent study by Bramer et al.,3 but has to the best of our knowledge not been found earlier. To date, no consensus exists on the type and number of electronic databases that should be searched, and it has been shown that the vast majority of relevant articles can be retrieved from a limited number of databases.4,5 Finally yet importantly, review authors have to balance the extensiveness of the search with the efficiency in use of time and budget.1
Dr Zhou and colleagues also raise concerns about the assessment of publication quality and risk of bias. We agree with Dr Zhou and colleagues that evaluating risk of bias is important to estimate the effect of the results. However, as outlined in the PRISMA Statement, this can be assessed by many methods, such as scales, checklists, and individual components.6 We utilized a checklist adapted from the REMARK guidelines7 to evaluate the reporting quality of the studies. These guidelines were designed to promote proper reporting in prognostic marker research and not as a tool to assess risk of bias. Therefore, we evaluated both the reporting quality and the risk of bias in a qualitative fashion without providing a quantitative grading score for the individual articles.
Dr Zhou and colleagues point out that our results should be interpreted with caution, and we have indeed stated that the conclusions derived from our review are uncertain.8 This is mostly because of reporting deficiencies on study design and conduct in the included studies, which inflict substantial risk of selection, performance and detection bias. Although CD163+ M2 macrophages and CD57+ natural killer cells were the most promising predictors of survival in oral cancer patients, we have concluded that these results should be validated in large, standardized studies before they are implemented in the clinic. We thank Dr Zhou and colleagues for valuable comments, but we do not believe that using a different methodological approach with regard to both literature search and assessment of risk of bias would change this conclusion. We hope that focusing towards proper and transparent reporting of biomarker studies will increase their quality and clinical usefulness. Summarizing the current knowledge, although it is hampered by uncertainty, may help direct future research.
References
Higgins, J. P. T. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 0 [updated March 2011]. (The Cochrane Collaboration, 2011).
Riley, R., Sauerbrei, W. & Altman, D. Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond. Br. J. Cancer 100, 1219 (2009).
Bramer, W. M., Rethlefsen, M. L., Kleijnen, J. & Franco, O. H. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst. Rev. 6, 245 (2017).
Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, DM. & Vandermeer, B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med. Res. Methodol. 16, 127 (2016).
Halladay, C. W., Trikalinos, T. A., Schmid, I. T., Schmid, C. H. & Dahabreh, I. J. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J. Clin. Epidemiol. 68, 1076–1084 (2015).
Liberati, A., Altman, DG., Tetzlaff, J., Mulrow, C., Gøtzsche, PC., Ioannidis, JP et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 6, e1000100 (2009).
Altman, D. G., McShane, L. M., Sauerbrei, W. & Taube, S. E. Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration. BMC Med. 10, 51 (2012).
Hadler-Olsen, E. & Wirsing, A. M. Tissue-infiltrating immune cells as prognostic markers in oral squamous cell carcinoma: a systematic review and meta-analysis. Br. J. Cancer 120, 714–727 (2019).
Author contributions
Both authors have contributed in writing of this letter.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Consent for publication
All authors have read and approved the final version of this manuscript.
Note
This work is published under the standard license to publish agreement. After 12 months the work will become freely available and the license terms will switch to a Creative Commons Attribution 4.0 International (CC BY 4.0).
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Hadler-Olsen, E., Wirsing, A.M. Reply to ‘Comment on “Tissue-infiltrating immune cells as prognostic markers in oral squamous cell carcinoma: a systematic review and meta-analysis”’. Br J Cancer 120, 1084–1085 (2019). https://doi.org/10.1038/s41416-019-0460-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41416-019-0460-3