Comparisons of population-based cancer survival between countries are important to benchmark the overall effectiveness of cancer management. The International Cancer Benchmarking Partnership (ICBP) Survmark-2 study aims to compare survival in seven high-income countries across eight cancer sites and explore reasons for the observed differences. A critical aspect in ensuring comparability in the reported survival estimates are similarities in practice across cancer registries. While ICBP Survmark-2 has shown these differences are unlikely to explain the observed differences in cancer-specific survival between countries, it is important to keep in mind potential biases linked to registry practice and understand their likely impact.
Based on experiences gained within ICBP Survmark-2, we have developed a set of recommendations that seek to optimally harmonise cancer registry datasets to improve future benchmarking exercises.
Our recommendations stem from considering the impact on cancer survival estimates in five key areas: (1) the completeness of the registry and the availability of registration sources; (2) the inclusion of death certification as a source of identifying cases; (3) the specification of the date of incidence; (4) the approach to handling multiple primary tumours and (5) the quality of linkage of cases to the deaths register.
These recommendations seek to improve comparability whilst maintaining the opportunity to understand and act upon international variations in outcomes among cancer patients.
Comparisons of cancer survival between or within countries, as derived from population-based cancer registry data, have been the subject of a number of large-scale and influential collaborative projects, including the International Cancer Benchmarking Partnership (ICBP) [1,2,3,–4]. As part of phase two of ICBP, Survmark-2 has sought to benchmark survival for eight cancer sites across 21 jurisdictions in seven countries, bringing together cancer clinicians, policy-makers, researchers and data experts [4, 5]. As well as documenting and elucidating cancer-specific survival differences [6,7,8,–9], one of the aims of Survmark-2 is to investigate if they might, in part, be explained by differences in cancer registration methodology [10,11,–12].
Most previous studies have concluded that, although differences in cancer registration practice could partially explain survival differences, they are unlikely to explain the large variations seen across countries [10, 12,13,14,15,16], as long as the data are not analysed too prematurely when both incident cases and links to death data might be lacking to a great extent. A number of international guidelines have been developed to ensure that the cancer registry data are collected, coded, processed and presented in a comparable way [17,18,19,20,21,22,–23]. There are, however, some differences in their interpretation and implementation between registries, as well as variations in the sources (of data) available to registries. Even though differences in registration practice are unlikely to explain large differences in survival, documenting what the differences are, understanding how they impact on survival comparisons, and how such sources of variation can be minimised could be of benefit to registries and to researchers seeking to examine international survival comparisons. In this paper, we therefore draw upon some of the lessons learned from the ICBP Survmark-2 study with respect to registry practices and offer five recommendations that seek to increase data harmonisation so as to improve the comparability of future studies of cancer survival [1,2,–3].
Data sources and the potential for missed cases
Cancer registries commonly receive information on new cancer cases from multiple sources, the two most common being pathology reports and hospital admission data. The completeness, i.e. the extent to which all incident cancers occurring in the population are included , can vary by source, and the average number of sources per case registered is a useful indicator of completeness . It is however equally important to distinguish between the sources and procedures used to identify new cases, and the sources which are subsequently used to find additional information about registered cases. Even though no cancer registry will capture all cancer cases, survival comparisons rely not only on the level of completeness, but whether those cases captured are truly representative. Cancer registries that rely heavily on pathology reports might miss cases that are not sent for a biopsy, for instance due to cases being of older age or having advanced stage (with a poorer prognosis and shorter survival time). In such circumstances, population-based survival estimates will be overestimated. On the other hand, cancer registries that rely more heavily on hospital admissions might instead miss cases with a better prognosis, e.g. those that are only treated in an outpatient setting, leading to cancer survival being underestimated. Thus while knowledge of sources used to identify new cancer cases is important, an understanding of the completeness of each source and the proportion of cases notified from each source is critical. Depending on the sources used for registering a new cancer case, registries might also be more or less likely to record a prevalent cancer of an individual that moves into the catchment area and has previously been diagnosed in another region, leading to duplicate records of the same individual but in different regions .
Capture–recapture methods can be used to investigate whether the proportion of cases notified by different sources varies according to registry variables [18, 25]. However, a drawback of the method is that it can not be used to estimate the number of cases that were not notified by one or more of the notification sources investigated . Cancer registries should therefore not only describe the underlying processes, but record the specific source used in registering a new case or in augmenting information to an existing case. An understanding of which cases might be missed is also important in reporting survival comparisons across countries or over time.
Use of death certificate to identify cases
Many cancer registries receive information on deaths in their target population for which “cancer” is mentioned on the death certificate, so-called death certificate notified (DCN) cases . Cancer registries typically check DCN cases against their registered cases to see if this is a case already notified to the registry through other sources. Depending on the methodology of the registry (e.g. the sources of information available to the registry, and the delay with which information from each source arrives), comparing deaths with the file of registered cases may require a “waiting period” to be established, to ensure that all source reports are duly received. The length of the waiting period prior to checking DCN cases against the registry file will differ between registries, depending on how often and quickly information from other sources is retrieved. It should however be of sufficient length to ensure receipt of the information on cases from all other sources.
For the DCN cases for which there is no other source in the register following the waiting period, a traceback procedure is initiated to verify the diagnosis and that the date of diagnosis precedes the date of death. The cases for which no dates of diagnosis are found are referred to as death certificate only (DCO) cases, and the DCO cases together with the cases for which extra information are found are referred to as death certificate initiated (DCI) cases; those that would not be in the register if it was not for the notification from the death certificate. The proportion of DCO cases is often used as a quality measure of a cancer registry, however, the proportion of DCI is rarely reported. In most registries, the proportion of DCI is usually not known. As a measure of quality, the proportion of DCI is much more informative than the corresponding DCO proportion, since the DCI gives more information as to how many cancer cases are missed through routine registration processes. By itself, the DCO% is not an indicator of completeness of registration, a low DCO% may indicate efficient case-finding, but it could equally well result from the efficient traceback of DCN cases (although since the DCI% will always be equal to, or greater than, the DCO%, an elevated DCO% is suggestive of incompleteness) .
DCO cases are often excluded from survival analysis, since these cases do not have any follow-up information, the underlying reason why registries attempt to minimise the DCO%. However, little consideration is given to the fact that inclusion of the DCI cases with positive follow-up time will lead to a downward bias in the survival estimates . This is because the DCI cases are a selective group of missed cases, that is, they are missed cases that die due to cancer, whereas the missed cases that are still alive or do not have cancer mentioned on their death certificates are still missed. Thus when comparing survival between registries with proportions of DCOs that are similar, comparisons are still biased if the proportion of DCIs are not. A registry that initially missed a large proportion of cases, but had very effective traceback and a registry that missed only a small fraction of cases with no traceback could have very similar proportion of DCO cases, even though their DCI proportion differ greatly. Another suggested bias resulting from inclusion of DCI cases is that the traceback procedures might not find the true date of diagnosis , although studies have shown that this can account for only a small part of the observed differences in survival estimates [12, 14, 16]. Differences in the amount of time and resources spent on traceback in establishing the date of diagnosis may lead to differences in accuracy and completeness between registries.
Simulation studies investigating errors introduced by DCI cases and incompleteness have shown however that any bias is likely to be small [12, 13, 16]. Even so, information on both the DCI and the DCO proportions are important when comparing survival across registries. Moreover, information on which cases are DCI cases can be informative in estimating completeness and in performing sensitivity analyses, and thus a key recommendation is that DCI cases are flagged in the registry database.
Date of incidence
Since cancer patient survival is estimated based on the time between the date of incidence (DOI) and date of death; the DOI recorded at the registry evidently has some influence on survival estimates. In order to ensure consistency between registries in how they determine DOI, a hierarchical set of rules have been established, prioritising which source of information from which to define DOI. There are several available: the European Network of Cancer Registries (ENCR) rules  the International Association of Cancer Registries (IACR) and International Agency for Research on Cancer (IARC) rules  and the Surveillance, Epidemiology and End Results (SEER) rules . The main difference between these rules is the priority given to different dates in selecting “date of incidence”. The IARC/IACR rules prioritise the first consultation or admission to a clinic or hospital for the malignancy in question, while the ENCR rules prioritise the date of histological or cytological confirmation. In a registry that uses the ENCR rules for instance, the DOI of a proportion of cases will be based on date of hospitalisation in the absence of morphological information. Even in registries using the same rules, there may be differences in the proportion of cases for which different dates are used, dependant on the information available. It is therefore recommended that registries record dates relevant to each information source—for example, date of (first) hospital admission, date of the pathology report, date of CT or MRI scan, date of first treatment—so that DOI can be standardised between the datasets being compared. In closing; however, we note that while DOI selection is often thought to be a problem in comparing survival, it has been shown to have a rather trivial influence, except on very short-term survival probabilities .
Differences in the definition of multiple primaries have been discussed as a potential problem in assuring comparability of cancer patient survival estimates across countries . Multiple primaries are primary tumours occurring in individuals who have been previously diagnosed with cancer, not extensions, recurrences or metastases. Such information on subsequent cancers, in the same or a different organ, can be important to record when investigating specific research questions, for example, studies on contralateral breast cancer or solid tumours following a diagnosis of a haematological malignancy . Several sets of rules define what constitutes a new reportable cancer [21, 22]. The rules developed jointly by IACR and IARC are commonly used internationally, while those of the SEER programme are used mainly by North American registries. However, when estimating population-based cancer survival for a specific cancer type, only the first diagnosis of that specific type for each individual should be included in the analysis; individuals who have previously had cancer at a different site should be included. And if an individual is later diagnosed with another cancer at the same site this new cancer diagnosis should not be included in the survival analysis, and the follow-up for this individual should be continued. For some sites, there could be multiple primaries recorded for an individual on the same date, which could in some situations lead to differences between registries in which type of cancers are included in survival analysis.
Hence, the definition of multiple primaries used by the registry has limited impact on survival estimates, since the definition of what is a subsequent primary does not alter the information for the first diagnosis. However, in a recently-established registry there will be cases recorded as first cancers in individuals who had previous primary malignancies not recorded as they occurred in previous years prior to the beginning of registration. This may lead to the inclusion of second cancer at the same site that would not have been included if first cancer had been known. Since the prognosis is usually worse for recurring cancers and second primary cancers, this could lead to differences in survival estimates between registries of differing maturity. However, multiple cancers of the same site are rare (according to the international rules, they are only reportable if they are of a different histological type ). As long as only multiple primaries of the same site are excluded in survival analyses, and not any multiple primaries, this will have little impact on the estimates.
Linking to mortality and vital statistics
In order to accurately estimate survival, the vital status of all cancer patients under study is required. In order to retrieve this information, many cancer registries link the cancer cases in the register to information from death registers or vital statistics offices. Most registries that use this approach to retrieve follow-up information assume that cases with no match are still alive, a situation that is often referred to as passive follow-up. If not all deaths are captured cancer survival will be overestimated. This is more likely a bigger problem for registries that do not have a unique identification or health care number for each individual, or where such a unique identifier is not always used, and the linkage therefore relies on matching on name, birth date, address, or other personal attributes based on probability matching. Another source of bias occurs when the registry does not capture vital status information on individuals that have emigrated from the country or moved out of the registry capture area, since these individuals are assumed alive, again leading to an overestimation of survival.
The problem with linking of death information and lacking emigration information leads to “immortals”: individuals for whom death information will never be retrieved and they appear in the register to live forever. Immortals are likely to have a larger impact on long-term survival than short-term survival, since problems with linkages gets larger over time as individuals change address or name, and a proportion of those still alive move out of the jurisdiction. It is still unclear as to the extent of this problem in international comparisons of survival. Registries can evaluate the potential for deaths to be missed, particularly when a country has a number of subnational registries. We recommend that registries review a random sample of long-term survivors, especially for cancers with poor prognosis, to try to establish if the vital status is correct. A review of all individuals alive beyond the age of 100 can also be done, and some registries do this as part of annual quality checks. Another indirect method to assess the accuracy of follow-up information is to estimate relative survival conditional on survival up to a number of years based on the number of years after diagnosis that statistical “cure” occurs for cancer under study. If the conditional survival is greater than 1, surviving cancer patients may have a better prognosis than would be expected, which could be due to missing linkages to deaths information.
International population-based cancer survival comparisons are important to benchmark the overall effectiveness of cancer management across countries. As differences in registration practice may impact on survival estimates and survival comparisons, registries should describe the specific situation with respect to the five areas of registry practice outlined herein studies to inform the results presented. Based on the ICBP SURVMARK-2 benchmarking studies  investigating the impact of these differences, they are only likely to explain a part of the observed survival differences. Even so, it is important to keep in mind potential biases based on registry practice, and understand how differences between registries likely impact on survival estimates. Standards and guidelines help to minimise the differences but factors such as legal aspects, national infrastructure or resources, e.g. for traceback, will lead to differences in registration practice. Since many sources of bias are due to missing information, such as cases not reported or cases assumed alive due to missing deaths information, it is not possible to reliably adjust or correct the data for the registration differences. Instead, one should try to understand how each issue dealt with in each registry, and undertake sensitivity analyses where possible.
Benchmarking cancer survival has been a driver for change in many individual jurisdictions; informing cancer service planning and policy leading to survival improvements over time. We have made some recommendations based on the experience of international survival comparison studies that seek to improve the quality of future benchmarking iterations, whilst also improving the validity of internal cancer survival estimates in each jurisdiction. A summary of the recommendations can be found in Fig. 1. These recommendations will improve comparability whilst maintaining the opportunity to understand and act upon international variations in outcomes among cancer patients.
Allemani C, Weir HK, Carreira H, Harewood R, Spika D, Wang XS, et al. Global surveillance of cancer survival 1995–2009: analysis of individual data for 25,676,887 patients from 279 population-based registries in 67 countries (CONCORD-2). Lancet. 2015;385:977–1010.
De Angelis R, Sant M, Coleman MP, Francisci S, Baili P, Pierannunzio D, et al. Cancer survival in Europe 1999–2007 by country and age: results of EUROCARE-5-a population-based study. Lancet Oncol. 2014;15:23–34.
Coleman MP, Forman D, Bryant H, Butler J, Rachet B, Maringe C, et al. Cancer survival in Australia, Canada, Denmark, Norway, Sweden, and the UK, 1995–2007 (the International Cancer Benchmarking Partnership): an analysis of population-based cancer registry data. Lancet. 2011;377:127–38.
Arnold M, Rutherford MJ, Bardot A, Ferlay J, Andersson TM, Myklebust TA, et al. Progress in cancer survival, mortality, and incidence in seven high-income countries 1995–2014 (ICBP SURVMARK-2): a population-based study. Lancet Oncol. 2019;20:1493–1505.
https://gco.iarc.fr/survival/survmark/index.html. Accessed 20201029.
Cabasag CJ, Butler J, Arnold M, Rutherford M, Bardot A, Ferlay J, et al. Exploring variations in ovarian cancer survival by age and stage (ICBP SurvMark-2): a population-based study. Gynecol Oncol. 2020;157:234–244.
Araghi M, Arnold M, Rutherford MJ, Guren MG, Cabasag CJ, Bardot A, et al. Colon and rectal cancer survival in seven high-income countries 2010–2014: variation by age and stage at diagnosis (the ICBP SURVMARK-2 project). Gut. 2020;70:114–26.
Morgan E, Soerjomataram I, Gavin AT, Rutherford MJ, Gatenby P, Bardot A, et al. International trends in oesophageal cancer survival by histological subtype between 1995 and 2014. Gut. 2020;70:234–42.
Pilleron S, Charvat H, Araghi M, Arnold M, Fidler-Benaoudia MM, Bardot A, et al. Age disparities in stage-specific colon cancer survival across seven countries: an ICBP SURVMARK-2 population-based study. Int J Cancer. 2020;148:1575–85.
Myklebust TA, Andersson T, Bardot A, Vernon S, Gavin A, Fitzpatrick D, et al. Can different definitions of date of cancer incidence explain observed international variation in cancer survival? An ICBP SURVMARK-2 study. Cancer Epidemiol. 2020;67:101759.
Andersson TM, Myklebust TA, Rutherford MJ, Moller B, Soerjomataram I, Arnold M, et al. The impact of excluding or including Death Certificate Initiated (DCI) cases on estimated cancer survival: a simulation study. Cancer Epidemiol. 2021;71:101881.
Andersson TM, Rutherford MJ, Myklebust TA, Moller B, Soerjomataram I, Arnold M, et al. Exploring the impact of cancer registry completeness on international cancer survival differences: a simulation study. Br J Cancer. 2020;124:1026–32.
Rutherford MJ, Moller H, Lambert PC. A comprehensive assessment of the impact of errors in the cancer registration process on 1- and 5-year relative survival estimates. Br J Cancer. 2013;108:691–8.
Moller H, Richards S, Hanchett N, Riaz SP, Luchtenborg M, Holmberg L, et al. Completeness of case ascertainment and survival time error in English cancer registries: impact on 1-year survival estimates. Br J Cancer. 2011;105:170–6.
Robinson D, Sankila R, Hakulinen T, Moller H. Interpreting international comparisons of cancer survival: the effects of incomplete registration and the presence of death certificate only cases on survival estimates. Eur J Cancer. 2007;43:909–13.
Woods LM, Coleman MP, Lawrence G, Rashbass J, Berrino F, Rachet B. Evidence against the proposition that “UK cancer survival statistics are misleading”: simulation study with National Cancer Registry data. BMJ. 2011;342:d3399.
Jensen OM, Parkin DM, MacLennan R, Muir CS, Skeet RG. Cancer registration: principles and methods. Lyon, France: International Agency for Research on Cancer; 1991.
Parkin DM, Bray F. Evaluation of data quality in the cancer registry: principles and methods Part II. Completeness. Eur J Cancer. 2009;45:756–64.
Bray F, Parkin DM. Evaluation of data quality in the cancer registry: principles and methods. Part I: comparability, validity and timeliness. Eur J Cancer. 2009;45:747–55.
European Network of Cancer Registries. Standards and Guidelines for Cancer Registration in Europe. 2003. Available from: https://www.iarc.fr/en/publications/pdfs-online/treport-pub/treport-pub40/IARC_Technical_Report_No40.pdf [accessed 18 April 2018].
Working Group R. International rules for multiple primary cancers (ICD-0 third edition). Eur J Cancer Prev. 2005;14:307–8.
Adamo M, Dickie L, Ruhl J. SEER program coding and staging manual. National Cancer Institute, Bethesda, MD 2016;20850–9765.
Young JLJ, Roffers SD, Ries LAG, Fritz AG, Hurlbut AA. SEER summary staging manual - 2000: codes and coding instructions. NIH Pub. No. 01-4969. Bethesda: National Cancer Institute; 2001.
Wohler B, Qiao B, Weir HK, MacKinnon JA, Schymura MJ. Using the National Death Index to identify duplicate cancer incident cases in Florida and New York, 1996–2005. Prev Chronic Dis. 2014;11:E167.
Tilling K, Sterne JA. Capture-recapture models including covariate effects. Am J Epidemiol. 1999;149:392–400.
Schouten LJ, Straatman H, Kiemeney LA, Gimbrere CH, Verbeek AL. The capture-recapture method for estimation of cancer registry completeness: a useful tool? Int J Epidemiol. 1994;23:1111–6.
Beral V, Peto R. UK cancer survival statistics. BMJ. 2010;341:c4112.
Eden M, Harrison S, Griffin M, Lambe M, Pettersson D, Gavin A, et al. Impact of variation in cancer registration practice on observed international cancer survival differences between International Cancer Benchmarking Partnership (ICBP) jurisdictions. Cancer Epidemiol. 2019;58:184–192.
Landtblom AR, Bower H, Andersson TM, Dickman PW, Samuelsson J, Bjorkholm M, et al. Second malignancies in patients with myeloproliferative neoplasms: a population-based cohort study of 9379 patients. Leukemia. 2018;32:2203–10.
The authors would also like to thank Lucie Hooper, Samantha Harrison, Charles Norell, Shanta Keshwala, Charlotte Lynch, Mathura Vithyananthan and Harriet Hall of Cancer Research UK for managing the programme. The ICBP SurvMark-2 Academic Reference Group for providing independent peer-review and advice, especially Mike Eden for valuable comments on the final draft of the manuscript. Finally, we are thankful to the ICBP Programme Board for their oversight and direction.
The ICBP is funded by the Canadian Partnership Against Cancer; Cancer Council Victoria; Cancer Institute New South Wales; Cancer Research UK; Danish Cancer Society; National Cancer Registry Ireland; The Cancer Society of New Zealand; NHS England; Norwegian Cancer Society; Public Health Agency Northern Ireland on behalf of the Northern Ireland Cancer Registry; DG Health and Social Care, Scottish Government; Western Australia Department of Health; Public Health Wales NHS Trust. Open access funding provided by Karolinska Institute.
Ethics approval and consent to participate
Consent to publish
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Andersson, T.ML., Myklebust, T.Å., Rutherford, M.J. et al. Five ways to improve international comparisons of cancer survival: lessons learned from ICBP SURVMARK-2. Br J Cancer 126, 1224–1228 (2022). https://doi.org/10.1038/s41416-022-01701-0