North American population-based validation of the National Comprehensive Cancer Network Practice Guideline Recommendations for locoregional lymph node and bone imaging in prostate cancer patients

Background The National Comprehensive Cancer Network (NCCN) guidelines provide recommendations for staging of prostate cancer patients in the objective regarding presence of locoregional lymph node metastases (LNM) and bone metastases. We tested the performance characteristics of these recommendations in a community setting. Methods Within the Surveillance, Epidemiology, and End Results database (2004–2014), we identified patients with available Gleason, clinical stage and prostatic specific antigen. Performance characteristics endpoints consisted of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NVP), overall accuracy and the number needed to image (NNI). Results Totally, 191,308 patients were assessable for the validation of the LNM staging recommendations. Sensitivity ranged from 80.6 to 86.3%, specificity from 74.7 to 79.3%, PPV from 7.8 to 8.0%, overall accuracy from 75.0 to 79.3% and NPV was 99.5%. The respective NNI values were 12.5 and 12.8. 197,408 patients were assessable for the validation of bone scan recommendations. These recommendations resulted in 90.8% sensitivity, 76.3% specificity, PPV of 5.7%, NPV of 99.8% and overall accuracy of 76.5%. The NNI was 17.5. Conclusion The NCCN recommendations for locoregional LNM miss few patients with clinical LNM (0.3–0.4%) and provide a virtually perfect NPV of 99.5%. Also, the recommendations for bone scan miss a marginal number of patients with established bone metastases (0.14%) and yield a virtually perfect NPV of 99.8%.


INTRODUCTION
Clinical staging in the context of prostate cancer (PCa) is important. 1 Treatment of individuals with locally advanced or metastatic PCa differs from that for individuals with localised PCa. The distinction between localised PCa vs. locoregional lymph node metastatic PCa and PCa metastatic to bone can be made using the recommendations of the National Comprehensive Cancer Network (NCCN), as outlined in the clinical practice guidelines in oncology. 2 Specifically, for staging of patients with suspected locoregional lymph node metastases (LNM) the NCCN PCa guideline (Version 2.2017) recommends a pelvic computed tomography scan (CT) or pelvic magnetic resonance imaging (MRI) if clinical tumour stage is T3 or T4 or in clinical tumour stage T1 or T2 when nomogram derived LNM probability is >10%. 2 Similarly, for staging of patients with suspected bone metastases, the NCCN PCa guideline recommends a bone scan in clinical tumour stage T1 patients when prostatic specific antigen value (PSA) is >20 ng/ml or if clinical tumour stage is T2 when PSA is >10 ng/ml or if clinical tumour stage is T3 or T4 or if Gleason score is ≥8 or if patients are symptomatic. 2 However, to the best of our knowledge these recommendations have never been validated in a community setting within a large epidemiological database, such as the Surveillance, Epidemiology, and End Results (SEER) database. Within the current manuscript, we tested the performance characteristics of the NCCN guidelines regarding presence of locoregional LNM, as well as regarding presence of bone metastases. Specifically, we hypothesised that the use of the NCCN guidelines will not result in more than a marginal proportion of patients with missed locoregional LNM or missed bone metastases. Testing of the NCCN recommendations We performed two separate analyses. First, we tested the NCCN guideline recommendations for identification of patients with positive clinical lymph node status (cN1). 4 The NCCN recommends a pelvic CT/MRI if clinical tumour stage is T3 or T4 or if clinical tumour stage is T1 or T2 and the nomogram derived probability of LNM is >10%. Since two different nomograms can be applied and no specific recommendation is made for the use of one vs. the other, we performed two separate analyses. 5,6 The first analysis relied on the use of the updated Briganti et al. 5 nomogram that includes biopsy core information. The second analysis relied on the use of the online Memorial Sloan Kettering Cancer Center (MSKCC) dynamic prostate cancer nomogram that includes biopsy core information. 6 Only patients with known positive clinical lymph node status (cN1) or negative clinical lymph node status (cN0) were included. These selection criteria resulted in 191,308 assessable patients for testing of the NCCN recommendations.
Second, we tested the NCCN guideline recommendations for identification of patients with bone metastases. The NCCN recommends a bone scan if clinical tumour stage is T1 and PSA > 20 ng/ml or if clinical tumour stage is T2 and PSA > 10 ng/ml or if clinical tumour stage is T3 or T4 or if Gleason Score is ≥8 (GGG ≥ 4). Only patients with known bone metastases (M1b) or without metastases (M0) were included. These selection criteria resulted in 197,408 assessable patients for testing the NCCN bone scan recommendations.

Statistical analyses
Descriptive statistics consisted of frequencies and proportions for categorical variables. Means, medians and ranges were reported for continuously coded variables. The chi-square tested the statistical significance of proportions' differences. The t test and Kruskal-Wallis test examined the statistical significance of means and medians differences, respectively.
The specific performance characteristics for each of the guideline recommendations consisted of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NVP), overall accuracy and the number needed to image (NNI). Finally, the positive (sensitivity/(1-specificity)) and negative likelihood ratios ((1 − sensitivity)/specificity) for each recommendation were calculated. R software environment for statistical computing and graphics (version 3.4.0) was used for all statistical analyses. All tests were two sided with a level of significance set at p < 0.05.
Validation with the use of the Briganti nomogram. Within the use of the Briganti nomogram to predict the LNM probability >10%,   Higher sensitivity of the MSKCC nomogram could be related to the fact that the Briganti nomogram was developed within a European tertiary referral center. 5 Conversely, the MSKCC nomogram originates from an American tertiary care center. 6 Despite these small differences, the performance characteristics recorded for both nomograms indicate an excellent balance between sensitivity and specificity that is expected from a robust testing recommendation. Moreover, the use of both nomograms resulted in similar NNI values, which were, respectively, 12.5 for the Briganti nomogram vs. 12.8 for the MSKCC nomogram. Finally, the positive and negative likelihood ratios with the use of the Briganti vs. the MSKCC nomogram were similar, 3.89 and 0.24 vs. 3.41 and 0.18, respectively. These results demonstrate an acceptable confidence for both nomograms regarding the NCCN recommendations to perform a pelvic CT/MRI. Taken together, our data validate the use of the NCCN guideline recommendations for identification of LNM with either the MSKCCC or the Briganti nomogram. The choice of a specific nomogram may be left to the discretion of individual physicians.
Second, regarding the NCCN guideline recommendations for presence of bone metastases, we relied on 197,408 assessable patients. Within those individuals the guidelines did not recommend a bone scan in 148,600 (75.3%) patients. Of those, only 281 or 0.14% did harbour bone metastases and would be considered as missed instances of bone metastases. This marginally low number and proportion of patients resulted in a virtually perfect NPV of 99.8%. In consequence, we confirmed that the NCCN guideline recommendations for bone metastases imaging are safe and result in a marginal proportion of omitted instances of imaging, despite the presence of bone metastases.
We also examined the guideline recommendations for bone imaging regarding their sensitivity and specificity. Here, a sensitivity of 90.8% was recorded in individuals with established bone metastases. Conversely, a specificity of 76.3% was recorded in individuals without bone metastases. These values indicate an excellent balance between sensitivity and specificity that can be expected from a well-balanced testing recommendation. The sensitivity in our analyses was noticeably higher than the one reported previously by Merdan et al. (82.3%), who tested the bone scan recommendations from the NCCN guidelines within the Michigan Urological Surgery Improvement Collaborative (MUSIC) clinical registry. 7 However, Merdan et al. relied on a smaller cohort (n = 1509) compared to the current study (n = 197,408). The recorded sensitivity in the current study was also higher than the one reported by Briganti et al. 8 (79.2%), who relied on an even smaller (n = 853) and more historical cohort (2003-2008) from a single European tertiary referral center. Taken together, the bone scan recommendations from the NCCN guidelines demonstrate a high sensitivity in our large patient cohort, which results in a low proportion of missed patients that harbour bone metastases. However, the NNI of 17.5 demonstrates that the staging recommendations from the NCCN guidelines are not perfect in distinguishing between patients with or without bone metastases. Finally, the positive-and negative-likelihood ratios of 3.83 and 0.12, respectively, demonstrate acceptable confidence for clinicians to rely on the NCCN guideline recommendations when to perform bone imaging.
Taken together, our analyses demonstrated that the NCCN guideline staging recommendations for prediction of LNM, as well as those for prediction of bone metastases result only in a marginal number of missed patients with either locoregional LNM or bone metastases in a large North American community setting, such as the SEER database. This is an important information, since it demonstrates that with the use of these recommendations most patients with metastatic disease can be identified. To the best of our knowledge, this is the first report of this kind and further studies are welcome to confirm our results.
Our study is not devoid of limitations. First and foremost, the SEER database only represents approximately 30% of the United States population, which is a limitation for itself. 9 Moreover, the NCCN guidelines do not provide recommendations for abdominal CT/MRI staging and thereby we were not able to include patients with visceral metastases or non-locoregional LNM in our analyses. However, such patients need also be identified and deserve to be studied, since their outcome and treatment differs from those of patients with localised PCa. Additionally, the SEER database does not capture information in regards of symptoms (e.g., bony pains) or laboratory abnormalities (e.g., elevated alkaline phosphatase) and as such, the current study cannot account for cases that might be diagnosed based on this information. Last but not least, both recommendations are based on gold standards that maybe suboptimal. 10,11 Specifically, bone scans have recently been shown to miss bone metastases. 12,13 Similarly, pelvic CT/MRI have recently been shown to miss locoregional LNM. [14][15][16] In consequence, future gold standards will need to be reassessed. However, to date only studies with very limited sample sizes are available, when new gold standards such as prostate specific membrane antigen scans are compared to routine imaging. 15 In consequence, population-based studies relying on prostate specific membrane antigen-based testing  cannot yet be performed. Limited availability of prostate specific membrane antigen testing will persist for next several years and will hamper such efforts for at least next decades.

CONCLUSION
The NCCN based recommendation for locoregional LNM miss very few patients with clinical LNM (0.3-0.4%), regardless of the applied methodology and provide a virtually perfect NPV of 99.5% and acceptable negative likelihood ratios (0.18-0.24). The NCCN bone scan recommendations also miss a marginal number of patients with established bone metastases (0.14%) and yield a virtually perfect NPV of 99.8% and a very acceptable negative likelihood ratio of 0.12. In consequence, both NCCN recommendations may be safely endorsed in clinical practice.