Correspondence Department of Surgery, University of Chicago, 5841 South Maryland Avenue, MC 5093, Chicago, IL 60637, USA

Email afichera@surgery.bsd.uchicago.edu

Original article

Fazio VW et al. (2004) Assessment of operative risk in colorectal cancer surgery: the Cleveland Clinic Foundation colorectal cancer model. Dis Colon Rectum 47: 2015–2024   PubMed

Synopsis

Background

Informed consent is an increasingly important aspect of surgery. Providing patients with an estimate of their operative risk allows them to make knowledgeable decisions as part of this process. Although there are several logistic regression models which provide an estimate of surgical risk, none have been designed specifically for patients undergoing colorectal cancer surgery.

Objective

To develop a statistical model for assessing operative risk in colorectal cancer patients.

Design and intervention

The study used outcomes data from colorectal cancer patients undergoing major surgery at the Cleveland Clinic Colorectal Cancer Center between October 1976 and July 2002, with 60% of the population randomly selected as the basis for the model and the remaining 40% used for validation. Patients lacking data on surgical outcome were excluded. Risk factors for operative mortality were determined by univariate logistic regression. Risk factors with a P value <0.25 underwent multilevel logistic regression analysis, with each factor added to the model in a stepwise fashion. Analysis of the odds ratio (OR), 95% CI and the change in log-likelihood statistic for each variable at every step indicated whether that variable should be included in the final model. The model was divided into two levels of hierarchy, with patient-dependent factors on the first level and surgeon-dependent variables on the second level. The model was validated using the Hosmer–Lemeshow statistic to determine the goodness of fit; the area under the receiver operator characteristic (ROC) curve to assess discrimination and a comparison of actual and predicted 30-day mortality rates to evaluate accuracy.

Outcome measures

The primary outcome measure was all-cause 30-day operative mortality (before or after hospital discharge).

Results

Data were obtained for 5,028 consecutive patients (median age 66 years). The annual 30-day operative mortality rate was 2.3% throughout the study period. Independent risk factors identified were age (OR 1.5 per 10-year increase), American Society of Anesthesiologists grade (ORs for grade II, III and IV-V vs I were 2.6, 4.3 and 6.8, respectively), mode of presentation (OR = 2.1 for urgent vs nonurgent), tumor, node, metastasis cancer staging (OR = 2.6 for Stage IV vs Stage I, II or III), cancer resection (OR = 4.5) and hematocrit (OR = 1.8 for <31 vs 36). Tumor size and fixity, site, operative procedure, history of resection, operative intent or carcinoembryonic antigen level did not significantly affect mortality. The level-two variance indicated no variation in outcomes between surgeons. The model showed a good fit with the data (Hosmer-Lemeshow statistic of 5.221; P = 0.734). The area under the ROC curve was 0.801, indicating sufficient discrimination. The likelihood of mortality predicted by the model correlated well with the actual operative outcome for different procedures.

Conclusion

The Cleveland Clinic Foundation Colorectal Cancer Model (CCF-CCM) provided an accurate assessment of operative risk in colorectal patients undergoing surgery at the Cleveland Cancer Clinic. External validation of the model should be obtained before successful application at other centers.

Commentary

Several multipurpose logistic regression models have been devised during the past two decades to provide an estimate of operative risk in surgical patients.^{1, 2} Fazio et al. are the first to describe a risk-assessment index specifically designed for use in colorectal cancer patients undergoing surgery. They propose an elegant and sophisticated statistical model, the CCF-CCM, as a unique tool to assess the probability of 30-day operative mortality in this population.

The model is based on a large and comprehensive colorectal cancer database of patients treated over a period of almost three decades.

Although the study was very well designed and the analysis based on very solid statistical principles, there are several problems intrinsically associated with such a model. The study centers on the general assumption that data obtained from a highly specialized tertiary center with a high volume of surgical procedures can be used as a measure of operative risk in the wider surgical community. Several studies have, however, demonstrated an inverse relationship between the number of surgical procedures conducted at a center and surgical mortality.^{3, 4} Because the study by Fazio et al. was conducted at an extremely high volume center, the results will need to be individually validated before this model could be applied to centers with a lower volume of surgical procedures. Furthermore, in order to provide a meaningful analysis, a very large study population was required in this study as mortality rates were so low. In addition, model discrimination as assessed by the area under the ROC curve could be biased, owing to the low prevalence of operative mortality in this study. Indeed, the authors suggest that external validation of the model will be needed before it can be applied to other centers with significantly different operative mortality rates.

The variables included in the analysis are easily obtainable preoperatively, even in emergency situations, and they all have prognostic significance; however, for reasons not clearly elucidated in the manuscript, nutritional and immunologic parameters included in other logistic models⁵ were not amongst the variables evaluated. Although such data might not be collected in the emergency setting, they should help to stratify surgical risk better.

Surgical mortality data are difficult to analyze and compare when a surgical study is carried out over three decades. Dramatic improvements in anesthesia, preoperative and postoperative management, surgical instrumentation (i.e. stapling devices) and the introduction of neoadjuvant therapy for rectal cancer over the period of the study complicate analysis of the data and applicability of the results. As valid data for a distinct patient population over a short period of time could only have been obtained from a multicenter source; a subgroup analysis dividing patients into decades could help validate the score in each period of the study.

On the other hand, the parameters included in the model by Fazio et al. are all an integral part of the preoperative evaluation of patients with colorectal cancer, even in an emergency situation. Therefore, after specific validation of this model, surgeons should be able to apply this model easily to their own practice.

The Cleveland Clinic Foundation Colorectal Cancer model needs to be specifically validated and calibrated by individual institutions before meaningful information can be obtained and offered to patients. More importantly, it should not be used as an absolute measure of patient outcomes or a surgeon's competency until a prospective multicenter validation study is carried out.

Acknowledgments

The synopsis was written by Chloe Harman, Associate Editor, Nature Clinical Practice.

References

1. Jones DR et al. (1992) Comparison of POSSUM with APACHE II for prediction of outcome from a surgical high-dependency unit. Br J Surg 79: 1293–1296 | PubMed | ChemPort |
2. Tekkis PP et al. (2003) Evaluation of POSSUM and P-POSSUM scoring systems in patients undergoing colorectal surgery. Br J Surg 90: 340–345 | Article | PubMed | ChemPort |
3. Kee F (1999) Number of cases operated on is important in volume-outcome debate for colorectal cancer. BMJ 319: 576–577 | PubMed | ChemPort |
4. Birkmeyer JD et al. (2002) Hospital volume and surgical mortality in the United States. N Engl J Med 346: 1128–1137 | Article | PubMed | ISI |
5. Knaus WA et al. (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100: 1619–1636 | Article | PubMed | ChemPort |

Contact the journal about this article

Subject areas under which this article appears: Cancer | Surgery