Introduction

Mortality is one of the most important outcomes for surgical procedures and one this easily defined, at least during a patient admission episode. However, perioperative mortality rates in themselves are difficult to interpret without risk stratification1,2. An accurate risk stratification tool is a key-player in the perioperative diagnostic/therapeutic pathway and enables meaningful comparison of surgical outcomes by adjusting for specific risk3. Almost 313 million surgical procedures are performed each year worldwide4,5, with at least 77,2 million disability-adjusted life-years (DALYs) avoidable by basic, life-saving surgical care6. Surgical needs account for 28–32% of the overall global burden of diseases: surgical activity and complexity will further increase in the near future mainly due to higher life expectancy and increase of chronic comorbidities7,8. In 2050, 16% of the world’s population will be older than 65 years, with ageing impacting surgical activity9. In this demographic transition, correct estimates of comorbidities and risk factors are expected to play a role in efficient decision-making. Technology advancements (Intensive Care Unit, minimally invasive surgery, robotics, hybrid interventions, and advanced anesthesiology procedures) make almost every patient fit-for-surgery10. Benchmarking of surgical outcomes across hospitals can help refining standards for estimating surgical risk but possible advantages are limited by different methods adopted for risk stratification and require appropriate adjustment models because postoperative inpatient mortality reflects individual risk factors, surgery complexity and data collection11.

Different scoring systems exist and are currently used both in clinical and perioperative settings (see Table 1)12,13,14,15,16,17,18, each considering different sets of pre-operative predictors. A predictive score should have some mandatory features: feasibility at the bedside, reproducibility and reliability. Moreover, a perioperative score should be able to stratify the risk according to different surgical specialties.

Table 1 Factors included in the different risk scores: ■factor shared in the different scores; uniquely present; included in comorbidities.

The aim of this study was to assess surgical mortality and generate a peri-operatory score system, SMATT score (Surgical Mortality Assessment & Stratification) pre-operative risk estimation using readily available and multidimensional patients, and structural characteristics in elective and emergency surgical procedure.

Methods

Study design and patient population

We carried a retrospective analysis of prospectively collected data from a large cohort of patients undergoing surgery in our Institution from 2002 to 2014. The study endpoint was perioperative hospital 30 days mortality for any cause following one or more surgical procedures. The inclusion criteria were: a) age 16 or more; b) surgery in one of the surgical wards of our hospital: cardiac, general and vascular surgery, neurosurgery, gynecology, obstetrics, ear-nose-throat surgery, urology, ophthalmology, and orthopedics.

The final aim was the generation of a risk scoring system (SMATT score) based on the analysis of pre-operative data predicting mortality in elective and emergency patient.

All patients included in the present study have signed the consent for using individuals data.

Data source

Data were retrieved from the 3 electronic registries: 1) the overall administrative database, where data pertaining to the hospital discharge form, including patients demographics, clinical data and procedures that were coded according to the ICD9-CM (International Classification of Diseases, 9th revision - Clinical Modification) are stored; 2) the operating theatre registry that collects data pertaining to the surgical act; 3) the anesthesiology registry with data regarding preoperative, intra- and perioperative clinical assessment of the patient19. Further perioperative deaths occurring after discharge but within 30 days were retrieved from the regional administrative databases from Italian Heath system.

Planned surgical complexity was defined according to NHS guidelines Preoperative Tests20; anesthesiology complexity was graded on the basis of the ASA physical status classification system21. The database structure and relationships are presented in the Supplemental Digital Content 2. As a single patient could have had repeated hospitalizations and more than one surgical procedure, each one of the latter was identified by a progressive number within a hospitalization nosology number and a patient’s unique personal identifier.

Statistical analysis

All statistical analyses were performed using STATA 13.1 statistical software (Stata Corporation; College Station, TX, USA). The level of significance was set at the two-tailed P-value <0.05. The Bonferroni correction was used for post-hoc comparisons within multiple categories variables.

Mean and standard deviation (SD), or median and 25th-75th percentiles were used to describe continuous variables, count, and percent for categorical variables. Perioperative mortality was reported with exact binomial 95% confidence interval (95%CI). For the purpose of the analysis, age was categorized into 4 groups and BMI into 3 groups. Comorbidities were considered both by type and after grouping by apparatus. A series of candidate predictors for mortality (Table 2) were assessed by means of univariable logistic models. The odds ratio (OR) and 95%CI was computed to measure the strength of the association. Given the large number of variables assessed, a consensus between the authors, including surgeons, hygiene, and public health specialists, bioengineer and biostatistician identified clinically meaningful candidate predictors (Table 3) to be included in a multivariable logistic model. The absence of collinearity between predictors was assessed before fitting the multivariable model. The statistical unit was hospitalization; thus Huber-White robust standard errors were computed while clustering on patients to account for the lack of independence of measures. Model discrimination was assessed with the area under the model Area Under the Curve-Reciever Operating Characteristic (AUC-ROC). Ten-fold cross-validation was performed to validate the model, and the discrimination AUC-ROC statistic was recalculated for confirmation. Finally, the linear predictor (or prognostic score) was calculated as the linear combination of the regression coefficients with the values of the co-variables for each patient in the dataset. For an easier calculation of the score, we multiplied each regression coefficient by 100 to obtain integers and built the score to a range from 0 to 100 using normalization. A new logistic model was then fitted, with the prognostic score as the sole predictor and model % of correctly predicted cases together with computed predictive values. Probabilities of death given the vector of predictors were derived from the model and plotted against the score.

Table 2 Univariable analysis.
Table 3 Multivariable Model, Number of obs = 171060; Wald chi2(42)= 7075.55 Prob> chi2 = <0.001 Model discrimination (10-fold cross-validation Model AUC-ROC) = 0.941, 95%CI 0.939–0.944.

Results

Study population and outcome

During the study, 121.290 patients underwent 173.017 surgeries in 164.153 admissions. The mean age at admission was 54 years (SD ± 19); 46% were male. The mean BMI was 25 kg/m2 (SD 4): 83% of patients were normal or over-weight, 4% underweight and 13% of patients were obese. Patient hospital mortality was 2.56% (95%CI 2.47% to 2.64–3,099 patients), with a mortality of 1.89% (95%CI 1.82–1.95%) over all the 164,153 admissions.

Correlates of mortality (univariable analysis)

Results of the univariable analyses are provided in Table 2. Most of the tested predictors were associated significantly with in-hospital death following surgery. Mortality was almost twice higher in males than in females; it increased with age and ASA score, but decreased over BMI classes. The presence of comorbidities increased the risk of death: overall, the risk was the highest in the presence of cardiovascular diseases (fourfold increase) and neurological diseases (threefold increase). Specifically, patients with heart failure, pulmonary hypertension, acute kidney injury, cirrhosis or coma showed the highest mortality, with a 6- to a 21-fold increase in risk, while patients with known drug intolerance demonstrated lower mortality (see table, Supplemental Digital Content 3). When considering surgery-related variables, mortality was the highest (8%) for cardiac surgery, followed by neurosurgery (4.1%), whereas it was lower (0.1%) for gynecology, obstetrics, and ophthalmology. Finally, the risk significantly increased with planned surgical complexity (grades 3: 2% and grade 4: 5%), procedure priority (emergency: 23.3%, urgent: 4.3%, urgent-off-time: 7.8%) and the number of re-do surgeries during the same admission ≥2 (from 10.4% to 25.5%).

Multivariable analysis and derivation of the prognostic SMATT score

The results of multivariable model are detailed in Table 3 and illustrated in Fig. 1. Age, BMI, ASA score, department, planned surgical complexity, surgical priority, previous surgeries in the same hospitalization, cardiovascular, pulmonary, hepato-renal comorbidities, known drug intolerance, cancer, and AIDS were independently associated with mortality after surgery. Model discrimination was optimal (AUC-ROC 0.940, 95%CI 0.938–0.943) and was confirmed after 10-fold cross-validation (AUC-ROC 0.941, 95%CI 0.9439–0.944). The prognostic score was then calculated from the estimated coefficient in the multivariable model, as described in the statistical analysis section. The final score ranged from 0 to 100. The SMATT score confirmed an excellent discrimination ability at 10-fold cross-validation (AUC-ROC 0.945, 95%CI 0.941–0.948). It was able to correctly classify 98.5% of admissions as having a probability of death>50%. The positive and negative predictive values, given the observed mortality rate of 1.89% of all admissions, were 61.4% and 97.99%, respectively.

Figure 1
figure 1

Forrest plot derived from the logistic multivariable model. The estimated odds ratios (OR) are shown as dots, and the 95% confidence intervals as whiskers. The reference category is shown with a dot only on the no effect line (OR = 1). Whiskers to the right of this line correspond to an increased risk of death, with respect to the reference category; whiskers to the left, correspond to a lesser risk than the reference category; whiskers crossing the no effect line denote lack of statistical significance.

Based on these results, we created a dedicated app to assist physicians in calculating the SMATT score at bedside and yielding the probability of perioperative death, together with a nomogram, providing graphical assistance to estimate the probability of death. The app is available on Google Play at following link https://play.google.com/store/apps/details?id=appinventor.ai_valentinafavalli.SMATT. Figure 2 shows the SMATT score calculated with the app together with the corresponding probability of perioperative death in three simulated patients with low, intermediate and high risk (panel A, B and C, respectively). As illustrated in the accompanying nomogram, the probability of death if minimal for SMATT scores up to 50; it increases steadily and linearly for scores above 50, and up to more than 95% for SMATT = 100.

Figure 2
figure 2

SMATT calculation and probability of death is obtained by the implementation of individual data in the app. Two hypothetical scenarios are shown, with the SMATT score calculated through the app. The derived probability of perioperative death is 0% in panel A (SMATT = 36); 3% (SMATT = 54) in panel B and 83% (SMATT = 88) in Panel C. Panel A). The patient is a 71-years-old woman, with a prior history of ischemic cardiomyopathy and COPD, chronic pancreatitis, peptic ulcer. She was hospitalized for a femoral fracture. Panel B). The patient is a 45-years-old woman, underweight, with a prior history of chronic gastritis, breast cancer, peripheral nervous system disease, osteoarthritis. She has an ASA score 3, she was hospitalized for an elective hysteroannessiectomy. Panel C). The patient is a 54-years-old man, normo-overweight, with a prior history of ischemic cardiomyopathy and COPD, cirrhosis, esophageal varices, AIDS, colon cancer, hyperparathyroidism. He has an ASA score 4, he was hospitalized for an emergency laparotomy for an intestinal obstruction.

Discussion

This large surgical cohort allowed us to build a prognostic score (Surgical Mortality Assessment & sTraTification - SMATT score) to be computed preoperatively at bedside, in order to stratify elective and emergency patients according to their risk of perioperative mortality. SMATT showed excellent discrimination and particularly, given the high negative predictive value, it was able to identify low risk patients who possibly would not require high intensity post-operative care. SMATT applies for surgical risk regardless the kind of surgery integrating also multiple patient’s clinical features. SMATT is easy to use, given the proposed associated app, which can be downloaded on a hand held device. Some concern may arise from the low positive predictive value of the SMATT score, making the identification of patients with higher risk more difficult. This issue is, however, shared by all risk scores in the field, given the low rates of perioperative mortality in Western Countries.

Age, BMI, ASA score, surgical specialty, planned surgical complexity, priority (elective, urgent and emergency) and previous surgeries during the same hospital admission, cardiovascular, pulmonary, hepato-renal comorbidities, unknown drug intolerance, cancer and AIDS were independently associated with mortality after surgery. Interestingly, whereas factors such as age and BMI were expected to be associated with surgical mortality22,23,24,25,26, our results confirm previous reports, showing that obesity was not a predictive factor for postoperative mortality, while underweight patients demonstrated a higher risk of mortality27,28,29. This is agreeing with those studies: a recent cohort study, including 401227 adult patients (National Adult Cardiac Surgery registry for all cardiac surgical procedures; April 2002- March 2013) and a meta-analysis through June 2015, including 557720 patients, showing that obesity was associated with lower risks after cardiac surgery30.

Although a number of risk-scoring systems have already been published12,13,14,15,16,17,18, most of them were related to specific procedures and surgical specialties. The collective assessment of both patient and structural characteristics may help in preoperative stratification of mortality risk that results from a combination of preoperative, intra-operative and postoperative factors, related to the current surgical event. Although each of these factors may have several standardized features and pathways (i.e. pre-operative diagnosis/indication and kind of surgical procedure), the patient’s features may vary widely and thus require a multidimensional evaluation. Also, using SMATT may lead to a physician-patient communication process more effective31.

A common issue when presenting a new score is its value in other settings. Our 10-fold cross-validation procedure confirmed the high discrimination ability of SMATT. Its extensive use in similar or different hospital settings needs further confirmation on independent case series. Given the results of our validation through simulation, we are confident that the prediction of perioperative mortality can be applied in multi-specialty hospitals, with a similar case-mix index as ours of 1.52, and an overall high complexity of organization and care20.

This is a retrospective single center study, however, a prospective data collection in three dedicated electronic registries for the administrative, anesthesiology and surgical data, has been systematically implemented in our institution since 2002, offering sufficient and exhaustive information on each single patient. Data analysis did not include costs and budget/resources assigned to the different surgical and anesthesiology teams that may have influenced the outcome. This could have been true in a multi-center study; it is however not the case in a single research hospital such as ours. We are planning to perform a multicentric study to apply the external validation.

We propose the SMATT score as a novel tool for stratification of surgical mortality, amenable for application in large multi-specialty hospitals. Its easy use through a hand-held device will allow a personalized scoring of risk at the pre-operative assessment, based on a multi-dimensional approach. Benchmarking evaluation could help further validate and confirm its potential role and easy application.

Ethics approval and consent to participate

All patient submit authorization for the processing of personal and sensitive data according to the local procedure approved by ethical board of IRCCS Policlinico San Matteo Foundation, Pavia, Italy.

All patients provided ‘informed consent’ for the use of their data in the present study.

The study design was approved by the Ethics Committee of IRCCS Fondazione Policlinico San Matteo with a ref. number 027/2017.

Consent for Publication

All the personal data included in the present work have the consent for publication.

Permission to use the SMATT score logos

We obtain permission to use SMATT score logos in Fig. 2 from Fondazione IRCCS Policlinico San Matteo, the logos do not have any copyright.