Development and Validation of a Risk Scoring System for Cephamycin-Associated Hemorrhagic Events

Cephamycin-associated hemorrhages have been reported since their launch. This research aimed to determine risk factors for cephamycin-associated hemorrhagic events and produce a risk scoring system using National Taiwan University Hospital (NTUH) database. Patients who were older than 20 years old and consecutively used study antibiotics for more than 48 hours (epidode) at NTUH between January 1st, 2009 and December 31st, 2015 were included. The population was divided into two cohorts for evaluation of risk factors and validation of the scoring system. Multivariate logistic regression was used for the assessment of the adjusted association between factors and the outcome of interest. Results of the multivariate logistic regression were treated as the foundation to develop the risk scoring system. There were 46402 and 22681 episodes identified in 2009–2013 and 2014–2015 cohorts with 356 and 204 hemorrhagic events among respective cohorts. Use of cephamycins was associated with a higher risk for hemorrhagic outcomes (aOR 2.03, 95% CI 1.60–2.58). Other risk factors included chronic hepatic disease, at least 65 years old, prominent bleeding tendency, and bleeding history. A nine-score risk scoring system (AUROC = 0.8035, 95% CI 0.7794–0.8275; Hosmer-Lemeshow goodness-of-fit test p = 0.1044) was developed based on the identified risk factors, with higher scores indicating higher risk for bleeding. Use of cephamycins was associated with more hemorrhagic events compared with commonly used penicillins and cephalosporins. The established scoring system, CHABB, may help pharmacists identify high-risk patients and provide recommendations according to the predictive risk, and eventually enhance the overall quality of care.

Therefore, this study aims to define the risk factors for cephamycin-associated hemorrhage using NTUH electronic medical record (EMR), and to establish a risk scoring system that can be adapted in clinical settings to enhance the safe use of cephamycin.

Methods
Data source. Data used in this study was obtained from the NTUH EMR. The database comprises person-level records of demographic information, medication utilizations in NTUH, medical history, laboratory examinations and imaging study results. This retrospective study was approved by Research Ethic Committee of NTUH with the waiver of informed consent. The research team was authorized to have the access to NTUHEMR (REC Number: 201312061RINB). All methods were carried out in accordance with relevant guidelines and regulations.
Study design and subjects. A retrospective cohort design was applied. Patients aged ≥20 years who continuously used injection forms of the study antibiotics for ≥48 hours in the emergency or inpatient department in NTUH during January 1 st , 2009 to December 31 st , 2015 were included. Study antibiotics included cephamycins (cefmetazole, flomoxef, cefoxitin and cefoperazone) and reference antibiotics (amoxicillin/clavulanate, ampicillin/sulbactam, cefuroxime, cefotaxime and ceftriaxone). Reference antibiotics were chosen under the consideration that they should have similar antibacterial spectrum and therapeutic roles with cefmetazole and flomoxef in treating infectious diseases, have no structures known to induce hypoprothrombinemia, and have rare adverse hemorrhagic events reported previously. Data were provided by Department of Pharmacy and Integrated Medical Database at National Taiwan University Hospital (NTUH-iMD).
The first prescription day of study antibiotics was assigned as the index date. The index date plus the following ten days composed the observational period 12,13 . We excluded patients who changed antibiotics from one study antibiotic to another during the observational period. Those who had observational period beyond the study period or prior use of study antibiotics within 30 days of the index date were also excluded. The treatment course was defined as the duration from the index date to the day of discontinuing study antibiotics.
We further divided all identified study subjects into two cohorts based on their index year. The 2009-2013 cohort was used for identifying risk factors and development of the scoring system while the 2014-2015 cohort was used for validation of the scoring system. outcome measurements. Study outcome was the occurrence of any hemorrhagic event during the observational period, which was defined through a two-step process. Firstly, we used the hemorrhagic-related diagnosis codes (International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)) documented within the observational period to detect the suspected hemorrhagic events. These events were further examined by chart review to see if the hemorrhage developed after the initiation of study antibiotics.

covariates.
Covariates including patient demographics, information of index stay, underlying diseases and conditions, concurrent medications/treaments, concurrent antibiotic, and indication of study antibiotics were also collected. Underlying diseases and conditions included chronic hepatic disease, chronic renal diseases, coagulopathy, history of operation, international normalized ratio (INR) prologation, hemorrhagic events, thrombocytopenia, hypoalbuminemia, hepatic dysfunction and renal dysfunction. History of hemorrhagic events was defined by hemorrhagic-related ICD-9-CM codes within 180 days prior to index date or positive stool occult blood test within 7 days prior to index date. Chronic hepatic/renal diseases, and coagulopathy were identified by ICD-9-CM codes coded before index date 14 and the available laboratory data acquired within 7 days prior to the index date 13,15 .
Concurrent medications/treatments within 7 days prior to index date include antiplatelets, anticoagulants, Vitamin K1, total parenteral nutrition (TPN), proton pump inhibitor (PPI), histamine-2 (H 2 ) blockers, tranexamic acid, clotting factors, steroids, nonsteroidal anti-inflammatory drugs (NSAIDs), packed red blood cells (PRBC) and fresh frozen plasma (FFP). We also retrieved any chemotherapy received by the study subjects within 180 days prior to the index date. Use of concurrent antibiotics was defined as any antibiotics other than study antibiotics prescribed for at least 72 hours during treatment course of study antibiotics. Indication of antibiotics was categorized based on infection sites identified through physicians' orders when prescribing antibiotics and diagnosis codes upon discharge.

Statistical analysis.
Descriptive statistics including Chi-square and Mann-Whitney U tests were estimated in both derivation and validation cohorts. Univariate logistic regression was used to examine the unadjusted association between independent variables and the outcome. The significance level in univariate analyses was set at 0.25. Factors shown to be statistically significant in univariate analyses were selected as candidates for multivariate logistic regression. Stepwise strategy with slentry level at 0.15 and slstay level at 0.05 was applied for model selection. Factors with p-value < 0.05 in the adjusted analysis were considered statistically significant and were incorporated into the risk prediction model. The multivariate logistic regression coefficients of each factor were transformed into a risk score by divided the regression coefficients by the smallest regression coefficient among the variables and then rounding the quotients to the nearest integer when developing a risk scoring system.
Model performance consisting of the area under the receivers operating curve (AUROC) and Hosmer-Lemeshow goodness-of-fit test was further assessed. Moreover, the Youden's index was applied in determining the optimal cutoff value of the risk score to best differentiate the outcomes. Cochran-Armitage trend test was used to evaluate if the score and occurrence of outcome were positively correlated. All data were analyzed using Microsoft Excel 2010 (Microsoft Corp., Redmond, WA, USA) and SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

Results patient demographics. The initial cohort included 193151 episodes with exposures to study antibiotics.
After applying all the inclusion and exclusion criteria, there were 69083 episodes identified. The cohort was divided into two sub-cohorts as the 2009-2013 derivation cohort (46402 episodes from 37094 individuals) and the 2014-2015 validation cohort (22681 episodes from 19201 individuals) according to the index date. In the derivation cohort, there were 18821 episodes exposed to cephamycins and 27581 episodes exposed to reference antibiotics. Among them, 227 (1.21%) and 129 (0.47%) hemorrhagic events occurred during exposure to cephamycins and reference antibiotics. Gastrointestinal bleeding was the most common type of hemorrhage events (303/356, 85.11%). In the validation cohort, there were 8615 episodes exposed to cephamycins and 14066 episodes exposed to reference antibiotics. Among them, 116 (1.35%) and 88 (0.63%) bleeding occurred during exposure to cephamycins and reference antibiotics (Fig. 1).
Compared to users of reference antibiotics, cephamycins users were younger (median age 61.5 years in cephamycins vs. 64.7 years in reference antibiotics, p < 0.0001) and more likely to have chronic hepatic diseases (7.91% vs. 4.74%, p < 0.0001). In terms of concurrent medications, cephamycins users were less likely to concurrently receive antiplatelets (4.08% vs. 10.84%, p < 0.0001) and anticoagulants (2.27% vs. 5.73%, p < 0.0001). On the contrary, cephamycins users were prone to concurrently receive vitamin K 1 (1.84% vs. 1.16%, p < 0.0001), TPN  Derivation of risk prediction model for cephamyins-associated hemorrhage. The final risk prediction model with AUROC of 0.8072 contained five variables that were all statistically significantly associated with hemorrhage (p < 0.0001) in the multivariate logistic regression analyses. The regression coefficients of all variables in the risk prediction model were transformed into integer risk scores demonstrated in Table 2. Patients using cephamycins, aged 65 years or above, afflicted with chronic hepatic disease, having history of bleeding   www.nature.com/scientificreports www.nature.com/scientificreports/ within 180 days prior to the index date, and showing bleeding tendency, which was indicated by prescriptions of PPIs, tranexamic acid or clotting factors, within 7 days prior to the index date had an increased risk score by one, two, or four points. A risk scoring system named CHABB (stands for use of Cephamycins, underlying chronic Hepatic diseases, Aged 65 years or above, Bleeding tendency, and Bleeding history in the preceding 180 days of the index date) with 5 items was established and the sum risk scores ranged from 0 to 9.
Validation of risk prediction model for cephamyins-associated hemorrhage. In the evaluation of predictive accuracy of the risk prediction model for cephamycins-associated hemorrhage, CHABB was able to achieve an AUROC of 0.8035 and 0.7550 in the derivation and validation cohorts, respectively, revealing that the sum risk scores lent satisfactory to great validity to the prediction of cephamycins-associated hemorrhage risk. The predictions made from the risk model were in alignment with the observed outcomes suggested by Hosmer-Lemeshow tests (Table 3) and turned out to be the most accurate when all five items were all included (Fig. 2). The cutoff score suggested by Youden's index was 6 points among cephamycins users across both cohorts. As depicted in Fig. 3, the counts of hemorrhagic events notably went up as the risk score increased in both cohorts. It indicated that the risk to developing hemorrhagic events was positively correlated to the risk score, which was statistically significant proved by Cochran-Armitage trend tests (p < 0.0001) as well. Moreover, we further divided patients in both cohorts into three risk groups, including low (with risk scores from zero to three), medium (with risk scores from four to five) and high risk (with risk scores equal to six or above) groups, using the event percentages of 1% and 4% as cut-offs. And the results revealing that the higher the score, the greater the possibility of developing hemorrhagic events had parallel in those who have not been categorized into groups.

Discussion
This is a longitudinal study attempting to identify risk factors for cephamycin-associated hemorrhagic events in a teaching hospital in Taipei. The identified risk factors further served as the foundation for the development of a risk scoring system connecting the use of cephamycins and the possible related hemorrhages. To the best of our knowledge, this is the first study with the largest population in evaluation of the risk factors for cephamycin-associated hemorrhagic events and in development and validation of the nine-score risk score system.
We have identified risk factors for cephamycin-associated hemorrhagic events and established a nine-score risk scoring system named CHABB. Our findings were similar to our previous study conducted by Chen et al. in 2014 using Taiwan's NHIRD 12 . In Chen's study, use of anticoagulants or history of bleeding events in preceding 180 days of the index date, malnourished status and hepatic failure were suggested as risk factors. Among those factors, nutrition status was determined using the surrogate indicator of TPN due to the nature of NHIRD that laboratory data was not available. In our study, nutrition status was determined by serum albumin level, and a normal value was assigned for patients who were lack of records, as was done in Strom et al.'s study 16 . However, hypoalbuminemia was not retained in the final multivariable model in our case. Use of anticoagulants and antiplatelet agents were included in stepwise selection for their clinical importance to the outcome, but they were not retained in the final model. This could be physicians' preference not prescribing cephamycins in patients under long-term anticoagulants or antiplatelet agents to avoid bleeding problems. Also, there may be a healthy-worker www.nature.com/scientificreports www.nature.com/scientificreports/ bias that patients under anticoagulants or antiplatelet agents persistently may be less likely to develop bleeding diatheses.
Some of the factors shown to be related to hemorrhages were interesting but not surprising. We expected anti-secretion agents of PPIs and agents affecting hemostasis such as clotting factors and tranexamic acid to be protective factors. Nevertheless, these factors were risk factors in our study. Similar results were observed in Brown et al. that the use of antiulcer therapy was associated with the highest odds ratio 10 . Considering PPIs, clotting factors and tranexamic acid essentially stabilized the hemostatic process but trended toward higher odds ratios in our regression, we assumed the underlying conditions resulted in the use of these agents were similar. Therefore, we merged the use of either kind of these medications into one variable named bleeding tendency. Judging from the AUROC, the new variable possessed similar ability in explaining the outcome of interest, meanwhile confirming our assumption to some extent.
We chose the scoring system as the presentation format because of its straightforwardness to be used in clinical setting. The indices we adapted to assess the model performance were widely used in building a scoring system [17][18][19][20][21][22][23][24] . Our study has large sample size with relative low incidence of the outcome. Although the calibration result in the 2014-2015 cohort had only a marginal p-value indicating the scoring system fit well, we need to understand that the test was based on the foundation of hypothesis testing. That is, the possibility that we make type II error may not be low and results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggested an appropriate fit of the scoring system.
Despite numerous methodological merits of our study, there were some limitations needed to be mentioned. First, due to its retrospective design, information bias might have occurred in outcome ascertainment. Clinicians may have handled minor hemorrhages before they progressed into serious events. Clinicians may not mention minor events in the discharge notes as well. On the other hand, certain hemorrhagic diagnoses might be part of medical history from previous visits, so they were actually not documented in medical chart. These patients were selected by hemorrhagic-related codes but were then excluded during chart review. Second, we might have included patients who had been prescribed with one of the study antibiotics at medical facilities other than NTUH during the screening period. Third, external validation by a future prospective study would be warranted as it would make this scoring system more relevant for clinical scenario.
In summary, we have established and validated a nine-score risk scoring system to identify potential high risk patients who may develop hemorrhagic events when taking cephamycin. For patients being classified in the high-risk group, we recommend close monitoring of INR and hemorrhagic signs to prevent subsequent catastrophic hemorrhages. In hopes of enhancing the patient care quality, we expect this risk scoring system to be adapted and fully exerted in clinical setting. Future refinement of the scoring system may be considered after the scoring system being prospectively applied clinically for some period of time.