Development and Validation of a Risk Scoring System for Cephamycin-Associated Hemorrhagic Events

Article metrics


Cephamycin-associated hemorrhages have been reported since their launch. This research aimed to determine risk factors for cephamycin-associated hemorrhagic events and produce a risk scoring system using National Taiwan University Hospital (NTUH) database. Patients who were older than 20 years old and consecutively used study antibiotics for more than 48 hours (epidode) at NTUH between January 1st, 2009 and December 31st, 2015 were included. The population was divided into two cohorts for evaluation of risk factors and validation of the scoring system. Multivariate logistic regression was used for the assessment of the adjusted association between factors and the outcome of interest. Results of the multivariate logistic regression were treated as the foundation to develop the risk scoring system. There were 46402 and 22681 episodes identified in 2009–2013 and 2014–2015 cohorts with 356 and 204 hemorrhagic events among respective cohorts. Use of cephamycins was associated with a higher risk for hemorrhagic outcomes (aOR 2.03, 95% CI 1.60–2.58). Other risk factors included chronic hepatic disease, at least 65 years old, prominent bleeding tendency, and bleeding history. A nine-score risk scoring system (AUROC = 0.8035, 95% CI 0.7794–0.8275; Hosmer-Lemeshow goodness-of-fit test p = 0.1044) was developed based on the identified risk factors, with higher scores indicating higher risk for bleeding. Use of cephamycins was associated with more hemorrhagic events compared with commonly used penicillins and cephalosporins. The established scoring system, CHABB, may help pharmacists identify high-risk patients and provide recommendations according to the predictive risk, and eventually enhance the overall quality of care.


Cephamycins have proven themselves as potential alternatives to carbapenems in treating infections caused by extended-spectrum beta-lactamase-producing isolates1,2. However, increased risk of bleeding with the use of cephamycins has been reported3,4,5,6,7,8,9,10,11. The potential mechanism of this risk may be attributed to that cephamycins may cause hypoprothrombinemia via inhibition of the growth of vitamin K – producing intestinal bacteria or inhibition of vitamin K – epoxide reductase and vitamin K – dependent carboxylase12. According to the internal statistics, cephamycins took part in 8% among all intravenous antibiotics during 2009–2013 in National Taiwan University Hospital (NTUH), which implied the significance of cephamycin-related bleeding risk.

In addition, our previous study using Taiwan's National Health Insurance Research Database (NHIRD) found cephamycin was associated with increased risk of hemorrhagic events. Patients on anticoagulants, in poor nutritional status, with underlying liver failure, and encountering previous hemorrhagic events were recognized as possible risk factors. Nevertheless, this study was limited to the lack of laboratory data due to the nature of NHIRD12. Also, patients with combination of two or more risk factors are frequently seen in daily practice. The cumulative effects brought by possible risk factors were not further discussed in this study. On top of that, in the era of emphasizing cost-effective medical care, precisely defining and monitoring patients with higher risk become much more important.

Therefore, this study aims to define the risk factors for cephamycin-associated hemorrhage using NTUH electronic medical record (EMR), and to establish a risk scoring system that can be adapted in clinical settings to enhance the safe use of cephamycin.


Data source

Data used in this study was obtained from the NTUH EMR. The database comprises person-level records of demographic information, medication utilizations in NTUH, medical history, laboratory examinations and imaging study results. This retrospective study was approved by Research Ethic Committee of NTUH with the waiver of informed consent. The research team was authorized to have the access to NTUHEMR (REC Number: 201312061RINB). All methods were carried out in accordance with relevant guidelines and regulations.

Study design and subjects

A retrospective cohort design was applied. Patients aged ≥20 years who continuously used injection forms of the study antibiotics for ≥48 hours in the emergency or inpatient department in NTUH during January 1st, 2009 to December 31st, 2015 were included. Study antibiotics included cephamycins (cefmetazole, flomoxef, cefoxitin and cefoperazone) and reference antibiotics (amoxicillin/clavulanate, ampicillin/sulbactam, cefuroxime, cefotaxime and ceftriaxone). Reference antibiotics were chosen under the consideration that they should have similar antibacterial spectrum and therapeutic roles with cefmetazole and flomoxef in treating infectious diseases, have no structures known to induce hypoprothrombinemia, and have rare adverse hemorrhagic events reported previously. Data were provided by Department of Pharmacy and Integrated Medical Database at National Taiwan University Hospital (NTUH-iMD).

The first prescription day of study antibiotics was assigned as the index date. The index date plus the following ten days composed the observational period12,13. We excluded patients who changed antibiotics from one study antibiotic to another during the observational period. Those who had observational period beyond the study period or prior use of study antibiotics within 30 days of the index date were also excluded. The treatment course was defined as the duration from the index date to the day of discontinuing study antibiotics.

We further divided all identified study subjects into two cohorts based on their index year. The 2009–2013 cohort was used for identifying risk factors and development of the scoring system while the 2014–2015 cohort was used for validation of the scoring system.

Outcome measurements

Study outcome was the occurrence of any hemorrhagic event during the observational period, which was defined through a two-step process. Firstly, we used the hemorrhagic-related diagnosis codes (International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)) documented within the observational period to detect the suspected hemorrhagic events. These events were further examined by chart review to see if the hemorrhage developed after the initiation of study antibiotics.


Covariates including patient demographics, information of index stay, underlying diseases and conditions, concurrent medications/treaments, concurrent antibiotic, and indication of study antibiotics were also collected. Underlying diseases and conditions included chronic hepatic disease, chronic renal diseases, coagulopathy, history of operation, international normalized ratio (INR) prologation, hemorrhagic events, thrombocytopenia, hypoalbuminemia, hepatic dysfunction and renal dysfunction. History of hemorrhagic events was defined by hemorrhagic-related ICD-9-CM codes within 180 days prior to index date or positive stool occult blood test within 7 days prior to index date. Chronic hepatic/renal diseases, and coagulopathy were identified by ICD-9-CM codes coded before index date14 and the available laboratory data acquired within 7 days prior to the index date13,15.

Concurrent medications/treatments within 7 days prior to index date include antiplatelets, anticoagulants, Vitamin K1, total parenteral nutrition (TPN), proton pump inhibitor (PPI), histamine-2 (H2) blockers, tranexamic acid, clotting factors, steroids, nonsteroidal anti-inflammatory drugs (NSAIDs), packed red blood cells (PRBC) and fresh frozen plasma (FFP). We also retrieved any chemotherapy received by the study subjects within 180 days prior to the index date. Use of concurrent antibiotics was defined as any antibiotics other than study antibiotics prescribed for at least 72 hours during treatment course of study antibiotics. Indication of antibiotics was categorized based on infection sites identified through physicians’ orders when prescribing antibiotics and diagnosis codes upon discharge.

Statistical analysis

Descriptive statistics including Chi-square and Mann-Whitney U tests were estimated in both derivation and validation cohorts. Univariate logistic regression was used to examine the unadjusted association between independent variables and the outcome. The significance level in univariate analyses was set at 0.25. Factors shown to be statistically significant in univariate analyses were selected as candidates for multivariate logistic regression. Stepwise strategy with slentry level at 0.15 and slstay level at 0.05 was applied for model selection. Factors with p-value < 0.05 in the adjusted analysis were considered statistically significant and were incorporated into the risk prediction model. The multivariate logistic regression coefficients of each factor were transformed into a risk score by divided the regression coefficients by the smallest regression coefficient among the variables and then rounding the quotients to the nearest integer when developing a risk scoring system.

Model performance consisting of the area under the receivers operating curve (AUROC) and Hosmer-Lemeshow goodness-of-fit test was further assessed. Moreover, the Youden’s index was applied in determining the optimal cutoff value of the risk score to best differentiate the outcomes. Cochran-Armitage trend test was used to evaluate if the score and occurrence of outcome were positively correlated. All data were analyzed using Microsoft Excel 2010 (Microsoft Corp., Redmond, WA, USA) and SAS 9.4 (SAS Institute Inc., Cary, NC, USA).


Patient demographics

The initial cohort included 193151 episodes with exposures to study antibiotics. After applying all the inclusion and exclusion criteria, there were 69083 episodes identified. The cohort was divided into two sub-cohorts as the 2009–2013 derivation cohort (46402 episodes from 37094 individuals) and the 2014–2015 validation cohort (22681 episodes from 19201 individuals) according to the index date. In the derivation cohort, there were 18821 episodes exposed to cephamycins and 27581 episodes exposed to reference antibiotics. Among them, 227 (1.21%) and 129 (0.47%) hemorrhagic events occurred during exposure to cephamycins and reference antibiotics. Gastrointestinal bleeding was the most common type of hemorrhage events (303/356, 85.11%). In the validation cohort, there were 8615 episodes exposed to cephamycins and 14066 episodes exposed to reference antibiotics. Among them, 116 (1.35%) and 88 (0.63%)  bleeding occurred during exposure to cephamycins and reference antibiotics (Fig. 1).

Figure 1

Study flow of cohort enrollment.

Compared to users of reference antibiotics, cephamycins users were younger (median age 61.5 years in cephamycins vs. 64.7 years in reference antibiotics, p < 0.0001) and more likely to have chronic hepatic diseases (7.91% vs. 4.74%, p < 0.0001). In terms of concurrent medications, cephamycins users were less likely to concurrently receive antiplatelets (4.08% vs. 10.84%, p < 0.0001) and anticoagulants (2.27% vs. 5.73%, p < 0.0001). On the contrary, cephamycins users were prone to concurrently receive vitamin K1 (1.84% vs. 1.16%, p < 0.0001), TPN (10.48% vs. 2.11%, p < 0.0001), PPIs (22.15% vs. 9.14%, p < 0.0001), H2-blockers (8.53% vs. 5.67%, p < 0.0001) and tranexamic acid (6.34% vs. 5.76%, p = 0.009) (Table 1).

Table 1 Patient characteristics of study antibiotics users.

Derivation of risk prediction model for cephamyins-associated hemorrhage

The final risk prediction model with AUROC of 0.8072 contained five variables that were all statistically significantly associated with hemorrhage (p < 0.0001) in the multivariate logistic regression analyses. The regression coefficients of all variables in the risk prediction model were transformed into integer risk scores demonstrated in Table 2. Patients using cephamycins, aged 65 years or above, afflicted with chronic hepatic disease, having history of bleeding within 180 days prior to the index date, and showing bleeding tendency, which was indicated by prescriptions of PPIs, tranexamic acid or clotting factors, within 7 days prior to the index date had an increased risk score by one, two, or four points. A risk scoring system named CHABB (stands for use of Cephamycins, underlying chronic Hepatic diseases, Aged 65 years or above, Bleeding tendency, and Bleeding history in the preceding 180 days of the index date) with 5 items was established and the sum risk scores ranged from 0 to 9.

Table 2 Development of scoring system for bleeding risk using multivariate logistic regression model.

Validation of risk prediction model for cephamyins-associated hemorrhage

In the evaluation of predictive accuracy of the risk prediction model for cephamycins-associated hemorrhage, CHABB was able to achieve an AUROC of 0.8035 and 0.7550 in the derivation and validation cohorts, respectively, revealing that the sum risk scores lent satisfactory to great validity to the prediction of cephamycins-associated hemorrhage risk. The predictions made from the risk model were in alignment with the observed outcomes suggested by Hosmer-Lemeshow tests (Table 3) and turned out to be the most accurate when all five items were all included (Fig. 2). The cutoff score suggested by Youden’s index was 6 points among cephamycins users across both cohorts. As depicted in Fig. 3, the counts of hemorrhagic events notably went up as the risk score increased in both cohorts. It indicated that the risk to developing hemorrhagic events was positively correlated to the risk score, which was statistically significant proved by Cochran-Armitage trend tests (p < 0.0001) as well. Moreover, we further divided patients in both cohorts into three risk groups, including low (with risk scores from zero to three), medium (with risk scores from four to five) and high risk (with risk scores equal to six or above) groups, using the event percentages of 1% and 4% as cut-offs. And the results revealing that the higher the score, the greater the possibility of developing hemorrhagic events had parallel in those who have not been categorized into groups.

Table 3 Validation of the risk scoring system.
Figure 2

Receivers operating characteristics (ROC) curves of risk score in derivation cohort (a) validation cohort (b).

Figure 3

Increasing risk of bleeding events with increasing risk score among cephamycin users.


This is a longitudinal study attempting to identify risk factors for cephamycin-associated hemorrhagic events in a teaching hospital in Taipei. The identified risk factors further served as the foundation for the development of a risk scoring system connecting the use of cephamycins and the possible related hemorrhages. To the best of our knowledge, this is the first study with the largest population in evaluation of the risk factors for cephamycin-associated hemorrhagic events and in development and validation of the nine-score risk score system.

We have identified risk factors for cephamycin-associated hemorrhagic events and established a nine-score risk scoring system named CHABB. Our findings were similar to our previous study conducted by Chen et al. in 2014 using Taiwan’s NHIRD12. In Chen’s study, use of anticoagulants or history of bleeding events in preceding 180 days of the index date, malnourished status and hepatic failure were suggested as risk factors. Among those factors, nutrition status was determined using the surrogate indicator of TPN due to the nature of NHIRD that laboratory data was not available. In our study, nutrition status was determined by serum albumin level, and a normal value was assigned for patients who were lack of records, as was done in Strom et al.'s study16. However, hypoalbuminemia was not retained in the final multivariable model in our case. Use of anticoagulants and antiplatelet agents were included in stepwise selection for their clinical importance to the outcome, but they were not retained in the final model. This could be physicians’ preference not prescribing cephamycins in patients under long-term anticoagulants or antiplatelet agents to avoid bleeding problems. Also, there may be a healthy-worker bias that patients under anticoagulants or antiplatelet agents persistently may be less likely to develop bleeding diatheses.

Some of the factors shown to be related to hemorrhages were interesting but not surprising. We expected anti-secretion agents of PPIs and agents affecting hemostasis such as clotting factors and tranexamic acid to be protective factors. Nevertheless, these factors were risk factors in our study. Similar results were observed in Brown et al. that the use of antiulcer therapy was associated with the highest odds ratio10. Considering PPIs, clotting factors and tranexamic acid essentially stabilized the hemostatic process but trended toward higher odds ratios in our regression, we assumed the underlying conditions resulted in the use of these agents were similar. Therefore, we merged the use of either kind of these medications into one variable named bleeding tendency. Judging from the AUROC, the new variable possessed similar ability in explaining the outcome of interest, meanwhile confirming our assumption to some extent.

We chose the scoring system as the presentation format because of its straightforwardness to be used in clinical setting. The indices we adapted to assess the model performance were widely used in building a scoring system17,18,19,20,21,22,23,24. Our study has large sample size with relative low incidence of the outcome. Although the calibration result in the 2014–2015 cohort had only a marginal p-value indicating the scoring system fit well, we need to understand that the test was based on the foundation of hypothesis testing. That is, the possibility that we make type II error may not be low and results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggested an appropriate fit of the scoring system.

Despite numerous methodological merits of our study, there were some limitations needed to be mentioned. First, due to its retrospective design, information bias might have occurred in outcome ascertainment. Clinicians may have handled minor hemorrhages before they progressed into serious events. Clinicians may not mention minor events in the discharge notes as well. On the other hand, certain hemorrhagic diagnoses might be part of medical history from previous visits, so they were actually not documented in medical chart. These patients were selected by hemorrhagic-related codes but were then excluded during chart review. Second, we might have included patients who had been prescribed with one of the study antibiotics at medical facilities other than NTUH during the screening period. Third, external validation by a future prospective study would be warranted as it would make this scoring system more relevant for clinical scenario.

In summary, we have established and validated a nine-score risk scoring system to identify potential high risk patients who may develop hemorrhagic events when taking cephamycin. For patients being classified in the high-risk group, we recommend close monitoring of INR and hemorrhagic signs to prevent subsequent catastrophic hemorrhages. In hopes of enhancing the patient care quality, we expect this risk scoring system to be adapted and fully exerted in clinical setting. Future refinement of the scoring system may be considered after the scoring system being prospectively applied clinically for some period of time.


  1. 1.

    Fukuchi, T., Iwata, K., Kobayashi, S., Nakamura, T. & Ohji, G. Cefmetazole for bacteremia caused by ESBL-producing enterobacteriaceae comparing with carbapenems. BMC Infect. Dis. 16, 427 (2016).

  2. 2.

    Lee, C. H., Su, L. H., Tang, Y. F. & Liu, J. W. Treatment of ESBL-producing Klebsiella pneumoniae bacteraemia with carbapenems or flomoxef: a retrospective study and laboratory analysis of the isolates. J. Antimicrob. Chemother. 58, 1074–1077 (2006).

  3. 3.

    McCloskey, R. V. Spontaneous reports of bleeding: comparison of N-methylthiotetrazole side chain (MTT) and non-MTT cephalosporins. J. Infect. Dis. 158, 1405 (1988).

  4. 4.

    Kaiser, C. W., McAuliffe, J. D., Barth, R. J. & Lynch, J. A. Hypoprothrombinemia and hemorrhage in a surgical patient treated with cefotetan. Arch. Surg. 126, 524–525 (1991).

  5. 5.

    Bertino, J. S., Kozak, A. J., Reese, R. E. & Chiarello, L. A. Hypoprothrombinemia associated with cefamandole use in a rural teaching hospital. Arch. Intern. Med. 146, 1125–1128 (1986).

  6. 6.

    Wong, R. S. M., Cheng, G., Chan, N. P. H., Wong, W. S. & Ng, M. H. L. Use of cefoperazone still needs a caution for bleeding from induced vitamin K deficiency. Am. J. Hematol. 81, 76–76 (2006).

  7. 7.

    Mueller, R. J., Green, D. & Phair, J. P. Hypoprothrombinemia associated with cefoperazone therapy. South. Med. J. 80, 1360–1362 (1987).

  8. 8.

    Baxter, J. G. et al. Clinical risk factors for prolonged PT/PTT in abdominal sepsis patients treated with moxalactam or tobramycin plus clindamycin. Ann. Surg. 201, 96–102 (1985).

  9. 9.

    Hooper, C. A., Haney, B. B. & Stone, H. H. Gastrointestinal-bleeding due to vitamin-K deficiency in patients on parenteral cefamandole. Lancet 1, 39–40 (1980).

  10. 10.

    Brown, R. B. et al. Enhanced bleeding with cefoxitin or moxalactam - statistical-analysis within a defined population of 1493 patients. Arch. Intern. Med. 146, 2159–2164 (1986).

  11. 11.

    Pakter, R. L., Russell, T. R., Mielke, C. H. & West, D. Coagulopathy associated with the use of moxalactam. JAMA 248, 1100–1100 (1982).

  12. 12.

    Chen, L. J. et al. Use of Hypoprothrombinemia-Inducing Cephalosporins and the Risk of Hemorrhagic Events: A Nationwide Nested Case-Control Study. PLoS One 11, e0158407 (2016).

  13. 13.

    Tung, M.-L. Use of Cephamycins and the Risk of Hypoprothrombinemia and Hemorrhagic Events: A Hospital-Based Retrospective Cohort Study. Master of Science thesis, National Taiwan University (2015).

  14. 14.

    Quan, H. et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med. Care 43, 1130–1139 (2005).

  15. 15.

    Department of Pharmacy, National Taiwan University Hospital (NTUH). Policy and Protocol: Recommendations on the Use of Warfarin by Anticoagulation Clinic of NTUH, approved by NTUH Pharmacy and Therapeutics Committee on April 8th, 2013.

  16. 16.

    Strom, B. L., Schinnar, R., Gibson, G. A., Brennan, P. J. & Berlin, J. A. Risk of bleeding and hypoprothrombinaemia associated with NMTT side chain antibiotics: Using cefoperazone as a test case. Pharmacoepidemiol. Drug Saf. 8, 81–94 (1999).

  17. 17.

    Hosmer, D. W., Lemeshow, J. S. & Sturdivant, R. X. Assessing the Fit of the Model in Applied Logistic Regression Ch. 5, 153–225 (John Wiley & Sons, Inc., 2013).

  18. 18.

    Mehran, R. et al. A simple risk score for prediction of contrast-induced nephropathy after percutaneous coronary intervention: development and initial validation. J. Am. Coll. Cardiol. 44, 1393–1399 (2004).

  19. 19.

    Khorana, A. A., Kuderer, N. M., Culakova, E., Lyman, G. H. & Francis, C. W. Development and validation of a predictive model for chemotherapy-associated thrombosis. Blood 111, 4902–4907 (2008).

  20. 20.

    Lipsky, B. A. et al. Developing and validating a risk score for lower-extremity amputation in patients hospitalized for a diabetic foot infection. Diabetes Care 34, 1695–1700 (2011).

  21. 21.

    Chan, P.-C. et al. Risk for tuberculosis in child contacts. Development and validation of a predictive score. Am. J. Respir. Crit. Care Med. 189, 203–213 (2014).

  22. 22.

    Lee, M. H. et al. Development and validation of a clinical scoring system for predicting risk of HCC in asymptomatic individuals seropositive for anti-HCV antibodies. PLoS One 9, e94760 (2014).

  23. 23.

    Hung, Y. C. et al. Development of risk scoring system for stratifying population for hepatocellular carcinoma screening. Hepatology 61, 1934–1944 (2015).

  24. 24.

    Altman, D. G. & Royston, P. What do we mean by validating a prognostic model? Stat. Med. 19, 453–473 (2000).

Download references


We sincerely thank Dr. Chi-Tai Fang of Department of Internal Medicine, National Taiwan University Hospital and Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan, for his assistance in providing statistical consultation for the manuscript. This work was supported by the research grants from Ministry of Science and Technology, Taiwan (MOST 104-2320-B-002-011). We also thank the staff of the Department of Medical Research, National Taiwan University Hospital for the Integrated Medical Database (NTUH-iMD).

Author information

The idea and study design were developed by Dr. Lin S.W., Dr. Hsiao F.Y. and Ms. Chien T.L. All authors were responsible for the preparation of the manuscript. Ms. Chien T.L. contributed to the data acquisition and statistical analysis. All authors participated in the analysis and interpretation of the data, reviewed and approved the manuscript for submission.

Correspondence to Fei-Yuan Hsiao or Shu-Wen Lin.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chien, T., Hsiao, F., Chen, L. et al. Development and Validation of a Risk Scoring System for Cephamycin-Associated Hemorrhagic Events. Sci Rep 9, 12905 (2019) doi:10.1038/s41598-019-49340-5

Download citation


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.