In 1999, quality-improvement initiatives were galvanized following the landmark Institute of Medicine report To Err is Human, which concluded that preventable errors were responsible for up to 90,000 deaths per year within the US health-care system.1 It signaled that those within the system, who had previously been confident of its value and service, could no longer ignore major shortfalls in the quality of care it provided. The report also served as official notice of severe deficiencies resulting in avoidable harm on a previously unrecognized level. Although clinicians have always prided themselves on delivering high-quality care, nationwide analysis returned a poor verdict on the performance of the US health-care system in general.

The resulting focus on performance improvement and evaluation of outcomes has placed new burdens on health-care systems and personnel. There is pressure to maintain or improve quality in the face of increasingly sophisticated diagnostic and treatment modalities. Attempts are being made to wring improvements from an already overloaded system, and numerous local and national programs have been set up to address the issue.

The Surgical Care Improvement Project (SCIP) is one such program. The aim of SCIP is to reduce complications by ensuring adherence to a number of processes proven to be efficacious in well-designed studies.2 Although implementation of the measures is voluntary, the Centers for Medicare and Medicaid Services have reduced hospital reimbursements by 2% for institutions failing to report compliance with SCIP recommendations.3 Despite the penalty associated with nonreporting, it remains to be seen whether adherence to these criteria actually translates to improvements in outcome.

A recent study by Stulberg et al.4 evaluated the relationship between postoperative infection rates and reported compliance with SCIP infection-prevention process-of-care measures, with a view to clarifying this relationship in practice. Their findings were mixed: although there was no relationship between adherence to individual measures and reduced postoperative infections, institutions that complied with all infection-prevention measures reported significantly fewer postoperative infections than those which had not.

The authors used an inpatient database that included information on SCIP infection-prevention process-of-care measures self-reported by 398 hospitals. A weighting strategy was used to make it possible to draw conclusions at a national level regarding compliance with the measures and associated effect. In order to determine whether compliance actually led to improved outcomes, the investigators performed a series of analyses that included compliance with individual recommendations and two different 'all-or-none' measures. The first 'all-or-none' measure was compliance with three original 'core' SCIP infection-prevention recommendations (that is, prophylaxis within 1 h before skin incision, appropriate choice of antibiotic for the procedure, and discontinuation of prophylactic antibiotics within 24 h). Patients were also assessed according to a second 'all-or-none' composite measure, which required compliance with at least two of the six infection-prevention measures included in SCIP (that is, glucose control for cardiac surgery patients, appropriate hair removal, and normothermia in colorectal patients, in addition to the three core recommendations). Demographic information, insurance status, and comorbidity indices were used to adjust for patient variability; procedures were categorized to stratify the risks of different types of surgery; and hospital size was analyzed to determine if volume and bed number contributed to any observed result.

...institutions that complied with all [SCIP] measures reported significantly fewer postoperative infections...

The investigators evaluated almost 406,000 patient discharges and 3,996 documented infections. Item level adherence to any of the six individual SCIP measures varied from 80% to 94%. Hospitals reported adherence to all three 'core' prophylactic recommendations for 78% of discharges, and 73% of hospitals reporting on at least two of the six measures declared 100% compliance with all six infection prevention measures. Of particular note, adherence to SCIP procedure in the second all-or-none composite was associated with a substantial decrease in postoperative infections, from 14.2% to 6.8%, yet compliance with the 'core' group of three original SCIP measures was not associated with statistically significant improvements.

The authors conclude that publicly reporting individual item performance, which ostensibly helps patients choose 'high-quality' hospitals, does not achieve its stated purpose. Aggregated measures of compliance, however, are associated with improved outcomes and thus may be better indicators of high-quality care. In spite of the weaknesses inherent in large administrative databases—including the likely under-reporting of infection rates, lack of a control group, and potential bias of hospitals contributing information to the database —the message is an important one: hospitals and health-delivery systems need to better organize care in order to improve compliance, preferably 100% of the time.

There is growing evidence that all-or-none measures of compliance are more appropriate for measuring quality of care.5 Processes often exert their effects synergistically or sequentially, and failure to comply with all measures highlights systematic weaknesses. Systematization of care requires coordination of the complex interactions between physicians, nurses, pharmacists and other care-givers. Procedures in intensive care units have demonstrated this principle, showing that 'bundling' of care can reduce the incidence of ventilator-associated pneumonia and central line infections.6,7 Adjunctive checklists are a proven means of ensuring that important but often overlooked details are not ignored during critical periods in a patient's hospitalization. More importantly, use of checklists can maintain results over time.8 Even 'high-quality' hospitals often fail to maintain basic standards of safety, and a properly employed perioperative checklist can improve compliance with procedures that most surgeons and anesthetists expect to occur routinely, but which might actually be overlooked.9

Scientific research is driven by cause and effect. Researchers tend to identify and manipulate a single variable and determine its effect while keeping all other variables constant or controlling for them in some way. The science of quality improvement often demands the opposite: small changes made concurrently and with rapid feedback.10 This does not mean that scientific rigor cannot be applied to improvement efforts, rather innovative methodologies and clinically meaningful outcomes need to be considered in concert to evaluate effect.

With the increasing frequency of unfunded mandates and demands for reporting and compliance—such as those underpinning the premise of Medicare reimbursement strategies—administrators and care providers are forced to seek savings in other ways while instituting resource-intensive changes. Happily, efforts at quality improvement frequently result in cost savings, but the process of implementation is often painful and time consuming, and require the participation of all the numerous caregivers who interact with a patient. Patients expect high quality and adherence to basic standards of care. SCIP compliance certainly ought to be part of a high-quality system of care delivery, as it is patients who suffer the consequences of our omissions.