Main

Clinical practice guidelines (CPGs) are recognised and used as the standard for treatment and care. Although studies show that implementing oncology CPGs does improve clinical outcomes in both overall and recurrence-free survivals (Hébert-Croteau et al, 2004; Varga et al, 2010; Wöckel et al, 2010; Schwentner et al, 2012), the adherence of clinician decisions with CPGs remains low (Bloom et al, 2004). For instance, Wöckel et al (2010) reported a 51.9% guideline adherence rate for the complete treatment received by patients with primary breast cancer. The sole dissemination of CPGs as texts demonstrated a limited effect on practice (Giguère et al, 2012). Organisational measures such as multidisciplinary tumour boards (MTBs) were thus introduced to promote quality in care delivery (Patkar et al, 2011). However, the effect of MTBs is controversial. Kesson et al (2012) showed a positive impact on adherence to CPGs and clinical outcomes, while Keating et al (2013) observed varying effects including no effect. In fact, MTBs are often not completely efficient (Patkar et al, 2012) because a high case load is discussed and individual cases usually only receive a very limited amount of time for review.

Clinical decision support systems (CDSSs)—defined as any software in which individual patient characteristics are matched to formalised knowledge to generate patient-specific recommendations—are considered as tools that might alter physician behavior and improve CPG adherence (Shojania et al, 2010; Roshanov et al, 2011). While reminding physicians of patient-specific recommendations, CDSSs should solve the problem of physician awareness of CPG contents when making decisions. OncoDoc2 (Séroussi et al, 2001) is a CDSS based on local guidelines (CancerEst, 2008) for the management of non-metastatic invasive breast cancer. From 2005 to 2006, a preliminary uncontrolled before-after intervention study was performed with OncoDoc2 in MTBs of the Tenon hospital, Paris, France (Séroussi et al, 2007). The compliance rate of MTB decisions with CancerEst CPGs was significantly higher when the system was used (increase from 79.2% to 93.4%, P<0.0001). Although the improved compliance could not be attributed to the system (due to the lack of a control arm), MTB participants collectively decided to continue using OncoDoc2.

The aims of this study were to evaluate the compliance rate of MTB breast cancer therapeutic decisions with CancerEst CPGs when OncoDoc2 was used routinely, and to determine patient factors still associated with non-compliance despite the use of the CDSS reminding clinicians of the recommended treatment plan.

Materials and methods

CancerEst guidelines for early breast cancer management

CancerEst CPGs have been developed according to the framework of decision tables (Shiffman and Greenes, 1994), providing a treatment plan for all possible clinical situations of breast cancer. Where evidence exists, CancerEst CPGs are aligned with the International state of the art and French national guidelines. When evidence is missing or when scientific results are controversial, the editorial committee in charge of CancerEst guideline development decided to complete guideline coverage to provide treatment options established from local consensus (professional agreement) to promote clinical practice harmonisation. In particular, CancerEst guideline coverage has been extended to integrate the provision of alternative options to the recommended treatment plans to take into account patient preferences (e.g., when a patient refuses to undergo a mastectomy although recommended) or co-morbidities (e.g., when surgery is recommended but contraindicated because of a high anaesthetic risk). CancerEst CPGs are the acknowledged reference guidelines for all hospitals of Eastern Paris grouped in the CancerEst network including the Tenon hospital. The principles of CancerEst CPGs are given in Table 1 along with the main therapeutic options.

Table 1 Summary of the main therapeutic options adopted in CancerEst guidelines for the management of non-metastatic invasive breast cancer
Full size table

The OncoDoc2 CDSS

As opposed to risk predictors, such as Adjuvant! Online, OncoDoc2 (Séroussi et al, 2001) is a knowledge-based CDSS embedding CancerEst CPGs that provides patient-specific guideline-based care plans. Its knowledge base is structured as a decision tree. Paths represent patient clinical profiles as sequences of criteria. The use of OncoDoc2 consists in interactively navigating through the knowledge base. Clicking appropriate answers for closed-ended questions allows the building of a patient profile, and recording of corresponding data. Once the navigation is completed, relevant patient-specific guideline-based therapeutic options are displayed as recommended care plans (Figure 1). The knowledge base currently includes 83 criteria (e.g., postmenopausal status=yes/no). The decision tree integrates a total of 47 618 different clinical profiles, each of them leading to 1–11 recommended care plans (average 3). Clinical profiles are created out of 2–27 variables (average 19).

Figure 1
figure 1

A screenshot of OncoDoc2’s interface. Display of adjuvant care plans recommended according to CancerEst guidelines, for a postmenopausal woman with an invasive breast tumour of 23 mm, after lumpectomy and axillary lymphadenectomy, with clear margins, three positive nodes, and HR+/HER2+ status. A recapitulative of the decision criteria that characterise the patient clinical profile and a summary are displayed above the treatment plan proposals.

Full size image

Study design and setting

The study was conducted from February 2007 to September 2009 at the Tenon hospital (AP-HP, Paris, France). Following the prior before-after study (Séroussi et al, 2007), a new study was conducted whose aim was to identify and analyse for which patient profiles MTB decisions remained non-compliant despite the use of OncoDoc2. MTB participants consented by vote to participate in the study, which was acknowledged in the meeting minutes. Patients gave informed written consent to therapeutic procedures and to the analysis of data related to their pathology in accordance with institutional policies. OncoDoc2 has been routinely used during breast cancer MTBs for all patients with confirmed non-metastatic breast cancer diagnosis, including invasive, microinvasive, and in situ tumours. OncoDoc2 navigation was performed while the case was orally presented by the patient’s attending physician, and the system’s output was displayed on a large screen to allow MTB participants to follow the navigation and review the patient-specific guideline-based recommendations. Decisions were made by MTB participants who remained responsible. When non-compliant, decisions were recorded together with the reason of non-compliance.

Clinical variables and study population

For each decision, data were prospectively collected. Since medical records are mostly paper based at the Tenon hospital (AP-HP, Paris, France), we collected most data from OncoDoc2’s navigation to reduce manual input. Study variables are mainly clinical, such as age at diagnosis, menopausal status, tumour size, multifocality, microinvasion, prior neoadjuvant treatment, contraindications to surgery, prior surgical procedures (breast and axillary), margin clearance, node status, hormonal receptors (HRs), human epidermal growth factor receptor 2 (HER2) status, and Scarff-Bloom-Richardson (SBR) histological grading. The frequency of patient profiles was also studied. A patient profile (i.e., a clinical situation) was considered rare when it was used <1 out of 250 in our data set.

Because treatment modalities and decision variables taken into account differ according to the stage of patient treatment, the study population was divided into three groups (i) pre-surgery, (ii) re-excision (when prior surgery was not satisfactory), and (iii) adjuvant. Decisions for in situ cancers were discarded (no adjuvant therapy). When a patient had a sentinel node procedure followed by an axillary lymphadenectomy (for instance, in the case of a positive sentinel node, or in the case of the discovery of additional invasive foci while the tumour was initially supposed unifocal, or when the tumour size was finally >20 mm), the prior axillary surgical procedure is ‘sentinel node procedure’ for the re-excision decision, whereas it is ‘axillary lymphadenectomy’ for the adjuvant decision. The same applies for initial lumpectomy followed by re-excision with mastectomy.

Because the knowledge base is structured as a tree, where required information is limited based on the prior knowledge, the questions asked downstream depend on the answers given upstream. Thus, variables are not systematically asked for or documented for a given patient and decisions are not made from exactly the same set of criteria. For instance, when the tumour size is >20 mm, chemotherapy is recommended and SBR grade is not asked. Variables not asked for within the navigation are not relevant for decision-making adhering to the 10 commandments for effective clinical decision support (Bates et al, 2003). According to the group of decisions, they may be logically impossible (e.g., <3 mm margins in the pre-surgery group of decisions), logically possible but incidental (e.g., SBR grade in the pre-surgery group), or logically possible and customary (e.g., postmenopausal status in all groups of decisions). Logically possible and customary data have been manually collected in paper-based medical records to complete the data set before statistical analyses.

Compliance definition and unit of analysis

OncoDoc2 provides patient-specific care plans. Each plan is composed of a sequence of detailed care procedures. We stated that an MTB decision complies with CancerEst CPGs when it is strictly identical to one of OncoDoc2’s propositions, that is, exactly the same therapeutic procedures in exactly the same order: same surgery (breast and axillary), same chemotherapy (drugs, dosage, and rhythm), same radiation areas, and same endocrine drugs. Every time a patient case was discussed in MTB, the resulting decision was considered as a new decision. The decision is the unit of analysis. It is characterised by OncoDoc2’s navigation, the corresponding set of variables enriched by the added customary variables, the selected care plan, and its compliance status with CancerEst CPGs.

Statistical methods

The goal of the study was to identify patient factors associated with MTB decision’s non-compliance with CancerEst CPGs. Analyses were performed for each of the three groups. Continuous variables (e.g., age and tumour size) were categorised following clinical arguments. Univariate analyses between non-compliance and the studied variables were performed with Fisher’s exact tests. Variables associated with non-compliance with P<0.20 were entered in a multivariate-adjusted logistic regression model with a backward selection procedure and a significance level of P=0.05. Two-way interactions between significant variables were studied. Adjusted odds ratios (ORs) and 95% confidence intervals (95% CIs) were estimated. The discrimination obtained with the final multivariable model was assessed by the C-statistic and its 95% CI, and the calibration was assessed by the Hosmer–Lemeshow goodness-of-fit test. All tests were two-sided at the 0.05 significance level. Analyses were performed using the SAS software (version 9.2; SAS Institute, Cary, NC, USA).

Results

Between February 2007 and September 2009, 1886 MTB decisions were collected. In all, 262 decisions concerning in situ cancers were excluded, leading to the analysis of 1624 decisions for invasive breast cancers: 692 (43%), 198 (12%), and 734 (45%) in the pre-surgery, re-excision, and adjuvant groups, respectively.

Description of patients and decisions

The main characteristics of patients and decisions are described in Table 2. Breast cancers were mainly unifocal (82.4% in pre-surgery and 74.3% in adjuvant), while their proportion was lower in the re-excision group (64.1%) denoting the underdiagnosis of additional foci. They were mostly not microinvasive (97.8% in pre-surgery, 92.4% in re-excision, and 96.7% in adjuvant). Almost half of the tumours were of small size (T1) at diagnosis in pre-surgery and adjuvant groups, but T1 tumours represented 77.8% of the re-excision group, denoting an underestimation of surgery for a priori small size tumours. Prior lumpectomies represented 54.0% of the adjuvant group, but 89.9% of re-excisions which supports the a priori underestimation of surgery. The same phenomenon was observed with the prior sentinel node procedure (31.2% of the adjuvant group, but 64.6% of re-excisions) and axillary lymphadenectomy (65.5% of the adjuvant group, but 18.2% of re-excisions).

Table 2 Patient and tumour characteristics by group for the 1624 MTB decisions
Full size table

Factors associated with non-compliance

The overall non-compliance rate of MTB decisions with OncoDoc2 was measured at 8.3% (135 out of 1624, 95% CI: 7.0–9.8). The non-compliance rate is significantly different in the three groups (P=0.0006, Fisher’s exact test).

In the pre-surgery group, non-compliance rate reached 5.8% (40 out of 692 decisions, 95% CI: 4.1–7.8). Results of univariate and multivariate analyses are reported in Table 3. Only age greater than 80 years at diagnosis was independently associated with non-compliance. The C-statistic was 0.67 (95% CI: 0.59–0.75) and the model had good calibration (Hosmer–Lemeshow: P=1.0).

Table 3 Factors associated with non-compliance in the pre-surgery group (N=692)
Full size table

Non-compliance rate was highest in the re-excision group with 14.1% (28 out of 198 decisions, 95% CI: 9.6–19.8). As reported in Table 4, three variables were independently associated with non-compliance: microinvasive tumour, presence of microinvasion associated with a unique invasive tumour, and prior neoadjuvant treatment. The C-statistic was 0.67 (95% CI: 0.57–0.77) and the model had good calibration (Hosmer–Lemeshow: P=1.0).

Table 4 Factors associated with non-compliance in the re-excision group (N=198)
Full size table

In the adjuvant group, non-compliance rate was equal to 9.1% (67 out of 734 decisions, 95% CI: 7.1–11.4). Three variables were independently associated with non-compliance (Table 5): age <35 years at diagnosis, no prior axillary surgery, and HR+/HER2+ score. The C-statistic was 0.76 (95% CI: 0.70–0.83) and the model had good calibration (Hosmer–Lemeshow: P=0.63).

Table 5 Factors associated with non-compliance in the adjuvant group (N=734)
Full size table

Two-way interactions were studied between significant variables in all multivariate analyses. However, no significant interaction was found at the level of P=0.05.

Discussion

There exists a variety of physician barriers to guideline adherence, for example, lack of awareness, lack of familiarity, lack of agreement, lack of outcome expectancy, or inertia of previous practices (Cabana et al, 1999). By providing patient-specific guideline-based recommendations, CDSSs should solve the problem of the physician’s lack of guideline knowledge (awareness and familiarity). In our case, OncoDoc2 should also give answers to issues regarding lack of agreement and inertia of previous practices as (i) MTB participants were all involved in the development of local CancerEst CPGs and in the choice of recommendations for clinical situations where evidence was missing, thus they should agree with CancerEst CPG content and (ii) MTB participants consented to integrate OncoDoc2 in the MTB decision workflow to follow a quality improvement process. Consequently, the last barrier, not controlled in the study, is related to the lack of outcome expectancy of the recommended care, not in general, but in given particular clinical cases. Therefore, the analysis of non-compliance despite the use of OncoDoc2 reminding physicians of agreed upon best practices provides insights into the limits of guideline-driven care.

Some studies have analysed patient factors associated with non-compliance with guidelines in breast cancer care, but did not include decision support. These studies mainly identified elderly patients (Lebeau et al, 2011), multifocal tumours, occurrence of micrometastasis on lymph-node involvement (Chéreau et al, 2011), and patient choice (Landercasper et al, 2006). When using OncoDoc2, we found that similar patient factors were associated with non-compliance. However, we detected differences according to the stage of the care management.

In pre-surgical situations, factors related to non-compliance were elderly patients. Several retrospective studies already reported that increased patient age is independently associated with decreased guideline compliance (Giordano et al, 2005; Van Leeuwen et al, 2011; Kiderlen et al, 2012; van de Water et al, 2012). Aged populations are indeed subject to under-treatment when compared with standards of care (Lavelle et al, 2007; Hancke et al, 2010) with a negative impact on survival (Owusu et al, 2007; Schonberg et al, 2010). In this study, MTB clinicians decided to not follow the recommendations for surgery provided by OncoDoc2, choosing to undertreat elderly patients. However, in some cases, they also chose to overtreat them (e.g., decide a mastectomy instead of the recommended lumpectomy) to avoid a probable re-excision.

Variability in re-excision following breast surgery is commonly observed (McCahill et al, 2012). In our study, non-compliance was related to microinvasion, either within an in situ tumour or discovered in addition to an a priori unique invasive tumour. In these situations, non-compliance mainly consisted in the absence of re-excision, either of breast re-excision, when margins were ‘almost clear’, 2 mm instead of the recommended 3 mm (Aziz et al, 2006), or of axillary re-excision as recommended in the case of multifocality (Koukouras et al, 2010). In fact, today’s literature associates a spontaneous good prognosis with microinvasive cancers (Bianchi and Vezzosi, 2008; Sánchez-Muñoz et al, 2010), and subsequent axillary invasion is rare (Guth et al, 2008; Lyons et al, 2012) that challenges the need for re-excision. For instance, French national CPGs recommend to handle microinvasive tumours as in situ tumours with no need for re-excision. However, since there is no evidence to support this decision, it is also reasonable to handle microinvasive tumours like invasive tumours as CancerEst guidelines do recommend. In fact, although it is clear for all MTB participants that CancerEst guidelines are the reference, they conscientiously did not systematically decide the re-excision when microinvasion was a surgical discovery. The third factor of non-compliance was the existence of a prior neoadjuvant treatment. Indeed, surgical modalities after neoadjuvant treatment remain controversial (Kaufmann et al, 2012).

In the adjuvant group, three factors were identified as related to non-compliance: patients younger than 35 years, histological profile of the tumour, particularly HR+/HER2+, and nature of prior axillary exploration. With respect to the first two factors, main discordance affected the endocrine treatment, consistently with the reported uncertainty concerning the optimal treatment strategy (Goel et al, 2009). Some MTB decisions involved only tamoxifen as antihormonal treatment instead of the tamoxifen-agonists association in younger women. Similarly, positive HRs prevail over HER2+ for the assessment of intermediate risk tumours because HRs positivity predicts efficacy of endocrine agents, but HER2 overexpression is possibly associated with resistance to tamoxifen (Prat and Baselga, 2008). The type of endocrine treatment remains unclear for this population (Bauer et al, 2010) and could explain observed discrepancies. As for older women, suboptimal treatments mentioned in the literature and observed in our pre-surgery group were not observed. Rather, advanced age seemed to protect from non-compliance and this is consistent with the strong evidence that healthy older women tolerate adjuvant chemotherapy and stand to gain the same benefits as younger women from treatment (Kimmick, 2011). Axillary information, whose role in the further management is also controversial (Straver et al, 2010), was the third factor of non-compliance. Cases of non-compliance in the absence of axillary surgery (for the current care episode) were considered as borderline non-compliant since such decisions were made in a context of second, or more, local recurrences for patients that, finally, do not fit well the CPG cases and scope.

Although our definition of compliance was very restrictive, for it involved multiple and detailed therapeutic modalities as well as their sequencing, the observed compliance rate of MTB decisions with CancerEst CPGs when using OncoDoc2 was high (91.7%). It appears higher than prior published rates that only focus on a single step of early breast cancer management plan, often expressed at a high level of abstraction (‘surgery’, ‘adjuvant chemotherapy’, or ‘radiotherapy’). For instance, considering only adjuvant decisions, we observed a compliance rate of 90.9% with OncoDoc2, while taking into account all detailed steps. This compares with the less constrained compliance based on the presence of a ‘systemic adjuvant chemotherapy’ of 71.4% in the cohort of Wöckel et al (2010) or the even more general ‘systemic adjuvant therapy’ (including chemotherapy and hormonotherapy) of 63% in an earlier study by Landercasper et al (2006). The most appropriate compliance data to which our results could be compared are the data from Wöckel et al (2010) who reported a ‘100%-guideline adherence’ of 51.9% when considering the four main therapeutic steps together (surgery, radiotherapy, chemotherapy, and endocrine treatment) for managing invasive-only breast cancers. On earlier 2003 and 2004 data, Lebeau et al (2011) reported that the overall treatment sequence was fully compliant with CPGs in 12% of the patients. The high compliance rate obtained in our study could be explained by the fact that many usual barriers to CPG compliance have been discarded in our study setting and by the fact that CancerEst guidelines integrate alternative options to standard recommended treatments, which allows MTB decisions to be compliant in the case of contraindications or to take into account patient preferences.

The 83 variables used to describe patient profiles with OncoDoc2 widely explore the set of patient characteristics including preferences and co-morbidities, for example, contraindications to surgery (in the case of a bad general condition contraindicating general anaesthesia), contraindications to chemotherapy, and more specifically to anthracyclines (in the case of heart failure), contraindications to tamoxifen (in the case of antecedents of venous thromboembolism), and contraindications to radiotherapy. These variables were poorly represented in the population of analysis, except contraindications to surgery (cf. Tables 2 and 3), and not significantly associated with non-compliance. Thus, medical co-morbidities that usually play into many non-compliant treatment decisions were taken into account in the analyses, but showed to be non-significantly associated with non-compliance in our study. Indeed, compliance was enforced by the granularity of CancerEst CPGs that provide treatment plans for all clinical conditions including co-morbidities.

In a context, where MTB efficiency in guaranteeing quality of care is questioned (Keating et al, 2013), our results with OncoDoc2 suggest that the use of CDSSs during MTBs could help time and resource strapped MTB clinicians to maintain a high level of guideline compliance. However, the results obtained in a single institution, despite the 3-year time period, may be of limited generalisability. Another limitation is the restriction of the study to MTB decision compliance with guidelines without analysing how frequently the final treatment plan was actually compliant with guidelines. A multi-centered randomised clinical trial is currently being conducted with OncoDoc2 to provide answers to these questions.

This study demonstrated that while the use of a CDSS may improve the recall of recommended care plans, it cannot compensate the weaknesses of guideline content. Guidelines are but suggestions or recommendations for care, not rules or laws. They cannot be exhaustive and cover every case, and there are always individual patients who should be managed differently. Thus, there would always remain a residual non-compliance that cannot be shortened for situations where even consensually adopted local agreements may be collectively criticised. Therefore, a 100% compliance rate cannot be expected and should not be the target to allow for some flexibility in integrating nuances of a particular patient that might not be captured in a theoretical guideline-based formalised profile. Beyond patient preferences that cannot be totally foreseen, evolution of medical knowledge based on recent scientific publications makes CPGs always lagging behind ‘last’ best practices. The analyses of patient factors associated with MTB non-compliance despite the use of OncoDoc2 retrieved unclear patient conditions already identified in the literature that challenge CPG content rather than CDSS use. Rare situations are related to microinvasion (Schnitt, 1998) and neoadjuvant situations (Fumagalli et al, 2012). More frequent non-compliance points out the need for a better integration of guideline-based care in oncogeriatry (Biganzoli et al, 2012; Parks et al, 2012) and the need for specific clinical research (Mohile et al, 2012). They further support the importance of patient choice in shared decisions (Leonard et al, 2011), and illuminate populations for which management is still unclear (e.g., HR+/HER2+) (Prat and Perou, 2011).

Beyond fostering CPG compliance, the systematic use of CDSSs should help identify situations supported by low evidence as candidate profiles for prospective studies. Practically, such cases should be discussed as priority at the beginning of MTBs. Besides, repeated non-compliance should trigger the revision of guidelines to follow the evolution of practices, or the development of new guidelines to manage special cases whose increasing frequency makes them regular, usual, cases, for example, oncogeriatric guidelines, and consequently the update of CDSSs in an iterative process part of quality improvement procedures.