Introduction

Checklists have become widely recognised as a means of contributing to safer healthcare provision. The checklist model, as used by the aviation industry for many years, has only relatively recently been adopted within medical settings, and evidence suggests that the use of checklists in certain settings can improve patient safety.1

The use of systemic immunosuppression or immunomodulation for the management of severe oral mucosal inflammatory disease is one of the highest risk areas in the practice of oral medicine. With the possible exception of pemphigus vulgaris, the majority of oral mucosal inflammatory disorders have the potential to impact quality of life, rather than present a threat to life. As such it is imperative that patients are not exposed to excessive risk in the management of conditions which have low mortality. However, the effects of severe oral mucosal disease on quality of life and nutritional status can be significant, and some patients will require intensification of treatment in order to obtain control of the disease.2,3,4,5 Systemic immunosuppression is a means of doing this, with modest support from the literature but a large volume of anecdotal clinical experience attesting to its effectiveness.6,7,8,9

Immunomodulatory or immunosuppressive medications used in oral medicine practice include azathioprine, mycophenolate mofetil and dapsone. These drugs have the potential to cause significant adverse effects including bone marrow suppression, hepatotoxicity and haemolysis, and these risks must be mitigated.10,11,12,13,14,15,16,17,18

In order to improve the safety of our clinical practice, we developed checklists for the pre-treatment workup and on-treatment monitoring for patients prescribed azathioprine, mycophenolate mofetil and dapsone. We performed a retrospective study assessing for improvement in our compliance with the checklists at 6 and 12–24 months following their introduction.

The aim of this study was to determine whether the use of checklists led to an improvement in our compliance with safe prescribing protocols for these medications. The discussion will also consider the potential benefits of the use of checklists in oral medicine practice.

Method

Checklists for pre-treatment workup and on-treatment monitoring for azathioprine, mycophenolate mofetil (MMF) and dapsone were developed from resources including the drug manufacturers' guidance, the British National Formulary, and available information in the literature, such as the British Association of Dermatologists' Guidelines for the use of azathioprine.16

The checklists contained guidance regarding pre-treatment counselling to patients, investigations required before commencement, significant drug interactions, and important cautions or contraindications to prescribing.

The second part of the checklists related to on-treatment monitoring for patients who had commenced the drugs. These included information about screening for adverse effects, blood testing and other investigations for monitoring (see online supplementary appendices 1, 2 and 3 for complete checklists – please note that the MMF checklist has been updated.)

The checklists were made available for the use of clinicians caring for patients who had been prescribed any of these three medications. Clinicians were informed regarding the introduction of the checklists via email. Formal education sessions regarding checklist use were not carried out. The checklists were to be filed in the patient's notes for future reference.

Baseline data regarding our clinical practice before checklist introduction were collected for the period 2010–2012. Following this, compliance with the recommendations of the checklists were audited. For azathioprine, data were collected at six months and 12–24 months following checklist introduction. For MMF, data were collected at six months following checklist introduction. For dapsone, data were collected at six months and 24 months following checklist introduction. Compliance with the checklists was compared between these time points. Data were gathered by retrospective review of paper and electronic clinical notes by three clinicians (MKS, CVN, PT). The results were recorded in an Excel spreadsheet and this program was used to calculate percentage compliance with each point on the checklists.

Results

The number of patient records included for each drug in each stage of the study is presented in Table 1.

Table 1 Number of patient records included in each phase of the study
Full size table

Results: azathioprine group

Compliance with pre-treatment viral serology screening improved from <10% to over 80% at six months after checklist introduction (Fig. 1). Compliance with tuberculosis risk assessment improved from 5% to 50% at six months. Documentation in the clinical notes of counselling provided to patients regarding side effects and risks improved from 61% to 85% at six months (Table 1). Compliance with monitoring blood tests for the first six weeks of therapy improved slightly (from 10% to 15%), and for the second six weeks of therapy improved significantly (20% to 66%) at six months after checklist introduction (Fig. 2).

Figure 1
figure 1

Compliance with pre-treatment workup – azathioprine

Full size image
Figure 2
figure 2

Compliance with monitoring blood tests – azathioprine

Full size image

In the 12–24 month iteration of the audit, compliance with pre-treatment viral serology screening for HIV, hepatitis B and hepatitis C had improved to 100% (Fig. 1). There was 92% compliance with varicella zoster virus serology screening, compared to 7% before checklist introduction and 85% at six months post introduction. Documentation of TB risk assessment had declined to 4%, from 50% at six months post checklist introduction. All cases had some form of patient counselling documented in the notes (Table 2). Compliance with monitoring blood tests for the first six weeks of therapy improved further to 20%, however, compliance with monitoring blood tests in the second six weeks of therapy had decreased to 25% (Fig. 2).

Table 2 Documentation of pre-treatment counselling (% compliance)
Full size table

Results: mycophenolate mofetil group

Compliance with viral serology screening increased from nil to 100% following introduction of the checklist (Fig. 3). Compliance with basic blood tests (full blood count [FBC], renal function [EUC] and liver function [LFT]) increased from 80% to 100%. One hundred percent of patients had documented evidence of pretreatment counselling after introduction of the checklist, compared to 20% previously (Table 1). Monitoring blood test compliance improved for the first six weeks of therapy but decreased for the second six weeks (Fig. 4).

Figure 3
figure 3

Compliance with pre-treatment workup – MMF

Full size image
Figure 4
figure 4

Compliance with monitoring blood tests – MMF

Full size image

Results: dapsone group

Compliance with pre-treatment workup blood tests (FBC, LFT, glucose–6-phosphate dehydrogenase level) was 100% before the introduction of the checklists, and remained at this level in both the six month and 24 month audits (Fig. 5) Documentation of assessment of contraindications to the medication improved from nil to 100% by 24 months. Documentation of patient counselling improved from 25% pre-checklist, to 50% at six months post introduction, and 60% at 24 months (Table 1).

Figure 5
figure 5

Compliance with pre-treatment workup – dapsone

Full size image

On-treatment monitoring had improved at six months post checklist introduction. The percentage of cases with full compliance with FBC testing had improved from 16% to 33%, and the percentage of cases with full compliance with LFT testing had improved from 25% to 33% (Fig. 6). However, full compliance with monitoring blood tests had fallen at 24 months post introduction of the checklists, to 14% for both FBC and LFT testing. The percentage of cases with 50–100% compliance with the recommended blood testing regimen improved at six months and maintained a slight improvement at 24 months.

Figure 6
figure 6

Compliance with monitoring blood tests – dapsone

Full size image

Discussion

Checklists are cognitive aids which assist users to complete tasks accurately.19 Widely used in high-reliability industries such as aviation, they have only relatively recently become commonly used within the field of medicine. The late and somewhat reluctant adoption of safety checklists within medicine is thought to be due to a number of factors. The inherent complexity of clinical medicine, individual patient variation and rapidly changing clinical circumstances has been perceived to be incompatible with checklist-based standardised systems. Culturally, a high value is placed on clinical autonomy and clinical judgement in decision-making, and it has been felt that checklists remove this autonomy or significantly reduce it. Similarly, the use of checklists or aide memoirs has been perceived by clinicians as an admission of weakness or a lack of knowledge.19,20

Following the introduction of the WHO surgical safety checklist in 2008 the use of checklists in medicine has become more widespread.1,21 Difficulties exist in accurately collecting data on the effectiveness of checklist use. Audits displaying high rates of checklist completion and compliance do not necessarily indicate an improved safety culture and in many situations, correct usage is more important than the actual contents or completion of the checklist.16,22 Awareness has arisen of the potential for the development of 'checklist fatigue' whereby clinicians are overwhelmed by the sheer volume of checklists required to complete tasks, leading to failure to utilise them correctly.19,23 An analysis of reasons for poor adherence to prescribing guidelines for dapsone in a dermatology service found that significant factors were the lack of a standardised hospital-wide protocols, poorly functional IT systems, a variable level of knowledge and awareness of prescribing protocols among clinical staff, and a lack of clinical time.19,20,24 In this case a checklist existed, but utilisation was poor.

There are no published guidelines for the use of immunosuppression in oral medicine clinics, however, these may be extrapolated from other specialities such as dermatology and rheumatology. Documents such as the British Association of Dermatologists' guidelines on the safe prescribing of azathioprine are directly relevant to oral medicine practice, with similar dosages and indications.16 Less material exists regarding safe prescribing of MMF and dapsone, and our checklist development was guided by information in the drug manufacturer's documents and published material as available.24,25

As such, a limitation of the speciality of oral medicine is the lack of available, commonly used, standardised guidelines for pre-treatment workup and on-treatment monitoring for these drugs, particularly dapsone and MMF. It is possible that the checklists we developed omit important items or tend towards over-investigation. It is important that formal consensus is reached based on the clinical experience of clinicians in different specialities as well as in oral medicine, in order to standardise the guidelines for pre-treatment workup and on-treatment monitoring for these drugs. This should ensure that important items are not missed, and that workup and monitoring are cost-effective and use the minimum resources required for patient safety.

Our study demonstrated a significant improvement in our compliance with pre-treatment workup recommendations for azathioprine, MMF and dapsone, as defined by the checklists we produced. Improvements in on-treatment monitoring were less significant and in some cases were not sustained over time.

An important limitation of our study is a potentially large amount of missing data, due to the lack of locally-performed blood test results being filed in the patients' hospital notes. This may have led to an incorrect assessment of on-treatment monitoring compliance, which may have actually been better than as recorded in our data set. This may also be considered a limitation of the current health system, in which there is a lack of robust electronic systems for communication of investigation results between health professionals. In addition, the quality of clinical note-taking may have also negatively impacted the results. Prior to the introduction of the checklists, clinicians did not routinely document pre-treatment counselling given to patients, although anecdotally this did take place on most occasions.

We have found that ensuring compliance with on-treatment monitoring is the most challenging aspect of systemic therapy. This is often due to the fact that patients have their monitoring blood tests performed locally, and the results are not available on our internal computer system. We rely on the patient's GP to review and act upon abnormal results or send the results to us for monitoring. This process can lead to difficulties in always being aware of the current status of patients on these drugs. The distance patients travel to clinics and the current funding arrangements necessitate this arrangement. In order to mitigate the risk, staff members within our department routinely follow-up on external blood test results.

Shared care protocols are a useful means of managing on-treatment monitoring for patients taking immunosuppressive medications. These are formal agreements established between primary care providers and the specialist clinic which initially commenced the medication. Following the initial stage of stabilisation on the medication, the responsibility for monitoring investigations is transferred to the primary care provider, with guidance and support available from the specialist clinic as required. Specialities such as dermatology and rheumatology regularly use shared care protocols and these may be effectively utilised by oral medicine units in the same region.26

The use of checklists in surgical and procedural specialities has led to changes in practice and improvements in patient safety.21 However, a number of studies have been performed which failed to demonstrate significant improvements in safety with the routine use of checklists.27 In order for checklists to be effective they must be used appropriately, and this may necessitate a cultural change within clinical practice.28 Additional factors which appear to be important for successful use of clinical checklists are strong institutional support, appropriate staff training, adaptability of the checklists to staff feedback, avoidance of duplication of tasks, time-efficient functionality and allowing clinicians to still feel empowered in clinical decision-making.20,23,29

Most of the published literature regarding clinical checklists relates to procedural specialities such as surgery, critical care and anaesthesia. The use of computerised clinical decision support systems relating to prescribing has been reviewed in a number of publications, and studies have been published regarding the use of these systems for dermatological indications.30,31 A dermatology service in Singapore introduced a safe prescribing system for dapsone as an add-on to their existing electronic prescribing system. The system automatically prompts the clinician to order workup and monitoring blood tests according to their safe prescribing checklist, and generates reminder questionnaires related to drug adverse effects. This system was found to be effective in improving compliance with safe prescribing for dapsone.24

The clinical usefulness of prescribing checklists is also supported by the large number of published guideline documents relating to drugs used in rheumatology and dermatology.16,25,32

Our study demonstrated that there was an initial improvement in compliance with some aspects of the checklists, but that this was not sustained over the longer term. This may partially be due to the Hawthorne effect (behaviour improves when people are aware they are being watched) as clinicians were aware that the first stage audit was taking place, whereas the second stage audit was not announced. Further reasons may include staff turnover, with new staff members not appropriately trained in checklist use; checklist fatigue, with established staff members no longer looking at the checklists and relying on recall; and inappropriate standards in the checklist – the number of monitoring blood tests required may not match clinicians' experiential awareness of the frequency of adverse effects and how to mitigate these appropriately.

In our experience, the provision of checklists has assisted our clinicians in improving the consistency of workup and monitoring for patients taking potentially harmful medications. Clinicians have commented favourably on the usefulness of the checklists as an 'aide memoir' when prescribing these medications. However, in the absence of clear information on the exact requirements for frequency of monitoring blood testing, individual consultants have tended to follow their own monitoring regimens rather than the checklists, which has potentially contributed to the poor compliance identified in on-treatment monitoring in our audits.

Future steps for improvement of patient safety in oral medicine clinics related to immunosuppressant or immunomodulatory drugs include the use of computerised prescribing safety systems, the utilisation of clinical nurse specialists in oral medicine clinics as is standard in specialities such as gastroenterology and rheumatology, further refinement and standardisation of prescribing checklists, and education of clinicians in order to establish a safety culture and the appropriate use of checklists.

Conclusion

The use of checklists in medicine has led to improvements in patient safety. Utilisation of checklists in the safe prescribing of medications used in oral medicine practice is a useful strategy for ensuring high quality practice and minimising adverse effects in a patient population who are being treated for non-life threatening conditions. Further work is required to develop the most effective and appropriate standardised checklists for the use of these medications and to engage with clinicians in order to ensure appropriate use of checklists within a culture of safe practice.