Abstract

Introduction

Oral surgery services are progressively moving out of traditional hospital departments and into primary care. This necessitates accurate methods of triaging referrals, so patients of varying complexity are managed in the most suitable environment. The latest NHS commissioning proposal identifies 'level 1' procedures as simple extractions which do not require referral. We developed a model for quantifying how accurately these simple extractions can be predicted from information in standard referral letters.

Methods

Experienced clinicians (N = 10) were independently asked to predict whether extractions (N = 25) were likely to be simple-forceps or surgical procedures, from information provided in specially developed standardised referral letters. One oral surgeon had previously completed all extractions. The triaging clinicians were asked to comment on reasons for each decision and state their level of confidence in their predictions.

Results

Only 67% (range 52–76%) of extractions were correctly predicted as either simple or surgical with a significant propensity to underestimate the complexity of surgical extractions rather than overestimating simple procedures (p <0.05). High levels of confidence reported by the clinicians in their decisions correlated with more accurate predictions (p <0.05).

Conclusions

This is the first attempt to develop a model for clinical decision-making in oral surgery triage services. Our findings suggest there is significant scope for improvement and highlight areas for development.

Introduction

Trends in NHS dental activity data suggest that referrals for oral surgery services are increasing.1 It is also known that a significant proportion of referrals turn out to be for simple extractions2 although the primary reason for referral is an anticipated difficulty in extraction.3 This might suggest there is a discrepancy in the ability to predict difficulty of extractions either by the referring dentist or at the accepting referral centre. Conventionally, referrals are made from primary care dentists to hospitals where extractions are carried out under the care of a consultant in oral and maxillofacial surgery or, more recently, oral surgery. Each referral is triaged by the consultant and the extractions allocated to the most appropriately experienced clinician in their team; therefore complex cases are treated by more experienced clinicians. It could be argued that accuracy of triage decision-making isn't overly important as there is always consultant cover to manage complex cases, so patient safety isn't compromised. Inappropriate allocation after triage, in this situation, would theoretically result in less than optimal patient throughput but also potentially more experience for trainees. The latest NHS England Guide for commissioning oral surgery and oral medicine specialties seeks to remove the divide between primary and secondary care settings to ensure improved access, quality of care and better patient outcomes.1 The proposed approach is to undertake more dentoalveolar surgery in primary care with reduced reliance on traditional hospital services, except for the most complex cases. This undoubtedly places more emphasis on accurate triaging of referrals as cases deemed suitable for primary care won't have the fall-back of consultant cover if they turn out to be more complex than expected. Stratification of services is proposed by a draft classification of extraction complexity. Level 1 is 'extraction of erupted tooth/teeth' and 'of buried roots (whether fractured during extraction or retained root fragments)' whereas level 2 comprises 'surgical removal of buried roots and fractured or residual root fragments'. Therefore, this suggests the difference between non-wisdom tooth extractions classified as level 1 and level 2 could be simplified as simple-forceps versus surgical procedures.

The comprehensive commissioning document also helpfully provides the definitions: 'Level 1 care complexity outlines the skill set and competencies a dentist covers on completion of undergraduate and dental foundation training. Therefore commissioners would expect that level of competence as a minimum standard for performers on the NHS performer list' and 'Level 2 care is defined as procedural and/or patient complexity requiring a clinician with enhanced skills and experience who may or may not be on a specialist register. This care may require additional equipment or environment standards but can usually be provided in primary care'.1 This is important, as although not explicitly stated, it is likely that only level 2 and above procedures will be referable by general dental practitioners and all level 1 extractions will have to be completed by the dentist as part of their NHS contract. Again not specifically stated, but experience from previous schemes indicates that inappropriate referrals will be sent back to the referrer,4,5,6 so presumably extractions deemed as level 1 by the triage system will be rejected and the referring dentist required to complete the procedure. Clearly, it is fundamental that the assessment of referrals is reliable, consistent and, most importantly, accurate, so that incorrect triaging outcomes don't put referring dentists and their patients in difficult situations. Surprisingly, there is no literature on the validity of the process of interpretation of referrals. For the assessor to accurately determine if a referral is appropriate they must have all the information required to predict whether an extraction is simple (level 1) or surgical (level 2), yet there is no evidence to confirm exactly what information is needed. Studies on wisdom tooth surgery associate difficulty with increasing patient age, ethnic background, male gender, increased patient weight, bone impaction, horizontal angulation, depth of the application point, unfavourable root formation and the surgeon's experience.7,8 Given this information, it should be determinable whether a surgical wisdom tooth extraction is likely to take longer than average but the relevance of this to non-wisdom tooth and particularly level 1 extractions is unknown.

We present a model for the triage decision-making process, developed to determine how accurately the difficulty of non-wisdom tooth extractions can be predicted from information provided in standard referral letters. This is fundamental to understanding how the referral process can best used to triage exodontia patients requiring varying complexities of care.

Methods

As with all attempts to model clinical practice a balance needs to be achieved between accurately replicating the clinical situation (validity) while permitting the necessary standardisation such that outcomes can be quantified and attributed to specific identifiable variables (utility).

To model a level 1 procedure we defined a simple extraction as requiring only forceps or elevators. In contrast level 2 procedures were modelled as surgical extractions where either (1) a flap was raised or (2) a surgical drill was required to remove bone or section teeth even if completed with a flapless technique.

Patient cases were retrospectively selected from the treatment list of one specialist oral surgeon (FM) to minimise variation in surgical technique and clinical decision-making. To incorporate teeth likely to be extracted by general dentists and/or enhanced practitioners (level 1 or 2 procedures) the majority were pre-radiotherapy extractions in oncology patients, indicated for osteoradionecrosis prevention. Specifically chosen were cases that would potentially have been referred back to the patient's own dentist but only completed in the hospital to minimise delay in starting radiotherapy. Additionally, all wisdom teeth were excluded, to avoid the controversial issues around which should be extracted by general dental practitioners. Similarly, all cases were treated under local anaesthetic (LA) only, to avoid potential issues with different treatment approaches under sedation or general anaesthetic (GA). All cases included a diagnostic quality radiograph. The outcome of whether the extraction was completed as a simple or surgical procedure was obtained from the clinical notes.

To generate standardised referral letters for clinicians to assess, the chosen cases were anonymised but all details known or suspected to influence the assessor's decision were unchanged. This included the gender, ethnicity and age while all medical histories were stated as clear. All radiographs were scanned with a high-resolution scanner and printed on plain paper to simulate the typical real-life clinical situation in the NHS Trust.

Ten clinicians independently assessed each referral letter. Each clinician was familiarised with the process and definitions before starting and asked to record on a standard pro forma whether each referred extraction was likely to be a simple or surgical extraction and their level of confidence (low, medium or high) in their prediction. Although highly subjective it would be expected that only when absolutely confident in their decision would a clinician declare high confidence and conversely, where they really couldn't decide would be marked as 'low' or 'medium' depending on personality. Therefore analysis of the high confidence decisions may provide an insight into what makes a good referral letter in terms of accuracy of clinical decision-making as the clinicians were also asked to provide brief comments on factors that influenced their decision-making for each case.

With no literature to guide the effect size likely to be observed or the level of inaccuracy that is clinically relevant, it was inappropriate to determine a sample size by either power or precision methods. Therefore a pragmatic approach was utilised with a total 40 minutes of each clinician's time taken being the maximum allocated. This would need to include 15–20 minutes to read and understand the specific instructions and clarify any issues with the lead investigator (WS). With approximately 20 minutes to actually triage the referrals it was decided a total of 17 referral letters with 25 teeth would simulate the realities of a normal clinical scenario.

The study was approved for conduct and ethics by the Clinical Effectiveness Unit of the NHS Trust (CEU #1714). Data handling and statistical analysis was completed with Excel 2003 (Microsoft, USA) with the StatPro add-on (Kelley School of Business, Indiana University).

Results

In total 17 referral forms were assessed independently by ten clinicians, with a total of 25 teeth requiring predictions. Of these, 15 were actually simple extractions and the remaining ten required surgical intervention. The assessing clinicians had varying levels of training and experience in exodontia and comprised six specialist oral surgeons, one speciality registrar, one final year postgraduate clinical Master's student, one speciality doctor and one dentist with a special interest (DwSI). The clinicians' number of years since qualification, ranged from seven to 35 years.

Numbers of correctly predicted simple or surgical extractions ranged from 13 (52%) to 19 (76%) out of 25 teeth (Fig. 1) with mean accuracy of 67%. Of the teeth, 14 were predicted correctly by eight or more of the assessors, while two teeth were incorrectly assessed by this majority (Fig. 2). The most common comments made by clinicians to justify their decision-making were poor radiograph quality, patient's ethnicity, extent of caries and complex or simple root morphology, but there was no qualitative difference to explain the variance in extremes of prediction accuracy. There was also no significant difference in the number of correct predictions for clinically confirmed simple versus surgical cases (p = 0.27, U = 54.5, Mann Whitney U test, Table 1).

Figure 1
figure 1

The narrow range (13/25–19/25) of total correctly predicted teeth for the ten clinicians assessing referral letters independently

Full size image
Figure 2
figure 2

Number of clinicians correctly predicting each case varied widely. Some teeth (for example, referral letters 6 and 13) were correctly assessed by all ten clinicians while everyone wrongly evaluated case 3

Full size image
Table 1 Actual against predicted outcomes for 25 teeth based on the information provided in referral letters. No significant difference in the number of correct predictions for simple against surgical cases (^Mann Whitney U Test). Significantly more surgical extractions were underestimated than simple extractions overestimated (*Chi squared)
Full size table

Erroneously predicting surgical extractions to be simple represents underestimation of complexity, which could result in a clinical scenario where a difficult extraction is recommended for an inexperienced clinician. This occurred in 43% of decisions made by the triaging clinicians (Fig. 3). Conversely, 25% of decisions overestimated the complexity of a simple extraction, which could lead to inappropriate use of resources in an efficiency-driven system. Importantly, significantly more surgical extractions were underestimated than simple procedures overestimated (p <0.05, Chi Squared test, Table 1).

Figure 3
figure 3

In total the ten clinicians assessed 25 teeth making 250 decisions. 10/25 cases were actually surgically removed and 43/100 times these were underestimated and predicted to be simple extractions

Full size image

In assessing clinicians' confidence in their decision-making the outcomes were pooled per tooth. Cases where more than seven out of ten were highly confident in their decisions were significantly more likely to be predicted correctly than cases where seven or more assessors reported low or medium confidence in their predictions (p = 0.032, U = 11.5, Mann-Whitney U test).

Discussion

This is the first study to model and quantify the accuracy of the triage clinical decision-making process for non-wisdom tooth extractions. Only 67% of extractions were correctly predicted as simple (level 1) or surgical (level 2) with a significant propensity to underestimate the complexity of surgical extractions rather than overestimating the difficulty of simple procedures. This suggests that a proportion of referrals to any exodontia triage service are likely to be assessed as suitable for level 1 practitioners when better suited to level 2 providers, which may have implications for safe and effective service delivery.

Validity and development of the triage model

A valid model system for triaging oral surgery referrals will allow potential improvement to be trialled in a safe environment rather than retrospectively auditing changes made to clinical practice to determine their possible benefits and safety. We attempted to control variables identified from the published literature that may influence the accuracy of the clinical decision-making process. A significant risk of this process is a loss of validity of the model system whereby it does not adequately represent the clinical situation and therefore any conclusions cannot be generalised and applied to improve clinical practice. In an attempt to assess validity we can compare our model to the reported features of oral surgery triage systems currently on trial in the UK. A frequently reported issue is the poor quality of radiographs included with referral letters.2,9 This was mirrored in our model but as all radiographs included in the study would have been diagnostically acceptable at the time of surgery the loss of fidelity could be attributed to the administrative processes involved in preparing and conveying the referral. Fortunately, this should be improvable with increasing digitisation or electronic correspondence and our model system could be utilised to show what qualitative improvement in clinical decision making this simple administrative adjustment may effect.

Improvements to the model could be made by including wisdom teeth. Lower mandibular 8s are arguably the most studied tooth in the mouth, but with few general dental practitioners attempting their extraction their relevance is questionable. In the current study all patients had no medical conditions likely to complicate dental extractions, for example, bisphosphonate or antiplatelet therapy. However, it is clear from the literature that a complex medical history is one of the most frequent reasons for referral.2,3,10 It would therefore be valuable to develop a medical history component to the model.

Improving accuracy of decision-making

Accuracy of decision-making appears to be independent of the assessing clinician as long as they have adequate experience (Fig. 1), indicating that the properties of the referral letter are likely to be important. This is important as a better referral could result in more appropriate patient management via the triage system. Furthermore, our model could be used to optimise the referral process as it permits quantification of improvements such as better quality imaging in referrals.

Our findings indicate that higher confidence in clinician's decision-making equates to more correct decisions. This needs further investigation, as it could be important for developing more robust triage systems where referrals are only rejected if the assessing clinician has a high level of confidence in predicting the extraction as simple. As the tendency appears to be towards underestimating surgical more than overestimating simple extractions (Table 1) this confidence-based assessment should potentially be useful for determining which extractions are most likely to be simple.

Limitations of the study

Use of one specialist oral surgeon's treatment list as the source of patients will not represent the ability of most level 1 or even level 2 practitioners. The likely impact is from a specialist potentially being able to extract teeth, without needing to resort to a surgical procedure, more often than a less experienced dentist. Conversely, it may be argued that someone specifically trained in surgical extractions may resort to this approach more than a generalist; however the patients in this study were pre-radiotherapy cases where surgical extractions should ideally be avoided to permit rapid healing and avoid excessive delay to oncology treatment. Taken together this would suggest that the number of simple extractions might potentially be higher than achieved by a general dental practitioner. In terms of model validity this may reduce sensitivity to triggers for surgical extractions by general dentists but the impact should be minimal as consistency of decision-making is more important than the relative level. Additionally, it is likely that the assessors would be predicting each case to be a simple or surgical extraction if they themselves undertook it, so as they were either specialists or experienced in exodontia the actual performing surgeon's ability would have been closely matched in this study.

Use of pre-radiotherapy patients in the model may not represent typical referrals. This can be seen in terms of the lack of anterior teeth in the study as radiotherapy fields usually involve the posterior mandible or maxilla.

Despite these limitations the validity assessment of our model suggests that it is an acceptable representation of the clinical decision-making process fundamental to triaging exodontia referrals.

Conclusion

Development of a model system for clinical decision-making in oral surgery triage services is important for improving service delivery. Iterative changes and novel innovations can be piloted in a safe environment without the need to impact patient care and retrospectively audit outcomes. Our data suggest there is significant scope for improving accuracy of clinical decision-making and highlights areas for development, to optimise care of patients referred for extractions.