Introduction

Part two of this article looks at the frequency of use of IOTN among registrants, training and influencing factors on accuracy of use of the IOTN in both primary and secondary care.

Following the introduction of the IOTN in 19891 the benefits of this index have been widely recognised. The use of the IOTN continues to rise in secondary care services with studies reporting that 33% of hospitals were using IOTN in 19912 which increased to 75% by 1994.3 A more recent survey of UK dental public health consultants found 80% of respondents used the IOTN as an instrument for planning, contracting and monitoring NHS orthodontic services.4 In a study exploring the utility of the IOTN, British orthodontists' most common description of the IOTN included 'quick', 'simple' and 'easy to use'.3

When the index was initially developed it was proposed for use by orthodontists in secondary care. Since then surveys have shown acceptability of the index by orthodontic specialists in practice4 but few have looked at patterns of use by general dental practitioners in primary care.

Aims

To understand:

  • Frequency of use of IOTN among dental registrants, working in either primary or secondary care

  • When registrants last had training in the IOTN and if it was considered verifiable or non-verifiable

  • Which factors influence 'accuracy' of use of IOTN?

Method

Within this study participants were asked to complete a short tick box questionnaire at the same time as being asked to score the IOTN for 14 preselected cases. The questionnaire was developed by the authors of this study to understand registrants' group, year of qualification, place of work, frequency of use of IOTN, last episode of IOTN training and type of training.

Statistical analysis

Data was extracted manually from the questionnaires and entered into SPSS software (version 21) for statistical analysis. Simple descriptive statistics (frequency and percentages) were used to determine the distribution and ranges of participants' response to each question. Mean kappa scores for the DHC and AC for individuals variables were calculated using SPSS.

Multiple linear regression analysis was used to reveal which predictor factors best described a registrant's knowledge of IOTN. Two models were used, one for DHC and the other for AC. To ensure that linear regression was an appropriate statistic model to assess for the influence of predictor variables on the outcome variable (kappa scores), the residuals within the model were checked to see if they followed a normal distribution.

Results

Registrant groups

A large number (n = 229) of registrants took part in the study. The number of participants within each group is shown in Table 1.

Table 1 Number of participants per registrant group
Full size table

The group with the largest number of participants was the general dental practitioner group (n = 62) with recruitment taking place at the British Dental Association's conference. This is also where the majority of Dental Foundation Trainees (n = 50) participants were recruited. The specialist orthodontists (n = 49) and the qualified orthodontic therapists (n = 21) were mainly recruited at the British Orthodontic Society's annual conference in Edinburgh (2014). The groups with the least participants included the student orthodontic therapists (n = 19) followed by the qualified orthodontic therapists (n = 21) group.

The overall participation count was 229 with a substantial sample size in each group.

Place of work

The spread of place of work among the different registrant groups is shown in Figure 1.

Figure 1
figure 1

Bar chart to show distribution of place of work within each registrant group

Full size image

The results showed that the mean kappa scores were the highest for participants working in secondary care (K = 0.68), which was considered acceptable agreement. Participants working in primary care had the lowest mean kappa scores (Table 2).

Table 2 Distribution of participants' place of work and its effect on mean DHC kappa scores
Full size table

Frequency of use of IOTN

The specialist orthodontic and postgraduate orthodontic student groups used the IOTN most frequently with over 95% using it at least once a week (Fig. 2). There was a more even distribution of use of IOTN among the general dental practitioner and Dental Foundation Trainee groups. The majority of the qualified orthodontic therapist group (62%) used the IOTN at least once a week, with 3% rarely or never using it. Within the student orthodontic therapist group 37% used the IOTN at least one a week and the majority (63%) rarely or never used it.

Figure 2
figure 2

Bar chart showing the distribution of frequency of use of IOTN within the different registrant groups

Full size image

The effect of frequency of use on DHC kappa scores was studied for all participants.

The mean kappa score between the different categories of use ranged from 0.22 to 0.66 (Table 3). Only the participants that used the IOTN everyday achieved an acceptable mean DHC K >0.60. There is an increasing trend of mean DHC kappa scores evident with increasing frequency of use of IOTN indicating that the participants that used IOTN more frequently were better in its use.

Table 3 Distribution of participants' frequency of use of IOTN and its effect on mean DHC kappa scores
Full size table

Training

Last episode of training in IOTN

All participants had received training in the use of IOTN with the majority (n = 181) having their last episode of training within the last five years (Table 4).

Table 4 Last episode of training of participants within each registrant group
Full size table

There does not appear to be an obvious trend with DHC kappa scores and when the participants undertook their last episode of training in IOTN (Table 5).

Table 5 Distribution of participants' last episode of CPD and its effect on mean DHC kappa scores
Full size table

Type of IOTN training

Participants were asked to classify their training in IOTN as either verifiable CPD or non-verifiable CPD.

To count as verifiable CPD, an activity must have:

  • Concise educational aims and objectives;

  • Clear anticipated outcomes

  • Quality controls (opportunity to give feedback).

The majority of the training received in IOTN was classified as being verifiable (n = 154/229) with the remainder classifying their training as non-verifiable. Our findings indicate that the type of training the registrants received appeared to have no effect on the DHC or AC agreement kappa scores in this study.

A much larger than expected proportion of participants (32%) had received training in IOTN classified as non-verifiable which could imply a lack of available courses or training days in the use of IOTN.

Influencing variables on the DHC kappa scores

Two multiple linear regression models were considered, one for DHC and one for AC. Within these models the dependent variable was considered to be the observer DHC and AC kappa scores. Two predictor variables were found to have a significant influence on the DHC kappa scores (Table 6).

Table 6 Results for MLR model looking at the influence of predictor variables on the main outcome measure (DHC kappa scores)
Full size table

Place of work

The base category was primary care in this regression model so when you compare secondary care to primary care the results indicate that you would expect the kappa score to be higher by 0.169 for those that work in secondary care. When you compare those that work in both primary care and secondary care with those that work in primary care alone, you would expect the kappa score to be higher by 0.054.

Registrant group

The base category decided for this predictor variable within the multiple linear regression model was the Dental Foundation Trainee group. This model indicates that when you compare registrants in the qualified orthodontic therapist with Dental Foundation Trainee registrants you would expect their DHC kappa scores to be higher by 0.277. Specialist orthodontists would also score higher by 0.247 when compared to a Dental Foundation Trainee registrant. Student orthodontic therapist registrants would score higher by 0.224 when compared to the Dental Foundation Trainee group.

Only one of the predictor variables had a statistically significant influence on the AC kappa scores. This was the type of CPD.

The results indicate that in participants who had received non-verifiable CPD compared with verifiable CPD in the use of IOTN you would expect them to have a lower kappa score by 0.56

Discussion

Place of work

The results suggest that place of work had an influence on the participant's accuracy of use in the DHC of the IOTN looking at agreement. Participants working in secondary care achieved an overall mean DHC kappa greater than 0.60 indicating acceptable agreement with the expert scores. The groups with the majority of participants working in secondary care included the specialist orthodontist and postgraduate orthodontic student group. Both these groups are either orthodontic specialists or those training to become orthodontic specialists and hence one would likely expect these groups to be the most accurate in the use of IOTN.

On the other hand participants working in primary care achieved an overall mean DHC kappa = 0.34, indicating only fair agreement. The DFT and GDP groups had the majority participants working in primary care. These findings are supported by our multiple linear regression looking at predictor variables to explain at DHC kappa scores, which showed that when you compare secondary care to primary care you would expect the kappa score to be higher by 0.169 for those registrants working in secondary care.

There was no link found between place of work and AC agreement.

Frequency of use of IOTN

The specialist orthodontist and GDP groups were found to be the groups with the largest proportion of participants using IOTN on a daily basis. Our findings indicate that 52% of general dental practitioners, 42% of Dental Foundation Trainees and 98% of specialist orthodontist groups used the IOTN at least once a week.

Overall 28% of all participants rarely or never used the IOTN, with the student orthodontic therapist and qualified orthodontic therapist group comprising the largest proportion of participants within this category. This may be explained by the fact that currently therapists, either qualified or students, may not be requested to carry out the task of using IOTN in their work setting, however this may change as all newly qualified orthodontic therapists since 2013 will have been trained in the use of IOTN as the scope of practice of orthodontic therapists has changed.

Given that GDPs are considered the 'gatekeepers' in recommending orthodontic specialist treatment, it is not surprising that the GDP and DFT groups used the IOTN most frequently after the specialist orthodontists (Table 3).

A previous study looking at the orthodontic referral behaviour of West Sussex dentists showed that 76% did not routinely use the IOTN when making referrals.5 They also reported of the 229 dentists that completed their questionnaire, 5.7% of GDPs always used it, 17% often using it and 5.2% had never heard of it. When comparing these results to our results it can be seen that the proportion of GDPs that used the IOTN at least once a week was 52% indicating a greater acceptability of the IOTN within primary care over time.

In a more recent study looking at use and knowledge of IOTN among GDPs in Scotland6 61% of GDPs indicated they did not use the IOTN, while 39% reported they did. The most common reported reason for not using the index was that the GDPs thought it was only suitable for secondary care. The Scottish study did not look at how often those that did use IOTN actually used the index, they purely reported on whether they used it or not.

Our findings, which show a difference in use of IOTN between GDPs and the SO group, are supported by a study looking at the use of IOTN in primary care in Scotland which showed 10% of GDPs used IOTN in comparison to 50% of specialist orthodontists.7

These lower percentages within the Scottish study can be explained by the fact that IOTN was later implemented in NHS Scotland in 2011, compared to 2006 in England and Wales. Because of the later implementation we could expect acceptability of the index to be lower by GDPs in Scotland when compared to the UK.

The results from our study show a positive correlation between participants' DHC kappa scores with frequency of use of IOTN, indicating that those participants that used the IOTN more frequently were more accurate in its use. Participants that used the IOTN everyday achieved a mean DHC kappa greater than 0.60, indicating an acceptable level of agreement with the expert scores. There was no link found between frequency of use of IOTN and accuracy in the use of the AC.

Last episode of training in IOTN

All the participants that took part in the study reported having received training in IOTN. For the GDP and DFT group the participants' last episode of training in IOTN mostly corresponded to the period of their undergraduate training.

Both the student groups' (postgraduate orthodontic student and student orthodontic therapist) last episode of training also related to their training period. For the qualified orthodontic therapist group the last episode of training correlated well to when they qualified indicating that their training in IOTN was undertaken during their training period. It is known that in the main the IOTN training provided as part of the orthodontic therapy course before May 2013 was very limited as the role of the orthodontic therapist in the use of this index was minimal before the introduction of direct access by the GDC.

The specialist orthodontist group's last episode of training was varied, following the pattern of when they obtained registration as specialists with the GDC indicating their training in IOTN was undertaken during their speciality training.

There did not appear to be a trend of when a participant's last episode of training in IOTN was with participants' kappa scores for DHC or AC. While it may have been expected that those who have had more recent training would be better in the use of IOTN, the results of this study did not support this.

Many studies have indicated that different methods of training in IOTN can have a positive influence on agreement when using IOTN.8,9,10,11,12 It has been shown that dentists who received IOTN training referred patients more appropriately with a greater proportion of patients having definite treatment need.13

Given the requirements stated in the GDC document Preparing for practice – Dental team learning outcomes for registration,14 it would be expected that recently qualified dentists (DFTs), with 98% of this group registered with the GDC within one year, should have completed appropriate undergraduate training in the use of IOTN and as such should be competent in its use. The results from this study showed that 66% of DFT group had training in IOTN within the last year and 100% of them completed IOTN training within the last five years. Unexpectedly this group achieved the least agreement with the expert score, with a mean kappa of 0.25, indicating only fair agreement. Our results found that the time period from when training in IOTN was last undertaken has no correlation with accurate use of IOTN.

Type of Training

Participants were asked to classify their training in IOTN as either verifiable or non-verifiable CPD. Verifiable CPD is defined as having concise educational aims and objectives with clear anticipated outcomes.

The majority of the training in IOTN was classified as verifiable CPD (n = 155/229). The remainder classified their training as non-verifiable (Table 3). A much larger than expected proportion of participants (32%) had received training in IOTN classified as non-verifiable which could imply a lack of available courses or training facilities in the use of IOTN.

In this study the type of training the participants received appeared to have no bearing on the DHC or AC agreement kappa scores.

Factors contributing to use of IOTN

The factors that were most likely to explain accurate agreement when using DHC of the IOTN were found to include the following variables:

  • Registrant group

  • Frequency of use of IOTN

  • Place of work.

This study does not attempt to criticise undergraduate courses, dental foundation training or the practising standards of GDPs; rather it is hoped it will highlight to the appropriate bodies the need for more tailored training and tools to help those using the IOTN to do so more accurately. If dentists in primary care are to act effectively as gatekeepers of orthodontic provision more needs to be done to ensure they are using the index to an acceptable level.

Currently the outcomes outlined in the GDC document Preparing for practice – Dental team learning outcomes for registration14 do not specify that a dentist should be competent in using the IOTN to an acceptable level, but that a dentist should undertake an orthodontic assessment including indication of treatment need. This perhaps is understandable as the GDC regulates the whole of dentistry, not just NHS dentistry, and as such may not be willing to specify the use of IOTN, although the original purpose of the index was to highlight unnecessary treatment rather than a form of NHS rationing.

The final draft of the Dental Foundation Training Curriculum published by COPDEND in 2015 clearly states in outcome 9.5 that the foundation dentist must demonstrate to an appropriate standard the ability to use orthodontic indices.15 The results of this study show that the DFT group scored the lowest mean DHC kappa score of 0.25 SD 0.23, indicating only fair agreement. As a group they achieved acceptable sensitivity (75%) and the lowest specificity score of (26%). These results suggest that they achieved appropriate standard of diagnostic validity in identifying patients that needed treatment, but did not achieve the acceptable level of identifying those that that did not require treatment. These results suggest that patients not needing treatment could potentially be inappropriately referred for treatment having cost implications on NHS services and to patients.

This study has highlighted a need for greater support in either resources or tools for GDPs to help them use the IOTN more accurately.

Further specialised training or aids could help this group of registrants fulfil their competencies to lead to successful completion of their dental foundation training. Deaneries are ultimately responsible for the ensuring that foundation trainees are competent in the competencies outlined in their curriculum it is essential that they work closely with trainers ensuring appropriate resources and facilities are provided to enable the trainees to achieve, their requirement. The deaneries also have a responsibility in assessing these competencies to ensure appropriate standards are achieved before successful completion.

NHS England is responsible for commissioning NHS orthodontics and they too need to work closely with postgraduate deaneries as part of Local Dental Networks (LDNs) to improve access to verifiable IOTN CPD to all registrant groups. Currently there is nothing in primary care dental contracts specifying dentists' requirement to be calibrated in the use of IOTN, and even if this was implemented there is currently not enough capacity to train and award calibration in the use of IOTN. To date there is only one IOTN calibration course, by Cardiff University in the UK, which runs annually, and consideration may need to be given to increase the access to calibration courses.

Consideration also needs to be given to cases with borderline need for treatment. Currently an IOTN DHC of 3 will require an AC of 6 or above to qualify for treatment on the NHS. This study has shown only poor to fair agreement was achieved with the AC for the different registrant groups. Therefore a decision to offer a patient treatment for DHC 3 will rely on the AC being accurately determined which has shown to be difficult to achieve. Even though agreement was low for the AC, all groups were considered to achieve an acceptable level for specificity (identifying patients not needing treatment to the acceptable level). On the other hand none of the groups achieved acceptable level for sensitivity (identifying those that needed treatment) with the AC. These results indicate that patients that would potentially qualify for treatment according to AC may potentially be inappropriately refused treatment basis of inaccurate AC assessment.

Conclusion

The current level of use of IOTN among dental registrants is varied with 38% of all dental registrants achieving 'acceptable' agreement.

Three of the registrant groups studied achieved a mean kappa >0.60; these included the specialist orthodontists, postgraduate orthodontic students and qualified orthodontic therapist groups.

The Dental Foundation Trainee and general dental practitioner groups achieved the lowest level of agreement perhaps highlighting the need for more tailored and specific training and tools to help these registrants achieve acceptable levels of accuracy in use of the IOTN. This would improve the effective use of this index which is now being used to allocate state funded orthodontic care to patients in the UK

Place of work, frequency of use of IOTN and registrant group were the main factors which were found to influence accuracy of use of IOTN.