Continuing development of an oral health score for clinical audit

Key Points

• Provides a composite measurement of oral health status.

• Suggests the online audit facility in DEPPA allows the average oral health status to be reported to dental teams periodically so that they can benchmark their outcomes against the average.

• These audits can help to inform required staffing levels, the balance of skills needed in the team, and the oral health policy for a practice.

Abstract

Aim To compare the outcomes of a contemporary oral health status (OHS) scoring system with national oral health data from the 2009 Adult Dental Health Survey, and to explore the utility of the OHS in audit and service development.

Methods An OHS scoring system was developed as part of a previously reported comprehensive on-line patient assessment tool. The assessment tool also measured future disease risk and indicative capitation fee grading. The modified OHS score component was developed over 20 years of research and experience from the original Oral Health Index (Burke and Wilson 1995). The online tool was piloted by 25 volunteer dentists on 640 recall patients and qualitative and quantitative feedback provided. Anonymised data from the inputs and scores generated were collected centrally and analysed using descriptive statistics.

Results The modified OHS was reported to have good validity by the pilot group. Submitted data confirmed a mean age for the recall patients examined as 53 ± 15.8 years and an average oral health status score of 79.5 ± 10.8 where a score of 100 equates to perfect oral health. A breakdown of the scores into the eight principal components provided evidence of cross validation with the Adult Dental Health Survey (2009).

Conclusions Scoring oral health status electronically offers valuable opportunities for clinical audit. The reported benchmark oral health score of 79.5 for recall patients can be updated as increased numbers of patients enter the centralised data recording system. Audit can be facilitated by this move from a paper-based system to an on-line tool with central data collection.

Background

There is some dispute as to whether it was actually Galileo (1564-1642) who stated 'Measure what is measureable and make measureable what is not so'. Harrington¹ observed that 'Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it.'

Healthcare has entered an era in which service development, delivery and evaluation have come to rely upon the analysis of data. Moreover, that data permits the use of analytics to embrace patient-centred outcomes as important within the overall outcomes of care provided, and to help inform care pathways. There would therefore seem to be merit in providing patients with an overall 'oral health score' as well as specific disease/health outcomes in order to improve patient communication. This provides biofeedback and engages patients in behaviour change to improve their own oral health.

Oral health has been defined by the Department of Health England as: 'A standard of health of the oral and related tissues, which enables an individual to eat, speak and socialise without active disease, discomfort or embarrassment and which contributes to general well-being'.² This therefore suggests that any composite measurement of oral health must take into account patient perceptions of the life impacts of oral disease, as well as the professional assessment of oral disease status.

There are a variety of methods available for measuring different aspects of oral health that are in use in general dental practice and oral epidemiology. For example, Slade³ developed the Oral Health Impact Profile-14 (OHIP-14), which measures patient perceived life impacts of oral disease. This was employed in the most recent 2009 Adult Dental Health Survey of England and Wales⁴ (ADHS). In the ADHS 'tooth condition' was assessed by recording the number of teeth, the number of sound and untreated teeth, the number of restored teeth that are sound and the number of decayed or unsound teeth for each patient examined.

Ireland and co-workers⁵ proposed that a minimum of ten factors needed to be assessed in order to provide a composite picture of the oral health of patients. These ten factors are listed in Figure 1.

Figure 1

The minimum data set for primary dental care⁶

In 1995 Burke and Wilson⁶ described the Oral Health Index (OHX), which was a composite measurement with a maximum score of 100 equating to perfect oral health. The factors assessed in order to calculate the OHX are shown in Figure 2 and comprise a similar set to those proposed by Ireland et al. in Figure 1.

Figure 2

Factors assessed in the OHX and the OHS

The OHX was modified to produce the Denplan Excel Oral Health Score (OHS) during 1999 and 2000. Burke et al.⁷ published a summary of a pilot study of the OHS, which was conducted by 329 volunteer dentists, validating the weightings allocated to each element of the score. In addition, it was found that patients appreciated receiving their oral health status shared with them via the OHS.⁸ Delargy and co-workers⁹ subsequently demonstrated strong inter- and intra-examiner reproducibility with the OHS.

By 2011 the OHS was in use with 1,000 dentists as part of their commitment to the statement of principles of an established accreditation scheme (Denplan Excel Accreditation; www.denplan.co.uk). In 2011 at conferences held for accredited dentists relating to the development of this system, participants recommended the addition of future disease risk assessment to the OHS. This led to a collaboration with Oral Health Innovations Ltd (UK licence holders for PreViser at www.previser.co.uk), which produced The Denplan Excel PreViser Patient Assessment (DEPPA). DEPPA requires dental teams to input patient lifestyle and clinical data online to produce an 'oral health status' and 'disease risk' report. The development and pilot evaluation of DEPPA has recently been published by Busby et al.¹⁰ DEPPA incorporates:

• 1 PreViser^™ risk scores for periodontal disease, caries, non-carious tooth surface loss and oral cancer

• 2 The original OHS modified to incorporate PreViser's disease severity measurements for periodontal health and tooth health

• 3 A fee indicator for Denplan Care capitation contracts.

The aim of this paper is to describe how DEPPA's modified OHS was developed, building on the heritage of the OHX and OHS and how it was evaluated in a pilot study in primary care practices. The headline DEPPA pilot outcomes have been previously reported¹⁰ and so this paper focuses specifically upon the OHS component of DEPPA and its potential role in clinical audit.

Methods

Development of the modified OHS for DEPPA

The eight components of the original OHS and an indication of how the score is calculated are shown in Table 1. Where there are three options the scores are essentially:

Table 1 Original OHS factors and scoring options

0 = significant problem

4 = minor problem

8 = no problem/health

Where the scores are allocated in sextants, the health of each sextant is scored either out of 2 for wear (total score = 12) or 4 for caries and periodontal disease (total score = 24).

When adapting this protocol to the online OHS within DEPPA¹⁰ it was considered important to preserve the relative weightings as these had previously been validated⁷ and the reproducibility was found to be strong.⁹ For the first five components inputs were provided in DEPPA to score these in line with the original protocol. For the last three components, scored originally in sextants, modifications were made as follows:

• 1 Dedicated inputs were provided in DEPPA for a summary non-carious tooth surface loss as follows: (a) Normal/none – score 12; (b) More than expected for age – score 6; (c) Much more than expected for age - score 0. The maximum score was therefore maintained at 12.

• 2 For tooth health the existing disease severity score produced by the PreViser algorithms from inputs in DEPPA on carious lesions, defective restorations and existing restorations was utilised as summarised in Table 2. The maximum score for this domain was therefore maintained at 24.

  Table 2 Tooth health scoring in DEPPA

• 3 For periodontal health the existing disease severity score produced by the PreViser algorithms from inputs in DEPPA on pocketing, bone height and bleeding was utilised as shown in Table 3. The maximum score was therefore maintained at 24.

  Table 3 Periodontal health scoring in DEPPA

Once all inputs are made by the dental team and submitted on line, a report is returned within 60 seconds. This includes future disease risk scores, the modified version of the OHS described above and fee category recommendations. A mock-up of the patient version of this OHS report is shown in Figure 3. The dentist's version for the patient record also details the inputs. This OHS report is colour coded so that dark green broadly represents health, amber represents mild-moderate problems and red indicates severe problems.

Figure 3

A mocked Up OHS patient report from DEPPA

Comparison of pilot data with 2009 Adult Dental Health Survey

The primary aim of the reported analysis was to benchmark the OHS data collected via a centralised and anonymised electronic recording system by 25 general dental practitioners for 640 patients, with a national comparator of the 2009 ONS Adult Dental Health Survey. A secondary aim was to explore the potential of such an electronic database for use in audit to help inform developments in service quality and delivery.

DEPPA was piloted by 25 volunteer dentists¹² who provided qualitative and quantitative feedback. Anonymised data from the inputs and scores were collected centrally and analysed for 640 recall patients.

The development, practitioner and patient evaluation of DEPPA has been previously reported,¹² but the published study did not report on the full data submitted on patient oral health status during the trial and this is now presented. In this paper we compare and contrast a breakdown of the eight component scores making up the OHS to data from the ADHS⁴ as a measure of cross validation.

Results

Busby et al.¹⁰ reported the feedback from the 25 dentists piloting DEPPA in full. The aspects of this feedback relevant to the validity and development of the OHS as part of DEPPA were covered by two statements put to the pilot dentists for scoring:

• 1 'The revised oral health score gives a valid measurement of each patient's oral health status'

• 2 'This revised oral health score is an improvement on the old score'.

A score between zero and ten was permitted where ten equated with total agreement and zero with total disagreement. The 25 pilot dentists scored both of the above statements at an average of 8.6.

Table 4 presents a summary of the oral health scores recorded by the 25 participating dentists for the 640 recall patients and Table 5 shows the percentage of patients recoded in each 'band' of health for each component of the OHS. Finally, Table 6 compares aspects of the oral health status of the DEPPA group of patients with the oral health status of a sample of the general population recorded in the ADHS 2009.

Table 4 Summary of oral health scores across the 640 recall patients

Table 5 Percentage of patients in each band of health for each aspect of the OHS

Table 6 DEPPA aspects of oral health status compared with oral health status recorded in the Adult Dental Health Survey 2009

The results in Table 6 have been rounded to the nearest whole number apart from the number of 'unhealthy' teeth per patient. The average age of dentate patients in the ADHS group was 45. The whole DEPPA group was dentate, whereas in the ADHS, 6% were edentulous. 'Healthy teeth' in the DEPPA group are categorised as teeth that have neither caries nor defective restorations. 'Healthy teeth' in the ADHS refers to the lack of obvious decay in the crowns or roots of teeth.

Discussion

The OHS is a composite outcome measurement of current oral health status, taking into account patient perceptions of life impacts (on three aspects) and professional assessment of disease status (on five aspects). The development, practitioner and patient evaluation of DEPPA has been previously reported,¹⁰ but the published study did not report on the full data submitted on patient oral health status during the trial. The OHS does not measure future disease risk; this is the function of the risk calculator in DEPPA, based upon the PreViser algorithms. The importance of disease risk assessment and its incorporation into DEPPA has been discussed previously.¹⁰ Patients with high risk scores do not necessarily see these risks translate into current disease and life impacts because risk does not imply cause and effect,¹¹ more the likelihood of a given disease developing in a given individual. The OHS component of DEPPA, which is the focus of this report, measures current disease status and its life impacts.

From the outset the OHS was designed as a tool to support patient communication through biofeedback. Biofeedback has been shown to positively impact upon lifestyle behaviour change in primary care dental practice.¹² In this paper we report its outputs relative to a national benchmark, the 2009 ADHS and therefore indicate its potential value in audit and in surveys of the oral health demographics of practice populations where DEPPA scores are available for that population. An outcome measurement of oral health status such as the OHS enables changes in oral health status to be monitored for individual patients. It therefore helps the clinical team to establish whether the care plan for a particular patient is effective. If the patient already enjoys good oral health this would be demonstrated by the OHS being maintained. For the patient with poor oral health this would be demonstrated by the score being significantly raised by their oral health care programme. Practitioners' attention should therefore be drawn particularly to recall patients who have a declining OHS.

The OHS also allows changes in the oral health of 'groups' of patients to be audited by examining average scores, providing valuable utility in monitoring the outcome of practice oral health policies. The pilot study involving 25 dentists and 640 recall patients with an average age of 53 could be deemed to have established an initial oral health score benchmark for the DEPPA version of the OHS at 79.5 (see Table 4). If a practice wished to audit its performance against this benchmark it would be important, when reflecting on their result, to consider factors such as the average age of patients audited, as there is a tendency for scores to fall with age (vide infra). It would also be important to consider the percentage of new patients audited, as this initial benchmark only includes recall patients. There may even be a case for looking at post code deprivation indices. However, more than 75% (n = 19) of the dentists in this study were within five points of this average and only 8% (n = 2) were more than five points below it. The data in Table 5 effectively provides a benchmark for the individual components of the OHS designated in the national reference sample (NRS). Deeper analysis into the individual components may give insight into which aspect(s) of oral health need more attention in a practice oral health policy where scores are below average. These data could help with planning staffing levels needed to support care for a specific patient base in a practice. Clearly, with centralised electronic data recording, the accuracy of the NRS can be improved with time as more recall patients are entered on to the system. In this respect, the move to an online facility with the central collection of data provides a significantly improved opportunity to support practices through the clinical audit of oral health outcomes. It is intended that practices will receive regular reports (illustrated in Table 7) containing the contemporary NRS benchmark and this report will also contain data from the other two elements of DEPPA on future disease risk (perio, caries, wear and soft tissues/cancer) and capitation fee indication (A-E).

Table 7 Mock up of proposed practice report

As more practices start engaging in the use of DEPPA the NRS will be constantly updated by the inputs received, providing an invaluable public health monitoring tool for dental teams choosing to use it.

The previous version of the OHS⁷ and the new DEPPA version¹⁰ have both been perceived as valid tools by dentists piloting their use. The data in Table 6 could be held to support these views by cross validation of the DEPPA results with similar data from the ADHS 2009.⁴ Examination protocols were similar but not identical for the two samples, providing some limitations to the benchmarking process. The DEPPA group of 640 were 6 years older on average than the ADHS group of 6,469 patients. Furthermore, the DEPPA group were recall patients attending primary care dental practices whereas the ADHS group were designed to be a representative sample of the population of England, Wales and Northern Ireland. The OHS data reported here and collected as part of the full DEPPA are essentially derived from a convenience sample of patients within Denplan Excel practices, rather than a randomised sample of community dwelling adults, as in the ADHS. Therefore, the reported outcomes of the OHS are not generalisable to the UK population. However, as the number of patients entering the DEPPA centralised data repository increases, the value and utility of the OHS data for service planning and population health monitoring within that recorded population will increase substantially.

Despite the limitations discussed above, the two datasets are broadly comparable; differences are of an expected order of magnitude. For example, it might be predicted, as illustrated, that the DEPPA group would have healthier teeth. However, they have fewer teeth (24 versus 25) than the younger ADHS group. The age sensitivity of this aspect is illustrated in the ADHS by the finding that in the 55-64-year-old age group the average number of teeth had fallen to 23. The DEPPA group also have fewer sound, unrestored teeth (15 versus 18), but again this is age sensitive, with the ADHS finding that the number of sound and unrestored teeth in the 55-66-age-group was 12. A similar situation can be observed in the case of periodontal disease. The older DEPPA group have a higher percentage of patients with clinically significant pocketing (52% versus 45%). This could be predicted from the ADHS finding that 61% of 55-64-year-olds have significant periodontal pockets. Importantly, however, a significant ongoing treatment need, particularly with regards to chronic periodontal disease, is highlighted within the DEPPA recall population.

Conclusions

This study has shown that the OHS scoring systems within DEPPA provided an oral health profile consistent with that of the ADHS 2009. It also presents valuable opportunities to audit patient outcomes progressively, either for individuals or for groups of patients, against a national reference sample benchmark of 79.5 for recall patients with an average age of 53 years. Audit is facilitated considerably by this move from, essentially a paper based system, to a secure on-line tool with central data collection. The benchmark will be continuously revised as more dentists use the system and enter patient data, providing a powerful resource for oral health planning and implementation of oral health strategies at a population level for dental teams choosing to use it.