A computer-assisted learning programme to reduce caries detection variability

Abstract

Design A randomised controlled trial using block design was devised.

Intervention A computer-assisted learning (CAL) programme, with feedback, was tested.

Outcome measure Sensitivity and specificity of caries diagnosis, and the summary receiver operating characteristic (SROC) method for summarising true-positive ratio (sensitivity) and false-positive ratio (1 — specificity), were used to analyse the dichotomous data.

Results The mean sensitivity for dentine caries detection was 76.3% (standard deviation (SD), 13.0%) for the experimental group and 66.9% (SD, 14.8%) for the control group (P=0.005). Mean false-positive ratios were similar (experimental 28.1% and control 28.7%; P=0.5). The area under the SROC curve was 0.832 for the experimental group and 0.773 for the control group (P=0.002).

Conclusions The CAL programme does improve diagnostic performance. Improving the cognitive feedback provided by the programme should be considered before implementation.

Commentary

Caries detection is an important subject to study: many papers have reported the large variations between observers for detecting lesions. Part of this variability is due to the use of low diagnostic accuracy tests (visual inspection, sharp explorers and radiographs) and part due to a poor understanding of the caries process together with interpretation of detectable changes. The authors of this paper have concentrated on developing and testing a method for improving the interpretation of radiolucencies as an indirect method for detecting caries. This article offers the opportunity to comment on a programme of evidence-based teaching of radiographic diagnosis and to comment on an evidence-based evaluation of that programme.

The CAL programme

The CAL programme had two sections: calibration and validation (and a section: evaluation for evaluating the programme in this experiment). The two sections of the programme itself offer the student, teacher-independent, ways of improving their diagnosis against a published gold standard. In the case of the validation section there is a histological gold standard diagnosis in which the user is shown the surface condition and a histological section of the surface, which has just been viewed on the radiograph.

The experimental design of the programme evaluation

The CAL programme was evaluated in a randomised control trial design. The authors used evaluation radiographs with a histological gold standard from a previous independent publication. The data was collected by computer during the study and the gold standard applied independently of the authors. Data analysis included checking for selection bias and analysing data using the S-ROC summary technique.

Part of the trade-off in designing a CAL is deciding how realistic to make the diagnostic process versus exposing the students to sufficient disease examples to decrease the training time. The authors elected to use in section calibration radiographs of 118 surfaces of which 18 surfaces had dentine caries (prevalence 15%), in section validation they used radiographs of 105 surfaces of which 45 surfaces had dentine caries (prevalence 43%) and in the evaluation section radiographs of 56 surfaces of which 23 had dentine caries (prevalence 41%). Such high disease prevalence might be found in some low socio-economic groups but would not be representative of middle-class regular dental attendees. The impact of disease prevalence on the predictive value of positive test findings (an interproximal lucency) can be seen from the fact that a test with both a sensitivity and specificity of 80% in a population with a disease prevalence of 15% would have a positive predictive value of only 41%. This suggests the teaching experience would not be representative for the prevalence found in a true general practice population. However in order to reproduce this experience in this CAL programme user time would have been unacceptably long.

One of the problems of observer detection of caries is the false-positive interpretation of ‘burn-out’ and cement-enamel junction regions as caries. It did not appear from the paper that these misinterpretations of normal anatomy as false disease were explicitly addressed in the CAL method and it may improve the student's knowledge if examples were included in the programme. In addition, the larger issue of treatment outcomes related to this programme should be addressed. This CAL programme improved average sensitivity by 9% without an increase in false-positive interpretation. However if a CAL were to produce increased lesion detection, which results in premature unnecessary operative intervention, then such a programme would result in negative health outcomes in the form of inappropriate treatment. To prevent this it would be wise to ensure that caries detection CAL programmes are designed to incorporate treatment decision modules that take into account patient caries risk, lesion severity (depth), activity (change over time) and cavitation state, to ensure that lesion detection does not automatically result in operative care.

In summary, however, the authors are to be commended for developing a tool for improving diagnostic validity and for reducing caries detection variability.

Practice point

• A CAL programme can help to improve diagnostic validity and reduce caries detection variability.