Introduction

The early detection and treatment of enamel-only proximal caries can preserve tooth structure and prevent the subsequent cycle of treatment and re-treatment that is involved with more invasive treatment. Recent guidelines from the NHS encourage preventive care in dental practices, especially for young children. Patients differ widely on their willingness to pay for preventive therapies,1 but nearly all parents value a healthy dentition for their children and are willing to invest resources to maintain this.2 However, it is well documented that preventive care in children and adults in the UK is offered less frequently than it should be and therefore if prevention is to be adopted more widely in adults, caries detection must be time-efficient and accurate. Only then can the ideal, personalised caries assessment of adult patients and their preventive care be developed.

Radiographic examinations can increase the number of carious lesions that are detected over those that would be detectable by clinical examination alone. The Department of Health recommends that dentists use the Faculty of General Dental Practice (UK) guideline document Selection criteria for dental radiography in determining the frequency of use of bitewing radiography.3 Nevertheless, systematic reviews have consistently shown that detection of proximal caries on bitewing radiography has a low sensitivity.4 A number of studies have reported poor diagnostic sensitivity for radiographic detection of demineralisation by dentists. In a classic study by Mejàre et al.,5 premolar and adjacent teeth surfaces were examined radiographically and visually. The premolar teeth were then extracted for orthodontic reasons. They found that the sensitivity of detection of enamel-only proximal caries was 36.7%. Other studies have found similarly low sensitivity values.6,7

The introduction of artificial intelligence (AI) methods allows routine tasks to be conducted more quickly and efficiently. AssistDent is an AI software product, developed by Manchester Imaging Limited, and uses machine learning algorithms to search for evidence of enamel-only proximal caries on bitewing radiographs.8 It is an aid to the dentist, assisting their clinical decision-making by providing on-screen prompts for potential locations of enamel-only proximal caries. The final judgement about whether enamel-only proximal caries is present, or not, is a decision for the clinician.

The null hypothesis of this research was that there is no difference in the performance of dentists in diagnosing the presence of enamel-only proximal caries on bitewing radiographs with and without the use of AssistDent.

Methods

A pilot study9 was conducted with dental students to assist in developing the methodology and to provide initial data for a sample size calculation. The final study protocol, participant information sheet and consent forms for the study were approved by the Manchester University Research Ethics Committee (Ref: 2020-9892-15955).

Participants were recruited from two sources: 1) dentists practising as general dental practitioners who, in addition, provide tutorage for dental students within the University of Manchester Dental School; and 2) practising dentists undertaking postgraduate training within the University of Manchester Foundation NHS trust. All signed informed consent forms.

The dentists were randomly divided into control and experimental arms by pairing the participants within the recruitment sources according to the order in which they were enrolled. The first of each pair of participants were randomly assigned to either the control or experimental arm, with the second assigned to the other group. This method ensured random assignment while maintaining even arm sizes equally balanced between the recruitment sources as the study progressed.

Both arms examined the same images using the same graphical user interface as shown in Figure 1. In the control group (n = 11), the caries prompting function of AssistDent was disabled in order to measure the ability of the group to detect enamel-only proximal caries without the use of AI software. In the experimental group (n = 12), the caries prompting function of AssistDent was enabled in order to assist the participants.

Fig. 1
figure 1

The AssistDent graphical user interface. Orange arrows indicate the presence of enamel-only proximal caries and purple circles indicate that the tooth has been detected and analysed by the AI algorithm. The experimental group were provided with an initial set of indicators generated by the AI software while no initial indicators were generated for the control group. Controls in the lower left menu enable users to add and delete indicators. Indicators can be moved by dragging them to the desired position. Once the analysis of the image is complete and the user is satisfied that every enamel-only proximal caries is marked by an arrow, the participant pressed the save button, bottom left, and proceeded to the next image

Full size image

So that the study would have applicability to general practice, a total of 1,446 bitewing radiographs were collected from a range of different sites (one teaching hospital site and nine general dental practitioner [GDP] sites). Separate ethical approval had been received from the Integrated Research Application System (IRAS project ID: 248306, REC reference: 18/NI/0111). As illustrated in Figure 2, a test set of 103 images were selected randomly, but with the same proportion across the image acquisition sites, and excluded from all AI model training and evaluation. A further subset of 24 images were selected form the test set for the study, again representative of the acquisition sites but with the criterion that there was at least one enamel-only proximal caries in each image. Images from one of the GDP sites were excluded due to their poor quality. Another GDP site lacked images with one enamel-only proximal caries; therefore, two study images from this site had no enamel-only proximal caries. For practical purposes, the prevalence of caries was higher in our study set than the general population. However, this did not affect the measures of sensitivity and specificity. The images were presented to each participant in the same order, grouped according to the acquisition site.

Fig. 2
figure 2

Selection of images for the study. All images were divided into a training and test set, with the test set excluded from the training and evaluation of the AI models. A subset of training set images was chosen for the study

Full size image

Gold standard annotation of all classes of proximal caries was obtained from a panel of five dento-maxillofacial radiologists and one professor of restorative dentistry, each of whom annotated the location and grade of caries on a set of images independently of each other. Each image was annotated by at least three members of the panel. These individual expert annotations were consolidated by retaining, in the gold standard set, any region identified by any expert as enamel-only proximal caries while removing duplicate annotations, resulting in a gold standard set of 1,972 examples of enamel-only proximal caries for algorithm training and evaluation.

The caries annotations entered by each participant were collected remotely via a web application and analysed to determine whether they were true positives (correct identifications of enamel-only proximal caries) or false positives (annotations not corresponding to the location of the gold standard enamel-only proximal caries). Annotations corresponding to dentine proximal caries were recorded but excluded from this analysis. The true positive rate or sensitivity of diagnosis is a measure of how well a participant detected the enamel-only proximal caries and was calculated as the sum of true positives divided by the sum of the gold standard caries. True negative rate or specificity is a measure of how well the participant identified healthy surfaces and did not mark them as carious. The probability of a false detection is quoted in terms of a false positive rate, calculated as the sum of false positive detections divided by the sum of healthy surfaces, which is equal to 1-specificity.

Results

A per-participant breakdown of the evaluation scores and performance measures, together with the aggregate scores and measures for each arm, is presented in the online supplementary information. The data demonstrate that 23 dentists were recruited, 11 in the control arm and 12 in the experimental arm. These were balanced within the arms between the two recruitment sources of general dental practitioners and practising dentists undertaking postgraduate training. All participants analysed all 24 images.

Figure 3 presents the mean true positive and true negative rates over all participants and the 95% confidence intervals, for each arm. The improved mean true positive rate of the experimental arm participants (75.8% with AssistDent) compared to the control arm (44.3% without AssistDent) is clearly visible. This is accompanied by a decrease in true negative rate from 96.3% to 85.4%. Table 1 presents the statistical analysis for the performance measures within each arm together with the result of a student t-test. The t-tests demonstrate that the improved true positive rate and reduced true negative rate of the experimental compared to the control arm were significant, with p values below alpha of 0.01.

Fig. 3
figure 3

Bar charts showing the mean true positive and true negative rates together with their 95% confidence intervals, for each arm

Full size image
Table 1 Statistical analysis of the mean per-participant performance measures for each arm
Full size table

Table 2 presents the results of an odds ratio comparison of true positive and true negative rates for each arm. The ratio between the experimental and control groups with a value greater than 1 indicates that use of AssistDent increased the ability to detect enamel-only proximal caries by 71%. Similarly, the ratio of less than 1 for true negative rate indicates that the experimental arm participants were 11% less likely to correctly identify healthy proximal surfaces as non-carious. A per-participant and per-group breakdown of the results are presented in the online supplementary information together with additional summary statistics for each group.

Table 2 Odds ratio of true positive and true negative rates of the experimental group compared to the control group with 95% confidence intervals
Full size table

Discussion

Small areas of proximal surface enamel demineralisation are difficult to detect visually on radiographs.10 In a review of diagnostic studies reported by Keenan and Keenan,11 the authors found lower sensitivities for the detection of proximal lesions in clinical studies than in vitro studies. The mean sensitivity in the clinical studies was 0.24 and for the in vitro studies was 0.43.

Other studies that have measured sensitivity and specificity of caries detection using radiographs have found a large variation. This may be due to variation in the range of caries depth or the use of in vitro studies. In our in vivo study, we mainly used a sample of radiographs from a range of dental practices that were taken in the course of the examination of patients. The low sensitivity and high specificity of our control group in detecting caries on bitewing radiographs is comparable with the above studies, and with that reported in a systematic review.12

Low sensitivity in the detection of enamel-only caries is problematic in the context of preventive dentistry. A significant increase in sensitivity is required over the values reported in these studies, even at the expense of an increased number of false positive detections. Enamel lesions are treated in a conservative manner with dietary and interdental cleaning advice, fluoride treatments and, potentially, resin infiltration. This, in conjunction with cooperation from the patient, will prevent operative intervention and the entry of the tooth into a restorative cycle of increasingly larger restorations and eventual extraction. An increase in sensitivity of detection is important as it provides a better assessment of the full extent of the patient's caries experience. The dentist should create an individually tailored dental care plan with the patient based on a true picture of their oral health. Some aspects of this risk assessment are based on the findings from bitewing radiographs obtained during the process of screening for early, enamel-only caries. In those individuals who suddenly develop many early lesions, this would merit an investigative effort to determine any underlying medical or social causes followed up by more frequent recall intervals. The number of lesions present is an important variable when predicting future caries experience.13 However, increased sensitivity carries with it the likelihood of an increased number of false positive detections. This trade-off is inevitable, but it is clearly desirable that the false positive count should be kept as low as possible. The risk to patients arising from false positive detections is minimal provided clinicians follow guidelines and do not restore teeth with enamel-only caries. In a situation where a false positive detection is made when no caries is present, an individual may, as a result, receive preventive advice without any obvious clinical benefit. The decrease in specificity observed from the use of AssistDent accompanies a much larger increase in sensitivity.

There has been a growing interest in general dental publications and online forums in the use of AI in dentistry. A recent review by Schwedicke et al.14 has described the basics of AI and explored its potential use in diagnostics, treatment planning and conduct in dentistry. Computer image analysis is a particular application of AI in dentistry (and medicine more generally). An earlier, scoping review15 refers particularly to the application in image diagnostics. There is a relatively small number of studies related to caries detection. For example, Lee et al.16 investigated the performance of a convolutional neural network on classification of caries in images of individual teeth isolated from periapical images. The study did not focus specifically on enamel-only caries. Srivastra et al.17 also trained a convolutional neural network for fully automatic detection of caries in bitewing images.

To our knowledge, AssistDent is the only commercially available AI system for use in the clinical diagnosis of enamel-only proximal caries, acting as a prompting system to support dentists' diagnostic decisions. Schwendicke et al.15 recommend that 'the dental community should appraise [the AI systems] against the rules of evidence-based practice'. This study is an example of such an appraisal.

It is important that dentists receive appropriate training with any new diagnostic system. Qudeimat et al.18 investigated the effect of ICDAS training and found a significant increase in over-treatment recommendations. The reproducibility of any diagnostic system depends on the experience of the clinician and this is especially so where the diagnosis is a visual score rather than AI-based.13 How should our profession address the issue of poor sensitivity in detection of enamel-only proximal caries? Audit, reflection with peer review and evaluation of past performance are essential parts of dental practice. An audit of caries diagnosis will rely on identifying opportunities for improvement, comparison with an accepted standard of care and implementing change. The increased sensitivity arising from AI-supported detection may provide a useful standard for audit in caries detection. Testing will also reveal clinicians with unacceptable variation in bitewing analysis competency assessments, with direction towards further training. More efficient caries detection may help in identifying patients with higher caries risk, while the associated display of detected caries can provide a basis for encouraging a detailed discussion with the patient of their oral health and the factors affecting it. The intuitive graphical screen display of AssistDent may be more readily understood by patients in comparison to other algorithm-based tools such as the Cariogram. The latter uses a pie chart of important factors to illustrate the probability that future caries may be prevented.19,20 However, once the high-caries risk individuals are identified, it is important to further investigate all the factors predisposing to caries, such as a high sugar intake and infrequent brushing.21

Caries assessment by dentists using AssistDent is compatible with the ICCMS caries management system of the ICDAS Foundation and could be used in conjunction with it. AssistDent provides feedback on the proximal enamel surfaces using radiographs, whereas ICCMS is a mainly visual assessment of the non-proximal surfaces for caries. Both systems aim to maintain tooth structure and encourage preventive care by developing a caries assessment for each patient. At this stage of its development, this AI software is used as an aid to diagnosis, but future developments could include monitoring the progression of caries.

Conclusion

AssistDent AI software increases dentists' sensitivity when assessing enamel-only proximal caries. This increased sensitivity is accompanied by a smaller decrease in specificity. The increase in false positive diagnoses may occasionally result in unnecessary preventive treatment and associated use of limited healthcare resources.

The increase in sensitivity in detecting enamel-only proximal caries should enable better informed targeting of preventive treatments. This would contribute to avoiding the requirement for later restorative treatment, resulting in an overall saving of resources and an improvement in the dentition of patients.