Introduction

Artificial intelligence techniques are fast developing and applied to various industries. Healthcare service is one of the most potential fields to take advantage of those techniques because of the fast accumulation of massive and complex features of medical data. Especially with the development of deep learning techniques, the convolutional neural network (CNN) model and its variances are used in many fields of image analysis, such as classification and object detection. Because many medical data are collected in image formats, such as skin image datasets1 and ophthalmoscope2 images, artificial intelligence techniques are widely applied to various medical fields and tasks. Especially, regional convolutional neural network (RCNN) models are widely applied to the medical field in order to detect some medical signs from a given image to detect the region of breast cancer3, pneumonia4, and so on.

Dentistry is also one of the hospital branches where artificial intelligence techniques are vividly applied because it takes and uses a lot of medical images during a clinical routine. Patients admitted to hospitals are usually examined by radiographs. From simple intra-oral periapical x-rays to computed tomography, there are various tools to examine patients’ oral health and status. Panoramic dental radiography is one of the most commonly performed imaging techniques by dentists and oral surgeons in everyday practice. It shows good patient acceptance because it is simple and painless and also uses a small amount of radial dose5. Panoramic dental radiography produces an image that includes not only maxillary and mandibular dental arches, which are mainly examined by dentists, but also the surrounding structures as the maxillary sinus, nasal fossa, temporomandibular joints, styloid processes, and hyoid bone6. Radiographic findings from these structures are suggested for diagnostic features of many diseases from dental anomalies to systemic diseases like hypoparathyroidism, hyperparathyroidism, and osteoporosis7. However, orthodontic and surrounding areas shown in panoramic dental radiography are complicated regions so correct diagnoses of anomalies can be very laborious and time-consuming, also potentially inaccurate8. Also, dentists might only concentrate on teeth of symptoms and regions of interest due to lack of time.

One of the methods to reduce this burden is computer-aided anomaly detection techniques. Diagnosis of dental anomalies with computational analysis of panoramic dental radiography is not a novel concept. Before the recent development of deep learning techniques, scientists tried to diagnose dental anomalies through image texture calculations9,10 or abnormality thresholding11. And with the recent advances of deep learning techniques, it has been quickly applied to increase the performance of analyzing panoramic dental radiography. Many of the studies are focused on detecting some signs of dental diseases from panoramic dental radiography. Especially, a large proportion of previous studies mainly selected carious lesions as their detection targets9,10,11,12,13,14, and few studies are focused on another dental anomaly, such as periodontal bone loss15, odontogenic cyst, tumor16, osteoporosis17, impacted tooth8, and so on. These studies used variations of convolutional neural network models developed for image classification or object detection tasks, such as MobileNet V212, single-column deep convolutional neural network (SC-DCNN)17, regional convolutional neural network (RCNN)13.

These previous studies could detect targeted disease with satisfying performance, however, most of the works targeted only one or a small number of diseases or abnormal signs. This limitation makes it hard to take one of the major advantages of panoramic dental radiography, which covers a wide area from individual teeth to mandibular and maxillary regions. Also, to reduce the burden of dental clinicians, it is needed to filter out healthy images to reduce the number of images that need to be manually examined.

Here, we selected fine-grained 17 anomalies that are closely related to patients’ quality of life and also can be detected from panoramic dental radiography. We categorized them into four groups: carious lesions, calcifications, anomalies in dental regions, and anomalies in surrounding regions. To distinguish the dental region and surrounding regions, we used the method suggested by Langland, et al.18. Features and objects shown in zone 1 are included in the formal category, while others shown in the rest part of the image, from zone 2 to zone 6, are included in the latter category.

A detailed list of anomalies, their main features, and criteria for labeling in the image are shown in Table 1. We built and trained a deep learning model to detect signs of those 17 diseases from an image. It covers a wide region of panoramic radiography and will reduce the burden of clinicians in hospitals and help the prevention and early diagnosis of diseases.

Table 1 Category, name, and labeling criteria of 17 fine-grained dental anomalies.
Full size table

Result

Dataset generation

A large amount of high-quality datasets is one of the essential factors to take advantage of machine learning techniques. To build a model that is directly applicable to real clinical practices, we used panoramic dental images taken from local dental clinics. In this study, a total of 22,999 panoramic dental images were collected from 30 local dental clinics during a year, from July 2020 to July 2021. The size of image datasets used in previous studies varies from 87 to 3,000 according to the review14. Compared with those datasets, the size of our dataset exceeds the size of the others. The dataset is manually labeled by a dental radiography expert. Figure 1 shows some examples of labeled images.

Figure 1
figure 1

Examples of labeled 17 dental anomalies. (a) Calcified carotid atherosclerotic plaque, (b) lymph node calcification, (c) ossification of stylohyoid ligament, (d) tonsillar calcification, (e) cervical caries or abrasion, (f) dental caries or coronal defect, (g) proximal caries, (h) secondary caries, (i) periapical radiolucency, (j) external root resorption, (k) tooth overlapped with mandibular canal, (l) impacted tooth, (m) residual root, (n) supernumerary tooth, (o) mucosal thickening on maxillary sinus, (p) radiopacity in jaw, and (q) retention pseudocyst on maxillary sinus.

Full size image

Anomaly detection

Our detection model consists of four parts (Fig. 2a). First, we convert DICOM formatted images directly transferred from dental clinics into PNG format. Then it detects anomalies through trained faster R-CNN models19. Basically it is an object detection model, we got object boxed regions of high possibilities of anomalies (Fig. 2b). Next, we filter out some boxes that are not located in a predetermined region (Fig. 2c). For example, if detected boxes are about carious lesions, it is obvious that the boxes should be located in the dental region. However, if some of the boxes are not located in the region but in the surrounding region, we can be sure that those boxes are absurd, so we can filter out those boxes. In the final stage, we narrow down the region of abnormal signs from a box form to a polygon (Fig. 2d). Through this stage dental clinicians can get high-resolution information not only the location and the type of anomalies found in the image, also the specific regions that show the feature of the anomalies. We used a prebuilt library, Detectron220, in the polygon shaper stage. Figure 2e is an example of our detection model. There are two signs of dental anomalies, proximal caries and periapical radiolucency. The proximal caries in the figure is in asymptotic stage so it may be stay undetected without careful examination. This model successfully detected it which means that the model can help early detection of selected anomalies.

Figure 2
figure 2

Anomaly detection process of our model. (a) Overall process of this model, (b) result of disease detection stage. Boxes are found from the faster R-CNN architecture. (c) Example of position verifier, carious lesion, for example, is known to be located in dental region, so boxes which are located in surrounding region are deleted. (d) Result of polygon shaper. To notice dental clinicians why the regions are detected to be abnormal, this model narrows down the region into polygon shape. (e) Example figure of final outcome of the model. Proximal caries in upper tooth is in asymptomatic stage but this model can find this. This model can contribute the early detection and diagnosis of several dental anomalies before they go severe, improving outcomes dental healthcare and patients’ quality of life.

Full size image

We tested our detection model using part of our dataset. We divided images to training, validation and test datasets. Train and validation datasets are used to training phase and test dataset are used to evaluate our model. Table 2 shows the class-wise performance of our detection model applied to the test dataset and Table 3 contains information about detailed numbers of test datasets. Our model shows very high specificity, in most classes over 95%, which means it can filter out most of the normal or healthy images. High specificity means that this model can be used to reduce the burden of examination of dental clinicians because it successfully filters out healthy images so that clinicians can focus on other images. Precision and sensitivity vary depending on the type of anomalies but generally the score exceeds or is similar to that from previous studies.

Table 2 Class-wise performance score of the trained model.
Full size table
Table 3 Class-wise performance score of the trained model.
Full size table

Discussion

As we mentioned above, panoramic dental radiography is one of the most widely and frequently used imaging techniques in dentistry. It is safer than other imaging tools and quickly takes a wide range of dental structures so it is essential to make diagnoses and further treatment plans for patients, but due to the complicated structure of the dental region and lack of time, clinicians mainly focused on small parts of the images. If that neglected information is fully detected and noted to clinicians, it will improve the overall quality and consequences of the treatment. To achieve that goal, we applied an artificial intelligence technique which is widely used in the image analysis field to automatically detect regions of anomalies. We trained our model focused to increase the specificity to help clinicians filter out healthy panoramic dental radiography so as to decrease the number of images to be examined and to alleviate the burden of clinicians. First, we selected 17 major dental anomalies which are closely related to patients’ oral health. These anomalies can lead to serious outcomes if ignored or are related to other systemic diseases. It means that the early detection of those anomalies can prevent severe outcomes and can be used as a marker to suspect other systemic diseases. We categorized anomalies into four groups corresponding to their clinical features and locations.

First, carious lesions are considered the most prevalent problem in dentistry14. Because carious lesions cause more serious problems if ignored, prevention and early diagnosis are very important.

Many previous pieces of research applied artificial intelligence techniques to detect carious lesions13. Depending on its progress and location, various treatments can be used to treat carious lesions.

Here we divided carious lesions into four categories corresponding to their clinical features for fine-grained diagnoses; dental caries, cervical caries, proximal caries, and secondary caries. These subtypes of carious lesions show unique features which are related to their diagnoses and treatment. First, cervical caries is considered the most dangerous type because it leads to the rapid loss of tooth due to its location. Proximal caries is a type of carious lesion which is located on the surfaces between adjacent teeth. They are the most difficult type to detect because they cannot be visually or manually detected. Finally, secondary caries is a disease that occurs on the tooth after the filling. Because it takes a lot of burdens to detect21. This fine-grained diagnosis of carious lesions is important to early detection of caries before their progression to severe stages and to prevent further loss of dental tissues.

The second category of our fine-grained model, calcifications, occurs when calcium accumulates in body tissue. The diagnostic criteria of calcifications are their anatomical locations, distributions, numbers, sizes, and shapes22. Calcifications in maxillofacial areas can be found through examinations of panoramic dental radiography but there are very few studies conducted regarding them23. Though the presence of calcifications on panoramic dental radiographs is uncommon, their detection is important to prevent the further progression of diseases. We selected four calcification anomalies; Calcified carotid atherosclerosis plaque, lymph node calcifications, calcifications of the stylohyoid ligament, tonsillar calcifications (tonsilloliths)24.

Our third category is dental anomalies. We included dental disease features and abnormal structures shown in the dental region18 to this category. Dental anomalies are abnormal forms or structures of teeth in the dental area. We selected six dental anomalies which are critical factors of dental health; external root absorption, impacted tooth, periapical radiolucency, residual root, supernumerary tooth. Some of these anomalies often cause symptoms such as pain, halitosis, and bleeding, and can be used as diagnostic markers, and anatomical factors when planning further dental surgeries. For example, Periapical radiolucency is the radiographic changes around the apex of the tooth and is the sign of inflammatory bone lesions. Recent studies present that periapical radiolucency may be caused by several diseases such as cirrhosis25. External root resorption is an undesirable dental injury that causes a loss of some parts of a tooth and can be seen radiographically. This type of anomaly damages the underlying tissues and causes a number of complications including infection, loss of teeth, pain, and so on26. The positional relationship between the mandibular canal and corresponding tooth is a key anatomic factor to make surgical plans such as extraction of the mandibular third molar because damage to the inferior alveolar nerve affects the function of the stomatognathic system and the quality of life of patients27. Panoramic dental radiography is one way to evaluate the risk of nerve injury before the extraction28. Impacted teeth can cause several symptoms such as swollen gums, halitosis, and pain when opening the mouth. If ignored, it causes severe complications such as infection, cysts, absorption, and many gum diseases. A recent study presented that an impacted tooth might have some association with a large central osteoma29. A residual root is a leftover of a tooth in the jaw after an extraction. It sometimes causes infections and pain. Usually, it is recommended to extract with a local anesthetic. Finally, supernumerary teeth may lead to many severe problems like displacement, crowding, root resorption, dilaceration, loss of vitality of adjacent teeth, and even ameloblastomas and odontomas in severe cases. So, clinicians should aware of the existence of the occurrence so that they can formulate treatment plans30.

The last category is anomalies located in surrounding regions of the dental area. These anomalies are rarely related to oral health but may be used as potential markers to diagnose other related diseases. These radiographic anomalies are signs of inflammatory processes of that region and are known to be related to several diseases. For example, previous studies showed that retention pseudocysts of the maxillary sinus may have some associations with allergic and inflammatory processes, trauma, periapical and periodontal infections31, radiopacity in jaws with many osteoblastic and osteoclastic activities32, mucosal thickening of the maxillary sinus with apical periodontitis, alveolar bone loss, and so on33.

To train a deep learning model to detect these many types of anomalies, it is essential to accumulate datasets including enough number of objects for every class. In fact, the most important factor of using artificial intelligence techniques is the quality and quantity of data. We built the system to collect panoramic dental images directly from local dental clinics and manually labeled them by a dental radiography expert. For a year, we accumulated a large and high-quality dataset compared to previous studies. This dataset is also still growing, so it has great potential in this field. Our tool successfully detects given anomalies with high performance especially for specificity and demonstrates that artificial intelligence can reduce the burden of dental clinicians by reducing the number of images that should be examined manually through detecting potential anomalies and filtering normal images.

Method

Data acquisition and labeling

Images are taken from 30 local dental clinics by panoramic x-ray cameras model PaX-i, Rayscan alpha from VATECH, Papaya 3d from Genoray, and RealScan from PointNix. Images are fully anonymized removing all patient identifiers. We installed a data transfer module in imaging devices to automatically collect panoramic dental images from clinics. Images are collected from July 2020 to July 2021. During the data collection period, data collection and labeling anomalies are done in parallel. Ethics approval was not required for this study because panoramic dental image data used in this study is obtained fully anonymized and no identifiable private information is included, and the study is performed in accordance with the Declaration of Helsinki. For each image, above 17 anomalies are manually labeled one by one by a dental radiology expert. Significant features and diagnostics used to annotation of every anomaly are described in Table 1. Image regions showing features of anomalies are first labeled in box form and then in the polygonal form to specify the exact region of the anomalies. Supplementary table 1 contains detailed numbers of objects and images of every anomaly.

Model architecture and training

Because images are originally transferred in DICOM format, we converted images to PNG format. We used faster R-CNN architecture for dental anomaly detection19. We trained a model for every 17 dental anomalies. We divided our dataset into a train/validation set and a test set. All images taken from July 2020 to March 2021 are used as a train/validation set, and others which are taken from April 2021 to July 2021 are used as a test set. We built a position verifier to filter out wrongly detected object boxes from the R-CNN model. It takes each object class and its location and examines whether it is well detected or not. Two classes, Carious lesions and anomalies in the dental region, are expected to be in the dental region and other classes are expected to be in the surrounding region. Finally, in the polygon shaper stage, we used Detectron220 to narrow down the detected regions from box form to polygon shape.

Detection and evaluation

To evaluate our model, we used an intersection over union (IOU) value to determine correctly-detected lesions from the output of the model. IOU is a widely used metric to evaluate the performance of object detection models by calculating the intersection over union between the ground-truth area and the prediction area. We determined the predicted box is correctly-detected when its IOU value is over 0.5. After the detection, we calculated precision, sensitivity, and specificity for evaluation. Table 3 shows detailed numbers of detection result on test dataset.