Detecting 17 fine-grained dental anomalies from panoramic dental radiography using artificial intelligence

Panoramic dental radiography is one of the most common examinations performed in dental clinics. Compared with other dental images, it covers a wide area from individual teeth to the maxilla and mandibular area. Dental clinicians can get much information about patients’ health. However, it is time-consuming and laborious to detect all signs of anomalies because these regions are very complicated. So it is needed to filter out healthy images to save clinicians’ time to examine. For this, we applied modern artificial intelligence-based computer vision techniques. In this study, we built a model to detect 17 fine-grained dental anomalies which are critical to patients’ dental health and quality of life. We used about 23,000 anonymized panoramic dental images taken from local dental clinics from July 2020 to July 2021. Our model can detect these abnormal signs and filter out normal images with high sensitivity of about 0.99. The result indicates that our model can be used in real clinical practice to alleviate the burden of clinicians.


Result
Dataset generation. A large amount of high-quality datasets is one of the essential factors to take advantage of machine learning techniques. To build a model that is directly applicable to real clinical practices, we used panoramic dental images taken from local dental clinics. In this study, a total of 22,999 panoramic dental images were collected from 30 local dental clinics during a year, from July 2020 to July 2021. The size of image datasets used in previous studies varies from 87 to 3,000 according to the review 14 . Compared with those datasets, the size of our dataset exceeds the size of the others. The dataset is manually labeled by a dental radiography expert. Figure 1 shows some examples of labeled images.
Anomaly detection. Our detection model consists of four parts (Fig. 2a). First, we convert DICOM formatted images directly transferred from dental clinics into PNG format. Then it detects anomalies through trained faster R-CNN models 19 . Basically it is an object detection model, we got object boxed regions of high possibilities of anomalies (Fig. 2b). Next, we filter out some boxes that are not located in a predetermined region (Fig. 2c). For example, if detected boxes are about carious lesions, it is obvious that the boxes should be located in the dental region. However, if some of the boxes are not located in the region but in the surrounding region, we can be sure that those boxes are absurd, so we can filter out those boxes. In the final stage, we narrow down the region of abnormal signs from a box form to a polygon (Fig. 2d). Through this stage dental clinicians can get high-resolution information not only the location and the type of anomalies found in the image, also the specific regions that show the feature of the anomalies. We used a prebuilt library, Detectron2 20 , in the polygon shaper Table 1. Category, name, and labeling criteria of 17 fine-grained dental anomalies. Soft tissue calcification   Calcified carotid atherosclerotic plaque  Irregular linear radiopacity between mandibular angle or hyoid bone and cervical vertebrae   Lymph node calcification  One or multiple irregular or cauliflower-like radiopacity in the lower or rear of the mandibular  angle or between the mandibular and cervical vertebrae   Ossification of the stylohyoid ligament  Long and thin radiopacity in anteroinferior direction between styloid process and hyoid bone   Tonsillar calcification  One or multiple radiopacities in the dorsal surface of the tongue overlaps with the  www.nature.com/scientificreports/ stage. Figure 2e is an example of our detection model. There are two signs of dental anomalies, proximal caries and periapical radiolucency. The proximal caries in the figure is in asymptotic stage so it may be stay undetected without careful examination. This model successfully detected it which means that the model can help early detection of selected anomalies.

Name of anomaly Descriptions and diagnostics
We tested our detection model using part of our dataset. We divided images to training, validation and test datasets. Train and validation datasets are used to training phase and test dataset are used to evaluate our model. Table 2 shows the class-wise performance of our detection model applied to the test dataset and Table 3 contains information about detailed numbers of test datasets. Our model shows very high specificity, in most classes over 95%, which means it can filter out most of the normal or healthy images. High specificity means that this model can be used to reduce the burden of examination of dental clinicians because it successfully filters out healthy images so that clinicians can focus on other images. Precision and sensitivity vary depending on the type of anomalies but generally the score exceeds or is similar to that from previous studies. www.nature.com/scientificreports/

Discussion
As we mentioned above, panoramic dental radiography is one of the most widely and frequently used imaging techniques in dentistry. It is safer than other imaging tools and quickly takes a wide range of dental structures so it is essential to make diagnoses and further treatment plans for patients, but due to the complicated structure of the dental region and lack of time, clinicians mainly focused on small parts of the images. If that neglected information is fully detected and noted to clinicians, it will improve the overall quality and consequences of the treatment. To achieve that goal, we applied an artificial intelligence technique which is widely used in the image analysis field to automatically detect regions of anomalies. We trained our model focused to increase the specificity to help clinicians filter out healthy panoramic dental radiography so as to decrease the number of images to be examined and to alleviate the burden of clinicians. First, we selected 17 major dental anomalies which are closely related to patients' oral health. These anomalies can lead to serious outcomes if ignored or are related to other systemic diseases. It means that the early detection of those anomalies can prevent severe outcomes and can be used as a marker to suspect other systemic diseases. We categorized anomalies into four groups corresponding to their clinical features and locations. First, carious lesions are considered the most prevalent problem in dentistry 14 . Because carious lesions cause more serious problems if ignored, prevention and early diagnosis are very important.
Many previous pieces of research applied artificial intelligence techniques to detect carious lesions 13 . Depending on its progress and location, various treatments can be used to treat carious lesions.
Here we divided carious lesions into four categories corresponding to their clinical features for fine-grained diagnoses; dental caries, cervical caries, proximal caries, and secondary caries. These subtypes of carious lesions show unique features which are related to their diagnoses and treatment. First, cervical caries is considered the most dangerous type because it leads to the rapid loss of tooth due to its location. Proximal caries is a type of www.nature.com/scientificreports/ carious lesion which is located on the surfaces between adjacent teeth. They are the most difficult type to detect because they cannot be visually or manually detected. Finally, secondary caries is a disease that occurs on the tooth after the filling. Because it takes a lot of burdens to detect 21 . This fine-grained diagnosis of carious lesions is important to early detection of caries before their progression to severe stages and to prevent further loss of dental tissues. The second category of our fine-grained model, calcifications, occurs when calcium accumulates in body tissue. The diagnostic criteria of calcifications are their anatomical locations, distributions, numbers, sizes, and shapes 22 . Calcifications in maxillofacial areas can be found through examinations of panoramic dental radiography but there are very few studies conducted regarding them 23 . Though the presence of calcifications on panoramic dental radiographs is uncommon, their detection is important to prevent the further progression of diseases. We selected four calcification anomalies; Calcified carotid atherosclerosis plaque, lymph node calcifications, calcifications of the stylohyoid ligament, tonsillar calcifications (tonsilloliths) 24 .
Our third category is dental anomalies. We included dental disease features and abnormal structures shown in the dental region 18 to this category. Dental anomalies are abnormal forms or structures of teeth in the dental area. We selected six dental anomalies which are critical factors of dental health; external root absorption, impacted tooth, periapical radiolucency, residual root, supernumerary tooth. Some of these anomalies often Table 2. Class-wise performance score of the trained model.  Table 3. Class-wise performance score of the trained model. www.nature.com/scientificreports/ cause symptoms such as pain, halitosis, and bleeding, and can be used as diagnostic markers, and anatomical factors when planning further dental surgeries. For example, Periapical radiolucency is the radiographic changes around the apex of the tooth and is the sign of inflammatory bone lesions. Recent studies present that periapical radiolucency may be caused by several diseases such as cirrhosis 25 . External root resorption is an undesirable dental injury that causes a loss of some parts of a tooth and can be seen radiographically. This type of anomaly damages the underlying tissues and causes a number of complications including infection, loss of teeth, pain, and so on 26 . The positional relationship between the mandibular canal and corresponding tooth is a key anatomic factor to make surgical plans such as extraction of the mandibular third molar because damage to the inferior alveolar nerve affects the function of the stomatognathic system and the quality of life of patients 27 . Panoramic dental radiography is one way to evaluate the risk of nerve injury before the extraction 28 . Impacted teeth can cause several symptoms such as swollen gums, halitosis, and pain when opening the mouth. If ignored, it causes severe complications such as infection, cysts, absorption, and many gum diseases. A recent study presented that an impacted tooth might have some association with a large central osteoma 29 . A residual root is a leftover of a tooth in the jaw after an extraction. It sometimes causes infections and pain. Usually, it is recommended to extract with a local anesthetic. Finally, supernumerary teeth may lead to many severe problems like displacement, crowding, root resorption, dilaceration, loss of vitality of adjacent teeth, and even ameloblastomas and odontomas in severe cases. So, clinicians should aware of the existence of the occurrence so that they can formulate treatment plans 30 . The last category is anomalies located in surrounding regions of the dental area. These anomalies are rarely related to oral health but may be used as potential markers to diagnose other related diseases. These radiographic anomalies are signs of inflammatory processes of that region and are known to be related to several diseases. For example, previous studies showed that retention pseudocysts of the maxillary sinus may have some associations with allergic and inflammatory processes, trauma, periapical and periodontal infections 31 , radiopacity in jaws with many osteoblastic and osteoclastic activities 32 , mucosal thickening of the maxillary sinus with apical periodontitis, alveolar bone loss, and so on 33 .

Name of anomaly Positive labeled objects Predictions True positives Negative images True negatives
To train a deep learning model to detect these many types of anomalies, it is essential to accumulate datasets including enough number of objects for every class. In fact, the most important factor of using artificial intelligence techniques is the quality and quantity of data. We built the system to collect panoramic dental images directly from local dental clinics and manually labeled them by a dental radiography expert. For a year, we accumulated a large and high-quality dataset compared to previous studies. This dataset is also still growing, so it has great potential in this field. Our tool successfully detects given anomalies with high performance especially for specificity and demonstrates that artificial intelligence can reduce the burden of dental clinicians by reducing the number of images that should be examined manually through detecting potential anomalies and filtering normal images.

Method
Data acquisition and labeling. Images are taken from 30 local dental clinics by panoramic x-ray cameras model PaX-i, Rayscan alpha from VATECH, Papaya 3d from Genoray, and RealScan from PointNix. Images are fully anonymized removing all patient identifiers. We installed a data transfer module in imaging devices to automatically collect panoramic dental images from clinics. Images are collected from July 2020 to July 2021. During the data collection period, data collection and labeling anomalies are done in parallel. Ethics approval was not required for this study because panoramic dental image data used in this study is obtained fully anonymized and no identifiable private information is included, and the study is performed in accordance with the Declaration of Helsinki. For each image, above 17 anomalies are manually labeled one by one by a dental radiology expert. Significant features and diagnostics used to annotation of every anomaly are described in Table 1. Image regions showing features of anomalies are first labeled in box form and then in the polygonal form to specify the exact region of the anomalies. Supplementary table 1 contains detailed numbers of objects and images of every anomaly.
Model architecture and training. Because images are originally transferred in DICOM format, we converted images to PNG format. We used faster R-CNN architecture for dental anomaly detection 19 . We trained a model for every 17 dental anomalies. We divided our dataset into a train/validation set and a test set. All images taken from July 2020 to March 2021 are used as a train/validation set, and others which are taken from April 2021 to July 2021 are used as a test set. We built a position verifier to filter out wrongly detected object boxes from the R-CNN model. It takes each object class and its location and examines whether it is well detected or not. Two classes, Carious lesions and anomalies in the dental region, are expected to be in the dental region and other classes are expected to be in the surrounding region. Finally, in the polygon shaper stage, we used Detectron2 20 to narrow down the detected regions from box form to polygon shape.
Detection and evaluation. To evaluate our model, we used an intersection over union (IOU) value to determine correctly-detected lesions from the output of the model. IOU is a widely used metric to evaluate the performance of object detection models by calculating the intersection over union between the ground-truth area and the prediction area. We determined the predicted box is correctly-detected when its IOU value is over 0.5. After the detection, we calculated precision, sensitivity, and specificity for evaluation. Table 3 shows detailed numbers of detection result on test dataset.