Accuracy and efficiency of automatic tooth segmentation in digital dental models using deep learning

This study evaluates the accuracy and efficiency of automatic tooth segmentation in digital dental models using deep learning. We developed a dynamic graph convolutional neural network (DGCNN)-based algorithm for automatic tooth segmentation and classification using 516 digital dental models. We segmented 30 digital dental models using three methods for comparison: (1) automatic tooth segmentation (AS) using the DGCNN-based algorithm from LaonSetup software, (2) landmark-based tooth segmentation (LS) using OrthoAnalyzer software, and (3) tooth designation and segmentation (DS) using Autolign software. We evaluated the segmentation success rate, mesiodistal (MD) width, clinical crown height (CCH), and segmentation time. For the AS, LS, and DS, the tooth segmentation success rates were 97.26%, 97.14%, and 87.86%, respectively (p < 0.001, post-hoc; AS, LS > DS), the means of MD widths were 8.51, 8.28, and 8.63 mm, respectively (p < 0.001, post hoc; DS > AS > LS), the means of CCHs were 7.58, 7.65, and 7.52 mm, respectively (p < 0.001, post-hoc; LS > DS, AS), and the means of segmentation times were 57.73, 424.17, and 150.73 s, respectively (p < 0.001, post-hoc; AS < DS < LS). Automatic tooth segmentation of a digital dental model using deep learning showed high segmentation success rate, accuracy, and efficiency; thus, it can be used for orthodontic diagnosis and appliance fabrication.


Results
We analysed the reliability of measurements and found that the intraclass correlation coefficients (ICCs) for the intra-rater reliability were 0.987-0.997 (Table 1). In the final assessment, we reviewed the intra-rater agreement (success vs failure) as well as Cohen's kappa statistic and its p-value. Table 2 shows the statistical analysis results of the degree of agreement within the evaluator on the success and failure of tooth segmentation with nominal variables. The range of kappa values in the three different segmentation types ranged from 0.885 to 1.000, showing very high reproducibility between the first and second assessments. www.nature.com/scientificreports/ The success rates for automatic segmentation were 97.26% and 97.14% in the AS and LS, respectively, which were higher than the 87.86% success rate of the DS (Table 3). These rates were significantly different between the three groups (p < 0.001, post hoc test: LS, AS > DS).

Discussion
In orthodontics, artificial intelligence, including deep learning, can be applied to diagnosis and treatment planning for orthodontic extractions or orthognathic surgery [21][22][23] , automated cephalometric landmarking 24,25 , diagnosis of impaction 26 , determination of skeletal maturity for growth stage evaluation 27 , and automatic segmentation and setup of digital models 14,19,28 . Reducing the time and effort required for simple tasks for diagnosis and appliance fabrication allows the users to focus more on making decisions.
In this study, we verified the MD width and CCH, presented as continuous variables, for intra-rater reliability using ICC. The success and failure of segmentation, presented as nominal variables using Cohen's kappa, indicated very high evaluation reproducibility: ICC and Cohen's kappa were 0.987-0.997 and 0.885-1.000, respectively. In addition, this study showed statistically significant differences in segmentation success rate, Table 5. Comparison of MD width and CCH among the three methods by tooth group. The means of the MD width error ranged from −0.31 to −0.08 mm, −0.09 to 0.68 mm, and −0.35 to 0.61) mm in the LS, DS, and AS groups, respectively. There were statistically significant differences in all tooth groups (p < 0.001, post hoc: DS > AS > LS in upper incisal, lower incisal, lower canine, and lower premolar; DS, AS > LS in upper canine, upper and lower molar, DS > LS, AS in upper premolar). The means of the CCH error ranged from −0.03 to 0.00 mm, −0.21 to −0.09 mm, and −0.11 to −0.06 mm in the LS, DS, and AS groups, respectively. There were statistically significant differences in all tooth groups (p < 0.001, post hoc test: LS > DS, AS). Data are given as the mean (95% confidence interval). a p values were derived from Friedman test; Shapiro-Wilk's test was employed to test the normality assumption; *p < 0.05.  www.nature.com/scientificreports/ time, and size of segmented teeth using three different orthodontic CAD/CAM programs; thereby, rejecting the null hypothesis. We designed the DGCNN-based segmentation model in two stages to prevent degradation of the segmentation performance due to differences in number of vertices between tooth and gingiva. In the first stage, the digital dental model was segmented into gingiva and dentition using the two-class DGCNN model. In the second stage, the digital dental model was segmented into individual tooth and gingiva using the seventeen-class DGCNN model after adjusting the number of gingiva vertices, which were segmented in the first stage, not to exceed twice the number of individual tooth vertices.
DGCNN using point clouds is advantageous for semantic segmentation and classification of a digital dental model, but it suffers from poor resolution for tooth margins. We attempted to obtain a clear tooth margin by supplementary use of curve-based mesh segmentation using skeleton/pruning algorithm. However, in some cases, the closed loop may not be formed due to the unclear curvature of the scanned data, or a closed loop may be formed in the wrong area, such as the tooth groove. Therefore, the segmentation and classification by DGCNN was optimized by supplementing the curvature-based mesh segmentation.
To compare and evaluate the accuracy and efficiency of the DGCNN-based segmentation model, we used two existing commercially available software. Both software packages used for comparison (i.e., OrthoAnalyzer and Autolign) were selected for the following reasons: the first reason is popularity; both are popular with orthodontists. Second, we considered their functionality for the tooth segmentation; OrthoAnalyzer is characterized by the need to set precise MD points for tooth segmentation, and Autolign has the need to set approximate MD points. Third, we considered their versatility. Some orthodontic software packages create closed working environments that prevent exportation of segmented teeth. OrthoAnalyzer and Autolign can export segmented teeth as stereolithography files, which can be imported to Meshmixer and Geomagic Control X software for success/ failure determination and tooth-size measurement.
This study presented the digital dental model segmentation success and failure for its clinical applications. As the purpose of tooth segmentation is to diagnose and fabricate orthodontic appliances, such as custom brackets, clear aligners, and indirect bonds, accurate tooth surface models are essential to fabricate orthodontic appliances, and defects cannot be allowed. Considering the width and height of the bracket base and the undercut needed to obtain the retention of the removable application, the cervical ± 25% line was set as the success baseline. Therefore, criteria for determining whether the segmentation was successful or not include the cervical margin of the segmented tooth not deviating beyond ± 25% of the cervical margin of the actual tooth and finding no defects in the occlusal or incisal edge of the segmented tooth.
A high segmentation success rate increases user convenience by reducing the time and effort required to modify segmentation splines. This study showed high segmentation success rate (Table 3) in all three groups. However, the success rates of LS and AS (97.14% and 97.26%) were significantly higher than the success rate of DS (87.86%). These findings imply that there are differences in the success rates of different segmentation methods. In contrast to segmentation of general objects, tooth segmentation has to work on the complex intersection of concave regions (e.g. tooth-gingival margin, tooth groove, and interproximal area), for which traditional geometry-based segmentation is typically used. Various tooth segmentation algorithms have been introduced to overcome these limitations. However, region-based and feature curve methods still involve some limitations, such as difficulty obtaining high-quality segmentation results and reduced efficiency due to complex implementation procedures. The recent developments in deep learning require little manual intervention and have low algorithm complexity and high accuracy. In this study, the AS method based on DGCNNs exhibited satisfactory success rates.
The MD width of the experimental group used the results provided by each software program. Consequently, results varied depending on different measurement and calculation methods. OrthoAnalyzer, used in the LS, set the mesial and distal points of individual teeth before tooth segmentation and calculated the MD width using the virtual plane formed by the screen view at the MD point setting. In contrast, Autolign and LaonSetup used in the DS and the AS, respectively, were normalised after tooth segmentation to calculate the MD width. Even if the software user sets the MD points precisely on an unsegmented dental model, they are impossible to set in the occlusion area of the interproximal region. Owing to the location characteristics of the measurement points, the MD width measured before tooth segmentation is likely to be measured more conservatively than when measured on the segmented teeth. In this study, the MD width of LS also showed an error of −0.35 mm and −0.23 mm when compared with DS and AS, respectively.
The MD widths of upper molars were recorded as larger in DS and AS groups than in the LS group. This may be because of the characteristic shape of the upper molars and the method of MD width measurement. An upper molar often forms a parallelogram in occlusal view, in contrast to other teeth whose height of contour on the MD surface is clear. When an upper molar is in the shape of a parallelogram, the heights of contour in mesial and distal surfaces show many differences in the bucco-lingual position. Without considering these morphological features, the MD width of a normalised upper molar can cause errors in measurement, leading to inaccurately large sizes. Therefore, the MD width measurements of upper molars require corrections.
A limitation of digital models is that measuring accurate MD width is difficult because of the presence of occlusions in the interproximal area. When using a plaster model, the adjacent surface is reproduced naturally during the teeth section, but in the case of a digital model with only surface information, the occlusions in contact with the adjacent area during segmentation remain empty after segmentation. To solve this problem, Kim et al. 14 proposed an image reconstruction method for an adjacent occlusion using a generative adversarial network and obtained an average improvement of 0.004 mm compared with the conventional method.
CCH can affect vertical bracket positioning and thus requires evaluation of accuracy. Compared with the DS and AS, LS had fewer errors in the whole tooth group, and DS and AS produced shorter CCHs than reference group (REF). However, according to the CCH of each tooth group ( www.nature.com/scientificreports/ −0.21 mm, which is not likely to cause problems in diagnosis and appliance fabrication, and it is considered clinically acceptable. This study performed repeated measurement of digital dental models conducted with three different segmentation methods, and there were within-subject correlations that were not independent. In addition, segmentation failure resulted in a missing value because the MD width and CCH could not be measured. Therefore, we used GLMM as a statistical method to analyse the main effects and the first-order interactions of the MD width and CCH errors. The segmentation success rate of the DS was lower than that of the remainder of the group, but the mean error according to the segmentation method was the lowest in the DS. This was the result of evaluating successful teeth segmentation and excluding missing values due to failure of segmentation. In addition, the mean error value results between groups of post hoc tests were within 0.12 mm and thus clinically acceptable.
The mean segmentation time of the AS was 57.73 s, which was shorter than those of the DS and the LS, with means of 150.73 and 424.17 s, respectively, presenting significant differences in efficiency. This was attributed to the differences in the segmentation processes; in all three experimental groups, the digital model was orientated to the coordinate system, but the subsequent process differed for each group. In the case of LS, which required precise marking of the mesial and distal points of all teeth, the segmentation time was the longest due to the necessity of axes specification of each tooth. Similarly, the DS also required marking of mesial and distal points for all teeth but did not require precise marking, so the time required for point designation was short. In the case of AS, segmentation and classification were performed without manual intervention after orientation; thus, it took the shortest time for segmentation. As the convenience of using segmentation increases and the requirement of manual intervention decreases, it is important to automate segmentation and reduce the time and effort required for correction by decreasing the segmentation failure rate.
A limitation of this study is that we used digital models of permanent dentition in good condition without teeth and gingival defects. Therefore, it is not possible to determine the tooth segmentation ability in cases such as a missing tooth, severe wearing, dental caries, partial eruption, and third molar. In subsequent studies, it will be necessary to use digital dental models of various conditions.
Because the OrthoAnalyzer software used in LS was used to obtain the reference data, bias may have occurred. To reduce the potential bias, the splines of all segmented teeth in the REF method were corrected. Moreover, the MD width of REF was measured using Geomagic Control X software. There was no correction after segmentation across experimental groups (i.e., LS, DS, and AS), and raw MD width data provided by software were used. Therefore, depending on the segmentation accuracy and the method of calculating MD width, values may differ from REF.
This study used different automatic tooth segmentation software in different groups. Because the three software programs were not developed and distributed simultaneously, their performances may vary depending on the software version. Moreover, the software continues to be updated to new versions; thus, improvements in accuracy, convenience, and speed of segmentation and classification can be expected. In addition, because the CAD/CAM software for orthodontics used in this study was distributed for commercial use, the detailed algorithms were not disclosed, making a direct comparison of the segmentation methods impossible. Therefore, this study focused on comparing the software from the point of view of a user and compared the use of the software and results of tooth segmentations.

Digital model selection. This retrospective study was approved by Yonsei University Dental Hospital
Institutional Review Board (IRB No. 2-2021-0033) and passed the exemption review of informed consent on the use of patients' intraoral scan data. All clinical examinations were conducted in accordance with the Declaration of Helsinki.
Among the 1005 digital dental model sets of patients treated by the Department of Orthodontics at Yonsei University Dental Hospital between January 2010 and February 2019, we selected 546 digital dental models satisfying the following criteria.
The inclusion criteria of the digital models were as follows: • Over 14-year-old orthodontic patients with second molars eruption.
• Digital dental models without any defects in the teeth and gingiva.
• Mild and moderate crowding.
The exclusion criteria of the digital models were as follows: • Congenital tooth deformity.
• Severe dental caries and tooth wear.
• Congenital or acquired missing tooth.
We used 516 dental models for deep learning-based tooth segmentation training, and the 30 dental models to evaluate the accuracy and efficiency of segmentation and classification of digital tooth models based on DGCNN model using two commercially available software programs. www.nature.com/scientificreports/ Deep learning process. We performed segmentation and classification of digital dental models based on DGCNN 20 as shown in Fig. 1. We extracted vertices from the digital dental model and converted into a point cloud model. When performing the two-class DGCNN model method that segments dentition and gingiva, we performed uniform sampling so that the number of points classified as gingiva was approximately twice that of the dentition. In addition, the seventeen-class DGCNN model was implemented to segment and classify individual teeth and gingiva. The hyperparameters of the DGCNN model used in this study can be found as Supplementary Fig. S1 online. As a post-processing process to determine precise tooth proximal and cervical margin, we divided the interdental area based on the mesh curvature 29 . We established the tooth margin, that showed a negative value as compared to the mean curvature of the tooth, as a feature vertex, and formed a closed loop by applying the skeleton pruning algorithm 30 . We obtained segmentation results with clear margins by merging the teeth and gingiva segmented by the DGCNN model and the teeth segmented by a mesh curvature closed loop.
Sample size and power calculation. Thirty digital dental models were used in the experiment. As 28 segmented teeth were evaluated per digital dental model, 840 teeth were evaluated in each of the four methods. With an observed sample size of n = 30 per group, power analysis of variance (two-tailed) conducted a posteriori using G*Power software version 3.1.9.2 (Franz Faul, Universität Kiel, Kiel, Germany) indicated > 99% power was needed to detect a medium effect size (Cohen's d = 0.25) at a significance level of 0.05. Fig. 2. Thirty identical digital models were segmented by three different methods (Fig. 3). (1) In the landmark-based tooth segmentation method (LS) using OrthoAnalyzer (ver.1.7.1.3, 3shape, Denmark), after orienting the digital model in the virtual coordinate system, the precise MD points of the teeth were set, and the segmentation proceeded. After the original dental model was converted to a point cloud model, the dentition and the gingiva were segmented using the two-class DGCNN model. To increase the accuracy of semantic segmentation, we segmented individual teeth and gingiva using the seventeen-class DGCNN model with the vertices of the point cloud of the gingiva reduced to less than twice that of the tooth group. The gingival vertices were then restored. Since some of the segmentation results using DGCNN showed an unclear teeth margin, curvature-based mesh segmentation was used as post-processing to segment the teeth margin. The images of the digital dental model used in this figure were obtained using MeshLab (ver. 1

Measurements.
To evaluate the accuracy and efficiency of automatic segmentation using the three types of software, we measured the MD width and CCH of segmented teeth, segmentation time, and success rate. We used a total of 3, 360 teeth, 28 teeth from 30 identical digital models, each segmented in three experimental groups and one control group, for evaluation. Orthodontic specialist (J.I.) measured ten randomly selected digital models twice within a two-week interval to confirm the reliability of the measurements.
MD width and CCH of automatically segmented teeth. We extracted the segmented teeth of the three experimental groups (LS, DS, and AS) and the control group (REF) as individual teeth using Meshmixer (ver.11.5.474, Autodesk, USA) and imported into the Geomagic Control X software (3D Systems, USA), where we superimposed the corresponding teeth using the best fit method. Overlapped teeth were imported into Meshmixer and rearranged to measure the MD width and CCH. We established a virtual occlusal plane and positioned the incisal tip and buccal cusp of the overlapping teeth. Then, we adjusted the rotation, angulation, and inclination of the teeth to measure the MD width and CCH. The accuracy of the MD width of the three experimental groups was evaluated using the values output from each program. For comparison, the control group values were measured by Meshmixer. For measuring CCH, the segmented teeth of all groups re-orientated in Meshmixer were loaded into Geomagic Control X software, and the distance between the virtual occlusal plane and the lowest point of the gingival margin of the clinical crown was measured (Fig. 4).
Segmentation success rate. The case where the segmented tooth margin did not deviate from ± 25% of the cervical margin of REF group, was set as the success criterion for segmentation. When a segmentation fault was found on one or more occlusal, labial, lingual, or distal surfaces of the most posterior molars, the segmentation of the tooth was considered a failure (Fig. 5). For each experimental group, we determined the segmentation success rate for 30 digital models (i.e. 840 segmented teeth).
Segmentation time. We measured the time required for segmentation in seconds after loading the digital model into the software used in each experimental group from the orientation stage to the output of the value of the MD width. In the LS group using OrthoAnalyzer and the DS group using Autolign, it was necessary to mark the mesial and distal points of each tooth (28 teeth) of the digital model after orientation. In the AS group using LaonSetup, the software automatically classified and segmented the teeth.

Figure 2.
Schematic diagram indicating the study flow. Thirty digital dental models were segmented using three types of software (i.e., OrthoAnalyzer, Autolign, and LaonSetup). The manually corrected segmented tooth was used as the reference group. The size of the segmented teeth such as MD width and CCH, success and failure of tooth segmentation, and tooth segmentation time were evaluated.   www.nature.com/scientificreports/ lowed a normal distribution. Independent variables affecting the response (tooth size error) were simultaneously evaluated by a GLMM. GLMM jointly considered the main and the first-order interaction effects. Bonferroni's test was used for post hoc multiple comparisons. All statistical analyses were performed using SPSS version 26.0 statistical software, and p values of less than 0.05 were considered as indicating statistical significance.

Data availability
The data underlying this article cannot be shared publicly to protect the privacy of individuals that participated in the study. The data will be shared at reasonable request to the corresponding author.