Three-dimensional soft tissue landmark detection with marching cube algorithm

Current method of analyzing three-dimensional soft tissue data, especially in the frontal view, is subjective and has poor reliability. To overcome this limitation, the present study aimed to introduce a new method of analyzing soft tissue data reconstructed by marching cube algorithm (Program S) and compare it with a commercially available program (Program A). Cone-beam computed tomography images of 42 patients were included. Two orthodontists digitized six landmarks (pronasale, columella, upper and lower lip, right and left cheek) twice using both programs in two-week intervals, and the reliability was compared. Furthermore, computer-calculated point (CC point) was developed to evaluate whether human error could be reduced. The results showed that the intra- and inter-examiner reliability of Program S (99.7–100% and 99.9–100%, respectively) were higher than that of Program A (64.0–99.9% and 76.1–99.9%, respectively). Moreover, the inter-examiner difference of coordinate values and distances for all six landmarks in Program S was lower than Program A. Lastly, CC point was provided as a consistent single point. Therefore, it was validated that this new methodology can increase the intra- and inter-examiner reliability of soft tissue landmark digitation and CC point can be used as a landmark to reduce human error.


Scientific Reports
| (2023) 13:1544 | https://doi.org/10.1038/s41598-023-28792-w www.nature.com/scientificreports/ the most protruded area. Digitizing 3D soft tissue landmarks via contour lines seems to reduce the errors that occur in previously existing software programs. These errors included soft tissue appearing differently when the volume-rendered image was rotated or when the contrast, opacity, or brightness was manipulated. Therefore, we hypothesized that using the marching cube algorithm would increase the reliability of digitizing soft tissue landmarks.
If soft tissue analysis is performed using a 2D computer monitor for 3D structures, the lack of experience of orthodontists and surgeons accustomed to 2D diagnosis can affect the analyses of the results. To reduce and solve human errors in 3D data analysis, evaluating landmarks by employing a pre-educated artificial intelligence program or computer-aided analyzer might be recommended. These study results have led to the introduction of programs such as the YOLOv3 algorithm 14,15 and WEBCEPH 16 . However, these programs are being used to measure 2D lateral cephalometric radiographs, and computer-aided 3D soft-tissue diagnostic methods are still insufficient.
The aim of this study was to introduce a method to analyze 3D CBCT soft tissue data reconstructed using the marching cube algorithm and compare it with landmarks digitized with the current commercially available software program. The null hypothesis was that there were no differences in reliability between the two programs. Moreover, we further evaluated whether human error could be reduced by proposing a specific computer-aided landmark named the computer-calculated point (CC point).

Materials and methods
Ethics statement. This study protocol was conducted with the approval of the Institutional Review Board (IRB) of Pusan National University Dental Hospital (IRB No.: PNUDH-2020-033) and was performed in accordance with the Declaration of Helsinki and relevant guidelines and regulations. This study passed the exemption review for informed consent on the use of patients' CBCT images and medical records.

Subjects. This study was conducted using CBCT images of patients at Pusan National University Dental
Hospital from January 2019 to May 2022. Patients who were diagnosed with impaction of wisdom teeth or temporomandibular disorder without condylar bone changes were selected for this study. CBCT (Viso G7; Planmeca Oy, Helsinki, Finland) was performed with the following scanning parameters: 120 kVp, 11 mA, 36 s, voxel size of 0.3 mm, and field of view (FOV) of 16 × 16 cm. Images without blurring, severe metal artifacts, or surgical defects were selected. Images of patients whose eyelids to the tip of the chin were not captured and those whose nose or lips were cut from the image were excluded from the study.
The sample size was calculated using G * Power 3.1.9.7. (G * Power, version 3; Heinrich Heine University, Düsseldorf, Germany) with a p value < 0.05 indicating statistical significance, a power of 80%, and an effect size of 0.5 for detecting differences between programs 17 . The minimum sample size of 34 was required, and the present study included 42 participants.

Methods.
To perform the following process, we used a software using MATLAB (MathWorks, Matick, MA; 2022) for algorithm development, data analysis, visualization, and numerical calculation.
Soft tissue binarization. Automatic binarization of soft tissue was performed on 42 CBCT images using the Otsu method 18 (Fig. 1). As a result of binarization, the soft and hard tissue images were inverted, and the soft tissue image could thus be obtained after separation.
Contour line formation using marching cube algorithm. The 3D surface reconstruction was done using the marching cube algorithm to form contour lines (Fig. 1C). A total of 42 files with enhanced soft tissue informa-  www.nature.com/scientificreports/ tion in the the binary MATLAB files (.mat) were created using the soft tissue information obtained from the patient's CBCT. By using the marching cube algorithm, since CBCT images used in this experiment had a spatial resolution of 0.3 mm with an FOV of 16 × 16 cm, the soft tissue data was reconstructed to have stepped contour lines and mountain tabletop effects. However, the reconstruction process caused block noise; this was resolved by simplifying the entire surface and the contour lines and converging the vector direction of the soft tissue landmark to simple values.
It was confirmed that the normal vector on the contours of the protruded soft tissue points with a convex shape, as shown in Table 1, had a constant value between patients. This is because the anatomical definitions of these points correspond to the direction of the normal vector. That is, because the pronasale and both lip points are the most protruding points, they have a normal vector in the forward direction, and the columella and cheek points are located at the midpoint of the curved surface; thus, they have a normal vector inclined at 45° downward and laterally, respectively.
Manual coordinate determination of soft tissue landmarks. According to the definition in Table 1, two orthodontists manually digitized six landmarks twice in two-week intervals on the center of the tabletop surrounding the protruded soft tissue points using current commercialized software [Program A (INVIVO6 software, version 6.5.0, Anatomage, San Jose, CA)] and homemade software using the marching cube algorithm (Program S).
Unlike in previous studies, the (0, 0, 0) reference point or reference plane was not set separately 6,9,19 . This was a method for reducing the error caused by setting the reference point and plane according to the examiners. The 3D coordinates were measured as (x, y, z), and Euclidean distance (d) between the coordinates was calculated and recorded according to the conventional method.
In both programs, positive values were indicated as left on the x-axis, backward on the y-axis, and upward on the z-axis. The 3D coordinate values expressed by the two programs were different. Therefore, for comparison, coordinates obtained from Program S were converted to the coordinate system of Program A by subtracting 266.5 for x, y coordinates and subtracting 266 for z coordinates, then multiplying each by 0.3.
The soft tissue landmarks used in this study were those that protruded out of soft tissue structures. As shown in Table 1, six landmarks were digitized manually at the center of each tabletop (Fig. 2).
Obtaining the CC point of soft tissue landmarks. Because the landmarks were digitized manually, even if they were digitized meticulously to the center of the tabletop as closely as possible, human errors might still occur. Therefore, we developed an algorithm to automatically find all the coordinates making up the tabletop and determine the exact center point of protruded soft tissue landmarks using the center coordinate of the tabletop. The points determined by this method were called the computer-calculated (CC) points. The detailed method for obtaining the CC point is as follows: (1) Obtaining the tabletop plane containing the starting point: From the starting points obtained manually (Fig. 3A, 4A), points within 50 pixels of Euclidean distance were first obtained (Fig. 3B, 4B). Among them, the points within one pixel from the starting points in the direction of the normal vector, referring to Table 1, remained (Fig. 4C). (2) Obtaining the CC point: The average coordinates of the points on the tabletop obtained in step 1) (Fig. 3C, 4D) were determined as the CC points.
Using the 42 samples, the CC points (CC1, CC2) of each examiner were calculated using the measurements acquired in Program S, and the degree of agreement between CC1 and CC2 was investigated using Cronbach's alpha coefficient.

Consistency examination of CC point.
To confirm whether the points that comprise the tabletop consistently refer to the same CC point, new CC points were created and compared to the original point using the following method. After moving the points by 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, and 4 pixels in the up, down, left, and right directions from the CC point (Fig. 5), the second CC points were obtained from the moved coordinates as input.    www.nature.com/scientificreports/ marks at a two-week interval was assessed by each examiner (intra-examiner). Additionally, the reproducibility of landmarks was assessed between the two examiners using each program (inter-examiner). Cronbach's alpha coefficient was used to determine the intra-and interclass correlation coefficients. The difference (Δx, Δy, Δz, Δd) in the measured values between the two examiners (Ob1 and Ob2) according to each program was calculated. A paired t-test was performed to determine whether there was a significant difference in precision between Programs A and S among the operators for the six landmarks. Additionally, Cronbach's alpha coefficient was calculated to evaluate the reliability of CC1 and CC2.

Results
The intraclass correlation coefficient was statistically analyzed for differences between each examiner's first and second trials in measured values of each of the x, y, and z coordinates and distance (Table 2). In Program A, Cronbach's alpha coefficient of each examiner ranged from 64.0 to 99.9%. For the four landmarks (Pn, Co, ULP, and LLP), reliability was > 90%. However, for Ch_r (64.0%) and Ch_l (76.2%) at the x component by examiner 1, the reliability was < 80%. In Program S, Cronbach's alpha coefficient of each examiner was between 99.7 and 100%, meaning that each examiner's first and second trials were almost identical, which was statistically significant. Because the intraclass correlation coefficient value was reliable, the average values of the first and second trials were calculated for each examiner and used as a representative value of each examiner to analyze the interclass correlation coefficient (Table 3). In Program A, the reliability of the four landmarks (Pn, Co, ULP, and LLP) showed a high degree of agreement (> 90%). However, the reliability of the Ch_r and Ch_l points was < 90% (right cheek, 76.1%; left cheek, 88.2%). In Program S, the reliability was 99.9% to 100%, indicating almost perfect agreement between the two examiners.
To evaluate the difference in the precision of digitizing the landmarks, the difference in the coordinates (Δx, Δy, Δz) and distance (Δd) between the two examiners according to each program was calculated ( Table 4). The difference of the coordinates and the distance of all six landmarks using Program S were less than those using Program A (p < 0.05). For Ch_r and Ch_l, the difference in the coordinates was greater than 1.0, but less than 0.1 in Program S (p < 0.001).
The concordance between the measured values of CC1 and CC2 was investigated by calculating the difference arithmetically. The difference of distances (Δd) between CC1 and CC2 for all six landmarks of the 42 images were zero (Fig. 6), and Cronbach's alpha coefficient between CC1 and CC2 showed a value of 1 for all the landmarks. The CC points measured and calculated using Program S were 100% consistent for all six landmarks.

Discussion
The present study showed that the marching cube algorithm for soft tissue landmark detection had high intraand inter-examiner reliability. Moreover, a newly proposed coordinate, the CC point, was more consistent than manual digitation, and was thus validated as a more consistent and reliable landmark in clinical practice.
The marching cube algorithm was used in Program S to reconstruct the soft tissue data and form contour lines and tabletops to facilitate the digitization of the soft landmarks. The principle for the formation of contour lines via the marching cube algorithm is as follows. In 2D space, the marching cube algorithm divides the space into uniform cell units and divides the cells in and out of the circle (Fig. 7A). Boundaries can quickly be drawn while matching the dictionaries of the 16 possible cases (Fig. 7B). Subsequently, the distance from the center to the edge was approximated as the actual distance (Fig. 7C). All 16 2D marching cubes combinations are used to make the boundaries (Fig. 7D). In 3D space, this algorithm uses a cube as a unit to form an isosurface. The cube configurations formed during the triangulation step can generally be categorized into 15 unique patterns identified in the original marching cube algorithm (Fig. 7E) 20 . Therefore, the marching cube algorithm independently selects 1 of 15 patterns in each cell, enabling parallel processing, and then processes single-image results on a unique surface.
The degree of smoothing of the contours made by the marching cube algorithm is affected by the resolution of the 3D image: the higher the resolution, the smaller the size of the cube making up the surface, and as a result, more detailed surface images and dense contours can be obtained. Therefore, a smooth surface in which contours are not clearly visible is obtained after surface reconstruction using a marching cube algorithm, such as an intraoral scanner, for equipment with high spatial resolution. The CBCT images used in this experiment had a spatial resolution of 0.3 mm with an FOV of 16 × 16 cm, and this relatively low spatial resolution caused stepped contours and mountain tabletop effects to be generated during reconstruction with the marching cube Therefore, by this process, Program S showed a slight difference in the height of the soft tissue as a contour line and served as a guide when digitizing landmarks manually. Digitizing 3D soft tissue landmarks via contour lines reduces errors when evaluating 3D structures via 2D screens and resolves issues with soft tissue appearing differently when the rendered images are rotated or when the contrast, opacity, or brightness are manipulated. As a result, the intra-and interclass correlation coefficients were close to 1, indicating that the exact same point could be measured multiple times. Moreover, the difference in the measurement values between examiners (Δx, Δy, Δz, Δd) for Program S was less than that for Program A, and the differences were close to zero. In other words, the precision of the landmarks between examiners was higher when using Program S compared to Program A. Furthermore, in the present study, by calculating the average coordinates of the points on the tabletop, the CC point of the 3D soft tissue was obtained. There was no difference in the landmarks depending on the acquisition time and examiners, and it was possible to obtain completely consistent measurements. That is, it was possible to digitize the 3D soft tissue landmarks more reliably and objectively using Program S and CC points (especially cheek points on both sides). Since orthodontic and surgical treatments are mostly irreversible, treatment plans should be established not just by instinct, but with accuracy. In this study, we proposed a methodology for digitizing soft tissue landmarks that is highly reliable; therefore, by using this methodology, more accurate research can be conducted regarding soft tissue changes before and after treatment.
Using Program S, detailed digitization was possible even on other soft tissue areas and not on specific landmark points. For example, in Sample 23, two high contour lines were observed on the lower lip (Fig. 3). This meant that the patient's lips were slightly concave in the center. In Program A, it was difficult to observe these Table 4. Comparison of differences in measurement values (Δx, Δy, Δz, Δd) between the two examiners. Data are presented as p value calculated with paired t-test between differences in measurement values (Δx, Δy, Δz, Δd (mm)) between examiner 1 and 2 when measured using each program. Pn, Pronasale; Co, Columella; ULP, Upper lip point; LLP, Lower lip point; Ch_r, Cheek right; Ch_l, Cheek_left. SD, standard deviation; *p value < 0.05.  24,25 , and CBCT scans 6,9,19 , have been introduced to acquire 3D facial soft tissue data. In this study, CBCT was used for soft tissue data acquisition because the subjects underwent CBCT taking for the extraction of wisdom teeth. Likewise, orthodontists and oral-maxillofacial surgeons mostly modify the hard tissues (skeletal and teeth) to acquire the desired soft tissue change. Therefore, CBCT is the most convenient choice for acquiring both hard and soft tissue data simultaneously. Moreover, a previous study showed that CBCT is the most efficient and reliable imaging tool for soft tissues 26 , and so using CBCT for soft tissue evaluation was a reasonable choice for this study. However, CBCT inevitably exposes patients to radiation 27 . Therefore, in the cosmetic field, the degree of skin shrinkage or skin elasticity is often evaluated using 3D-stereophotogrammetry 21 and Moire topography methods 22 , limiting radiation exposure. If the clinician uses only soft tissue procedures, such as botulinum toxin or fillers, laser scanners are often used as a simple method 24,28 . Previous studies defined soft tissue landmarks based on the underlying hard tissue structures; consequently, if there is no hard tissue data, clinicians cannot detect soft tissue landmarks 29 . Moreover, soft tissue landmarks determined based on the underlying hard tissue are not always clinically meaningful, since as the soft tissue elasticity decreases or as the patients become more obese, the soft tissue becomes less associated with the underlying hard tissues [30][31][32] . This study provides a reliable method for digitizing soft tissue landmarks without the need for hard tissue. Therefore, this method can be applied to soft tissue acquisition methods other than CBCT. Further research on applying this method to other soft tissue acquisition methods will be conducted in the future, but similar results are expected as the marching cube algorithm can be applied to any data.
Until now, comparisons before and after treatment were inaccurate because there was a lack of studies that questioned the reliability of the current method of digitizing soft tissue landmarks. Therefore, most studies used 3D facial heatmaps to show changes in the soft tissue profile 33 . Using only heatmaps, clinicians are unable to determine the exact amount of soft tissue changes, leading to inaccurate treatment plans. Compared to heatmaps, digitizing landmarks and evaluating the change in these landmarks before and after treatment would allow clinicians to determine the exact changes.
In previous studies, most soft tissue landmarks were digitized on the lower facial area and along the midsagittal plane [6][7][8] . However, Ferrario et al. 35 in actual clinical practice, meaningful soft tissue landmarks are not www.nature.com/scientificreports/ necessarily located along the midsagittal plane. In particular, there are not enough standards and evaluations on features of the cheek area, which occupies a large portion of the face. Soft tissue of the cheek area relies on the structure of the underlying hard tissue to a certain extent; however, features such as protrusion and inversion of the cheeks vary depending on muscle development, amount of subcutaneous fat, patients' sex, age, and elasticity of the skin tissue 6 . As a result, the definition and method of measuring landmarks on the cheek area differed amongst each study. According to Menezes et al., it was defined as the intersection of Camper's plane and a line connecting the external eye canthus with the labial commissure 34,35 . In another study, it was defined as the intersection of the vertical line passing through the mid-canthus parallel to the z-axis and the horizontal line passing through the ala and perpendicular to the vertical line 36 . This area did not protrude from the center of the cheek, nor was it in the middle of the cheek area; neither point was clinically meaningful. Hence, beforeand-after evaluations of cheek area modifications such as MEDPOR augmentation 37 , soft tissue treatments with botulinum toxins [38][39][40] , malar-plastic surgery 41 , and changes in soft tissue when treated by mini-screw-assisted rapid palatal expanders 36 could not be precisely assessed using these landmarks. However, in this study, when using the marching cube algorithm to digitize dots on the cheeks (Ch_r, Ch_l) using Program S, the most protruding area could be evaluated, which is clinically significant, and can possibly be applied to the evaluation of the forementioned treatments. The developed algorithm can be applied not only to the field of medical imaging, but also to various studies that require reference points on various curved surfaces, such as animation, 3D land or remains surveys, and nanosurface measurements. The marching cube algorithm produces a mesh that is a rough approximation of the true isosurface and is not well suited to sharp corners or creases 42 . However, the developed algorithm was implemented on a soft, relatively simple face-shaped image with low enough resolution to show the mountain tabletops and contour lines, which is an acceptable use of the marching cube algorithm. Since mountain tabletops, which are the result of the marching cube algorithm, are objects that can be easily recognized visually, it is expected that the entire process can be fully automated through automatic segmentation using deep learning in the future.
There are a few limitations of this study. First, due to the scattered radiation, soft tissue data acquired via CBCT have low contrast resolution 43 . Despite the convenience of CBCT, applying any algorithm to reconstruct the CBCT soft tissue data has risks of analyzing inaccurate data. However, previous studies indicated that there were no clinical differences between other types of soft tissue acquisition methods 11,44 ; therefore, the results of this study could be considered acceptable. Nevertheless, future studies are needed to determine whether similar results are obtained for other acquisition methods. Second, only six landmarks were included in this study. More landmarks could have been included, such as soft tissue pogonion or glabella. However, in the process of immobilizing the patient during CBCT acquisition, deformation of the soft tissue pogonion and glabella was www.nature.com/scientificreports/ caused by the chin and forehead resting on the CBCT scanner. Therefore, based on the results of this study, future studies using CBCT data without deformation of the chin and forehead are required. In the case of soft tissue gonions, manual digitation by using Program S was also possible for most subjects. However, the soft tissue gonion was difficult to detect in subjects that lacked soft tissue elasticity. Moreover, because of the overlapping contour lines in the gonion area, it was difficult to develop the CC point using the current algorithm. Finally, only convex landmarks were evaluated in this study. Contour lines made by the marching cube algorithm made stable tabletops on convex points but not on concave points. An improved algorithm and methodology should be developed to overcome these limitations.

Conclusion
A newly developed Program S, which reconstructs soft tissue data using the marching cube algorithm, allows clinicians to manually detect and digitize six soft tissue landmarks more reliably than currently existing software. Moreover, from the contour lines formed by the algorithm, the computer can automatically detect the CC point, which is the center of the most protruded area for each landmark, and is consistent and reliable. This new methodology for detecting soft tissue landmarks will help clinicians analyze soft tissue changes more objectively and help establish a treatment plan more accurately.

Data availability
The data underlying this article cannot be publicly shared to protect the privacy of the individuals participating in the study. The data will be shared at a reasonable request to the corresponding author.