Development and virtual validation of a novel digital workflow to rehabilitate palatal defects by using smartphone-integrated stereophotogrammetry (SPINS)

Palatal defects are rehabilitated by fabricating maxillofacial prostheses called obturators. The treatment incorporates taking deviously unpredictable impressions to facsimile the palatal defects into plaster casts for obturator fabrication in the dental laboratory. The casts are then digitally stored using expensive hardware to prevent physical damage or data loss and, when required, future obturators are digitally designed, and 3D printed. Our objective was to construct and validate an economic in-house smartphone-integrated stereophotogrammetry (SPINS) 3D scanner and to evaluate its accuracy in designing prosthetics using open source/free (OS/F) digital pipeline. Palatal defect models were scanned using SPINS and its accuracy was compared against the standard laser scanner for virtual area and volumetric parameters. SPINS derived 3D models were then used to design obturators by using (OS/F) software. The resultant obturators were virtually compared against standard medical software designs. There were no significant differences in any of the virtual parameters when evaluating the accuracy of both SPINS, as well as OS/F derived obturators. However, limitations in the design process resulted in minimal dissimilarities. With further improvements, SPINS based prosthetic rehabilitation could create a viable, low cost method for rural and developing health services to embrace maxillofacial record keeping and digitised prosthetic rehabilitation.

Section 1: A Detailed Methodology

Phase A: Development of SPINS
The SPINS structure consisted of a turntable driven by a stepper motor, and an arc-shaped smartphone mount. The stepper motor was controlled by an Arduino UNO microcontroller board and a stepper motor driver. The Arduino board was programmed using an Arduino integrated Development Environment (IDE) software. The turntable was programmed to make a 360° turn in 24 steps, where each step was equivalent to 15° of rotation. The stepper motor used in this project had a revolution of 2048 steps per revolution in full-stepping mode. So, to make an exact 15° angle of turn, the stepper motor needed to make 85.33 steps of rotation. Since a stepper motor can only turn in an exact number of steps, the closest it could get to 15° angle of rotation was to make 85 steps of rotation, which produced 14.94°. The motor was programmed to stop for 500ms after each 85 steps (~15°), during which, the smartphone camera was wirelessly triggered by a Bluetooth shutter module to capture an image of the sample on the turntable. To trigger the image capture, the Arduino board was programmed to send a high signal for 100ms to the Bluetooth remote shutter.
The cycle of 15° rotation and the image capture was repeated until the turntable made a full 360° turn, which resulted in a total of 24 images captured. Details of the hardware used, and associated Arduino codes are mentioned in later on in Supplementary A, Sections 2 & 3. The images were projected in real-time on to the user's laptop using a screen mirror tool (Airdroid, Sand Studio). Each cycle of 24 images was controlled by an Arduino switch. 24 images of the model were taken at each of the three sleeve stops (25°, 55° and 345° on the arc) while the arc position was manually switched after each 24-image capture cycle. This resulted in a total of 72 images per model after moving across all three sleeve stops.
Corrugated white plastic sheets with white 15-diode 12V LED strips acted as primary diffused light source. A diffused ring light facing perpendicularly downward onto the model was also fixed on the crest of the arc to serve as secondary light source. The luminosity at the centre of the turntable was recorded at 1252 Lux (Lux Light Meter, Doggo Apps, Russia). A black sheet was placed in the background to prevent loss of camera focus in between shots.
Smartphones took focused images using their default smart camera systems. The images were transferred via cloud to Recap Photo (Autodesk Inc., USA); a software which automatically matched common points in each image and stitched the points to form a 3D model. The 3D models were scaled to actual size by measuring three successive linear reference distances on the physical model and entering the values for the stitched 3D model using dedicated software commands. The 3D model was then exported as STL with a maximum triangle budget of 200,000 ±10,000 triangles. The software commands have been detailed in Supplementary A, Section 4.1. The scans were kept in their original unoptimized format to prevent minute losses in feature and therefore the final 3D models contained more triangles (approximately 200,000 tris) as opposed to highly optimised models generated from proprietary dental scanning software (approximately 15,000 tris). Models derived from both laser scanner (NextEngine, Santa Monica, USA) and SPINS were decimated to maintain this budget and prevent an unfair mismatch during comparison.

Phase B: Preliminary test for precision of SPINS using different smartphones
Two physical models of simulated palatal defects (Model no. 2 & 18) were selected and laser scanned (NextEngine, Santa Monica) for pilot testing in phase B and software calibrations in Phase D. were scanned by the smartphones and later processed by ReCap to produce 3D models. MSA, VV, HD, and DSC were analysed for all 6 smartphone results.

Phase C: Test for accuracy of SPINS against standard laser scanning
The models were digitally captured using smartphone-4 attached to SPINS (n=18) and compared against their scanned counterparts from NextEngine laser-scanner (n=18). The two sets of models were compared for MSA, VV, HD & Area Discordance and DSC.

Phase D: Validation of functionality in the design of obturator bulbs
Digital bulbs were first designed using proprietary medical grade CAD software (3-matics, Materialise, Belgium). Physical obturators for models 2 and 18 were fabricated using conventional methods by a prosthodontist, laser scanned and compared with their digitally designed counterparts. The HD and DSC acceptability thresholds were met at default settings and therefore, no calibrations were made within the software. The bulbs were designed accordingly and labelled To ensure reliability of the prosthetic outputs; first, the extent of horizontal peripheral vertices (prosthetic flange) of each bulb was reviewed by two prosthodontic specialists for acceptability and progressed after both reviewers agreed on the outcome. Second, all designed bulbs were analysed using Meshlab, CloudCompare and Cura 4.6.1 (Ultimaker, Netherlands) following a simple set of software commands (Supplementary A, Section 4.7). The bulbs were scanned by all 3 open source platforms to ensure that they were 'watertight', error-free, and thus 3D-printable. Sets A, B and C were then evaluated for MSA, VV, HD & Area discordance and DSC.

Hausdorff's Distance by Cloud Compare
• Import 'reference' model (Laser scanned) and compute normal • Import 'aligned/compared' model (SPINS) model and compute normal • Select both STL models in the DB tree using Ctrl + Left mouse button > tools > registration > match bounding-box centers • Tools > registration > fine registration ICP > select reference and aligned models > apply and wait for 100% to complete (note: if bounding boxes are not symmetrical, use translate/rotate to roughly align both the models manually to overlap and then repeat fine registration ICP) • With both STLs selected in the DB tree > tools > distance > cloud/mesh distance > select compute on the distance computation window > record the mean distance (HD) from the 'console bar' on the bottom of the screen (do not close the window) > scroll sideways on the distance computation window and select approximate distances > select histogram image > select export histogram as image Note: positive and negative inclinations were disregarded as the objective was to observe the amount of discrepancy, not the direction of it

Dice Similarity Coefficient by Cloud Compare
• Should follow immediately after Hausdorff's distance in the same operation instance • Measure volume for both Laser scanned models and SPINS model individually and record • Select Plug-in > cork > select A ꓵ B Boolean Operation > wait for process to complete > select the newly generated 'intersect STL' in the DB tree and measure Mesh volume • Calculate DSC using the formula DSC = 2 X volume Intersect STL / (volume of Laser scanned STL + volume of SPINS STL)

Calibrate 3-matics design workflow using conventional bulb parameters
• Scan conventionally fabricated bulb using NextEngine laser scanner following 3.2.
• Compare volumetric parameters of conventional bulb against sample 2 to determine interpoint discrepancies (3.3.3) and spatial overlap ( • File > import > load saved file • select > roughly select the excess regions (outside of red ring margins) > discard (X) • Analysis > inspector > minimal fill > Auto Repair All (Note: select the excess peripheral horizontal vertices with the red ring as reference. High accuracy is not needed as the analysis tool will auto adjust and fix the mesh accordingly) • Sculpt > robust smooth > manually smoothen palatal face to obtain an even contour> done • File > export > choose destination > save

Filters > Quality Measures and Computations > Compute Geometric Measures > scroll the lower right hand box and observe for volume output.
If the mesh is not 'watertight'/ has non-manifold edges/ not 3D printable, the following output will be given instead of the volume: 'Mesh is not 'watertight', no information on volume, barycenter and inertia tensor.'

Edit > Mesh > measure volumes
If the mesh is not 3D printable, the following output will be given instead of volume: 'the mesh has holes' 4.7.3 Ultimaker Cura 4.6.1 • Open Cura > marketplace > select 'mesh tools' > agree and install • Open STL file > select model > extension > mesh tools > check models If the model is not 3D printable, the following message will be displayed: '.stl is not watertight and may not print properly'

Other issues during data acquisition
• A tripod base had to be mounted on the flat turntable for the software to be able to separate the apparently stationary turntable from the revolving 3D model. • Wide-angle action cameras were initially proposed alongside smartphones for image capture, but a subsequent pilot test demonstrated substantial distortion in the resultant 3D models. Action cameras were therefore discontinued from this study. • Bluetooth connectivity issues were experienced and likely due to the use of an older generation module (3.0). Using newer generation modules (>5.0) or other forms of wireless communications could resolve the issue in future improvements. • Latency of cloud transfer and (less frequently) image corruption was experienced on 4G network. This was resolved by utilising wi-fi connectivity and archiving the 72-image set as a single '.rar' file prior to cloud transfer. This issue can be resolved when 5G connectivity becomes widely available.