A super-grayscale and real-time computer-generated Moiré profilometry using video grating projection

By using the time-division multiplexing characteristics of the projector and the integral exposure characteristics of the charge coupled device (CCD) camera, a super-grayscale and real-time computer-generated Moiré profilometry based on video grating projection is proposed. The traditional digital static grating is of 256-grayscale at most. If an expected super-grayscale grating with a maximum grayscale of 766 is designed and divided into three 256-grayscale fringe patterns with balanced grayscale as far as possible, they can be synthesized into a repeated playing video grating instead of the traditional static grating. When the video grating is projected onto the measured object, as long as the exposure time is set to three times the refresh cycle of the video grating, the super-grayscale deformed patterns in the 766-grayscale can be captured with a 10-bit CCD camera, so that the deformed patterns are realistic. The digital error in computer-generated Moiré profilometry is effectively reduced. In addition, this method can expand the linear range of the deformed pattern by 20% in computer Moiré profilometry. Therefore, the proposed method has the perspectives of high accuracy and real-time measurement. Theoretical analysis and experimental results demonstrate the validity and capability of the proposed method.

www.nature.com/scientificreports/ and the projected grating is original of 256-grayscale at most due to the 256-grayscale standard of the digital projector. So the captured deformed pattern is of 256 grayscale at most. In this study, by skillfully using the time-division multiplexing characteristics of the digital projector and the integration characteristics of the CCD camera, a super-grayscale, and real-time computer-generated Moiré profilometry (SRCGMP) is proposed. Firstly, an expected super-grayscale grating with 766-grayscale is designed on a computer; then, it is divided into three traditional 255-grayscale fringe patterns with balanced grayscale as far as possible. These three fringe patterns are synthesized into a repeated video, which is called a video grating. When the video grating, which replaces the traditional static grating, is projected onto the measured object, a 10-bit CCD camera can capture the deformed pattern with 766-grayscale if the exposure time of the CCD camera is set to three times the refresh period of the video grating. Compared with the traditional technology, the video grating projection has the advantages of getting the super-grayscale deformed pattern with less digital error and higher resolution phase information. Meanwhile, the linear range of the super-grayscale deformed pattern can be expanded by 20%. The measurement accuracy of CGMP is improved originally, and the single-shot feature of real-time measurement is maintained. The experimental results demonstrate the validity and capability of the proposed method.

Principle
Principle of super-grayscale fringe pattern capturing. Firstly, four expected super-grayscale gratings with 766-grayscale are designed as follows: where a 766 0 , b 766 0 are both normally set to be 382. 5. The greyscale of each grating is in the scope of [0,765]. f 0 is the spatial frequency of the four gratings. n is the number of the phase-shifting step number.
Due to the 256-grayscale standard of the commercial projector, the above super-grayscale gratings can't be directly projected onto the measured object. Otherwise, it will automatically be compressed to 256-grayscale.
Based on the principle of time-division multiplexing, each grating is then divided into three 8-bit fringe patterns with 256-grayscale with balanced grayscale as far as possible: The divided fringe patterns I 256 n1 (x, y), I 256 n2 (x, y), I 256 n3 (x, y) are then composed into a repeated playing video vg n which we called the video grating, and its refreshes cycle t 0 is set to be consistent with the refresh cycle of the projector. Figure 1 gives the division and time integration procession of the fringe pattern. The more fitted between the digitized fringe pattern with the blue line and the analog sinusoidal grating with the red line, the more realistic the pattern information can be reflected.
The cutaway view of the expected super-grayscale sinusoidal grating and analog sinusoidal grating is shown in Fig. 1a. It shows that the digitized sinusoidal grating designed by the super-grayscale method is realistic because it is close to the expected analog sinusoidal grating. Figure 1b is gray values at partial pixels of the digitized grating in Fig. 1a. The cutaway views of the three corresponding divided fringe patterns with 256-grayscale I 256 n1 (x, y), I 256 n2 (x, y), I 256 n3 (x, y) and the analog sinusoidal grating are shown in Fig. 1c-e. The cutaway views show that they all deviate from the expected analog sinusoidal grating compared to that of the expected super-grayscale grating. Figure 1f-h are the corresponding gray values at partial pixels of digitized 256-grayscale fringe patterns in Fig. 1c-e. After setting the exposure time of the camera to be three times the refresh cycle (3 t 0 ) of vg n , and controlling the gain A m of the camera appropriately, a 10-bit CCD camera can capture the deformed supergrayscale patterns with 766-grayscale by making full use of the integral characteristics of the camera, which can be expressed as follows: The cutaway view of the captured super-grayscale pattern and the analog sinusoidal grating are shown in Fig. 1i, and Fig. 1j shows the gray values at partial pixels of the digitized fringe pattern. It shows that the expected super-grayscale pattern can be achieved, which is realistic and is helpful to improve the measurement accuracy originally.
Principle of SRCGMP. By projecting each of the four video gratings onto the measuring object instead of the traditional static grating, and setting the exposure time of the CCD camera to three times the refresh period of the video grating, and controlling the gain of the camera properly, the four corresponding super-grayscale deformed patterns I C766 n can be captured with a 10-bit CCD camera. Since the calculations below are based on the SRCGMP measurement process, the formula below omits the superscript 766 for simplicity. The captured super-grayscale patterns can be expressed as: where R 0 (x, y) stands for the reflectivity of the reference plane. φ 0 (x, y) denotes the phase distribution modulated by the reference plane. f is the spatial frequency of the fringe pattern captured from the reference plane. (1) (4) I C n (x, y) = R 0 (x, y) a 0 + b 0 cos[2πfx + 2(n − 1)π/4 + φ 0 (x, y) n = 1, 2, 3, 4 The other AC component of I C 2 (x, y) can be obtained by: These two AC components of the reference plane with a π/2 phase difference have been prepared and stored in the computer in advance.
In measuring, only one video grating vg 1 is projected onto the measured object. And only one corresponding super-grayscale deformed pattern is needed to be captured as: where R(x, y) stands for the reflectively of the measured object, φ(x, y) denotes the phase distribution modulated by both the measurement object and the reference plane.
The DC component R(x, y)a 0 of captured deformed pattern distorted by the measured object can be filtered out in the frequency domain, and the AC component left can be written as follows: where f x denotes the width of the filter window along x direction and f y denotes the width of the filter window along the y direction. When this AC component multiplied by the two previously prepared AC components respectively, the results can be written as: www.nature.com/scientificreports/ The second terms in Eq. (9) and Eq. (10) can be obtained by the low-pass filter, which are so-called computergenerated Moiré fringes. They can be written as: Both Eqs. (11) and (12) contain the phase distribution only modulated by the measured object φ(x, y) − φ 0 (x, y) , which can be obtained as: Due to the arctangent operation, the phase distribution of the measured object is wrapped within (−π , π ] . The unwrapped phase φ(x, y) − φ 0 (x, y) can be obtained by the phase unwrapping method 28 to be ��(x, y) . The height information can be calculated by the height-phase mapping algorithm 29 : where the four coefficients of a(x, y) , β 1 (x, y) , β 2 (x, y) and γ (x, y) are system parameters, and they can be obtained by calibration.
The measurement accuracy can be improved originally for the phase resolution of the super-grayscale deformed pattern is more realistic than that of the traditional deformed pattern. Moreover, the single-shot feature is also retained. It has a prospective application for real-time measurement.
Analysis of the linear relationship between the captured grayscale and the projected grayscale. In order to investigate the linear relationship between captured grayscale and projected grayscale by the traditional method and the super-grayscale method, respectively, the following experimental steps are designed:  (2), and then the three divided patterns are composed into a repeated playing video; 4. The repeated playing video is projected onto the reference plane, and a 766-super-grayscale pattern can be captured as shown in Fig. 2b as long as setting the exposure time of the camera to three times the refresh cycle of the video. 5. By calculating the gray values of each grayed block in the two captured patterns separately, the relationships between the output gray value and the input gray value in the traditional and super-grayscale methods can be figured out as shown in Fig. 2c,d, respectively.
Due to the nonlinear effect of the projector 30 , there are nonlinear phenomena in both methods. The linear interval of the grayscale using the traditional grayscale method is 38 to 210, the proportion of the linear interval is only about 67% with a linear regression coefficient of 0.9984. While the linear interval of the grayscale using the super-grayscale method is 128 to 738, the proportion of the linear interval can be expanded up to 80% with a better linear regression coefficient of 0.9997. It can be seen that the super-grayscale method has a stronger linear relationship between the input gray value and output gray value than that of the traditional method, and the linear range of captured pattern with the super-grayscale method can expand by 20%.
Principle of optimized SRCGMP. In order to reduce the nonlinear influence of the digital projector on the captured deformed pattern, the four designed super-grayscale gratings, as shown in Eq. (1), are optimized. That is to ensure the peak grayscale and valley grayscale of sinusoidal gratings are in the linear range of supergrayscale grating.
The maximal and the minimal gray levels of the linear interval of the captured super-grayscale pattern are G 766 max and G 766 min respectively as shown in Fig. 2d, so: I or 0 (x, y) = I r o (x, y)I r 1 = www.nature.com/scientificreports/ If the above conditions are satisfied and the linear section is fully utilized, the a 766 0 and b 766 0 can be calculated as: In this way, the designed a 766 0 and b 766 0 can guarantee the captured super-grayscale pattern within the linear interval, and the wide grayscale range of the super-grayscale patterns can also be achieved. So the four expected super-grayscale gratings can be optimized as: Each optimized grating is divided into three fringe patterns according to Eq. (2) for compositing the corresponding optimized repeated video. The four optimized video gratings are used instead of the previously designed four video gratings for projection. The optimized super-grayscale deformed patterns can be captured. After extracting and calculating the computer-generated Moiré fringes, the wrapped phase of the measured object can be obtained, and the height of the measured object can be efficiently reconstructed by phase-unwrapping and height-phase mapping. In this way, the measurement accuracy of the proposed SRCGMP can be improved further.

Experimental results and analysis
To verify the feasibility and validity of the proposed method, several experiments have been implemented. The experimental setup is shown in Fig. 3. It contains one projector (View Sonic PLED-W200) with a spatial resolution of 1200 × 800, and the maximum refresh rate is 75 fps. One camera (BEV-B1610 M) with a resolution of 1628 × 1236, which includes 8-bit, 10-bit, 12-bit, 14-bit four transmissions, and the 10-bit mode is used in the experiments; and one computer (Intel(R) Core (TM) i5-4590) is used for processing the information.  Fig. 4. The measured heart model is shown in Fig. 4a. Figure 4b is the deformed pattern of CGMP. The cutaway view in the middle row of Fig. 4b is shown in Fig. 4c. The grayscale of the captured deformed pattern ranges from 36 to 228. The reconstructed heart model with CGMP is shown in Fig. 4d. The captured deformed pattern in the SRCGMP-766 method is shown in Fig. 4e, and Fig. 4f shows the cutaway view in the middle row of it. The grayscale of the deformed pattern in the SRCGMP-766 method ranges from 31 to 765, which achieves the super-grayscale pattern capturing. It can be seen from Fig. 4c and Fig. 4f that the grayscale variation ratio in the SRCGMP-766 method is more distinct and realistic than that of the CGMP method. The heart model can be reconstructed efficiently at this condition, as shown in Fig. 4g. Further analysis of the measurement accuracy, the cutaway view in row 589 with CGMP, SRCGMP-766, and 8-PMP is shown in Fig. 4h. The experimental result with 8-step PMP (8-PMP) has high accuracy 28 and can be regarded as quasi actual value. The profile with the SRCGMP-766 method is closer to that of the 8-PMP method. The results show that the SRCGMP-766 method has higher accuracy than the CGMP method due to realistic super-grayscale deformed patterns, which help improve the measurement accuracy originally.
The feasibility of optimized SRCGMP. One face model is measured to verify the feasibility of the optimized SRCGMP with 610-grayscale (SRCGMP-610) in which the grayscale is in the linear interval of the measurement system. The measuring results are shown in Fig. 5. The measured face model is shown in Fig. 5a and the captured deformed pattern in Fig. 5b. The grayscale of the deformed super-grayscale pattern ranges from 128 to 738, which is in the linear interval of the projected intensity grayscale, and the grayscale in the middle row is shown in Fig. 5c. After extracting the AC component of the deformed pattern and multiplying it with two prepared AC components with a π/2 phase difference, respectively, the 0-degree Moiré fringe pattern and the 90-degree Moiré fringe pattern can be obtained, as shown in Fig. 5d,e respectively. The reconstructed result shown in Fig. 5f is obtained by unwrapping and phase-height mapping algorithm. The whole and detail parts of the face model, such as the nose and mouth, could be effectively reconstructed by optimized video grating.
The measurement accuracy between CGMP, SRCGMP, and optimized SRCGMP. To demonstrate the measurement accuracy of SRCGMP, one face mask is measured with CGMP, SRCGMP-766, and SRCGMP-610, compared with the 8-PMP. The measuring results are shown in Fig. 6. Figure 6a shows the deformed pattern of CGMP. The grayscale in the middle row of the deformed pattern is shown in Fig. 6b, ranging from 36 to 228. The reconstructed face surface with CGMP is shown in Fig. 6c. Figure 6d shows the deformed super-grayscale pattern of SRCGMP-610. Figure 6e shows the grayscale of the middle row of Fig. 6d. It ranges from 128 to 738, which is in the linear interval of the measurement system. The grayscale of the SRCGMP-610 method achieves the super-grayscale deformed pattern capturing, and the grayscale variation ratio is also more distinct than the CGMP method, which is more realistic and would reduce the nonlinear effect of the projector. The reconstructed face surface with SRCGMP-610 is shown in Fig. 6f. It is evident that the reconstructed surface with the SRCGMP-610 method is smooth and shows higher fidelity. Further analysis of the measurement accuracy of the proposed method, the cutaways at row 756 of the reconstructed face surface  Fig. 6g. Figure 6h is the magnified view of the dashed box from Fig. 6g. The SRCGMP-610 method has the highest measurement accuracy and is closest to the 8-PMP due to the realistic pattern and reduced nonlinear effect. In order to further verify the measurement accuracy of the proposed method, three known heights are measured by CGMP, SRCGMEP-766, and SRCGMEP-610. The measured results are shown in Table 1. h is the mean height of the measurement results, root mean square error (MAE) denotes measurement accuracy, and root mean square error (RMS) denotes the measurement repeatability. Compared to CGMP and SRCGMP-766, SRCGMP-610 has the highest measurement accuracy.
Real-time measurement. In order to verify the validity of the SRCGMP-610 method in real-time measurement, a mobile palm with gloves is used.
The camera continuously records the deformed super-grayscale patterns and saves per pattern out of every 50 ms, which is equal to the exposure time of the camera. Three different states of the palm are used to analyze measurement results in real-time. The measuring results of the moving palm in three different states are shown in Fig. 7. Three deformed patterns with three states of the palm are shown from figure Fig. 7a-

Discussion
The super-grayscale and real-time computer-generated Moiré profilometry using video grating projection proposed in this paper have the following advantages: Video grating projection. The video grating synthesized by three purposefully designed 8-bit patterns is used for projecting instead of the traditional static grating in this method. The morphology of the grating has changed qualitatively. It skillfully uses the time-division multiplexing characteristics of the digital projector and the integration characteristics of the CCD camera to obtain the super-grayscale pattern.
Super-grayscale. In this method, by projecting the designed video grating and appropriately controlling the exposure time and the gain of the camera, the super-grayscale pattern can be captured by using the timedivision multiplexing characteristics of the projector and the integral exposure characteristics of the CCD camera. The super-grayscale method has a stronger linear relationship between the input gray value and output gray value and can expand the linear range by 20%, which reduces the digitized errors in reconstructed results.
Accurate. The experimental results using the optimized video achieve both the deformed super-grayscale pattern and the whole captured intensity grayscale linear proportion to the projected intensity grayscale. That can obtain the realistic deformed pattern and reduced nonlinear effect, to originally improve the measurement accuracy.

Real-time.
This method only needs to project a repeated video onto the measured object and only needs to capture one corresponding deformed pattern for 3D reconstruction while measuring, so the single-shot feature for real-time measurement can be kept.

Conclusion
A super-grayscale and real-time computer-generated Moiré profilometry using video grating projection is proposed by skillfully using the time-division multiplexing characteristics of the projector and the integral exposure characteristics of the CCD camera. The video grating synthesized by three purposefully designed 8-bit patterns is used for projecting instead of static grating. After projecting the repeated video onto the measured object and appropriately controlling the exposure time and the gain of the camera, the deformed super-grayscale pattern with 766-grayscale rather than the traditional 256-grayscale can be captured for reconstruction. The supergrayscale pattern is realistic and can effectively reduce digital error in computer-generated Moiré profilometry, and the proportion of the linear range of the super-grayscale pattern is expanded by 20%. The proposed method can originally improve the measurement accuracy and keep the single-shot feature for real-time measurement.   (d-f) the 0-degree Moiré fringe patterns of (a-c); (g-i) the 90-degree Moiré fringe patterns of (a-c); (j-l) the reconstructed surface at three states.