Position and orientation measurement adopting camera calibrated by projection geometry of Plücker matrices of three-dimensional lines

A position and orientation measurement method is investigated by adopting a camera calibrated by the projection geometry of the skew-symmetric Plücker matrices of 3D lines. The relationship between the Plücker matrices of the dual 3D lines and the 2D projective lines is provided in two vertical world coordinate planes. The transform matrix is generated from the projections of the 3D lines. The differences between the coordinates of the reprojective lines and the coordinates of extracted lines are employed to verify the calibration validity. Moreover, the differences between the standard movement distance of the target and the measurement distance are also presented to compare the calibration accuracy of the 3D line to 2D line method and the point-based method. Furthermore, we also explore the noise immunity of the two methods by adding Gaussian noises. Finally, an example to measure the position and orientation of a cart is performed as an application case of this method. The results are tabled for the reproduction by the readers. The results demonstrate that the line to line method contributes higher calibration accuracy and better noise immunity. The position and orientation measurement adopting the line to line method is valid for the future applications.

Camera is an important measurement instrument as it bridges the scales from the 3D space to the 2D space 1,2 . Camera calibration is the bridge to estimate the transformation matrix of the camera from a captured photograph 3 . Consequently, the camera calibration is widely studied in the vision measurement and optical inspection, such as object reconstruction 4 , computed tomography 5 , pose estimation 6 , and robot arm positioning 7 . As the transformation matrix of the camera contains the position and orientation information of a measured object in a captured image, we focus on the position and orientation measurement technique using a calibrated camera by the projection geometry of Plücker matrices of three-dimensional lines.
Various methodologies have been explored to solve the camera calibration problem. These technologies are approximately classified by the methods based on 3D, 2D, and 1D calibration targets. The 1D target is firstly described by Zhang 8 . The target should rotate to a fixed point in the calibration. Qi 9 introduced a calibration method using the 1D object with three or more markers. The constraint equations of the camera parameters are provided by the rotation around one marker which is moving in a plane. 1D calibration method provides simple structure and easy operation. However, the accuracy of 1D calibration methods is generally low due to the insufficient information on the 1D bar. Consequently, many calibration methods are mainly based on the 3D or 2D targets. To promote the camera calibration accuracy, Ricolfeviala 10 proposes an optimal calibration method based on several images of a 2D pattern. The optimal conditions are proposed to resolve the calibration process accurately. Bethea 11 develops a camera calibration technique by employing three parallel calibration planes and two cameras. Heikkila 12 presents an approach to calibrate the camera by circular control points identified on two perpendicular planes. In this paper, 3D target is chosen to calibrate the camera due to the high calibration accuracy and the sufficient information of the calibration target. Various patterns are employed on the calibration objects, such as points, circles, lines, and color patterns. Most of the calibrations adopt the feature points of the target 13,14 . Point-based calibration method achieves the advantages of high speed and easy operation. However, it is easily affected by the image noises. The circle-pattern-based calibration technique also attracts many investigators due to the high noise immunity. Xue 15 describes a method using concentric circles and wedge grating for camera calibration. An improved calibration method is proposed by Rui 16 to increase the camera and projector calibration accuracy simultaneously by detecting the edge of the circles. Xu 17 investigates a camera calibration method using the perpendicularity of 2D lines in the target observations. A study is presented by Yilmaztürk 18 for full automatic calibration of color digital cameras using color targets. Nevertheless, the color distortion is an unavoidable element in the process of capturing the color photos. Although the circle-pattern-based calibrations contribute high noise immunity, the method shows low efficiency due to the low speed of extracting the circles. The line-pattern-based calibration method is selected in this paper considering the moderate speed of extracting lines and good noises immunity. The original line-pattern-based calibration method employs the geometry relationship between the 2D line on a planar calibration target and the 2D projective line in the image. The essence of the method above is a 2D line to 2D line homography. The 3D calibration target is chosen to calibrate the camera owing to the high accuracy. However, it is difficult to build the homography from the 3D line on the target to the 2D line in the image as the coordinates of a 3D line are generally indicated by the equations of two planes. Therefore, there is a lack of the calibration method adopting the projective geometry from 3D line to 2D projective line.
In the paper, the position and orientation of an object are obtained from the captured image of a calibrated camera. Therefore, we firstly explore the camera calibration method adopting the projection geometry from the Plücker matrices of 3D lines to the 2D projective lines. A projective line in the image is determined by the corresponding 3D line on the calibration target and the projective plane. The transformation matrix of the camera is generated from the geometrical relationship between 3D lines and 2D projective lines. The 3D line to 2D line method is compared with the point-based method to verify the measurement validity, the measurement accuracy and the noise immunity. Then, a cart with the 3D target is chosen as the application example. The transformation matrix of the camera is decomposed to the rotation matrix, translation vector and the intrinsic matrix. The position and orientation of the measured cart is generated from the translation vector and the rotation matrix, and verified by the absolute and relative errors of the reconstructed displacements.

Results
According to the 3D line to 2D line method, the transformation matrix P is generated from n 3D lines L i and n 2D projective lines l i . The coordinates of the 2D projective lines are extracted by the Hough transform 19 . The recognition results of the lines are shown in Fig. 1. The results indicate the Hough transform can extract the lines exactly.
The differences between the coordinates of the reprojective lines and the line coordinates extracting by the Hough transform are employed to evaluate the accuracy of the 3D line to 2D line method. The comparison of the average logarithmic errors adopting the 3D line to 2D line method and the point-based method 20 is shown in Fig. 2. The image resolution is 1024 × 768. Moreover, in order to explore the relationship between the errors and the movement distance, the calibration board is moved by 10 mm, 20 mm, 30 mm, and 40 mm respectively. In the first group of experiments, the images are captured at the measurement distance of 1000 mm. The mean of average logarithmic errors using the line to line method are 1.  Besides, the second group of experiments are performed at a smaller measurement distance. The images are observed by the camera at the measurement distances of 800 mm. Similar to first group of experiments, the 3D line to 2D line method is compared with the point-based method to verify the measurement accuracy in Fig. 3. The mean of average logarithmic errors using the line to line method are 1.01 × 10 −4 , 1.61 × 10 −4 , 2.50 × 10 −4 , and 3.99 × 10 −4 corresponding to the movement distances of 10 mm, 20 mm, 30 mm, and 40 mm. The mean of average logarithmic errors adopting point-based method are 3.12 × 10 −4 , 4.81 × 10 −4 , 8.08 × 10 −4 , and 1.34 × 10 −3 corresponding to the movement distance of 10 mm, 20 mm, 30 mm, and 40 mm. The results of the two groups of experiments both show that the errors increase with the increasing movement distance. Moreover, the errors of the 3D line to 2D line method are all smaller than the point-based method at the movement distances. The results indicate that the line to line method provides higher calibration accuracy. Furthermore, the errors of the images at the distance of 800 mm are smaller than the errors of the images at the distance of 1000 mm. The two methods achieve higher measurement accuracy in the small measurement distance.
Furthermore, three levels of Gaussian noises are added to study the effects of the noises. The 3D line to 2D line calibration method is also compared with the point-based method in the two groups of experiments. The measurement errors are evaluated by where L′ is the reconstructed movement distance of the calibration board from the first place to the next place.    In order to explain the application on how the technique can be used, a clear example is provided to measure the position and orientation of a cart. The details of the example are illustrated in Fig. 6, in which a cart is attached by a 3D target on the top and translated with the displacements of 10 mm, 20 mm, 30 mm and 40 mm, respectively.  The positions t x , t y , t z and orientations α, β, γ of the cart about the o-x, o-y, o-z axes of the camera coordinate system in 20 different places are shown in Fig. 7(a-f). As the orientations α, β, γ are stable in the three angles in Fig. 7(a-c), the cart is moved along a straight line. According to the position data of the cart in Fig. 7(d-f), the displacement of the cart can be solved by the norm of the difference between the translation vector at the first place and the translation vector at the second place in the camera coordinate system. The absolute and relative errors of the reconstructed displacements in Fig. 7(g-h) are considered as the indicators to verify the measurement results of the cart. The measurement results are listed in Table 1.
As the movement displacement is 10 mm, the maximums of translations t x , t y , t z are − 778.28 mm, 90.41 mm and 1121.14 mm, respectively. The minimums of translations t x , t y , t z are − 778.64 mm, 89.98 mm and 1120.76 mm, respectively. The maximums of rotation angles α, β, γ are 24.92°, 44.34° and 52.86°, respectively. The minimums of rotation angles α, β, γ are 24.54°, 44.05°, and 52.45°, respectively. The means of measurement errors Δ L and relative errors Δ L/d are 0.18 mm and 1.85%, respectively. When the movement displacement increases to 40 mm, the maximal translations t x , t y , t z are − 745.61 mm, 91.17 mm and 1123.78 mm, respectively. The minimal translations t x , t y , t z are − 749.82 mm, 88.27 mm and 1120.13 mm, respectively. The maximal rotation angles α, β, γ are 26.77°, 46.37° and 55.43°, respectively. The minimal rotation angles α, β, γ are 23.06°, 42.99°, and 51.55°, respectively. The means of measurement errors and relative errors grow to 1.78 mm and 4.45%, respectively.
According to the above analysis and data in Table 1, since the translations t y , t z do not vary obviously, the o-x direction of the translation t x is the major movement direction. Moreover, the rotation angles about the three axes vary a little due to the cart is moved by a straight line. The mean of measurement errors increases with the rising movement displacement. The relative error also increases as the displacement is on the rise. Finally, the relative errors are less than 5% in most cases of experiments. It reveals that the measurement method is valid in the applications to solve the orientation and the position of an object.

Discussion
According to the analysis above, the mean errors of the 3D line to 2D line method and the point-based method grow with the increasing noises. The mean errors of the 3D line to 2D line method are smaller than the point-based method. In the test without noises, the line to line method achieves the maximum relative error of 7.42% and minimum relative error of 0.68%. The ones of the point-based method are 7.83% and 1.13%, respectively. In the test with the noises, the line to line method provides the maximum relative error of 8.14% and minimum relative error of 0.30%. The ones of the point-based method are 8.17% and 0.55%, respectively. The results show that the 3D line to 2D line method contributes higher noise immunity. Moreover, the errors in the 800 mm measurement distance are smaller than the ones in the 1000 mm measurement distance in the experiments. The data reveal that the two calibration methods provide higher noise immunity in the near-camera measurement.
In the example to measure the position and orientation the of the object, the proposed method achieves the error means of 0.18 mm, 0.83 mm, 1.05 mm and 1.78 mm corresponding to the measurement displacements of 10 mm, 20 mm, 30 mm and 40 mm. It indicates that the method is workable and reliable in the measurement applications of the position and orientation.

Methods
The calibration method is interpreted in the Fig. 8. A 3D line L i on the 3D target is projected to the 2D image plane. The projective line l i in the image plane is denoted by  In the other way, the dual 3D line of the 3D line ⁎ L i is defined by the Plücker matrix 21 as Considering the relationship between the 3D line ⁎ L i and its dual L i as ref.   Table 1. The means, the variances, the maximums and the minimums of the rotation angles α, β, γ, the translations t x , t y , t z , the absolute errors and relative errors of the reconstructed displacements. The 3D line ⁎ L i can also be derived from From equations (6) and (10), we have  , The stacking of equations (11) and (12) is