JOINT CALIBRATION OF 3 D LASER SCANNER AND DIGITAL CAMERA BASED ON DLT ALGORITHM

Design a calibration target that can be scanned by 3D laser scanner while shot by digital camera, achieving point cloud and photos of a same target. A method to joint calibrate 3D laser scanner and digital camera based on Direct Linear Transformation algorithm was proposed. This method adds a distortion model of digital camera to traditional DLT algorithm, after repeating iteration, it can solve the inner and external position element of the camera as well as the joint calibration of 3D laser scanner and digital camera. It comes to prove that this method is reliable. * Corresponding author


INTRODUCTION
Three-dimensional laser scanning technique (ZHANG F., 2008) (also known as laser detection and ranging) is an important measure technology developed rapidly in recent years.It is widely used in the fields of digital city construction, reconstruction of ancient buildings, reverse engineering and deformation monitoring.As digital camera can obtain the image data of the shooting targets, if combined 3D scanning pointcloud and the 2D digital image, it can realize the complementary advantages of the two kinds of data, which can bring great convenience in target recognition, feature extraction, 3D modeling, and so on.
At present, many scholars have carried on the related research to the registration of two kinds of data.Liu Jia (Liu J., 2008) of Capital Normal University used self developed 3D laser scanning measurement system to improve the corner detection algorithm, and realized the calibration of camera's internal and external parameters, and determined the mapping relationship between 3D laser scanning data and 2D camera's texture image.Zhao Song (Zhao S., 2012) of information engineering university designed a three-dimensional calibration target, represented the rotation matrix in the joint calibration with Rodrigo matrix, and solved the nonlinear equation by using Newton iteration method.The calibration of scanner and digital camera was completed through simulation experiments.Deng Fei (Deng F., 2007) of Wuhan University, basing on ICP (iterative closest point algorithm), proposes a nearest neighbor iterative registration based on stereo pair matching points and 3D laser scanning point clouds.The algorithm iteratively eliminates the influence of stereo images on the gross errors of mismatch by using the iterative estimation of M estimation in the nearest neighbor search algorithm, so as to complete the matching of 3D laser scanning data and digital images.
The existing methods mainly focus on how to map the image directly on the point-cloud data.But if get the same view of images after scan the point-cloud of the target surface, which is to realize the joint calibration of scanner and camera, it will save a lot of work of subsequent mapping.As some kinds of scanner (such an VZ series of Riegl) can set up digital cameras, therefore, the main purpose of this paper is to calibrate the position relationship between 3d laser scanner and digital camera.
If put a digital camera on a scanner, the location of the camera's position relative to the scanner's coordinate system (external azimuth element) will be fixed when the first photograph is taken.Because the instrument coordinate system of scanner is fixed at single station scanning and the 360 degree panoramic image acquisition of digital camera is uniform shooting based on the overlap of adjacent images along the Z axis of scanner coordinate system, the angle of rotation of the camera along the Z axis of the scanner instrument can be determined on the basis of the total number of images obtained, therefore, the outer azimuth element of each phase can be calculated according to the outer azimuth element of the first photograph.
The essence of joint calibration of 3D laser scanner and digital camera is to solve the transformation relationship between scanner coordinate system and camera coordinate system, and complete the mutual mapping of data in the two coordinate systems.In order to realize the joint calibration of the threedimensional laser scanner and the digital camera, the conversion parameters between the two coordinate systems are required.In this study, based on the direct linear transformation (DLT) method, we improve to calculate the exterior orientation elements of the image coordinate system relative to the scanner coordinate system, clarify the location relation of the two coordinate systems in the space, and then complete the work of joint calibration.While the scanner and the digital camera are calibrated jointly, the camera calibration process can be completed according to the characteristics of the direct linear transformation method.

METHOD
Direct linear transformation (DLT) (Wang D H., 2006), is the most widely used linear calibration method at present, was proposed by Abdel and Karara in 1971 (Abdel-Aziz Y I., Karara, H. M., 1971).The method considers only the line forming image model, without considering the nonlinear distortion of the camera.Through solving a set of linear equations, the intermediate parameters are obtained, and then the final parameters of the camera are obtained from the decomposition of the intermediate parameters.

DLT algorithm model
The standard DLT model can be seen as the evolution of the collinear conditional equation (Formula 1): (2) Then: The DLT equation can be obtained by sorting (3) and multiplying the molecular as well as denominator at the same time by ' L in (2): (5) The matrix expression of the error equation is as follows: According to the equation ( 7): The external bit line element relative to the scanner coordinate system can be solved by ( 8): Considering the distortion factor of camera lens, the DLT model with distortion correction is used to correct the algorithm, and the DLT algorithm model with distortion correction is as follows:  are comprehensive error with distortion: (10) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium "Developments, Technologies and Applications in Remote Sensing", 7-10 May, Beijing, China This contribution has been peer-reviewed.https://doi.org/10.5194/isprs-archives-XLII-3-377-2018| © Authors 2018.CC BY 4.0 License.
The error equation of the DLT coefficient is: L is known, distortion parameter and 0 x , 0 y can be solved as well.
The internal parameters obtained above are brought into the DLT algorithm model with distortion correction.The distortion parameters are solved, and the resulting distortion parameters are reintroduced into the equation ( 9), and the new internal orientation elements are obtained.Keep continuous iterations until the amount of the calculated focal length is less than the predetermined threshold.

EXPERIMENTAL VERIFICATION
In this experiment part, a three-dimensional calibration rack was designed, which was pasted by the artificial reflection marks.Scanned the target by Rigel's VZ-400 3D scanner, and taken pictures by Nikon D700 camera with a 20mm focusing lens at the same time to complete the calibration of the VZ-400 scanner and the D700 camera, and then complete the fusion of the images and the point clouds.

Design of the frame and selection of reflection mark
In this experiment part, a three-dimensional calibration rack was designed.The calibration rack is made of metal steel, which has excellent stability and is not easy to deform.The calibrated frame is designed for 0.8m * 1m x 0.8m.The artificial reflection mark was pasted evenly on the calibration rack, avoiding setting the marks on the same plane to ensure the stability of the calibration.During the experiment, diamond material diameter 50mm circular light reflecting mark produced by the US 3M was used.As shown in Figure 1:

Process
The digital camera is Nikon D700, equipped with Nikon 20mm/F2.8DNikkor lenses.Scanner is VZ-400 3D laser scanner produced by Austria Riegl company.The instrument device is shown in Figure 2, The specific steps of the experiment are as follows: 1.The digital camera was fixing to the top of the scanner through the fixed rack, then scan the 3D calibration rack with the scanner, meanwhile, the calibration images are obtained.2. The image plane coordinates of the center of the mark point in the images taken by the digital camera, and the three-dimensional coordinates of the center of the mark point in the scanned point-cloud were extracted.3. Two sets of the coordinates were calculated by the joint calibration model.4. The calibration results of the relationship between the scanner and the digital camera position are verified through overall mapping.

Result
The coordinates of the mark point in the coordinate system of the laser scanner instrument, as shown in The pixel size of the D700 camera is 8.439 * 10 -3 millimeters, and the coordinates of the mark point in the image space coordinate system can be obtained.
The two sets of coordinates in Table 1 and table 2 are substituted into the direct linear transformation model above, and the inner orientation elements of the photo and the exterior orientation elements in the 3D laser scanner coordinate system are obtained, as shown in Table 3.  3 The result of inner and external position element When the internal and external elements are known, the threedimensional coordinates are brought into the collinear conditional equation ( 1), and the corresponding image space coordinates can be obtained.In order to verify the internal and external elements obtained from experiments, we evaluate the accuracy of point cloud and image fusion, bring the coordinates of point cloud markers into (1) form, and scale to image plane coordinates, and compare them with the corresponding plane coordinates of landmark points corresponding to table 2, and the results are shown in  4 The coordinate difference of reflector in image coordinate From table 4, it is known that The result shows that in addition to the influence of a few points due to the lower accuracy of the extraction, the accuracy of the fusion of point-cloud and images can reach 1 pixel.Meanwhile, according to the calibration results, the overall map of the full view point cloud in a laboratory has been realized, which has achieved good results, as shown in Figure 3.

CONCLUSION
A joint calibration model of 3D laser scanner and digital camera is proposed in this paper.On the basis of direct linear transformation, the method is added to the distortion correction, which makes the calibration model more complete and improves the precision of the joint calibration.In addition, the method can simultaneously solve the internal orientation elements of the digital camera, including the distortion parameters, and the joint calibration of the digital camera and the 3D laser scanner, that is, the solution of the exterior orientation elements.The experiment shows that the precision of the algorithm meets the basic requirements of joint calibration.

Table 1 (
only a part of the coordinates are listed).

Table 1 The
3D coordinates of reflectors measured by laser scanningUse digital camera to shoot the mark points in Figure1and extract coordinates in the image plane, as shown in Table2.

Table 2 The
2D coordinates of reflectors measured by digital camera