HIGH-DEFINITION POINT CLOUD MAP-BASED 3D LiDAR-IMU CALIBRATION FOR SELF-DRIVING APPLICATIONS

The multi-sensor fusion scheme has become more and more popular these days with its great potential to estimate reliable navigation information for the modern development in automated driving system (ADS) and mobile mapping systems (MMS). Since these systems are combined with numerous navigation sensors, thus their geometric relationship should be precisely known. This study focuses on practical aspects when calibrating LiDAR-IMU mounting parameters (lever-arms and bore-sight angles) in land-based MMS. This calibration model is based on expressing the mounting parameters within the direct georeferencing equation for each epoch time and conditioning a set of INS/GNSS and LiDAR navigation solutions to lie on it. There is no need for a required information about the planar features in the calibration field as part of the unknowns. Such conditions are only benefitable in the residential area where the presence of sufficient planes in form of building is abundant. We present an approach for recovery the mounting parameters by conditioning the high-definition (HD) point cloud map-based LiDAR information and INS/GNSS navigation solutions through the least-squares solutions. The presented results and discussion mainly focus on practical examples with data from land-based MMS. Preliminary results indicate that correct calibration parameters are not only capable to improve the performance of point cloud georeferencing but also dramatically provide reliable performance evaluation of navigation estimation. Moreover, these findings show that the studied method is not only applicable in the featureless environment but also in its practicality to the self-driving applications.


INTRODUCTION
To obtain an accurate georeferenced LiDAR scan of point clouds and navigation evaluation in MMS and ADS, LiDAR mounting parameters (i.e., lever-arm and bore-sight angles) need to be considerably estimated. There are several LiDAR calibration approaches have been proposed and developed over the past decade. For an example, the adopted approach (Skaloud and Schaer, 2007) based on the functional model for the recovery of calibration parameters through the automated detection and extraction of the planar section of the rooftops in airborne LiDAR system (ALS). This functional model is based on conditioning the georeferenced LiDAR target points to lie on surface planes (Skaloud and Lichti, 2006). Moreover, there is an approach that also followed an above functional model but adopted into their land-based mobile laser scanning system for estimating LiDAR mounting parameters (Tsai et al., 2018). As same as previously mentioned methods, most the existing strategies still relied on the specific planes (surface normal), key feature points (e.g., edges and cylinder object points) (Lv et al., 2020;Liu and Li, 2019), and locally conducted in the preferable laboratory and indoor environment (Le Gentil et al., 2018;Liu, 2017). Considering to such the procedures, LiDAR odometry (LO) is used to perform a single or multi-feature-based registration for LiDAR pose estimates and then formulate the calibration estimation model. For initialization in LO, only an Inertial Measurement Unit (IMU)-derived information is solely used to initialize LiDAR registration. In such a situation, LO accuracy may suffer from the features used , quality of initial pose, and prior point cloud map. Furthermore, another calibration framework is proposed in a structured environment using a specific plane to perform LO for parameter estimation (Li et al., 2021). To this end, the drawback of these existing methods for estimating the LiDAR-IMU mounting parameters are depend on the specific feature information, reliable navigation solutions, and its limit with practicality to ADS. To obtain more applicable calibration parameters for ADS, we propose the functional model for recovery the LiDAR-IMU calibration parameters through direct georeferencing of LiDAR measurement with the INS/GNSS and HD point cloud map-based LiDAR navigation solutions.
The organization of the paper is as follows. Firstly, we introduce the LiDAR-based navigation system with including INS/GNSS integration, direct georeferencing of LiDAR measurements, and the HD point cloud map-based LiDAR scan matching. Secondly, we propose the estimation model for recovery of calibration parameters. Thirdly, we describe an experiment including the sensor configuration and environment when applying this approach to land-based mobile mapping system vehicle. Next, the experimental results are presented and discussed. After that we conclude our practical experience.

LIDAR-BASED NAVIGATION SYSTEM
In order to obtain an accurate navigation estimation based on LiDAR measurements, there are three remarkable steps: initial pose from INS/GNSS integration, point cloud georeferencing, and LiDAR scan matching techniques. The first step is to synchronize the time between the LiDAR data and INS/GNSS navigation solution to be the same time domain and then interpolate the navigation information. The output from this process is the initial pose of LiDAR measurements. For the second step, the direct georeferencing of LiDAR measurements is conducted regarding an initial pose obtained from previous step. The third step is to refine the LiDAR navigation solution. This process uses the HD point cloud map-based LiDAR scan matching with NDT algorithm. The output from this process is used to formulate the functional model for calibration parameter estimation. The following sections describe the INS/GNSS integration, point cloud georeferencing, and HD map-based LiDAR scan matching in details.

INS/GNSS Integration
The integration of an inertial navigation system (INS) and a global navigation satellite system (GNSS) through the Kalman filter (KF) algorithm is considerable as the most applicable approach over the stand-alone system. The INS/GNSS integration systems are categorized into two schemes; loosely coupled (LC) and tightly coupled (TC) schemes. However, applying such the TC-INS/GNSS integration system would generally provide better navigation estimation due to its complexity.
The navigation state vector ( k x ) of the KF algorithm is expressed as follows: where  = the state transition model k H = the measurement matrix In addition to the outcome of TC-INS/GNSS integration system, the navigation solution was post-processed with the forward and backward smoothing process (Shin, 2005).

Direct Georeferencing of LiDAR Measurements
where , , , , = the lever-arm offset between the IMU and LiDAR centers expressed in the b-frame.

HD Point Cloud Map-Based LiDAR Scan Matching
In order to obtain an accurate position and orientation system (POS) of LiDAR sensor, the static map-based (i.e., scan-tomap) approach is applied in this paper. As already reported in (Magnusson, 2009), the LiDAR scan matching with the normal distribution transform (NDT) provides more robust and better pose estimate than using the iterative closest point (ICP) variants.
The proposed LiDAR scan matching algorithm follows the workflow depicted in Figure 2. As mentioned earlier, the need for LiDAR-based navigation estimation with scan-to-map approach arises from the LiDAR measurements and their corresponding transformation by the integrated INS/GNSS solution. To appropriately align a LiDAR scan to a prior map and prevent NDT algorithm from trapping into local minima, it needs the reliable navigation solutions to estimate an initial pose for LiDAR registration. It proceeds through the time synchronization and interpolation process. For simplicity, the proposed LiDAR scan matching algorithm proceeds in three main steps: pre-processing and georeferencing point cloud, map extraction, and NDT pose estimate. The outcome of this process is the LiDAR pose (i.e., translation vector and rotation matrix) with respect to the m-frame. More details on the NDT algorithm and its extension can be found in (Biber and Straßer, 2003;Magnusson, 2009).

Functional Model
The development of the functional model is based on conditioning the NDT-estimated LiDAR's POS and integrated INS/GNSS solution. When considering the use of a navigation grade INS and dual frequency GNSS receivers, the residual effects in TC-INS/GNSS navigation estimation should be lower than other cases in position and attitude. Regarding the better georeferencing, it is assumed that the LiDAR measurements are registered perfectly with a prior map by the NDT algorithm. As demonstrated by Figure 1, the lever-arm vector ( b l a ) and the rotation matrix ( b l R ) can be formulated herein as: where   , , The observations ( ( ) l t  ) at time t is a function expressed by the position and attitude of the INS/GNSS navigation solution and LiDAR scan matching results, respectively. While the calibration parameters ( x  ) of LiDAR and IMU are represented by the lever-arm and bore-sight angles: ( ) , , , , , , , , , , , , , , , , The functional model used for the parameter estimation:

Least Squares Solution
Since the functional model are non-linear, the solution is derived using the traditional method of non-linear least-squares adjustment. The linearized system of equations takes the form: Following standard procedures, the final form of the normal equations used herein are: Once the normal equations are formed, the corrections of calibration parameters are computed from an above equation. The parameter estimates are then improved iteratively. To start the iteration solution, the approximate calibration parameters can be set to the on-site measures and zero for the lever-arm and bore-sight angles, respectively. The principal advantage of proposed method are its computational efficiency, reliability, and practicality particularly during the alignment procedures with MMS or automated driving systems.

Configuration Description
A land vehicle-based mobile mapping system is adopted in our case for calibration parameter estimates. As shown in Figure 3, the experimental platform used in this study includes a navigation grade-IMU (iNAV-RQH), GNSS receiver and antenna (PwrPak7), and LiDAR (Velodyne VLP-16).
Considering the calibration parameters, only a LiDAR mounted on the horizontal plane is investigated in our study.
As mentioned earlier, the navigation solution was postprocessed using the TC scheme with forward and backward smoothing process through the commercial INS/GNSS software (Inertial Explorer, IE). Table 1 and Table 2 show the specification and performance characteristics of iNAV-RQH IMU and LiDAR, respectively.

Environmental Description
Regarding the calibration field, we only make use of a single test field depicted in Figure 4. As will become apparent, the calibration field is essentially the flat terrain of the road with regular trees, lighting poles, and some buildings. The principal advantage of this calibration field is not only its practicality but also demonstration the effect of road geometry and components on the quality of the calibrated parameters when applying the proposed method.
In addition to the GNSS conditions, they were almost optimal for this calibration field (indicated by the green-dashed circle in Figure 4) with favourable satellite geometry and satellite observation number. As a result, GNSS position and velocity are well observable and estimable. These facts, together with the navigation grade-IMU, contributed to the good estimation of the INS/GNSS solutions.

RESULTS AND DISCUSSION
To evaluate the proposed calibration method, the results are divided into three parts: calibration parameters, georeferenced point clouds, and HD map-aided LiDAR navigation estimation. For the first results, the parameters before and after calibration are presented. The second results depict the georeferenced LiDAR scan of point clouds with different calibration parameters. For the third results, the positioning differences of proposed LiDAR scan matching algorithm with different calibration parameters are analyzed. The following subsection describes the aforementioned results in details.

Calibration Parameters
As already mentioned, this study adopts the kinematic calibration model involving the INS/GNSS and NDT pose information through the direct georeferencing to construct the functional calibration model. Table 3 shows the estimated parameters before and after calibration including the lever-arm and bore-sight angles. As a result, it is clear that the calibration parameters totally differ from the initial values (i.e., before calibration) particularly a lever-arm in z-axis and all the boresight angles. Regarding the poor calibration parameters, this kind of error source would directly affect to the georeferenced point clouds and significantly lead to the wrong evaluation results especially in automated driving systems. In some case, the bore-sight angles can have a very small value but there should not be perfectly equal to zero. To this end, the preliminary results of proposed model seems to precisely estimate those unknowns and more practically bridge the gap of unreliable calibration parameters from manual measures.  Table 3. Comparison of parameter estimates before and after calibration.

Georeferenced Point Clouds
In terms of LiDAR scan matching (i.e., registration), LiDAR-IMU calibration parameters take an important role for the direct georeferencing. Therefore, we can make use of the georeferenced point clouds before and after calibration to visually evaluate the calibration parameters. Figure 5 illustrates that the georeferenced LiDAR scan of point cloud (indicated by the red points) with the initial values (before calibration) has a large amount of mis-alignment particularly in the height and all the bore-sight angles compared to the HD point cloud map (indicated by the grey points). On the contrary, Figure 6 indicates that the georeferenced LiDAR scan of point clouds (indicated by the green points) with our parameters (after calibration) perfectly aligned with the HD point cloud map. It is clear that our calibration model can precisely estimate and improve the LiDAR-IMU parameters.

HD Map-Aided LiDAR Navigation Estimation
As previously described, this study makes use of the HD point cloud map-based LiDAR navigation system to estimate the LiDAR navigation state (i.e., position, velocity and attitude) and formulate the LiDAR calibration model at the same time. Table  4 shows the statistical information of positioning differences before and after calibration with the proposed LiDAR navigation algorithm at the calibration field. As a result, it is clear that the poor LiDAR mounting parameters (before calibration) lead to wrong performance evaluation in terms of positioning differences compared to TC-INS/GNSS solution. The RMS of positioning differences can reach up to 0.040, 0.019, and 0.160 meters in the east, north, and height components, respectively. On the other hand, it is worth mentioning that there is a great amount of improvement with making use of our mounting parameters (after calibration). The RMS of positioning differences are only about 0.020, 0.018, and 0.056 meters in the east, north, and height components, respectively. Furthermore, the improvement is up to 50%, 5%, and 65% increases after calibration in the east, north, and height components, respectively.  Table 4. Statistical information of positioning differences before and after calibration with the proposed LiDAR navigation algorithm.

CONCLUSIONS
As publicly well-known, the multi-sensor fusion scheme has a great potential to be the core navigation estimator for the modern development in automated driving or self-driving systems. To accurately obtain the best performance with lowcost LiDAR sensor to such a system, this paper proposes the calibration model to estimate the LiDAR mounting parameters using the TC-INS/GNSS and HD map-aided LiDAR navigation solutions.
As a preliminary result, accurate calibration parameters can improve the performance of point cloud georeferencing and navigation evaluation dramatically. Considering the positioning difference results in the calibration field, the experimental results show that the positioning differences has a small amount based on the parameters after calibration. Hence, the benefit of the studied method is not only applicable in the featureless environment (surface normal) but also in its practicality and The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France suitability to the self-driving applications since they usually make use of HD map and integrated INS/GNSS information.

ACKNOWLEDGEMENT
The author acknowledges the financial supports provided by the Ministry of Science and Technology (MOST).