MOBILE LASER SCANNING SYSTEMS FOR GPS/GNSS-DENIED ENVIRONMENT MAPPING

Indoor 3D mapping provides a useful three-dimensional structure via an indoor map for many applications. To acquire highly efficient and relatively accurate mapping for large-scale GPS/GNSS-denied scene, we present an upgraded backpacked laser scanning system and a car-mounted indoor mobile laser scanning system. The systems provide both 3D laser scanning point cloud and camera images. In this paper, a simultaneous extrinsic calibration approach for multiple multi-beam LIDAR and multiple cameras is also proposed using the Simultaneous Localization and Mapping (SLAM)-based algorithm. The proposed approach uses the SLAM-based algorithm to achieve a large calibration scene using mobile platforms, registers an acquired multi-beam LIDAR point cloud to the terrestrial LIDAR point cloud to acquire denser points for corner feature extraction, and finally achieves simultaneous calibration. With the proposed mapping and calibration algorithms, we can provide centimetre-lever coloured point cloud with relatively high efficiency and accuracy. * Corresponding author. clwen@xmu.edu.cn


INTRODUCTION
In recent years, indoor three-dimensional (3D) map building becomes a hot research topic.Indoor 3D mapping provides a useful 3D structure via an indoor map for many applications, such as disaster searching and rescue inside buildings, virtual reality displaying, and restoring the complex internal structure of buildings etc. (Ellekilde et al., 2004).The traditional ways to get a 3D structural model of a building mainly rely on artificial hand-drawing or reconstructing based on design data, and these ways have limitations of manual error involved, low efficiency.Another solution is registering multi-scans from high precision Terrestrial laser scanner and RGB-D camera, and these ways are with low efficacy, and not suitable for indoor environment with multiple corners.
The robot-based indoor mapping system (Wen et al., 2014) and backpacked indoor mapping system (Wen et al., 2016) have been developed in the last several years.Different systems are suitable for different GPS/GNSS-denied environment mapping tasks.To unify the coordinate systems of the different sensors, the 3-D coordinate transformation relations between the LIDAR sensor and the camera must be calibrated accurately.At present, for most LIDAR sensor and multiple camera hybrid calibration algorithms, each camera and LIDAR sensor must be calibrated separately.In multi-beam LIDAR sensor calibration, it is difficult to find the corresponding target points or corner features from the sparse point cloud in the common view of the LIDAR sensors and cameras.Regarding the sparse point cloud and multiple cameras in the system, currently there are few calibration methods to determine the simultaneous extrinsic calibration between multi-beam LIDAR sensors and multiple cameras.
In this paper, to acquire highly efficient and relatively accurate mapping for large-scale GPS/GNSS-denied scene, we present an upgraded backpacked laser scanning system and a car-mount indoor mobile laser scanning system.Meanwhile, a simultaneous extrinsic calibration approach for multi-beam LIDAR sensors and multiple cameras is proposed for the above multi-sensor systems.To achieve simultaneous calibration of multi-beam LIDAR sensors and multiple cameras, Simultaneous Localization and Mapping (SLAM)-based algorithm is first used to acquire scene data for a large area.The point clouds acquired by the multi-beam LIDAR are relatively sparse.To acquire denser points for feature extraction, a terrestrial laser scanner scans the complete calibration scene acquired by the multi-beam LiDAR and registers it to the point cloud.This approach works well on both regular and fisheye lens camera images.Along with 3D mobile mapping process, a coloured 3D point cloud can be provided.

System Design
Two mobile laser scanning systems are designed for GPS/GNSS-denied scene use.The upgraded backpacked mobile laser scanning system (Figure 1

Intrinsic Calibration of the Cameras
The proposed approach is designed to apply, not only to cameras with regular lenses, but also fisheye and wide-angle lenses.The camera internal reference model is given by , where ( , )   xy ff is the focal length of the camera, and ( , )   xy cc is the main optical axis of the camera.A camera calibration method (Scaramuzza et al., 2006) is used to determine the internal parameters and distortion factors of the camera and obtain the camera internal reference model.

Multi-LIDAR Sensor Calibration
In our system, multiple LIDAR sensors are used to acquire 3D data.The calibration of the multi-LIDAR sensor is calculated recursively in the construction of the sub-map and its isomorphism constraint (Gong et al., 2018).

SLAM-based Mapping
For each scan of the multi-beam LIDAR sensor, the acquired point cloud is for a single scene and includes limited information about the environment.We first use a feature-based SLAM algorithm (Zhang et al., 2014) to acquire 3D mapping of multiple scans of point clouds by moving the multi-sensor mapping system.An example of a 3D map of an outdoor scene is given in Figure 2 ) ( ) ( ) where f = focal length; we make For each camera, nine directional cosine elements ( ) , , , , , ,  a a a b b b c c c are calculated for the three angles between the SLAM-based LIDAR point cloud frame and camera frame (φ,ρ,θ) as: (5)

Mapping Process
The 3D laser scanning frames acquired by the HDL-32E laser scanner and Velodyne VLP-16 Laser scanner are registered by a feature-based SLAM algorithm (Zhang and Singh. 2014).We select the feature points that are on sharp edges and planar surface patches in each frame for calculation.Considering that each frame acquired includes 32/16 individual beams, every point's smoothness can be evaluated through the spatial relationship with its surrounding points on the beams.Then, a 3D point cloud map and the platform trajectory can be reconstructed by matching these features.Closed-loops are detected using the distance between the frames.A possible closed loop is detected when two frames are discontinuous for a long time and their spatially distance is less than a threshold.
Regarding the accumulated mapping error, a G2O framework (Kümmerle et al., 2011) is used to optimize the trajectory and 3D map to finally obtain a consistent map.

EXPERIMENTS
Our main experiments related to 3D mapping are conducted in the following two aspects: (1) 3D mapping results of the proposed system and method.
(2) With the calibrated laser scanner and cameras, the coloured 3D point cloud map is shown in this section.More experimental results will be given using more complicated scenes.

3D Mapping Results
An example testing scene is shown in Figure 4.The backpacked and car-mounted mobile laser scanning systems explore the GPS/GNSS-denied scenes to acquire the 3D data.Based on our mapping algorithm, 3D map can be built (Figure 4

Car-mounted Mobile Laser Scanning Systems:
The proposed calibration method was tested also on a car-mounted multi-sensor system, consisting of a Velodyne HDL-32E LIDAR sensor, a Velodyne VLP-16 LIDAR sensor for 3D data acquisition, and four fisheye lens cameras.The Velodyne HDL-32 LIDAR sensor has a typical accuracy range of 2cm.Our calibration approach was tested on a large scale indoor basement scene using the car-mount system.A 200 x 200m size basement scene was used to test the proposed calibration approach.Results indicate the data from the calibrated sensors are well fused with different multi-sensor designs (Figure 6).The insufficient, inconsistent lighting situation results in an inconsistent color distribution of the calibration results.Our calibration approach also can be used in outdoor scenes.For the above scenes, a marker-based method, similar to the method (Zhuang et al., 2014), was used for an error analysis of the proposed calibration method.As shown in Table 1, the average distances between the corresponding center points of the marker and the total least mean square error were calculated.
For each scene, 10 groups of points were selected on the floor and walls, respectively.Results indicate that our approach achieves good accuracy and robustness for calibration.We observed smaller error is calculated for indoor scene regarding smaller error of manual point selection in indoor scene with closer range.

CONCLUSIONS
We present an upgraded backpacked laser scanning system and a car-mount indoor mobile laser scanning system for 3D data and camera image acquisition in GPS/GNSS-denied environments.And we proposed a simultaneous extrinsic calibration approach for multiple LIDAR sensor and multiple cameras.The approach uses the SLAM algorithm to achieve a large calibration scene by the mobile system and registers the acquired laser scanning point cloud to the TLS point cloud to acquire denser points for feature extraction.The proposed approach is designed to apply on cameras with regular, fisheye and wide-angle lenses.With the proposed mapping algorithm and the calibration method, a coloured 3D map can be provided.
sensor A at time (0~n) in the mapping algorithm, 2 n P is the point cloud of LIDAR sensor B at time, n. in T is the initial coordinate transformation between the LIDAR sensors. 1 n T and 2 n P are synchronized.Calibration is the calculation of the exact calibration matrix cal T by: the nearest neighbour point search algorithm.Using 1 n T and in T the point cloud, 1 n P , of LIDAR sensor A is first transformed to its location at time n in the sub-map M. Then the NN algorithm is used to search the sub-map for the nearest neighbour point, 1 n P .Finally, to obtain n cal T , an environmental consistency constraint is introduced.
(a).Considering that acquired point cloud data is relatively sparse, it is inaccurate to locate features directly in an original point cloud.To overcome this problem, a high-precision Terrestrial laser scanner (TLS), Riegl VZ 1000 is used to obtain the point cloud of the entire calibration scene with an accuracy of about 5mm.Then, using an ICP algorithm, dense TLS point cloud data is registered to the sparse SLAMbased point cloud.The registered dense point cloud also provides a relatively accurate estimation of the camera position in the SLAM-based point cloud coordinate system.Figure2(b) and Figure 2 (c) show the zoom-in views of a wall area before and after registration, respectively.

Figure 2 .Figure 3 .
Figure 2. (a) Point cloud data acquired by our system.(b) Dense point cloud before registration.(c) Dense point cloud after registration 2.2.4 Calibration of the LIDAR and Cameras In our method, highly reflective boards (Figure 3(a)), which are easily detected in the point clouds (Figure 3(b)), are used for calibration.For camera images, a corner detector extracts the candidate edge points of each square.For point cloud data, an intensity threshold-based method first extracts the high intensity areas of the targets.Then, a region growing method, along with a plane fitting method, are applied to obtain the high intensity block candidates on the reflective boards.Four pairs of correspondent corners are manually selected from the image corner candidates and intensity corner candidates.

(
Figure 4. (a) 3D map of the underground scene.(Wang et al., 2018) (b) A close look of 3D map of the underground scene.(c)3D map of multi-floor building scene.3.2Coloured Point Clouds3.2.1 Backpacked Mobile Laser Scanning Systems:Point cloud data and camera images of a three-floor building exterior were collected by carrying our backpacked multi-sensor system and walking along the corridors and staircases.The 3D indoor map was built based on the SLAM algorithm.With the proposed calibration approach, the LiDAR point clouds and images were well fused (Figure5).

Figure 6 .
Figure 6.Coloured point clouds fused by camera image and laser scanner point clouds.
Figure 7(c) displays an outdoor scene fusion result using the transformation matrix achieved by the proposed calibration method.As seen in Figure 7(a), some examples of images are selected from a sequence of corrected images.The 3D point cloud data built by our SLAM mapping algorithm is shown in Figure 7(b).Blue points represent an acquired higher reflectance intensity.Calibration results for outdoor scenes.(a) Camera images.(b) Point clouds.(c) Fusion result.

Table 1 .
Calibration errors of three scenes