GLOBAL REFINEMENT OF TERRESTRIAL LASER SCANNER DATA REGISTRATION USING WEIGHTED SENSOR POSES

Terrestrial laser scanner (TLS) sensor captures highly dense and accurate point clouds quite useful for indoor and outdoor mapping, navigation, 3D reconstruction, surveillance, industrial projects, infrastructure management, and others. In this paper, we present a global registration method that weights the sensor poses for refinement of TLS data registration. Our global refinement method assumes that the variance-covariance matrix that describes the uncertainty of sensor poses is available to refine the registration errors. The effectiveness of the proposed method is demonstrated with TLS dataset obtained into outdoor environment. Our results show that the weighting the sensor poses obtained in registration task improves the positional accuracy of TLS sensor


INTRODUCTION
TLS systems can provide 3D point clouds quite useful for many engineering projects, such as, modelling, infrastructure inspections, urban mapping, heritage documentations, and others.Basically, the laser device emits a beam towards the object which reflects the beam.It records its travelling time jointly with the horizontal and vertical scan angles of laser beam.As result it obtain the surface geometry in the form of point clouds.Such sensors can capture highly dense and accurate point clouds in a fast time period.
In order to recover large or complex areas, the TLS system captures point clouds from multiple views.A pre-requisite to ensure a consistent 3D model of the environment is the registration of the pairs of point clouds into a global reference frame.Thus, the challenge to build a consistent 3D map of the environment is twofold.First, a pairwise registration is achieved to estimate the sensor poses (three rotations and three translations), also known as transformation parameters, between every pair of point clouds.Second, a global refinement is used to align all point clouds into a single global reference frame.
In fact, the pairwise registration is prone to drift over time.In large areas, the pairwise registration errors accumulate and may result in an inconsistent model.Thus, a global refinement is important to minimize the registration errors.
In this paper, we present a global refinement of TLS data registration.Our global refinement method assumes that the variance-covariance matrix that describes the uncertainty of sensor poses is available to refine the registration errors.Given an unstructured set of pairs of point clouds, we included the weighted transformation parameter into global refinement solution to evenly minimize the accumulated errors that result from attaching pairwise registrations.This task is formalized with the help of a pose-graph approach, such as proposed by (Bormmann et al., 2008).We demonstrate that weighting the estimated transformation parameter improves the positional accuracy of TLS sensor.
The paper proceeds with related work in the Section 2. In Section 3, the methods for pairwise registration and weighting of global refinement solution are described.Section 4 describes the experiments and its results using the proposed method.The paper concludes with final remarks in Section 5.

RELATED WORK
Basically, TLS data registration methods are based on two key problems: pairwise registration and global refinement.We will cover the main related work about the techniques in our study by grouping them into two main different categories: (1) Pairwise registration: One of the most popular methods is the ICP (iterative closest point) for registration of 3D shapes.It was developed by (Besl and McKay, 1992).According to (Henry et al., 2012) "ICP has been shown to be effective when the two point clouds are already nearly aligned".(Chen and Medioni, 1992), (Masuda and Yokoya, 1994), (Eggert et al., 1998), (Okatani and Deguchi, 2002) and others have proposed some variations and improvements for the ICP algorithm, i.e. using point-to-plane and plane-plane approaches.
(2) Global refinement: There are few approaches proposed in the literature for global refinement of TLS data registration.(Lu and Milios, 1997) presented a method for global refinement of multiple 2D point clouds.An extension of Lu and Milios' proposed algorithm has been proposed by (Bormmann et al., 2008), which is able to registers the positions of a 3D point clouds.(Theiler et al., 2015) proposed a globally consistent registration of TLS via graph optimization.This task is formalized by combining ICP algorithm and (Lu and Milios, 1997) approach.They evaluated their algorithm on a TLS dataset and presented its effectiveness.In (Pathak et al., 2010) a global closed-form refinement using pose-graph relaxation technique for enhancing the consistency of 3D maps was proposed.(Ji et al., 2015) showed globally consistent 3D map of high-speed viaduct point clouds using closing condition and external geometric constraints.In the work of (Yang et al., 2013) a branch-and-bound strategy is used to globally solve the objective function.(Pavan and dos Santos, 2017) placed all the rotation parameters into a common coordinate system by play on a property of quaternions.Then, a global refinement procedure was applied to produce more accurate TLS data registrations.

METHOD
In this paper, we focus on two aspects for global refinement of TLS data: a plane-based approach for pairwise registration, and weighting of the sensor poses based on uncertainty of the transformation parameters for global refinement of TLS data registration.The generic structure of our proposed method is shown in Fig. 1.
Figure 1.Generic structure of the proposed method.
In the Figure 1, the framework of the proposed method consists of five key tasks.First, the outliers are detected and removed using the statistical outlier removal method (Rusu et al., 2008).Then we segment surface planes using a joint effort of RANSAC algorithm (Fischler and Bolles, 1981) and (Golub and Loan, 1980) method to fit the segmented planes are determines its normal vector and perpendicular distances from the origin.Thus, a plane-based correspondence algorithm is executed using the algorithm proposed by (Pavan and dos Santos, 2017).After, the least squares method is used to find the sensor poses (R and ).The mentioned steps are repeated until no more pairs of TLS data exists.Finally, the global refinement is executed using the graph-pose approach developed by (Bormmann et al., 2008).In this paper, the uncertainty of the sensor poses is used to weight the graph-pose.The following sections present the adopted strategy for global refinement of TLS data registration in this paper.

Plane-based correspondences
In this paper, the statistical outlier removal technique proposed by (Rusu et al., 2008) is used.Given a reference point cloud , the mean distance between each and its neighbors is computed.Then, a filtered point cloud is obtained using the mean , the standard deviation and the density restrictiveness factor of the distribution over space for entire .Then, the resultant point cloud can be estimated as follows (Rusu et al., 2008): (1) Note that to access the set of neighbors of a radius search must be defined.
The RANSAC is a broadly used algorithm at the plane segmentation task.Since the model hypotheses to be found represent a plane, it uses random samples of three non-collinear unique points from point cloud to compute the model coefficients.After, it computes the distances from all points belong to current point cloud to the plane model and the model coefficients are scored by a user specified threshold.The algorithm tasks are repeated for several iterations.Then, the set with the largest number of points is selected to estimate the planar model coefficients in a least-squares sense.
Once planes have been segmented, our plane-based correspondence algorithm is inspired on Dold and Brenner (2006).In contrast, our algorithm sorts the normal vectors into two classes: wall and floor.Then we can determine the true classified segments into each class based on some other considerations.For example, we know that floor is only one plane, so in the class of floor the segments must be in similar perpendicular distance of the origin.Second, the angle between normal vectors must be analyzed.
Assuming a minimum of three planes, that intersect in a point, appears in S and Ref.To start the correspondence procedure R (3D rotation matrix) and t (3D translation parameter) should be computed.The solution is given to each four pairs of correspondences.Then, initial combinations of four pairs of planes are created using a small subset of the planes in S and Ref.The correct combination is obtained using the centroid of a plane with their normal vector.
The estimated sensor poses must be used to find more plane correspondences.Then, the correspondences are used in pairwise registration task.In this paper, we follow the concept of pairwise registration using plane features extracted from point clouds.

Pairwise registration based on plane correspondences
Since a minimum of three planes that intersect in a point have been automatically found, the effect of the transformation on point-to-plane correspondence is that it rotates and translates a point in S to lie on its corresponding plane in Ref.This can be expressed as: (2) where , , denote the unit normal vector, is the distance to origin for plane , , , represent the centroid coordinates for correspondent plane and and denotes respectively the estimated sensor poses from S to Ref.

Global refinement solution
The mathematical model presented in Equation ( 2) is non-linear with respect to the transformation parameters.Thus, linear forms are derived using a Taylor series, and an iterative approach is required to estimate these parameters.The solution for and can be obtained by means of least-squares criteria: , where , A represents the Jacobian of the condition equation with respect to the unknown parameters, B represents the Jacobian of the condition equation with respect to the observations, w is obtained by evaluating the condition equation with the observation and the approximate values of the unknowns.
Once the pairwise registration task introduces drift errors during long-distance mapping, the global optimization goal is minimize the influence of random noise on the sensor poses via distribution of the residual errors consistently across the trajectory of the sensor (Pavan and dos Santos, 2017).
To address the global refinement task we used a pose-graph approach proposed by (Bormmann et al., 2006).For each pairwise registration we find edges within a point cloud graph and leave it to the global refinement task, as shown in Figure 2. The variables are pairs of point clouds and the domain of variable is its set of sensor poses , for all with denoting the number of pairs of point clouds.Our proposed global solution uses the variance-covariance matrix, which describes the uncertainty of the sensor poses, to weight the graph-based optimization.Thus, we introduced the variance-covariance matrix of the sensor poses in the graph optimization process.The variance-covariance matrix is obtained jointly with the pairwise registration task, as follows: (3) Note that, we introduced computed with respect to the residual vector and the redundancy, as well as the Jacobian of the condition equation with respect to the observations.For each pair of point clouds a difference between two poses and is obtained, as follows: (4) where and represent the rotation matrix and translation vector from linked to and the right-hand side of Equation ( 4) is a linear function of the unknown translation parameters .
To refine all poses to build a consistent 3D map of the environment, a system of linear equations ( contains the residuals values) can be formed to estimate the unknown translation parameters within the posegraph, as presented in (Bormmann et al., 2006).In contrast, we weight the pose-graph with and the least-squares solution for can be obtained as: (5) where and are obtained by stacking and for all nodes.
In this work, we have fixed the rotation parameters assuming that it is well estimated into pairwise registration task.Then, we only refine the translation parameters.

EXPERIMENTS
To show the effect of weighted sensor pose on the global refinement of TLS data registration we scanned eight point clouds using the FARO LS 800 TLS with a field of view of 360º in the horizontal direction and 120 in the vertical direction.Figure 3 shows the pairs of point clouds before the pairwise and global registration.As can be seen in Figure 3 the pairs of point clouds before the registration found misalignments.For each point clouds can be observed different levels of noise, point densities and different overlapping areas.In light of the proposed method the described tasks were executed.First, the point clouds were segmented using the RANSAC algorithm.Its normal vectors and perpendicular distances from the origin were obtained using a least-squares sense.Figure 4 shows some segmented planes obtained with RANSAC algorithm.
The surface planes were matched using the plane-based correspondences method proposed by (Pavan and dos Santos, 2017).Table 1 summarizes the number of scans (nodes), number of matched planes and number of edges.

Number of scans Number of matched planes
Number of edges 8 105 10 Table 1.Description of the experiment.
Table 1 indicate that the plane-based algorithm found 105 correspondences.Figure 5 shows the pairs of point clouds used in the pairwise registration task.In the urban environment surface planes are easily hidden or occluded by trees, cars and walking humans.These clearly impact both the plane-based correspondences algorithm and the pairwise registration task.As can be seen in Figure 5, the number of edges is 10 instead 8 showing that loop closing were detected.Figure 6 shows the pairs of point clouds used in the weighted pose-graph solution.
Figure 6.Pairs of point clouds used in the pose-graph optimization.
To evaluate the global registration accuracies the obtained result were compared with reference markers measured using a Leica TC2002 total-station.We measured corresponding control points in the consistent TLS data.Thus, we transformed the consistent TLS data of the whole area to total-station coordinate system using the 3D body rigid model.Finally, we compute the Euclidean distance between the ground points surveyed with total-station and the scan positions.The residual distributions were centered approximately at zero, implying that the weighted pose-graph solution is appropriate and that the overall adjustment estimation is probably reliable.Note that the estimated position of the sensor is more than 2 meters off its actual position when it has completed a loop.
Figure 7 shows the globally consistent TLS data obtained using the proposed method.It can be seen that a 3D map of urban environment is clearly formed.

CONCLUSIONS
When TLS system is used to recover large or complex areas, it requires the registration of multiple pairs of point clouds into a global reference frame to ensure a consistent 3D model of the environment.The main contribution of this paper is to exploit the covariance matrices that describe the uncertainties of the sensor poses parameters to weight the graph-based optimization task.
We showed that assigning weights based on uncertainties of the sensor poses in the global refinement improves the accuracy of the TLS sensor along the trajectory.The A drawback of iterative pairwise registration method is dependency of initial parameter values and it generates too low accurate alignments.Consequently, not accurate enough covariance matrices can be obtained.This accentuates the need to use a coarse-to-fine registration approach.We also point out that in the global refinement should be investigate a way to include the 3D rotation matrix into simultaneous adjustment, instead to fix it.

Figure 2 .
Figure 2. Connected graph with edges and nodes.

Figure 3 .
Figure 3. Pairs of point clouds before pairwise and global refinement.

Figure 5 .
Figure 5. Pairs of point clouds used in the pairwise registration.
assignment of unequal uncertainties to sensor poses is essential for global refinement of TLS data registration, once pairwise registration introduces drift errors during long-distance mapping.According Theiler et al. (2015) pose-graph optimization can be controlled by redundancies.However, in the urban environment surface planes are easily hidden or occluded by trees, cars and walking humans.These clearly impacted the plane-based correspondences algorithm used in this work.It was not able to detect loop closures between non-adjacent point clouds.These additional constraints would help the graph optimization task to improve the sensor pose estimates.This emphasizes the importance of more robust plane-based correspondences algorithm.

Table 2 .
Table 2 shows the minimum and maximum values for the distance error achieved.Minimum and maximum values for the distance error.