POLE-LIKE OBJECTS RECOGNITION FROM MOBILE LASER SCANNING DATA USING SMOOTHING AND PRINCIPAL COMPONENT ANALYSIS

With the spread of the Mobile Laser Scanning (MLS) system, the demands for the management of road and facilities using MLS point clouds have increased. Especially, pole-like objects such as streetlights, utility poles, street signs and etc. are in high demand as facilities to be managed. We propose a method for recognizing pole-like objects from MLS point clouds. Our method is based on Laplacian smoothing using the k-nearest neighbors graph, Principal Component Analysis for recognizing points on pole-like objects, and thresholding for the degree of pole-like objects. Our method can robustly recognize pole-like objects with various radii and tilt angles from MLS point clouds. For correctly segmented objects, accuracy of pole-like object recognition is on average 97.4%.


INTRODUCTION
With the development of inexpensive and high accuracy laser scanner devices, the Mobile Laser Scanning (MLS) system which installed these devices on a car has been widely used.
The MLS point clouds can be useful not only for building and city modeling, but also for managing various facilities in urban environments.In particular, pole-like objects such as streetlights, utility poles, street signs, and etc. are in high demand as facilities to be managed, and it is required to recognize them from urban MLS point clouds.However, manually recognizing these pole-like objects from large point clouds data requires a great deal of time and cost.Therefore, for efficient management of facilities, it is necessary to automatically recognize pole-like objects from MLS point clouds.
Much research on pole-like objects recognition from MLS point clouds has been conducted.Existing methods are based on machine learning (Golovinskiy 2009b, Lai 2009), or the arrangement and position of measurement points (Manandhar 2001, Lehtomaki 2010).These methods have some problems, such as they require a lot of training data, or that they cannot recognize pole-like objects with different radii, and tilt angles.
In this paper, we propose an algorithm to automatically recognize pole-like objects from MLS point clouds.Our method is based on Laplacian smoothing using the k-nearest neighbors graph, Principal Component Analysis (PCA) for recognizing points on the pole-like objects, and thresholding for the degree of pole-like objects.By using smoothing and PCA, robust recognition of the pole-like objects with various radii and tilt angles is realized.Details of the proposed method are described in chapter 3.

RELATED WORKS
Existing research on automatically recognizing or classifying objects, including pole-like objects, are introduced in this chapter.
Based on machine learning, Golovinskiy et al. (Golovinskiy 2009b) propose a method to classify various objects such as cars, streetlights, trees, fire hydrant, and etc. from the combination of MLS and ALS point clouds.A major feature of their method is high precision segmentation by the graph cut algorithm.To classify objects, they input feature quantities of objects into the Support Vector Machine (SVM).As learning data, they used a part of the input data which are classified manually.In order to increase the recognition rate, a sufficient amount of learning data must be required.They also indicated that the better shape descriptors and classifiers are required for better results.As a similar method using machine learning, Lai et al. (Lai 2009) attempted to classify objects in MLS point clouds by using a lot of 3D data that exists on World Wide Web as learning data.In the learning method, sufficient training data based on input point clouds is necessary for suitable learning.
Based on the arrangement and position of measurement points, Manandhar et al. (Manandhar 2001) detected vertical poles from MLS point clouds.In their research, the MLS point clouds consisted of vertical scan lines and vertical poles by extracting vertical line segments from individual scan lines.Their detection method was limited to the extraction of the vertical poles.Due to this limit, the method cannot detect tilted poles and is not applicable to arbitrary point clouds.Lehtomaki et al. (Lehtomaki 2010) extracted sweeps that were expected to be measured pole-like objects from MLS point clouds.Then they found another sweep either below the current sweep or above, and made them a cluster.However the method cannot recognize pole-like objects with a specific radii and limited point densities.
In this paper, we propose a robust recognition method of the pole-like objects with arbitrary tilt angles and radii from MLS point clouds without learning data, by using smoothing and PCA.

Overview of the Method
The proposed algorithm is shown in Fig. 1.In the method, we assume that ground points are already removed from given point clouds.The algorithm of our method consists of four phases.First, the input point clouds are segmented, as the result the points estimated on each object are grouped.Second, smoothing is applied to each segment.Third, each point is classified into the points on the pole-like objects, on the planar objects, and on other objects.Finally, the degree of the pole-like objects of each segment is evaluated, and the segments of the pole-like object are extracted by thresholding.

Segmentation
Many segmentation techniques of point clouds have already existed (Golovinskiy 2009a, Moosmann 2009, Wang 2008).In our implementation, we adopt the simple segmentation method by connecting the nearest neighbor points.As a result, the segment is composed of the point set of the connected k-nearest neighbors graph.The graph is generated to sequentially generate an edge among the k-nearest neighbors between the point i and points included in a sphere with a radius r centered at point i.Even if the neighbors included in the sphere are less than k, we do not expand the radius r.

Endpoint Preserving Laplacian Smoothing
The robust extraction of points on pole-like objects by PCA described in next section is difficult because of the measurement noises, the bias of the point distributions, and the differences of the pole-like object radius.Therefore, to improve the recognition rate of the points on the pole-like objects by PCA, endpoint preserving Laplacian smoothing is applied to the k-nearest neighbors graph in our method.
Generally, smoothing is applied for the purpose of removing the noise of the measurement data.On the other hand, recently, smoothing is used for other purposes.For example, some methods of skeleton extraction from point clouds have been proposed (Au 2008).The purpose of their research is to extract the skeletons of wire-objects, and the recognition of the objects is not focused on.
In our method, smoothing is applied to the MLS point clouds in order to improve the recognition rate of the points on the pole-like objects by the PCA and to distinguish points on the pole-like objects from the ones on the planar objects, and the others.We focus on the exaggeration and the degeneration of the object shape features based on Laplacian smoothing which is an operation that moves each point to the centres of the neighbours.Laplacian smoothing makes the pole-like objects into a thin pole shapes through shape degeneration.As the result, the point distributions of the pole-like objects come to be degenerated into a one dimensional distribution (Fig. 2).In addition, measurement noises are removed.Therefore, applying the Laplacian smoothing to the scan data raises the recognition rate of the pole-like object points by the PCA.Laplacian smoothing is done by applying equation ( 1).
(1) where = position of point i after smoothing = position of point i λ = smoothing strength (0 λ 1) = Laplacian, and it is given by the equation ( 2)


(2) where = positive weight ( ) = a set of the neighbors of point i The point clouds of pole-like objects with various radii can be degenerated into a one dimensional distribution by iteratively applying equation (1), and then pole-like objects with various radii can be recognized.
However, Laplacian smoothing has the problem that the branching structures of the pole-like objects are lost.This causes the decrease of the recognition rate in the following object recognition.To solve this problem, we propose the endpoints preserving Laplacian smoothing, which controls displacements during smoothing according to the distribution of the neighbors of a point.At the endpoint, the neighbors are distributed in one direction.On the other hand, the neighbors are distributed in all directions at the inner point.From this observation, the displacements during smoothing are controlled so as to preserve the endpoints according to the distribution of the neighbors.We evaluate whether point i is the endpoint or not using equation (3).
If the point i is the endpoint, angle(j,i,k) is small (Fig. 3 left)， then the e(i) becomes small.On the other hand, if the point i is far from the endpoint, some angle(j,i,k) are large (Fig. 3 right)， then e(i) becomes large.
Displacements in smoothing can be controlled by using the smoothing strength λ.Additionally in order to reduce influences of the distant points during smoothing, we use weight ij which is in inverse proportion to the distance between points.Endpoints preserving Laplacian smoothing is done by the equations (4), ( 5), and (6).

Point Classification
Each point is classified into three types which are the points on the pole-like object, the points on the planar object, and the others.The local point distributions are evaluated by calculating eigenvalues and eigenvectors of the variance-covariance matrix related to the point i and its neighbors.The variance-covariance matrix M i of the point i is shown in equation ( 7).When point i is on the pole-like object, the maximum eigenvalue is very large compared with other eigenvalues , , and the eigenvector represents the axial direction of the pole-like objects.On the other hand, when point i is on the planar object, and become relatively large compared with .When point i is on the other object, there are not so many differences between the three eigenvalues.

  
In order to investigate the magnitude relation of eigenvalues, we compute the three distribution features of each point i using equation (8).
where α, β = adjustment coefficient When point i is on the pole-like object or planar object or other object, the distribution futures , and becomes the maximum respectively.Fig. 5 shows the result of the point classification.Fig. 5(a)-(c) shows the utility pole, street sign, and tree respectively.The top row represents the input point clouds.The bottom row represents the point classification result after smoothing.
Vandapel, et al. (Vandapel 2004) classify points by using PCA similarly.They mentioned that it is difficult to hand-tune thresholding classification because those eigenvalues may vary considerably depending on the type of terrain, the type of sensor, and the configuration of the sensor and the vehicle.For this reason, they use machine learning to classify points.However, through experiments using some MMS data, we observed that the points are well classified after smoothing by comparing , and in equation ( 8), which adopts weights a α and β in the original definition of the saliencies (Vandapel 2004).Therefore, we use the direct comparisons of the , and in equation ( 8) to classify the points (α and β are set to 10, 100 respectively according our experiments).

Pole-like Object Recognition
Finally by evaluating the geometric properties and classified points of the segments, each segment is classified into pole-like object and others.As the minimum for the pole-like object, we assume that the height of the segment is more than 2m which is higher than average human height, and that the number of points of the segment is over 50.Hence, the segments which do not satisfy these conditions are recognized as other objects.In addition, the segment in which over 70% of points are the ones on the other objects is recognized as other objects, because the pole-like objects we intended consist of poles and plane surfaces.For each remaining segment, the degree of the pole-like objects is evaluated by equation ( 9).
where , = weights = a set of points of segment n = a set of points on the pole-like objects, and included in = a set of points that have almost vertical , and included in An example of the sets of points S n , C n and D n for a street sign is shown in Fig. 6.The first term of equation ( 9) represents the ratio of the points on the pole-like objects in the segment.The second term of equation ( 9) represents the ratio of the points whose neighbors are distributed vertically in the points on the pole-like objects.The degree of the pole-like object for each segment is evaluated by the weighted sum of the two terms.Fig. 7 shows the histogram of the value f n about various objects in MLS point clouds.The value f n becomes lager for the almost segments of pole-like objects (In the experiment, it was observed that the segments of the pole-like object had f n over about 45).Finally, the segments which have f n larger than the threshold are recognized as pole-like objects.Three kinds of urban point clouds acquired by MLS system shown in Table 1 were used in our experiments (Ishikawa 2009).As described in section3.1,the ground points were removed manually from the given data.
In our methods, each parameter is set as follows.

Pole-like Object Recognition Result
Fig. 9 shows the pole-like objects recognized by our method.In Fig. 9(a), there are a lot of various objects such as trees, buildings, cars, and pole-like objects.In Fig. 9(b), the pole-like objects such as streetlights and utility poles are recognized.However, some other objects such as trees are included.The recognition accuracy of the pole-like objects of the three data is shown in Table 2.The number of correct pole-like objects [B] was manually verified by using the photograph.[H] is the number of objects which were recognized as pole-like objects though they are not the pole-like objects.
The average accuracy of the pole-like object recognition is 63.9% for all segments.On the other hand, the average accuracy of pole-like object recognition is 97.4% for correctly created segments.The accuracy varies depending on the complexity of measured area.The Data I is the point clouds of simple area, but the Data III is the point clouds of complex area which includes overcrowded various objects.Some examples of the objects which were not correctly recognized in the Data III are shown in Fig. 10.In both cases, the segment includes pole-like objects and their neighboring objects, such as a hedge and tree.Our recognition method is designed for correctly segmented point clouds, therefore recognition failed for the incorrect segments.The [G] in Table .2shows our method works well for the correctly created segments.In the future, we will apply an appropriate segmentation method such as that proposed by Golovinskiy et al. (Golovinskiy 2009a) to the inputs and evaluate the recognition rate of the object.
Our method is implemented using standard PC (OS: Windows7 64bit, CPU: Intel Core i7 3.0 GHz, RAM: 6GB).Processing times of the object recognition are shown in Table 3.

CONCLUSION
In this article, we developed an algorithm that automatically recognizes pole-like objects with tilt angles and various radii from MLS point clouds in urban environments.Our method is based on the smoothing and principal component analysis for point clouds, and the evaluation of the degree of pole-like objects for the segments.The accuracy of pole-like objects was on average 97.4% for correctly segmented objects.
Future works are to improve the recognition rate by adopting or developing an appropriate segmentation method, and to classify pole-like objects into more detailed object classes, such as utility poles, streetlights, street signs and so on for supporting facility management from MLS point clouds.3. Running time of our method (sec)

Figure 1 .
Figure 1.Proposed algorithm of pole-like objects recognition

Fig. 4
Fig.4 shows the comparative result of the Laplacian smoothing and endpoint preserving Laplacian smoothing for the same utility pole.The number of iterations of the smoothing is 40.Fig.4(a) shows the input point clouds of the utility pole.Fig.4(b) and Fig.4(c) show the results of the Laplacian smoothing and endpoint preserving Laplacian smoothing respectively.After the endpoint preserving Laplacian smoothing, the branching structures still remain.During iterating smoothing, the distribution of the points on the pole-like objects with various radii becomes one dimension.
where = position of point i = a set of the neighbor points of point i = barycenter of i * We denote eigenvalues of the M i by , and ( ) and the corresponding unit eigenvectors by , and respectively.The local distribution of neighbors of point i is figured out by the magnitude relation of the eigenvalues.

Figure 6 .Figure 7 .Figure 5 .
Figure 6.Example of the point clouds of a street sign (orange: points on the planar object, green: points on the pole-like object, blue: points whose neighbors are distributed vertically) Number of neighbors: k = 15 Search radius: r = 0.5m Number of iterations in smoothing: n = 40 Threshold of degree of pole-like object: τ = 45 Weights in equation (9): =1.0, =2.0These parameters were determined based on the experiments for some data set.

Fig. 8
Fig.8 shows the results of point classification.Fig.8(a)-(c) represents a streetlight, street sign, and utility pole respectively.The top row represents the classification result without smoothing.The bottom row represents the classification result with smoothing.The results show that smoothing of the k-nearest neighbors graph improves the classification accuracy of the points by PCA and robustness for the differences of the radius of the object.
Figure 9. Result of the pole-like object recognition

Figure 8 .
Figure 8. Effectiveness of the smoothing (green: points on the pole-like objects, black: points on the planar objects and others) (c) utility pole (b) street sign (a) street light

Figure
Figure 10.Undetectable pole-like objects by our method

Table 2 .
Accuracy of the proposed method

Table 1 .
Data used in the article (without the ground points)