A CURVATURE BASED ADAPTIVE NEIGHBORHOOD FOR INDIVIDUAL POINT CLOUD CLASSIFICATION

: As a key step in 3D scene analysis, point cloud classification has gained a great deal of concerns in the past few years. Due to the uneven density, noise and data missing in point cloud, how to automatically classify the point cloud with a high precision is a very challenging task. The point cloud classification process typically includes the extraction of neighborhood based statistical information and machine learning algorithms. However, the robustness of neighborhood is limited to the density and curvature of the point cloud which lead to a label noise behavior in classification results. In this paper, we proposed a curvature based adaptive neighborhood for individual point cloud classification. Our main improvement is the curvature based adaptive neighborhood method, which could derive ideal 3D point local neighborhood and enhance the separability of features. The experiment result on Oakland benchmark dataset shows that the proposed method can effectively improve the classification accuracy of point cloud.


Introduction
Due to the rapid development of LIDAR technology, it's more convenient to obtain point cloud via various approaches such as Aerial Laser Scanning (ALS), Terrestrial Laser Scanning (TLS) and Mobile Laser Scanning (MLS) (Rodrí guez-Cuenca et al., 2015;Weinmann et al., 2015).The MLS can productively obtain precise point cloud from complex urban scene.And this makes the MLS widely used in the urban 3D scene analysis applications like the extraction of building structures (Demantké et al., 2012), the extraction of vehicle and road facilities (Rodrí guez-Cuenca et al., 2015;Serna and Marcotegui, 2014), the traffic signs detection (Soilá n et al., 2016(Soilá n et al., , 2017) ) and so on.As a key step in 3D point cloud scene analysis, point cloud classification become one of the major research fields in photogrammetry and remote sensing (Weinmann et al., 2015).
However, the defects in point cloud like partial loss and uneven density have brought tremendous challenges to the point cloud classification task.The general idea is to extract respective features and do classification task with supervised learning algorithms.The individual classification method assumes that each 3D point is independent and assigning a unique semantic label with local features automatically, which can be divided into three steps (Weinmann et al., 2014): the reconstruction of local neighborhood for each point, the extraction of low-level feature, and the point cloud classification based on point's local features.This kind of methods typically do classification task with machine learning algorithm such as GMM (Lalonde et al., 2005;Lalonde et al., 2006), SVM (Zhang et al., 2013), AdaBoost (Lodha et al., 2007) or Random Forest (Jie and Zulong, 2014) and treat every point as an individual entity which is very efficient.Unlike the individual classification method, since the nearby 3D points in point cloud tend to share a same semantic label, the contextual classification method takes the relationship between points into account (Triebel et al., 2006).And respective methods have been proposed with graphical model like conditional random field (Kumar and Hebert, 2003;Niemeyer et al., 2014), Associative Markov Networks (Triebel et al., 2006) and non-Associative Markov Network (Shapovalov et al., 2010) to generate a smooth result.But whether for individual classification methods or contextual classification methods, the robust and separable features are essential.
The meaningful features can express the local structure for objects in nature scene, which mainly derived by calculating the point's space distribution of points in local neighborhood (Weinmann et al., 2014).The recovery of 3D point local neighborhood can be summarized into fixed-scale methods (Filin and Pfeifer, 2005;Lee and Schenk, 2002;Linsen and Prautzsch, 2001;Niemeyer et al., 2014) and adaptive scale method (Demantké et al., 2011;Lalonde et al., 2005;Weinmann et al., 2014).The fixed scale method is limited to the prior knowledge of the scene, point cloud density and curvature (Weinmann et al., 2015).And the uneven density and scan lines in point cloud bring great difficulties for the estimation of the local neighborhood system.The adaptive neighborhood can mitigate the influence of uneven density and linear behavior in point cloud and can derive 3D point local neighborhood from the functional relationship between the size of points' neighborhood and point cloud distribution (Lalonde et al., 2005).But it requires ground truth information for the estimation of parameters.And the minimal entropy based method recovering neighborhood by varying scale parameter k (Weinmann et al., 2014;Yang et al., 2013) or radius r (Demantké et al., 2011) within fixed intervals, however, the radius r based method is easily influenced by the strong density variations within point cloud (Hackel et al., 2016), and the parameter k based method is limited to the linear behavior in high density region.Generally, the distance between objects and laser scanning sensor have a great impact on density of point cloud which is extremely uneven in the overall range.However, since the uneven distribution in point cloud not be considered, the robustness of adaptive neighborhood of whole 3D point cloud scene has not yet been confirmed.
In this paper, we classify point with individual point classification framework based on state-of-the-art classifier.The main contribution of our work is a curvature based adaptive neighborhood extraction method, which enhanced the separability of features and improved the performance of classification results.The paper is organized as follow.The proposed classification method is describe in Section 2. The experiments are reported in Section 3. Conclusions are given in Section 4.

Methodology
For the classification of point cloud, we propose a methodology which involves curvature based adaptive neighborhood (Section 2.1), feature extraction (Section 2.2) and supervised classification (Section 2.3).The framework is provided in Figure 1.

Curvature based Adaptive Neighborhood
The curvature () of each 3D point is defined as the surface variation at point p within a certain radius of spherical neighborhood, and extracted by analyzing the 3D covariance matrix of 3D point and its neighborhood (Pauly et al., 2002).
Denoting the eigenvalues of 3D covariance matrix by  1 , 2 , 3 , where  1 ≥  2 ≥  3 ≥ 0, the curvature is given by The curvature with a value field from 0 range to 1/3 where 0 indicate all points lie in plane and 1/3 means points represent a completely isotropically behavior.
Generally, the curvature of vegetation or object boundary provide a larger behavior, as can be seen from Figure 2c, which means there needs a small neighborhood to express the rich detail information.By contrast, the smaller curvature corresponds to larger neighborhood.Therefore, the adaptive scale method is essential for the distinctiveness of extracted feature.Figure 2d shows the k-minimal entropy based adaptive neighborhood (Weinmann et al., 2014) (  ) over the interval between   = 10 and   = 100 with ∆ = 1 (Weinmann et al., 2015), the neighborhood present a small and linearly behavior in region Ⅰ within high density regular range, which is not enough to extract robust feature.Figure 2e shows the result of r-minimal entropy based adaptive neighborhood (Demantké et al., 2011) (  ) over the interval between   = 0.25 and   = 2.0 with ∆ = 0.05, which remains the same problem with region Ⅰ in region Ⅱ.And as can be seen from region Ⅲ in Figure 2e, the size of points' neighborhood is quite larger for detail information in vegetation region.Since the uneven distribution of point cloud in complex scene not be considered, the current adaptive neighborhood recovering methods remain limited to recovering linearly behavior neighborhood within high density regular range.
On the basis of two types of minimal entropy based methods, a curvature based adaptive neighborhood extraction method (  ) is designed.The main idea of this method is to divide the point cloud into scatter and regular region by curvature threshold (  ) and handles separately, as shown in Figure 3.
For the outliers that with inadequate neighborhood point for curvature estimation were set to 1/3.The points are divided into scatter region when the curvature is greater than   or into regular region when the curvature less than cur.Then, the adaptive neighborhood in scatter region is recovered by   and in regular region with   .Since the prior knowledge of the object size in nature scene is unknown, the boundary of the adaptive neighborhood should be able to cover the most of object in scene while decrease the cost of computing.The adaptive neighborhood   needs a lower boundary of   = 10 to estimated 3D point local geometric features by PCA while the upper boundary is set with an ideal value to decrease the cost of computing.And the adaptive neighborhood   needs a lower boundary   = 0.25 to avoid the points within   centralized in one scan line within high density regions, and enclose too few neighbors in low density regions (Hackel et al., 2016).Since the adaptive neighborhood   has taken the prior knowledge of scatter and regular region into account.The upper boundary of parameter  is set to be 50 to decrease the cost of computing and  is varying between   = 10 and   = 50 with ∆ = 1.However, there are some points within scatter region that the density is a little bit small, which means the size of   tend to be very large to reach a stabilized state.Therefore, in our approach, the maximum size of adaptive neighborhood within scatter region is set to 3m which is a quite reasonable value for most of the objects in natural scene.The lower boundary of parameter  is set to be 0.5m for the objects within regular region which generally have a larger size, which aims to decrease the cost of computing.And the interval  ∈ [0.5, 2.0] has been sampled in 30 scales with ∆ = 0.05m Finally the point cloud's adaptive neighborhood   is recovered by combining these two types of adaptive neighborhood.
The   is extracted by k-means algorithm, that we split the points into two clusters, the initial cluster centers are the minimum and maximum curvature in point cloud.Assume that the center of the two clusters are cur1 and cur2 (cur2>cur1), and we get the   as follow: As can be seen from Figure 2e, the points' neighborhood on faç ade or ground typically have a large size for its relatively regular distribution in point cloud.And the points' neighborhood on natural vegetation has a small size for the relatively scattered distribution.

Feature Extraction
For the feature extraction, we only take the geometric features into consideration since most of the publicly benchmark point cloud datasets only contain space position information of 3D points.The meaningful feature can express the structure of local surface around 3D point, which mainly derived by calculating the point's space distribution within the local neighborhood.And we increase the distinctiveness of features with the curvature based adaptive neighborhood.In our approach, we mainly use the 3D  (Breiman, 2001) as representative individual classification algorithms.We use a RF classifier with 200 trees and a tree depth of √, where  is the dimension of the feature space.
Since most of the point cloud datasets for natural scene appear an unbalance distribution behavior of 3D point per class in datasets, which may cause trained classifiers tend to classify all point into majority class and may have a detrimental effect on the classification process.We apply a data level method for the unbalance issue in point cloud (Kotsiantis et al., 2006).We do random resampling of training data to obtain a uniform size of training examples pre class try to balance the class distribution.

Datasets
To evaluate the performance of the proposed method, we use the Oakland 3D Point Cloud Dataset (Munoz et al., 2009).The dataset was collected around CMU campus in Oakland by Naclab11 equipped with side looking SICK LMS laser scanners.It contains about 1.3 million manually labeled points and was divided into training set (1 file) and test set (15 files).And the classification task is to assign each 3D point one of the five semantic labels: vegetation, wire, pole/trunk, ground and facade.However, the dataset we use is imbalance data that almost 90% points are labeled as vegetation and ground, while less than 1% points are labeled as wire and pole/trunk.A number of solutions to solve the class-imbalance problem in data level, we randomly select 1,000 training examples per class from training set as the rebalanced training set for the training of model.

Experiments
In the experiments, first consider the impact of curvature threshold on classification results and verify the estimation of curvature threshold estimated by k-means algorithm.Subsequently, we focus on the impact of 3 different adaptive neighborhood on classification results: the k-minimal entropy based adaptive neighborhood   over the interval between   = 10 and   = 100 with ∆ = 1 , the r-minimal entropy based adaptive neighborhood   over the interval between   = 0.25 and   = 2.0 with ∆ = 0.05 , the proposed adaptive neighborhood   defined in formula (2).

Results and Discussion
For the evaluation, we use four commonly used measures: recall which represents the completeness of the classification result of classes, precision which represents the accuracy of the classification result of classes, F1-score which is harmonic mean of recall and precision for classes and overall accuracy which represents the overall performance of the proposed method on test set.Since the test set is imbalance data, the overall accuracy mostly dependent on the precision of ground and vegetation, we take the average of recall, precision and F1-score to evaluate the classification result of different categories.
Determining the curvature threshold is the first step in the proposed method.The curvature of the Oakland point cloud is estimated with radius  = 1m.We split the Oakland benchmark dataset into scatter and regular region with curvature threshold   = 0.03 which is achieve by equation 3 and the k-means algorithm with initial cluster centers 0 and 1/3 after 17 iterations.And we consider the distribution of each class over scatter and regular region.The improvement of faç ade with respect to recall values which means that more faç ade information is recognized (Table 2).Since the faç ade in 3D point cloud scene is quite complicated for its compositions of windows and walls, the robustness of features extracted from faç ade is limited, which makes the points classified with wrong semantic label.The windows with a relatively large curvature recovers adaptive neighborhood with   while fairly planar walls with   , which improved the robustness of features and further improved the recall value.The precision value of class wire and pole/tree-trunk is quite low which manly caused by their linear behavior.Since the point cloud is obtained by laser scanning sensors, which is formatted with scan lines, and this makes the points in facades or vegetation with low density is easily misclassified into class wire or pole/tree-trunk.As can be seen from Figure 5, the points in the middle of the window is misclassified into class pole/tree-trunk, since its provide a linear behavior.This may be solved by contextual classification method.Currently, the improvement of the proposed method is verified by benchmark datasets with only five classes which is quite simple, thus the classification results might vary for other increasingly complex scenarios.However, in-depth experiments and analyses for natural urban scene have not been done in our work, and it is still a challenging task for ensuring classification performance when the proposed method handles point cloud with pedestrians and vehicles.Considering that compared with Oakland dataset, the natural urban scene data may have more classes with a more complex spatial distribution, it should be noted that the applicability of the proposed method for natural urban scene with pedestrians, vehicles and others remains uncertain.
4 Conclusions High accuracy classification methods for complex unban scene point cloud data are critical for the complex spatial distribution.In this paper, we proposed a curvature based adaptive neighborhood for individual point cloud classification.The main improvement is a curvature based method for recovering adaptive neighborhood to enhance robustness of local 3D geometrical features.
The experiments on Oakland and Paris benchmark datasets indicate that the proposed method significantly improved the performance of classification result.However, the proposed method still have some problems in class wires and pole/treetrunk, which appear a fine behavior.Feature work should focus

Figure 1 .
Figure 1.Flow chart of the proposed point cloud classification method.

Figure 2 .
Figure 2. The distribution of the point cloud and the radius distribution for adaptive neighborhood.(a) Labeled point cloud (v: vegetation; w: wire; p/t: pole/trunk; f: facade; g: ground); (b) Density distribution for point; (c) Curvature distribution for point; (d) Result for k-minimal entropy based adaptive neighborhood; (e) Result for r-minimal entropy based adaptive neighborhood; (f) Result for curvature based adaptive neighborhood.

Figure 3 .
Figure 3. Sketch map of point cloud curvature based adaptive neighborhood.
classification results of class wires and pole/tree-trunk.Moreover, we intend to do contextual classification in natural scene analysis tasks, this cloud be based on probabilistic graphical model or deep learning.
, which represents the angle between normal and ground level,   is the third components of 3D point normal vector.The normalized height difference   according to The value   =   −   represents the height difference within neighborhood where   and   respectively represent the maximum and minimum z coordinate within neighborhood.And max and min respectively represent the maximum and minimum   in point cloud.The eigenvalue based features are extracted with features including verticality , maximum height difference   and eigenvalue based features   ,   ,   and   .Based on normal vector for each 3D point, the verticality  with  = 1 −   (4) can be estimated2.3Classification Given a labeled 3D point cloud data X= {(  ,   )} as training dataset, where   represent the feature vector of point   , and   ∈ { 1 ,  2 , ⋯ ,   } is the semantic label of point   , and m is class number.Our goal is to obtain classifier based on training data X and classify test point cloud  = { 1 ,  2 , ⋯ ,   } by assigning an object class label   to each point.The point in test dataset may have handle assigned semantic label, which can be used to evaluate the performance of classifier.For multi classification, we apply Random Forest (RF)

Table 1 .
Distribution of the classes (v.: vegetation, w.: wire, p./t.: pole/tree-trunk, g.: ground, f.: facade) for scatter and regular region with curvature threshold   = 0.03.In order to evaluate the impact of curvature threshold, the interval   ∈ [0.01,0.05]hasbeensampled in 9 scales with ∆ = 0.005.Figure4shows the obtained overall accuracy.Note that when curvature threshold   = 0.03, the proposed method can get the best performance classification result with overall accuracy of 96.1%, which confirmed the intuition that the estimation of   by k-means algorithm is quite appropriate.Figure 4. Overall accuracy (%) obtained by proposed method using different C value.Since the adaptive neighborhood   is the combination of   and   , we compared the classification results of different classes on three adaptive neighborhoods.The comparison of evaluation values are reported in table 2. The curvature based adaptive neighborhood is necessary when recovering neighborhoods in complex urban scene, since its do not use the empiric or heuristic a priori knowledge on scene and also avoids the influence of uneven density and curvature.And as can be seen from Table2, the curvature based adaptive neighborhood significantly improves the performance of each class with respect to recall, F1score and mean class evaluation value.Considering the mean class evaluation value (Table2), it becomes visible that the curvature threshold split the point cloud into scatter and regular region, which could exploit the full potential both   and   , and the classification result of   is significantly improved compared to   and   .