MULTIPLE-PRIMITIVES-BASED HIERARCHICAL CLASSIFICATION OF AIRBORNE LASER SCANNING DATA IN URBAN AREAS

: A hierarchical classification method for Airborne Laser Scanning (ALS) data of urban areas is proposed in this paper. This method is composed of three stages among which three types of primitives are utilized, i.e., smooth surface, rough surface, and individual point. In the first stage, the input ALS data is divided into smooth surfaces and rough surfaces by employing a step-wise point cloud segmentation method. In the second stage, classification based on smooth surfaces and rough surfaces is performed. Points in the smooth surfaces are first classified into ground and buildings based on semantic rules. Next, features of rough surfaces are extracted. Then, points in rough surfaces are classified into vegetation and vehicles based on the derived features and Random Forests (RF). In the third stage, point-based features are extracted for the ground points, and then, an individual point classification procedure is performed to classify the ground points into bare land, artificial ground and greenbelt. Moreover, the shortages of the existing studies are analyzed, and experiments show that the proposed method overcomes these shortages and handles more types of objects.


INTRODUCTION
Airborne Laser Scanning (ALS) point clouds have many benefits in contrast with the commonly used 2D remote sensing images for a variety of applications, such as ground point extraction (Sithole and Vosselman, 2004;Meng et al., 2010;Chen et al., 2016;Zhang and Lin, 2013;Yang et al., 2016), 3D city modelling (Sampath and Shan, 2007;Chen et al., 2014;Jarzgbek-Rychard and Borkowski, 2016;Sampath and Shan, 2010;Yu et al., 2016), etc.Over the past two decades, significant contributions to the consolidation and extension of ALS data processing methods have been witnessed (Yan et al., 2015).Among these processing methods, classifying the ALS point cloud data into categorical object instances is the first and most critical step for further data processing and model reconstruction (Guo et al., 2015).The existing studies of ALS point cloud classification could be divided into three strategies, i.e., individual point classification, segment-based classification, and multiple entity classification.
Recently, most majority of research works focus on the strategy of individual point classification.Individual point classification methods compute features for each point and classify point clouds considering individual point as the computational unit.Generally, features of each point are first computed, then, a classifier such as Random Forests (Breiman, 2000) is trained using a number of selected training samples which are labeled.At last, all the points in the input ALS point cloud are classified by the trained classifier.In this procedure, a neighborhood of each point is required to be determined, when it computes features for each point.There are three types of neighborhoods, i.e., spherical neighborhood (Lee and Schenk, 2002), cylindrical neighborhood (Filin and Pfeifer, 2005), k-closest neighborhood (Linsen and Prautzsch, 2001).To determine these neighborhoods, a scale parameter, either a fixed radius or a constant value is required.However, the 3D local structures and point densities are variable, a constant scale parameter cannot describe the local structural configurations.Therefore, a neighborhood optimizing step is performed to determine the scale parameter before feature computation.A number of neighborhood optimizing methods (Guo et al., 2015;Mitra and Nguyen, 2003;Lalonde et al., 2005;Pauly et al., 2003;Belton and Lichti, 2006;Demantke et al., 2011;Weinmann et al., 2014) have been proposed.Unfortunately, these neighborhood optimization methods are rather time-consuming (Wang et al., 2016), which is the main disadvantage of this kind of classification strategy.
For the strategy of segment-based classification, point cloud segmentation is employed.Local points with the same geometric structure (such as planar structure) are clustered into the same segment.After an input ALS point cloud data is segmented, features for each segment are extracted.To classify these segments, either several classification rules (Yang et al., 2015) or a fuzzy model classifier (Lee and Schenk, 2002) are utilized.The performance of these existing methods relies heavily on the result of the employed point cloud segmentation method.The existing point cloud segmentation methods focus mainly on planes or smooth surfaces extraction, and only one kind of segments is extracted.Unfortunately, 3D scenes are complex, and only one kind of segments can hardly depict them appropriately.Therefore, a point cloud segmentation method which extracts more kinds of segments is necessary.Although several shortages exist in the existing segment-based classification methods, they still has some benefits in contrast Commission III, WG III/6 with individual point classification ones.The benefits are shown as follows: (i) Segments are helpful to compute geometric features, and thus relieve the dependence on the neighborhood optimization.(ii) Segments give a new attribute of each point, and semantic rules are easier to be employed.(iii) Segments are considered as the computation unit, and reduce the computational burden when we extract features and make classification.
To solve the problem that only one kind of segments cannot depict the geometric properties of a 3D local structure, multiple entity classification (Xu et al., 2014)  To address these problems, a point cloud segmentation method which handles three kinds of segments is proposed in Ni et al. (2017), and a straightforward classification framework is presented.However, the method has the following two shortages:

METHODOLOGY
The proposed classification framework is a hierarchical procedure.First, the step-wise point cloud segmentation is performed, and the input ALS point cloud is divided into two subsets, i.e., planar and smooth surfaces, and rough surfaces.To simplify the expression, the planar and smooth surfaces are named regular surfaces.Next, points in regular surfaces are divided into ground points and building points by semantic rules.Then, several features of rough surfaces are extracted, and vegetation and vehicles are classified by a supervised classification procedure using Random Forests (RF).At last, individual point features of ground points are extracted, and the ground points are classified into bare land, artificial ground and greenbelt.The workflow of this procedure is shown in Figure 1.
In the following of the section, we will present the key steps of this procedure.
Figure 1.The workflow of the proposed multiple-primitivesbased hierarchical classification method.

Three Types of Primitives
Three types of primitives are extracted and utilized in the classification procedure.To extract these types of primitives, step-wise point cloud segmentation method (Ni et al., 2017) is employed.The segmentation method is able to handle three kinds of segments, i.e., planar, smooth, and rough surfaces.The performance of the segmentation method is shown in Figure 2.However, we consider the planar and smooth surfaces as one type of primitives, and name them regular surfaces for simplifying expression.Therefore, two types of primitives are extracted by the employed point cloud segmentation method.
Besides, individual points are considered as the third type of primitives.In other words, the three types of primitives that are utilized in this paper are regular surfaces, rough surfaces, and individual points.The three types of primitives are shown in Figure 3.

Classification of Regular Surfaces
The regular surfaces extracted by the step-wise point cloud segmentation method are classified into ground and building.Two semantic rules are utilized to make the classification, which are shown as follows: (i) The relative elevation between a query segment and its adjacent segments should be smaller than a given threshold.
To compute the relative elevation, we first determine the adjacent segments for of a query segment in XY-plane.Then, the nearest point pair in and is determined.At last, the relative elevation is defined as: for , where is the z-coordinate of , and is the zcoordinate of .(ii) The areas of ground segments are much larger than those of building segments.The rule is correct due to that the stepwise point cloud segmentation clusters the points in ground surfaces into a small number of segments with large areas in a 3D urban scene.In this paper, the number of points is defined as the area of a segment, because the point densities of building and ground are identical.
The classification result of regular surfaces in a small area is shown in Figure 4. Ground points and building points are correctly divided.

Classification of Rough Surfaces
To classify the rough surfaces into vegetation and vehicle, semantic rules lose the effectiveness.Therefore, we utilize a supervised classification procedure to reach the requirement.First of all, 15 segment-based features are extracted by considering rough surfaces as the computational units.The features are list as follows.(1) Average intensity The average intensity of a rough surface is the mean value of all the points' intensities.
(2) Area The point number of a rough surface. (

Point-wise Classification for ground points
To classify ground points into bare land, artificial ground and greenbelt, the RGB information and point intensities of the input ALS point cloud are utilized.A man may argue why the RGB information is not utilized in the previous steps.The main reason is that the RGB information has low geometric accuracy, especially for the small objects such as vehicles.If we utilize the RGB information to classify vegetation and vehicle, more misjudgements will be arose.In this procedure, five features are extracted for each individual point, i.e., R, G, B values, grey value , and point intensity.The grey value is derived from the RGB information as following: (9) where , , and are the R, G, B values, respectively.
After the point features are extracted, a supervised classification procedure is performed.In this procedure, RF classifier is utilized to make classification.The classification result of the ground points is shown in Figure 6.

The Testing Data
To validate the proposed multiple-primitives-based hierarchical classification method, we utilize the State of Utah Acquired Lidar Data.The dataset covers over 8380 with highresolution (0.5-1.0 meter).We select an urban area to test our proposed method.The urban area is shown in Figure 7 which is coloured by RGB information.The ground surface in the area is flat and smooth, and are composed of bare land, artificial ground and greenbelts.Buildings are composed of complex roof structures, and overlapped by trees.There is a small number of vehicles parking on the roads, or near buildings or trees.
Figure 7.The testing data which is coloured by RGB information.

Evaluation Metrics
For evaluation, we employ the confusion matrix and consider five commonly used measures: overall accuracy , Kappa coefficient , precision , recall , and -score.They are computed according to the confusion matrix as follows: (10) where is the main diagonal element in -th row, is computed from the sum of -th column, excluding the main diagonal element, is the sum along -th row, excluding the main diagonal element, is the number of classes, and is the number of all the points in an input point cloud.

Results and Discussions
The classification result and the interior results of the testing data are shown in Figures 8-11.The confusion matrix and the accuracies of the classification result are shown in Table 1.
From the aspect of different primitives, ground points and building points are well divided in the regular surfaces due to the ground is flat and smooth.There are some misjudgements in the classification result of rough surfaces.A close-up visual inspection shows that vehicles often parked near vegetation.Furthermore, some low vegetation has the same geometric structure with vehicles.When deviation exists in the objects' spectral information, misjudgements between vehicles and vegetation are inevitable.For the point-wise classification, there are noises in the classification result of the ground points, which is a common problem individual point classification methods encountered.From the aspect of different classes, the -score of class greenbelt (0.959) is higher than the other classes.The reason is that the RGB information of greenbelt is different with those of bare land and artificial ground which contributes to the accuracy value most.In contrast, the -scores of bare land and artificial ground are relatively low due the fact that the RGB information of most majority of bare lands is similar to artificial ground.For the class building, the most serious problem is that the building elements such as chimneys are often mislabeled as vegetation.However, the problem does not affect the accuracy of the class building too much.Vegetation makes the urban area much complex, it covers buildings and vehicles, and has various geometric structures.However, the base number of vegetation is large, the misjudgements between other classes affect the accuracy of it a little, and it achieves a high -score (0.959).In contrast, the base number of vehicle is small, and the misjudgements between vegetation affect the accuracy of it much.Besides, the RGB information of vehicles is noisy, and has no contribution to the classification of the class vehicle.Therefore, the class vehicle achieves the lowest -score (0.372).Experiments show that most of the classes achieve high classification accuracies.Besides, there is a limitation of this classification strategy.The problem how to improve the classification accuracy of small objects when the base number of them is rather small, is still remaining.
(i) It only uses geometric features of segments, and ignores spectral information and point intensity of an input ALS point cloud.Therefore, different types of objects with similar geometric properties are classified into the same class.(ii) Only five classes are extracted, and some classes such as ground should be classified into some sub-classes.In this paper, we propose a multiple-primitives-based hierarchical classification method for 3D urban areas.The stepwise point cloud segmentation method presented in Ni et al. (2017) is employed to extract three kinds of segments, i.e., planar, smooth, and rough surfaces.Classification rules and a fuzzy classifier are both utilized to classify these segments.Spectral information and point intensities are taken into consideration to point-wise classify ground points into bare land, artificial ground and greenbelt.Moreover, the proposed method is designed for urban areas, and we presume the ground surfaces are smooth and flat.The outline of this paper is shown as follows.Section 2 presents the methodology of our proposed multiple-primitives-based hierarchical classification.The experiments and discussions are shown in Section 3. At last, Section 4 summarizes the methodology and give a conclusion of this paper.

Figure 2 .
Figure 2. The result of the employed step-wise point cloud segmentation, (a) is the original ALS point cloud data which is colored by elevations, and (b) is the segmentation result which is colored by segments.

Figure 3 .
Figure 3.The three types of primitives.

Figure 4 .
Figure 4. Classification result of regular surfaces, (a) is the regular surfaces in a 3D local area, (b) is the top view of the classification result, and (c) is the side view of the classification result.The points coloured by red are building, and the points coloured by grey are ground.

5 Figure 5 .
Figure 5.The classification result of rough surfaces, (a) is the rough surfaces, and (b) is the classification result.The points colored by blue are vegetation, and the points colored by red are vehicles.

Figure 6 .
Figure 6.The classification result of ground points, (a) is the ground points colored by RGB information, (b) is the classification result coloured by classes.The points coloured by red, green, and blue are artificial ground, greenbelt, and bare land, respectively.

Figure 8 .Figure 9 .
Figure 8. Primitives extraction, (a) is the extracted planar or smooth surfaces, and (b) is the extracted rough surfaces.

Figure 10 .
Figure 10.The classification result of rough surfaces, the points colored by blue are vegetation, and the points colored by red are vehicles.

Figure 11 .
Figure 11.The classification result of ground points, the points colored by red, green, and blue are artificial ground, greenbelts, and bare land, respectively.

Figure 12 .
Figure 12.The classification result of the whole testing data.
primitives-based hierarchical classification strategy is proposed in this paper.Three types of primitives are utilized, i.e., regular surfaces, rough surfaces, and individual points.The step-wise point cloud segmentation method which extracts two types of primitives is first employed and performed.Next, the ground points and building points are classified in the regular surfaces based on two semantic rules.Then, Vegetation points and vehicle points are classified in the rough surfaces based on a supervised classification procedure.At last, the ground points are point-wise classified into bare land, artificial ground, and greenbelts based on RGB information and point intensities.

Table 1 .
The confusion matrix, and accuracies of the classification result.