SEMANTIC LABELLING OF URBAN POINT CLOUD DATA

Three dimensional urban reconstruction is gaining popularity with the paradigm shift from 2D maps to 3D maps. LiDAR remote sensing is emerging as the main source of 3D spatial data because of its very dense and discrete point cloud. The enormous amount of data collected over natural terrain calls for automatic methods for labelling the point cloud. Semantically labelling the urban point cloud into various features is essential for urban planning and development. In this study,we propose a new object oriented methodology for semantic labelling of urban point cloud data. In addition to the geometrical information from LiDAR, we have used the spectral information for labelling of the point cloud. The coloured point cloud was segmented using colour based region growing algorithm to produce 3D segments. Spectral and geometrical features were extracted from the segments created. The extracted features were classified using different classifier system into five urban classes. The proposed methodology has been tested on LiDAR captured over urban datasets .The results indicate the potential of object based classification for automated 3D point cloud labelling.


INTRODUCTION
Semantically identifying and labelling the minutest details of the real world (eg: Road, lawn, residential buildings, commercial buildings etc) resulting in a realistic 3D models has numerous applications -city planning, disaster response preparation,virtual tourism,cultural heritage documentation etc (Golovinskiy et al., 2009).However creating such models automatically remains a challenge.
Geospatial technology is considered to be the main source of data for producing such models ranging from ground survey to stereo images.Last decade has witnessed the power of Light Detection and Ranging (LiDAR) technique for accurately capturing the 3D of the various earth surface features.The ability of LiDAR in directly capturing the altitude information of ground and above ground objects through range measurements (point cloud) makes it readily usable for three dimensional city modelling.The dense point cloud obtained from LiDAR is useful not only for the generation of high resolution DTM (Digital Terrain Model) but also for many applications ranging from 3D urban reconstruction, biophysical parameter estimation,virtual reality models etc.Despite the ready availability of geometrical information from the LiDAR point cloud, many applications require the point cloud to be semantically classified into the various earth surface objects.This method known as point cloud classification is one of the important research topics in LiDAR data processing.The nature of data makes the automatic labelling of point cloud a challenge.
In the domain of point cloud labelling of urban scene, the earlier works were on separating the ground and non ground points.This technique, popularly known as filtering, has been well developed.Some of the popular filtering techniques include progressive densification based filtering (Axelsson, 2000) morphological based filtering (Vosselman, 2000) surface based filtering (Pfeifer, 2005) and segmentation based filtering (Filin and Pfeifer, 2006).A review of the ground filtering algorithms by (Meng et al., 2010) summarize that the current ground filtering algorithms use one * Corresponding author.This is useful to know for communication with the appropriate person in cases with more than one author of the following characteristics to separate ground points from non ground points: lowest feature in specific area, ground slope threshold, ground surface elevation difference threshold and smoothness.The review paper concludes that the filtering algorithms work well for flat terrain dataset but has certain limitations over undulating terrain.
Many studies have focussed on extracting one of the urban features from LiDAR dataset.Building prevail to be the objects of most interest amongst the researchers (Lari et al., 2011), (Rabbani et al., 2006), (Ghosh and Lohani, 2011), (Topographic Laser Ranging and Scanning:Priniciples and Processing, 2009).Clustering based segmentation was used by Shan and Sampath (Topographic Laser Ranging and Scanning:Priniciples and Processing, 2009) to extract building from point cloud data.(Sun and Salvaggio, 2013) has used graph cut methods for building extraction from point cloud.
Most of the existing studies used only the geometrical information from the LiDAR point cloud to identify various urban features in the scene.The spectral information from optical images could complement the geometrical information from LiDAR for better object discrimination.However, due to the different nature of the datasets, one being in three dimension and the other in the two dimension, it is not straightforward to use the datasets together for object recognition.Early researchers have converted the LiDAR to 2D Digital Surface Models (DSM) or Digital Terrain Models (DTM) and fused with the optical images (Rottensteiner et al., 2004), (Sohn and Dowman, 2007).However, this leads to loss of the rich geometrical information inherent to Li-DAR data (Wang and Shan, 2009).Very few studies have attempted using the coloured point cloud for urban scene analysis (Niemeyer et al., 2014).
Assigning labels to each points in the dataset can be either point based or object based.Commercial LiDAR processing softwares such as Terrasolid use point based methods which normally relies on the elevation information for point cloud labelling.However this involves significant human intervention and it is time consuming.With the popularity of Geographic Object Based Image Analysis (GeOBIA)approaches, few researchers have attempted labelling of LiDAR data using object based methods.Spectral information is not used in most of the cases.We propose a methodology for object oriented semantic labelling of point cloud derived from LiDAR data complemented with the spectral information obtained from optical data.The spectral information obtained from aerial images (R, G, B) was first integrated with geometric information from LiDAR to produce a colored point cloud (X, Y, Z, R, G, B).An improved color based 3D region growing segmentation algorithm was applied on the colored point cloud data.Various geometric and spectral features extracted from the 3D segments are then classified semantically using various classification algorithms.All the points inside a 3D segment were labelled with the same class as that of the segment.
We applied the proposed methodology on two airborne LiDAR datasets and the corresponding aerial images.

METHODOLOGY
Point cloud labelling can be either point based classification or object based classification approach.In point based classification points are classified mainly using the few features inherent to the point cloud such as distance from the ground, planarity.In most of the commercial softwares available such as Terrasolid, point labelling is based on point based classification.On the other hand, object based point cloud analysis is gaining momentum as it produces more realistic results.Similar to the GeOBIA (Geographic Object Based Image Analysis) concepts of 2D images, object based analysis for LiDAR point cloud has two stages.Segmentation is the first stage in which the point cloud is grouped to form objects (also called as segments), mainly based on geometric characteristics.Features are then extracted from the segments, for classification.The label assigned to each segment is then given to all the points belonging to the segment.This involves assigning a label ci to each of the 3D points i. Fig 2 .shows the methodology adopted in this research.The point clouds were assigned to one of the five classes 1. Pavement/Road 2.Lawn 3.Flat Roof Building 4. Gabbled Roof Building 5. Shrubs/Trees.The steps are explained in detail in the following sections.

Filtering
To reduce the complexity of classification, the LiDAR points were separated into ground and non ground points before further processing.This step is essential as the first two classes ie.,Pavement / Road and Lawn belong to the ground class whereas as the three classes belong to the non ground class.We have used have progressive TIN densification algorithm (Axelsson, 2000).In this algorithm, an initial TIN surface is generated from a set of seed points chosen initially based on the statistical information derived from the LiDAR data.The initial TIN surface is densified iteratively using the threshold that is obtained iteratively based on the threshold to the two parameters -angle to the nodes and the distance from the nodes.

Segmentation
Segmentation groups LiDAR points which are similar in some respect.This approach combines the inherent geometric properties of the LiDAR data with the spectral information from the optical image to produce disjoint 3D segments which are used for further processing.Since the LiDAR is captured in an unorganised random fashion, we have used KD tree algorithm to establish the neighbourhood of the point cloud.Color based region growing segmentation was then carried out on the organised dataset.There are three main steps in this algorithm : region growing, region merging and region refinement.Seed points were selected  (Rabbani et al., 2006).Seed points were selected which were grown using colorimetric distance as given in equation 1 (1) where R,G,B denote the spectral values obtained from the optical image.The roughly segmented regions obtained from the region growing process were checked for colorimetric similarity and are merged in the region merging stage.In the region refinement process the regions with less than acceptable number of points were merged with the neighbouring region.

Feature Extraction
The 3D segments created were used to extract features for further classification.Various spectral and geometrical features are computed in addition to the range of Z value in each segments.

Classification
Three different classifiers: Naive Bayes, K Nearest Neighbourhood, Support Vector Machines were used for classification.Naive Bayes is a probabilistic classifier which uses the prior and the likelihood to find the probability of an instance to be classified to a particular class.K Nearest Neighbourhood uses the distance/similarity function to relate the unknown class with the known class.Support vector machine is a mathematical tool which aims at finding the best hyperplane in the high dimensional feature space that separates the classes (Vapnik, 1995).We have used the 'Guassian Radial Basis Function (RBF)' as the kernel.SVM usually works for binary classification.We used multiclass SVM in this work as the entire dataset needs to be classified into five classes.
The performance of the classifiers was evaluated by generating confusion matrix and kappa coefficients.Ground truth map was generated based on complete enumeration.To assess whether the classification was due to chance, the popular statistical measure Cohen's Kappa coefficient was used.Kappa value was calculated using the equation 2 where p0 is the sum of the relative frequency in the diagonal of confusion matrix (proportion of true agreement) and pe indicates proportion due to chance agreement.The kappa value ranges between 0 and 1. Value of k between 0.6 and 0.8 indicate substantial agreement between the classification and ground truth data and the value of kappa above 0.8 and 1 indicate almost perfect agreement.

Point cloud labelling
The labels obtained from the classification for the segments assigned to all the corresponding points within the segment.The points were given one of the following labels 1. Pavement/Road 2. Lawn 3. Flat roof 4. Gabbled Roof 5. Shrubs/Trees.

RESULTS AND DISCUSSION
The methodology described in the previous section was applied to the two study sites and the results obtained is discussed in this section.The ground and the non ground data point were classified using the different algorithms discussed in the methodology section.The results are presented in Table 1 and Figure 8.It can be seen that the accuracy of classification is comparable among all the three classifiers.However, amongst the three classifiers, KNN performs relatively well for both the ground and non ground classes with an overall accuracy of around 96.67 % for ground classes and 84.48 % for non ground classes.Kappa coefficient also exhibits substantial to almost perfect agreement with the ground data.The overall accuracy for all the three classifiers is above 90% for the ground features whereas it is around 80% for non ground features.This is reflected also in the producers and users accuracy for the individual classes as shown in Figure 6 and 7.There is a drop in the accuracy for the classes 3 and 4 (Flat roof and gabbled roof).From the Figure 8 , it can be seen that there is misclassification between flat roof and gabbled roof.This can attributed to the fact that limited number of geometrical features

Figure 1 :
Figure 1: The general framework of object based point cloud labelling

Figure 3 :
Figure 3: LiDAR point cloud data of the test sites Area1 and Area 2 displayed based on elevation

Figure 6 :
Figure 6: Producers accuracy for different urban features

Figure 7 :
Figure 7: Users accuracy for different urban features

Figure 8 :
Figure 8: Labelled point cloud data representing various urban features

Table 1 :
Performance Evaluation of different classifiers used in this study.Use of more geometrical features such as shape and planarity int he classification may improve the classification accuracy.