MACHINE LEARNING FOR MOBILE LIDAR DATA CLASSIFICATION OF 3D ROAD ENVIRONMENT

3D road mapping is essential for intelligent transportation system in smart cities. Road features can be utilized for road maintenance, autonomous driving vehicles, and providing regulations to drivers. Currently, 3D road environment receives its data from Mobile Laser Scanning (MLS) systems. MLS systems are capable of rapidly acquiring dense and accurate 3D point clouds, which allow for effective surveying of long road corridors. They produce huge amount of point clouds, which requires automatic features classification algorithms with acceptable processing time. Road features have variant geometric regular or irregular shapes. Therefore, most researches focus on classification of one road feature such as road surface, curbs, building facades, etc. Machine learning (ML) algorithms are widely used for predicting the future or classifying information to help policymakers in making necessary decisions. This prediction comes from a pre-trained model on a given data consisting of inputs and their corresponding outputs of the same characteristics. This research uses ML algorithms for mobile LiDAR data classification. First, cylindrical neighbourhood selection method was used to define point’s surroundings. Second, geometric point features including geometric, moment and height features were derived. Finally, three ML algorithms, Random Forest (RF), Gaussian Naïve Bayes (GNB), and Quadratic Discriminant Analysis (QDA) were applied. The ML algorithms were used to classify a part of Paris-Lille-3D benchmark of about 1.5 km long road in Lille with more than 98 million points into nine classes. The results demonstrated an overall accuracy of 92.39%, 78.5%, and 78.1% for RF, GNB, and QDA, respectively.


Introduction
Light Detection and Ranging (LiDAR) is becoming a leading technology in data acquisition, LiDAR is significant with its rapid, direct and accurate technique in capturing different objects with point density that covering each object with a sufficient number of points. In the past years, mobile mapping scanning (MLS) has taken a lot of interest and importance due to its advantages over airborne and terrestrial scanning systems. MLS is distinguished with a high point density and cover the vertical structures such as buildings' facades. Mobile LiDAR scanning has a contribution in many applications that serve the government and private sectors such as city modelling, roadway detection, autonomous vehicles, etc. The point cloud extracted from MLS is meaningless and has no information about classes of objects. Therefore, an effective processing should be applied, especially in roadways design and management (Yadav et al., 2015). The point clouds classification is essential step for any MLS databased application. Thus, the classification step has a great interest in several researches (Munoz et al., 2009;Weinmann et al., 2015;Xiong et al., 2011).
The contribution of mobile point cloud in the extraction of road features such as ground surface, curb stone and light poles is a crucial and still under developing. The extraction went from individually search for each feature without considering any other features (Guan et al., 2014;Wang et al., 2015;Yan et al., 2016), to extraction of all features using machine learning (ML) classifiers (Hackel et al., 2016;Weinmann et al., 2017Weinmann et al., , 2014Weinmann et al., , 2013 . ML is the ability of the classifier to learn the experience from a pre-classified dataset automatically, and improves the classification ability of unknown dataset. It is also the process of solving any problem through getting known dataset, learning * Corresponding author from it and building a model that is able to expect unknown dataset (Burkov, 2019). In most cases, automatic labelling any point cloud through ML classifiers is based on three main stages; neighbourhood selection, features extraction and the classification stage.
In this paper, we evaluate three ML classifiers for mobile LiDAR point clouds data classification. A cylindrical neighbourhood selection method is first utilized to extract sufficient geometric LiDAR point features. The extracted features are then used an input to ML classifiers and finally, accuracy assessment of the three classifiers is presented.

METHODLOGY
Our proposed methodology is divided into four steps as shown in Fig. 1. The methodology starts with data pre-processing followed by features extraction based on cylindrical neighbourhood selection. These features could be 3D or 2D geometric features, and are utilized for the classification process using three ML classifiers. Finally, an evaluation of the three classifiers is conducted to check the applicability of each classifier.

Data Pre-processing
A pre-processing step is first conducted to handle the huge 3D point cloud dataset in the training and testing processes. Unclassified points are removed as they affect the processing time and classification results. In addition, this step is required in case of creating the model with any known dataset. Another preprocessing step is slicing of the dataset. Slicing is to divide the dataset into equal slices. The idea behind the slicing concept is reducing the processing time while finding neighbours of any point as described in the next section. Dataset slicing is based on either equal distance along the longitudinal direction of the road or equal number of points especially in the case of organized data structure. Dataset slicing has a disadvantage at edge points as they do not have sufficient neighbour points. Therefore, to best find the neighbours of edge points of slices, a side overlap is provided for each slice. Thus, the neighbourhood selection algorithm searches for all points within the slice, in addition to two sides overlap.

Neighbourhood Selection Method
Cylindrical neighbourhood selection is used to identify the surrounding points of a query point x in order to extract the required features as input of ML algorithms. The radius of the cylinder is defined in the 2D projection, while its height contains all points below and above the point x. (Zheng et al., 2018(Zheng et al., , 2017 implemented variant values of radius between 0.25 and 1.05 m. However, slight differences in results occurred between those different radius values. Increasing the radius would take more time to find the neighbours, and then more processing time which is not preferable. Figure 2. Cylinder neighbourhood selection definition; "r" is the radius of cylinder. The query point is in red colour, while the blue points are its neighbour points.

Features Extraction
From each point's neighbourhood, the covariance equation is formatted from whole points within the neighbourhood, constructing a 3x3 covariance matrix C as defined in Equation (1). Both Eigen values and Eigen vectors are calculated from the covariance matrix. First subset of features is the covariance features as shown in Table (1). Those features are defined by (Weinmann et al., 2014(Weinmann et al., , 2013 using the normalized eigenvalues. The sum feature, in this research, is calculated from the raw values of Eigen values as deriving it from the normalized values is meaningless. The verticality feature, defined by (Demantké et al., 2011), is also considered and added to covariance subset. Another subset of features, derived previously by (Hackel et al., 2016), is the moment subset. The third features subset is derived from height of points including the maximum difference in height within the neighbourhood as well as the standard deviation of the height values of whole points.  (Breiman, 2001), which combines multiple of weak learners to the sake of a stronger one (Weinmann et al., 2015). The simple and powerful GNB classifiers are predictive modelling and used for continuous data where the mean and standard deviation of the training data are used to show the distribution of the data. GNB algorithm also requires the same number of predictors and descriptors for linear parameters in a learning problem (Bishop, 2006;Gatziolis and Andersen, 2008). QDA detects a quadratic boundary between classes to isolate each class with single boundary. Shape of classes is also determined using the covariance of the classes, while their centre is the mean. Each classifier is used to construct a model from known dataset. The model is then evaluated using another part of the dataset which is not included in the training. The dataset is divided into two equal parts; training and testing. The training part is randomly divided into four equal portions and with equal contribution of each class in each portion. Four models are created with the four portions and they are locally evaluated using the rest of the training part. The best model is used to classify the testing part and evaluate the ML classifiers.

STUDY AREA AND DATASET
The used dataset in this research is a part of Paris-Lille-3D benchmark belonging to NPM3D Benchmark suite research project which is available at http://npm3d.fr/paris-lille-3d. Paris-Lille-3D dataset was acquired with the MLS prototype of the centre for robotics of Mines Paris Tech: L3D2. It is a Citroën Jumper equipped with an IMU (Ixsea PHINS in LANDINS mode), a GPS receiver (Novatel FlexPak 6), and a Velodyne HDL-32E LiDAR mounted at the rear of the truck with the axis of rotation with angle 30 degrees to the horizontal (Roynard et al., 2018).
Lille part is a scanning of 1.5 km long road in Lille with more than 98 million points and available as two parts; Lille 1 and Lille 2. It consists of nine classes including ground, building, pole, bollard, trashcan, barrier, pedestrian, car and vegetation. Table  (2) provides the percentage of each class. The point clouds density is between 1000 and 2000 points per square meter.

RESULTS AND DISCUSSION
Dataset slicing was implemented using equal number of points (250,000 pts./slice) in addition to two sides overlap with 50,000 points each and a cylindrical neighbourhood was carried out through whole points to find their neighbours within a 0.20 m radius. The three subsets of features were then derived from the neighbourhood of each point and replaced the raw coordinates of points. The whole dataset with the new features was divided into two equal parts (i.e., training and testing). The training part was divided into the four portions as explained in the methodology to obtain the best model, while the testing part was used to evaluate that model for the three classifiers.
For the evaluation process, four scores were used including the overall accuracy, precision, recall and f1-score. Overall accuracy was used as a primary score for the models. Other scores (precision, recall and f1-score) were used to individually evaluate the results of each class. Mobile LiDAR point clouds data of road environments are usually imbalanced, and two or three classes may consist more than 90% of the whole dataset. Consequently, any ML model may be confused and tend to classify most of points to the major class or classes. Therefore, the role of using precision, recall and f1-score is to evaluate the classifiers with datasets that have low percentages of classes.
In the three classifiers, the overall accuracy of RF, GNB and QDA was 92.39%, 78.47% and 78.18%, respectively. Figure 4 shows the precision, recall and f1-score of the three different classifiers for each class.
The three classifiers revealed, in terms of precision, recall and f1score close scores for ground class and relatively close scores for building class. This is attributed to that the ground and buildings classes contain the most LiDAR points. For instant, the precision of ground class was 98.11%, 96.66% and 97.68%, while the recall was 91.35%, 88.1% and 87.95% and f1-score was 94.61%, 92.18 and 92.56% for RF, GNB and QDA, respectively. For building class, the precision was 95.60%, 76.27% and 81.21%, while the recall was 92.65%, 90.25% and 87.14% and f1-score was 94.1%, 82.67 and 84.07% for RF, GNB and QDA, respectively.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-M-3-2021 ASPRS 2021 Annual Conference, 29 March-2 April 2021, virtual Figure 4. Precision, recall and f1-score of the three ML classifiers.
Generally, the results showed that RF is much more effective than other classifiers. RF is also suitable to classes with less number of points such as poles, barrier, and trashcan. Therefore, classes with variant geometric characteristics require large scale of features to best distinguish between them. However, not all classifiers are able to handle all classes with huge number of samples as well as a lot of features. GNB and QDA classifiers were not able to achieve high scores for all classes. Only ground and building classes achieved close results for different classifiers. Ground and building were clearly distinguished due to their geometric shapes 2D planes, which were determined using Verticality feature. Thus, this is helpful for any classifier to best find most of points that belong to those classes.

CONCLUSIONS
The use of machine learning in 3D mobile LiDAR point cloud classification is still interesting to many researchers. In this research, we used a cylindrical neighbourhood selection method to identifying surrounding LiDAR points for features extraction. Three sets of point features, based on data geometry, were then extracted. Three different ML classifiers were finally evaluated for mobile LiDAR data classification.
The RF, GNB and QDA revealed an overall accuracy of 92.39%, 78.47% and 78.18%, respectively. Machine learning classifiers achieved outstanding results in extraction of multiple road features at the same time, coming over the traditional researches that focused on one road feature. Although they achieved relatively close scores for ground and building classes, RF achieved the highest scores for all classes. Taking the great difference in the amount of points between different classes into consideration, we evaluated the three classifiers using accuracy measures for each class, including precision, recall and f1-score. RF showed sufficient results especially for classes with low points' portion in the whole dataset.