Classification of mobile laser scanning point clouds of urban scenes exploiting cylindrical neighbourhoods

This paper presents our work on automated classification of Mobile Laser Scanning (MLS) point clouds of urban scenes with features derived from cylinders around points of consideration. The core of our method consists of spanning up a cylinder around points and deriving features, such as reflectance, height difference, from the points present within the cylindrical neighbourhood. Crucial in the approach is the selection of features from the points within the cylinder. An overall accuracy could be achieved, exploiting two bench mark data sets (Paris-rue-Madame and IQmulus & TerraMobilita) of 83% and 87% respectively.


INTRODUCTION
Around the year 2003 mobile laser scanning (MLS) systems became operational for surveying and 3D mapping of road scenes.Today MLS systems are used for capturing roads and their vicinity aimed at road inventories on a regular basis.The point acquisition rate as well as the number of commercial MLS systems in operation are steadily increasing.However, data is not yet informationthe conversion of data to information requires careful processing.
The automated classification of point clouds, either generated by Lidar systems or computed from overlapping imagery using dense image matching methods, is in demand for many different applications.One of the driving forces of today's research in automated classification is the need for autonomous navigation of self-driving cars.However, there are many other application domains, which may highly benefit from automated classification or where automation is even a necessity to handle and make tractable the massive amount of points.Coming from a geomatics background, we are, at present, particularly interested in 3D mapping as an application domain of automated classification.Indeed, the demand for 3D maps of cities and road networks is steadily increasing and mobile mapping systems are often the preferred geo-data acquisition method for capturing such scenes (Lemmens, 2017).The demand for automated mapping methods has been a great stimulus for research in photogrammetry, remote sensing and computer vision over the last decade.In our present research, we particularly focus on point clouds generated by Mobile Laser Scanning systems so that in the present context 3D mapping means the outlining of objects along road scenes which are of interest for a particular task at hand and the assignment of class labels to these objects.
To test our approach we use two available bench mark data sets.The first is the benchmark dataset generated by the Robotics laboratory (CAOR) at MINES ParisTech, France, covering a 160m-traject in Paris acquired February 8, 2013 (Serna et al., 2014).This data set has also been used in our previous experiments.The second bench mark data set is the IQmulus & TerraMobilita Contest bench mark dataset (Vallet et al., 2015).This paper is structured as follows.In Section 2 we briefly present our cylindrical approach and the features derived and used in the classification.In Section 3, the two Bench Mark Data Sets are introduced.In Section 4 we present the experimental setup and performance in terms of classification accuracy.In Section 5 we compare our results with two other classification approaches.The first approach is based on a simple box classification using heuristics based on scene knowledge.The second approach has been presented by (Weinmann et al., 2014) who used the same bench mark data.Future work and Conclusions separately are given in Section 6 and 7.

Cylinder description
A detailed description of our method and its underlying motivation can be found in (Zheng et al., 2017).The cylinder is spanned up around each and every point of the point cloud.The axis of the cylinder is defined by the X, Y coordinates of the point under consideration and the vertical.Our approach is based on a point-based classification, i.e. first computing and assigning features and next classes to each and every point.In order to obtain suitable features we place cylinders around points of consideration resulting in a cluster of points for each cylinder from which features can be derived (Figure 1). Figure 1.Cylinder placed around a point of consideration with pre-specified radius; the height is determined by the lowest and highest point.

Feature description
In our earlier work (Zheng et al., 2017) we used the reflectance value of a point of consideration ( R ) as a digital number ( DN ) in a range from 0 to 255, the height difference between the highest and lowest point in the cylinder ( max min ) and the number of points within the cylinder ( N ).For point in different classes, different vertical heights in corresponding cylinders should be significative for classification.The computation of these features can be done fast.A further rationale underlying this method is that objects captured in an MLS point cloud may have different reflectance values and different height ranges.For example, with respect to the latter a building facade varies in range which may start at eight meters, or higher, depending on the urban area, while the height of a traffic sign mounted on a pole from ground level upwards does usually not exceed three metres.The presence of a high number of points within the cylinder indicates an object with a large vertical extension.Of course, the height of a cylinder and the number of points that cylinder are highly correlated.However, in MLS point clouds (and also point clouds acquired by terrestrial laser scanners) the point density decreases with increasing distance from the sensor.This makes it feasible to explore both point density and height difference simultaneously as features in the classification process.The first experiments, reported in our earlier paper (Zheng et al., 2017), exploiting cylinders with a fixed radius of 0.25m, resulted in an overall accuracy of 73%.In this paper, a range of radii will be tested to reveal the influence of the number of points ( N ) as a feature, further exploring the significance behind the point density.Five classes are selected.
In an attempt to improve the overall accuracy we extend the three features exploited earlier, (that is,

Classifier description
As we do not aim at finding the best classifier but to test the feasibility of exploring features derived from the points present in a cylindrical neighbourhood, we use a popular classification scheme which is available as open source software and with which do have good experiences: LIBSVM (Chang and Lin, 2011).LIBSVM is an integrated software for support vector classification, and supports multi-class classification.

BENCH MARK DATA SETS
To test our approach we use two available bench mark data sets.
The first is the benchmark dataset generated by the Robotics laboratory (CAOR) at MINES ParisTech, France, covering a 160m-traject in Paris acquired February 8, 2013 (Serna et al., 2014).This data set, which we used also in our previous research, contains 20 million points of which the X,Y,Z coordinates, the reflectance value per point and the object class are given.
The second bench mark data set is the IQmulus & TerraMobilita Contest bench mark dataset (Vallet et al., 2015) on which we carried out the same classification approach.The database contains 3D MLS data from a dense urban environment in Paris (France), composed of 300 million points.The acquisition was carried out in January 2013.
Since algorithms are available to classify point clouds into two classes: ground points and non-ground points, and we want to refine the classification of non-ground points, we removed the ground points prior to classification.This was done by removing the points indicated in the bench mark data set as ground points.
For the Paris-rue-Madame database (Serna et al., 2014), class facades, cars, pedestrians, motor cycles and traffic signs are selected.The number of points per class are 9978435, 1835383, 10048, 98867, and 15480, respectively.The points belonging to other classes consist of 0.05% of the total number of points.We skip these points, which may be considered as semantic noise, at forehand, because leaving them out does not significantly influence the final classification accuracy and eases experimentation.For the IQmulus & TerraMobilita Contest bench mark dataset (Vallet et al., 2015), we select the following five classes: Building, Cars, Trees, Scooters without driver, and still pedestrians.Also for this data set the points belonging to other classes consist of 0.05% of the points and we skip these points.

Results and discussion
The first experiments, reported in our earlier paper (Zheng et al., 2017), exploiting cylinders with a radius of 0.25m, resulted in an overall accuracy of 73%.To investigate the influence of the setting of the radius we vary the size within the range [0.45m, 1.05m] with an interval of 0.15m in the present investigation.The parameter setting is based on scene knowledge.
Table 1 shows the confusion matrix of the Paris-rue-Madame dataset showing also the precision (P), the recall (R) per class and the overall accuracy (OA) using a radius of 0.45m.An overall accuracy of 0.83 could be achieved.Increasing the radius affects the overall accuracy slightly but not significantly (Figure 2).This is in contrast with the results of the IQmulus & TerraMobilita Contest bench mark dataset.Figure 2 shows that for this data set the accuracy increases with increasing radius.This is due to the fact that the point density of this data set is considerably lower than the point density of the Paris data set.This result shows that there is a relationship between the size of the radius and point density: the lower the point density the larger the size of the radius should be.

Introduction of performance adjustment
To judge the performance of the cylindrical approach proposed, we compare the classification results with (1) a simple box classification carried out by ourselves and (2) the classification results for the same bench mark data set as we use, by the approach developed by (Weinmann et al., 2014) based on the analysis of eigenvalues for the Paris-rue-Madame dataset.

Box classification
In order to adjust the performance of the proposed method, we compared the results with the results of a box classification on two features: reflectance value and height above ground, i.e.R and H.For example, points with a height above ground larger than 2.25m are classified as facades.It is obvious that the height value of objects in classes other than Facade is less than 2.25m.This is a rapid approach, easy to implement, and thus appropriate for testing the gain of using advanced methods.Figure 3 shows the two-dimensional feature space of our box classification approach and the thresholds set on the features H and R. All points which have a height above the threshold 2.25m (H3) above ground are classified as faç ade.As a further example, points with thresholds H < 2m and R <170 are either classified as faç ade or as pedestrians.The assignment of faç ade or pedestrian to the point is done in a random way using a random number generator.The setting of the threshold is based on heuristics using general knowledge about the given urban scene and the geo-data acquisition method.

FUTURE WORK
The tests presented in this paper show that our cylindrical approach is feasible and worthwhile to investigate further.This doesn't mean that there would be no room for improvement and modifications.First of all our results show that the classification accuracy depends on the point density.This is because objects appear in different sizes and may even differ in shape.So, for the future we aim to carry out experiments with a hierarchy of point densities.
Secondly, we want to modify the cylinders into cubes, that means the base is not anymore a circle but a square tile.In the cylindrical approach, presented here, a cylinder is constructed around each and every point and each point is classified separately.In the cube approach, the tile as a projection of the cube on the ground surface, will be the unit of classification.This brings in the advantage of reducing computational effort as well as easy combination with multispectral (satellite) images, which considerably expands the available data from which features may be computed.Added, to this it should be recognized that the objects distributed along roads and highways are not arbitrarily distributed but show regularities in their distribution, particularly in the form of a zonal appearance.This zonal distribution has been earlier described in (Lemmens, 2017) from which we quote the following: 'Therefore, to improve classification results we can introduce scene-specific rule number one: roads and their vicinity are manmade, meaning that the placing of objects, their shape, size and orientation, have to obey road traffic regulations, master plans and other official restrictions.As a result road objects, such as guardrails and traffic signs, appear in zones which are approximately parallel to the main direction of the road, while the distance to the road only varies within a certain range.Added to this, the orientation of traffic signs mounted on poles is usually perpendicular to the road direction.' Recently , Yang et al. (2017) have shown that the use of scene knowledge improves the accuracy of the classification pipeline they developed compared to other methods.

CONCLUSION
This paper presents our work on automated classification of Mobile Laser Scanning point clouds of urban scenes and builds upon the method developed earlier.This work extends the previous work by adding additional features and by testing the effects on the classification accuracy of the setting of various radius sizes of the cylinder.The tests presented in this paper show that our cylindrical approach is feasible for 3D mapping.
The above mentioned extension resulted in and improvement of the accuracy compared to earlier work.So, the features selected are appropriate, but we want to explain them in a cubic approach rather than in a cylindrical approach, as explained in the further work paragraph.

Figure 2 .
Figure 2. Overall accuracy using a range radius size for two point cloud bench mark datasets.

Table 1 .
Confusion matrix (i.e.numbers of points) of the five classes for Paris-rue-Madame database showing also the precision (P), the recall (R) per class and the overall accuracy (OA) with a radius of 0.45m

Table 2
shows the confusion matrix of the Paris-rue-Madame dataset showing assignment situations per class on two features and random selection, and the overall accuracy (OA) using the box classification.The box classification of the Paris-rue-Madame dataset resulted in an overall accuracy of 0.79.Combining the constraints that height value is between 1.5 m and 2m, and reflectance value is higher than 190, 375358 points are assigned to class Car.Same to the situation, 212708 points are classified to class Facade.Finally, Chance in Table2represents the assignment results of random select when two classes meet same constraint conditions at the same time.

Table 3 .
Recall and Precision between existing work and proposed work and overall accuracy (OA: overall accuracy)

Table 3
shows that two methods perform well for class facade and car, but not for the other classes.The large differences in classification accuracy are due to the number of points belonging to facade and car, which is larger than 90% of the total points.For example, 294,730 points in class Facade are incorrectly assigned to class Traffic signs (Table1), which largely results in the low precision of class Traffic signs.The same occurs for classes Pedestrians and Motorcycles.