PERFORMANCE EVALUATION OF DIFFERENT GROUND FILTERING ALGORITHMS FOR UAV-BASED POINT CLOUDS

Digital Elevation Model (DEM) generation is one of the leading application areas in geomatics. Since a DEM represents the bare earth surface, the very first step of generating a DEM is to separate the ground and non-ground points, which is called ground filtering. Once the point cloud is filtered, the ground points are interpolated to generate the DEM. LiDAR (Light Detection and Ranging) point clouds have been used in many applications thanks to their success in representing the objects they belong to. Hence, in the literature, various ground filtering algorithms have been reported to filter the LiDAR data. Since the LiDAR data acquisition is still a costly process, using point clouds generated from the UAV images to produce DEMs is a reasonable alternative. In this study, point clouds with three different densities were generated from the aerial photos taken from a UAV (Unmanned Aerial Vehicle) to examine the effect of point density on filtering performance. The point clouds were then filtered by means of five different ground filtering algorithms as Progressive Morphological 1D (PM1D), Progressive Morphological 2D (PM2D), Maximum Local Slope (MLS), Elevation Threshold with Expand Window (ETEW) and Adaptive TIN (ATIN). The filtering performance of each algorithm was investigated qualitatively and quantitatively. The results indicated that the ATIN and PM2D algorithms showed the best overall ground filtering performances. The MLS and ETEW algorithms were found as the least successful ones. It was concluded that the point clouds generated from the UAVs can be a good alternative for LiDAR data.


INTRODUCTION
3D point clouds have been used to generate DEMs, which are required for various remote sensing applications.As known, DEM represents the bare earth surface.Hence, the very first step of generating a DEM is to remove the non-ground points from the point cloud, which is called ground filtering.After ground filtering, the ground points are then interpolated to generate the DEM.Many ground filtering algorithms have been introduced in the literature.If there are abrupt changes in topography, then the filtering process would be more challenging (Meng et al., 2009).
LiDAR sensors generate multiple returns (first return, last return etc.).This is a huge advantage when separating the ground and non-ground points.Hence, ground filtering algorithms were mainly developed to filter LiDAR point clouds.However, the use of a LiDAR point cloud is not always possible due to its high cost (Wallace et al., 2012;Díaz-Varela et al., 2015).As an alternative for LiDAR point clouds, it is possible to generate very dense point clouds by using overlapped aerial photos taken from UAVs.The aim of this study is to investigate the performances of ground filtering algorithms, mainly developed for LiDAR point clouds, for UAV-based point clouds.The density of a point cloud effects the filtering result.Hence, in this study, denser point clouds were also generated to investigate the effect of point density on filtering performance.

GROUND FILTERING
In the study, the Progressive Morphological 1D (PM1D), Progressive Morphological 2D (PM2D), Maximum Local Slope (MLS), Elevation Threshold with Expand Window (ETEW), and Adaptive TIN (ATIN) algorithms were used to filter the point clouds extracted from the aerial images taken from a UAV.All these algorithms were implemented in the ALDPAT v.1.0software.

PM1D and PM2D Algorithms
The PM algorithm was developed by Zhang et al., (2003) to remove the non-ground points from the point cloud.This algorithm removes the different-sized non-ground objects and keeps the ground points by using gradually increasing window and elevation difference threshold.PM algorithms depend on the dilation and erosion operations, which are widely-used in mathematical morphology.These operations are used to increase or decrease the size of the objects (Haralick et al., 1987).Combinations of the erosion and dilation operations leads to opening and closing operations (Zhang et al., 2003).Dilation follows erosion in opening operation; whereas erosion follows dilation in closing operation.A regularly spaced minimum surface grid was generated by using the minimum elevation values in each grid cell.The elevation value of the nearest point is assigned to the grid if a grid does not contain any elevation values.Opening operation, which is the most important component of the PM filtering algorithm, is performed to the grid surface.In the first iteration, minimum elevation surface and initial window size are used as input data.Following iteration uses the increased window size and filtered surface, which are generated in the previous iteration.The window size is increased and elevation difference threshold is calculated.Iterations continue until the size of the filtering window is greater than the previously defined maximum threshold.Let ℎ , is the elevation difference threshold, and ℎ , is the elevation difference between the filtered and the original surface at a point  in k th iteration.In each k th opening operation, if ℎ , ≤ ℎ , , the point is considered a ground point.Otherwise, the point is labelled as a non-ground point.The iterations continue until the size of the window is greater than a predefined maximum value, which is generally slightly larger than the size of the largest non-ground object (Zhang et al., 2003).The only difference between the PM1D and PM2D algorithms is that the PM2D algorithm uses a two-dimensional square window to perform erosion and dilation.PM1D and PM2D algorithms use the same parameters.Vosselman (2000) developed the MLS algorithm, in which the ground points are detected by comparing the local slope differences between the adjacent LiDAR points.Irregularly distributed LiDAR points are overlaid with a regularly spaced grid network.

MLS Algorithm
Each grid represents the elevations of LiDAR points.Each LiDAR point   (  ,   ,   ) is assigned to the grids with respect to  and  coordinates.In case where more than one point falls into a grid, the point with the minimum elevation is chosen.MLS algorithm calculates the slope ( 0, ) between a  0 ( 0 ,  0 ,  0 ) point and the points in a defined radius. 0, is the maximum slope value between the point  0 and its neighbours in the radius.If this value is smaller than a predefined threshold (), then the point  0 is labelled as ground.Otherwise, it is labelled as non-ground and removed.

ETEW Algorithm
This algorithm uses a gradually increasing search window to separate the ground and non-ground points (Zhang and Whitman, 2005).First, the data is divided into grid cells.In each grid, the points, whose elevations are higher than the elevation of the point with the minimum elevation, are removed.In the next iteration, the size of the grids is increased and the minimum elevation value in each grid is recalculated.Then, the points whose elevations are higher than the point with the minimum elevation with respect to a predefined threshold are removed.Iterations continue by increasing the grid sizes and threshold values until there is no point to remove in the previous iteration.Let  , is the elevation of the point  , in  th iteration and  th grid,  , is the minimum elevation in this grid, and ℎ , is the elevation difference threshold.If  , −  , > ℎ , , then the point  , is removed (Zhang and Cui, 2007).

ATIN Algorithm
ATIN algorithm, developed by Axelsson (2000), identifies the ground points with respect to the distance between each point and generated TIN (Triangular Irregular Network) surface.First, the data is divided into square grids.Then, the points which have the minimum elevations in the initial ground data are chosen as seeds.Reference TIN surface is then generated by using these seeds (Zhang and Cui, 2007).Each unclassified ground point (candidate point) is added to each of the triangles in TIN.Candidate point is classified with respect to its distance to the triangular surface and to the angle with the vertices of the triangle.Candidate point is said to be a ground point if the calculated distance and angle are smaller than predefined thresholds.This process is repeated until all points are labelled as ground and non-ground (Axelsson, 2000;Zhang and Cui, 2007;Zhang and Lin, 2013).

Study Area
A small part of the Karadeniz Technical University (KTU) campus was chosen as the study area.The campus is in the city of Trabzon, which is located on the northeast of Turkey.The study area, which has a dimension of 178 m x 410 m and elevation ranges from 18 m to 83 m, contains flat, sloping and rough regions.There are also non-ground objects such as trees and buildings.Study area can be seen in Figure 1.

Data Preparation
The aerial photos of the study area were taken by using the RICOH GR DIGITAL IV digital camera, which was mounted on the Gatewing X100 UAV.With a 40-minute flight, 256 aerial photos were captured along 9 flight lines.Before the flight, 12 ground control points (GCP) were established in the campus to obtain the georeferenced 3D point cloud.The ground control points were evenly distributed over the entire campus.The aerial photos were processed and the raw point cloud was generated by using the Agisoft PhotoScan Professional software.The generated raw point cloud was then densified to form the medium-density and high-density data.The densities of the raw, medium-density and high-density data were 0.1 point/m 2 , 4.2 point/m 2 and 16.5 point/m 2 , respectively.As a final step, a 25 cm orthophoto was produced for the study area.

Ground Filtering
PM1D and PM2D algorithms were used to filter the point clouds in PM filtering stage.Main parameters of the PM algorithm are as follows; Cell size () is the size of each grid dividing the point cloud into square arrays.This parameter can be chosen smaller than the average distance among the points in data (Montealegre et al., 2015). () parameter specifies the elevation difference threshold.This parameter can be chosen as the average slope of the study area (Zhang and Cui, 2007).Initial Threshold () is the initial elevation difference threshold used to approximate the error of the points (Zhang and Cui, 2007).Maximum Threshold () is the maximum elevation difference threshold.Window base () is the base of the exponential function in the window size equation.Table 1 shows the parameters used in the PM1D and PM2D algorithms.The optimum parameters used in the MLS algorithm are given in Table 2. Experiments revealed that the search radius and maximum slope parameters are the most effective ones.The parameters used in the MLS algorithm are; Cell size () is the size of the grids and generates the minimum elevation grid.Minimum distance () refers to the minimum separation between the points allowed in slope computation (Zhang and Cui, 2007).Maximum slope () is the maximum threshold value for slope.A point is labelled as ground if the maximum slope with its neighbours is smaller than .
Otherwise, it is labelled as a non-ground point (Zhang and Cui, 2007).Search radius () specifies the neighbours of a point.The optimum parameters used in the ETEW algorithm are shown in Table 3.The slope and iteration number parameters were found to be the most effective ones with regard to the performance of the ETEW algorithm.The parameters used in the ETEW algorithm are as follows; Cell size () is the initial size of each grid and can be chosen as the average distance among the points in data (Montealegre et al., 2015).Slope () is the threshold parameter for slope.
Iteration number () indicates how many times the algorithm will iterate (Zhang and Cui, 2007).Table 4 shows the optimum parameters used in the ATIN algorithm.Experiments indicated that the cell size, Z difference and initial grid size parameters effect the performance of this algorithm most.The parameters used in the ATIN algorithm are as follows; Cell size () is the size of each grid and is used to divide the point cloud into grids.Z difference () is the threshold for the elevation difference between each point and triangular surface (Zhang and Cui, 2007).Initial grid size () parameter generates the initial grid network.This parameter chooses the seed points corresponding to each grid.Axelsson (2000) indicated that this parameter can be set to the size of the largest non-ground object.Angle threshold () is the threshold parameter for the angle between the candidate point and the vertices of the corresponding triangle (Axelsson, 2000).Since the Delaunay triangulation is a time consuming process for large-scaled areas, the ATIN algorithm uses the   −  ℎ () parameter to divide the data into tiles with dimensions  and .

Accuracy Assessment
Performance of each filtering algorithm was evaluated qualitatively and quantitatively.In qualitative evaluation, filtered point clouds were overlaid with the orthophoto to investigate whether or not the filtering algorithms achieved to separate the ground and non-ground points.In quantitative evaluation, some of the points in point clouds were chosen as test points.These test points were overlaid with the filtered point clouds and very high-resolution orthophoto image to investigate whether or not each point was filtered properly.
Ground filtering process causes two types of errors, namely the commission and omission error.Commission error is related to the classification of the non-ground points as ground; whereas omission error refers to the removal of the ground points.It may not always be practical to investigate the performances of ground filtering algorithms by using all points in data.Hence, Zhang et al., (2003), and Zhang and Whitman (2005) proposed to evaluate the filtering result by using randomly selected test points.In this approach, some random points are selected.These points and all points whose distances to the selected points are smaller than a predefined threshold are also selected as test points.All test points are then used in quantitative evaluation.The approach proposed by Zhang et al., (2003) and Zhang and Whitman (2005) was used to evaluate the ground filtering results.Hence, 100 random points were selected from each point cloud (i.e. the raw, medium-density and high-density data).Radius threshold was chosen as 3 m, 1 m and 0.5 m for the raw, medium-density and high-density data, respectively.In total, 688, 662 and 637 test points were chosen for the raw, medium-density and high-density data, respectively.The metrics proposed by Sithole and Vosselman (2004) were used for accuracy assessment of these test points.These metrics are the type I error, type II error and total error, and are given as; where,  is the number of ground points classified as nonground (omission error) and  is the number of non-ground points classified as ground (commission error). and  stand for the total number of ground and non-ground points, respectively (Montealegre et al., 2015).
Each filtering result was also evaluated by using the McNemar test in which the statistical significance of the differences between two proportions are measured (Foody, 2004;Kavzoglu and Colkesen, 2011).More specifically, this test calculates the chi-squared value between two data sets.Difference between two data sets is considered significant if the calculated chisquared value is greater than the critical value, which is 3.84 at 95% confidence interval (Kavzoglu and Colkesen, 2012;Yilmaz and Gungor, 2016).Cohen's Kappa index (Cohen, 1960) has been used in various studies to assess the filtering result (Silván-Cárdenas and Wang, 2006;Meng et al., 2010;Chen et al., 2013;Pingel et al., 2013;Montealegre et al., 2015).Kappa index generally ranges from 0 to 1. Obtaining negative Kappa values is also possible.Landis and Koch (1977) indicated that a Kappa index below 0.40 indicates a poor agreement, a Kappa index between 0.40 and 0.75 indicates a good agreement.A Kappa index greater than 0.75 presents an excellent agreement (Montealegre et al., 2015).In this study, the same test points used in the computation of the type I, type II and total error were used for calculation of Kappa and chisquared values.

Qualitative Evaluation Results
Study area includes non-ground objects like trees and buildings on various flat and rugged surfaces.Figure 2 shows the orthophoto draped over the filtered raw data (Figure 2a), medium-density data (Figure 2b) and high-density data (Figure 2c).concluded that the commission error committed by this algorithm was very small.MLS and ETEW algorithms, which were found to be the least successful ones, yielded similar results.Visual inspection of the filtering results indicates that the ATIN algorithm yielded the least average errors.
As seen in Figure 2b, the commission error committed by the filtering of the medium-density data with the ATIN algorithm was greater than those in the results of the PM1D and PM2D algorithms.The omission error of the ATIN result was found to be smaller than those of the PM1D and PM2D results.It can be concluded from Figure 2b that the MLS and ETEW algorithms committed similar errors and found to be the least successful ones in terms of omission and commission errors.The ATIN algorithm caused least omission error.
Figure 2c indicates that the ATIN algorithm committed least error when filtering high-density data.However, the commission error of the ATIN result was greater than those of the PM1D and PM2D results.The MLS and ETEW algorithms caused similar commission errors.However, the MLS algorithm committed more omission errors than the ETEW algorithm.
Hence, the MLS algorithm can be considered the least successful one.

Quantitative Evaluation Results
Optimum parameters were chosen for ground filtering in light of the approach suggested by Hu et al., (2014).According to this approach, parameters minimizing the total error are chosen to filter the point cloud.Quantitative evaluation results are given in Table 5. Quantitative evaluation results indicate that the type I, type II and total error ranges from 1.9% to 69.35%, from 2.5% to 28% and from 5.9% to 50.08%, respectively.The smallest type I error (1.9%) was committed by the PM2D algorithm when filtering raw data; whereas the smallest type 2 error (2.5%) was achieved by the ETEW algorithm when filtering the highdensity data.Filtering the raw data with the PM2D algorithm resulted in the smallest total error (5.9%).It was also concluded that the greatest type I (69.35%), type 2 (28%) and total error (50.08%) were obtained by the MLS, PM2D and MLS algorithms when filtering the high-density data, raw data and high-density data, respectively.
The highest Kappa (77.5%) was achieved by filtering the raw data with the ATIN algorithm; whereas the smallest Kappa (24.6%) was obtained by filtering the high-density data with the ETEW algorithm.The ATIN algorithm resulted in the highest chi-squared value (415.90) when filtering the raw data and the MLS algorithm yielded the smallest chi-squared value (36.20) when filtering the high-density data.
As seen in the filtering results of the raw point cloud, the best Kappa and chi-squared results were achieved by the ATIN algorithm.The PM2D algorithm kept the ground points better than the ATIN algorithm in terms of the type I error.However, the performance of the ATIN algorithm in classification of the non-ground points decreased the total error and increased the Kappa.Although the qualitative evaluation results revealed that the ATIN algorithm yielded the smallest commission error among all filtering results, the quantitative evaluation results show that the MLS algorithm resulted in the smallest commission error.The worst Kappa (61.8%) and chi-squared (280.50)results were obtained by the ETEW algorithm when filtering the raw point cloud.
Examination of the filtering results of the medium-density data depicts that the best Kappa (64.5%) and chi-squared (284.81)results were achieved by the PM2D algorithm.The ATIN algorithm follows the PM2D algorithm.As happened when filtering the raw data, the Kappa and chi-squared values, obtained by filtering the medium-density point cloud with the MLS algorithm, are better than those obtained by the ETEW algorithm.The ETEW algorithm yielded the worst Kappa (43.7%) and chi-squared (159.01)results.The commission error of the ATIN result was found to be greater than that of the PM2D algorithm, which also justifies the qualitative evaluation results.
As seen in the filtering results of the high-density point cloud, the best Kappa and chi-squared results were achieved by the ATIN algorithm.The PM2D algorithm classified the ground points better than the ATIN algorithm, with respect to the type I error.However, the ATIN algorithm caused less commission error when filtering the high-density data.Hence, the ATIN algorithm performed better than the PM2D algorithm in terms of total error.In contrast with the filtering results of the raw and medium-density point clouds, the metric results show that the ETEW algorithm performed better than the MLS algorithm when filtering the high-density point cloud.The MLS algorithm was found to be the worst one with a Kappa of 15.9% and a chisquared value of 36.20.
Filtering results revealed that the performances of the filtering algorithms decreased as the densities of the point clouds increased.The MLS algorithm was found to be the one that was effected from the increase in the densities of the point clouds most.The MLS algorithm performed better than the ETEW algorithm when filtering the raw and medium-density data.However, this algorithm was the least successful one in filtering the high-density data.These results depict that it is better to use the MLS algorithm to filter the sparse point clouds.In general, the PM2D algorithm performed better than the PM1D algorithm.However, the performances of these algorithms get closer to each other as the point density increases.The results of the PM2D algorithm were generally found to be closer to those of the ATIN algorithm.In general, the ATIN algorithm was found to be the most successful one and can be used to filter the dense point clouds.
There are a few reasons that the performances of the filtering algorithms decrease as the point density increases.Since the densification of a point cloud generates erroneous points in the vicinity of non-ground objects, errors increase in the areas close to non-ground objects such as trees and buildings.On the other hand, filtering algorithms use some parameters specified by users.These parameters play an important role in the performances of filtering algorithms.Increase in the point density makes it hard to use optimum parameters.The use of inaccurate parameters leads to greater errors when filtering dense point clouds.

CONCLUSIONS
In the literature, there are various studies in which LiDAR point clouds have been filtered.However, the acquisition and processing of LiDAR data is not always affordable.In this study, the performances of ground filtering algorithms, which were mainly developed to filter LiDAR point clouds, were investigated by using the point cloud extracted from the aerial images taken from a UAV.The generated raw point cloud was then densified to generate the medium-density and high-density point clouds to investigate the effects of point density on filtering performance.This study achieved to reveal the effects of point cloud density on filtering performance thanks to the considerable differences in the densities of the used point clouds.The main conclusion drawn from the results was that the filtering performance decreases as the point density increases.
The results show that the ATIN algorithm was found to be the most successful one.The PM2D algorithm follows the ATIN algorithm with a small margin.The MLS algorithm was found to be the least successful one.It can be concluded from the results that the use of UAV-based point clouds can be considered an alternative for LiDAR point clouds to separate the ground and non-ground points.Future studies will focus on generating DTM from these filtered point clouds.

Figure
Figure 1.Study area

Filt
The parameters used in the ETEW algorithm (M.D stands for medium-density and H.D means high-density) Figure 2. Filtering results of the raw (a), medium-density (b) and high-density (c) point clouds

Table 2
. The parameters used in the MLS algorithm (M.D stands for medium-density and H.D means high-density)

Table 4 .
The parameters used in the ETEW algorithm (M.D stands for medium-density and H.D means high-density)

Table 5 .
The parameters used in the ETEW algorithm (M.D means medium-density and H.D means high-density)