VALIDATION OF A UAV-DERIVED POINT CLOUD BY SEMANTIC CLASSIFICATION AND COMPARISON WITH TLS DATA

Recent years showed a gradual transition from terrestrial to aerial survey thanks to the development of UAV and sensors for it. Many sectors benefited by this change among which geological one; drones are flexible, cost-efficient and can support outcrops surveying in many difficult situations such as inaccessible steep and high rock faces. The experiences acquired in terrestrial survey, with total stations, GNSS or terrestrial laser scanner (TLS), are not yet completely transferred to UAV acquisition. Hence, quality comparisons are still needed. The present paper is framed in this perspective aiming to evaluate the quality of the point clouds generated by an UAV in a geological context; data analysis was conducted comparing the UAV product with the homologous acquired with a TLS system. Exploiting modern semantic classification, based on eigenfeatures and support vector machine (SVM), the two point clouds were compared in terms of density and mutual distance. The UAV survey proves its usefulness in this situation with a uniform density distribution in the whole area and producing a point cloud with a quality comparable with the more traditional TLS systems.


INTRODUCTION
Geomatics is intimately connected with environmental surveying and monitoring and is becoming increasingly used in geosciences for digitization of outcrops; once this is accomplished, enhanced segmentation techniques can be applied to select, and measure selected features. Traditionally this type of surveying is performed from the ground with topographic total stations and GNSS receivers and consisted in the measurement of a necessarily limited set of points; however these systems have gradually given way to laser scanners (Buckley et al., 2008). The terrestrial laser scanner (TLS) has the capability to measure thousands of points with a density and an accuracy that has become increasingly higher over last years. Besides, as the last TLS generations mount optical cameras, they have the capability to automatically generate 3Dcoloured points clouds that significantly helps geological interpretation. Due to the relevance of photo-interpretation, terrestrial photogrammetry was also largely used for outcrop surveys especially thanks to the introduction of digital cameras and data processing by computer vision algorithms (Firpo et al., 2011). In recent years, the general trend is to perform integrated surveys where the strengths of each systems are combined to produce most reliable and accurate products; some examples of TLS and photogrammetry integration are reported by (Assali et al., 2014;Sturzenegger and Stead, 2009). The principal advantages of an integrated approach are the simplicity of use, a well-known experience background and the possibility to survey inaccessible steep and high rock cliff faces, reducing the risks for workers as the survey can be undertaken in a remote safe location (Sturzenegger and Stead, 2009). Nevertheless, there are some impediments: the presence of elements, like bushes and trees, that introduce perspective obstructions and can hinder the instrument placements, and the complexity of the survey when the extension and the shape of the outcrops need many stations.
The arrival of UAVs has partially allowed overcoming these limitations; drones are flexible, cost-efficient and can support outcrops surveying in some inaccessible conditions (Bemis et al., 2014;Vollgger and Cruden, 2016). The use of multicopters is particularly convenient as it allows flying over and bypassing obstacles and permits to follow the shape of the outcrop thanks to a careful flight planning; moreover, the capability to mount the camera with a tilted angle ease the acquisition on vertical rock cliff faces. Nevertheless, the presence of both low and high vegetation can obstacle the surveying especially considering the stereoscopic nature of photogrammetric measurements. This complementarity between TLS and UAV has made their comparison an object of some research in the past years (Ruggles et al., 2016;Wilkinson et al., 2016). In this context, the present paper aims to evaluate the quality of the point clouds generated by laser and by a UAV system to evaluate the shape and spatial distribution of calcite concretions developed within the sandprone fill of a turbidite channel (Marini et al., 2019). The paper aims to evaluate if the two techniques have comparable results in terms of precision and reliability allowing a potential integrated use in order to exploit their mutual strengths. The comparison will be done in terms of density, mutual distance and distance from some control points located on the outcrop. However, to conduct a correct comparison, only bare rock areas were considered excluding the points belonging to vegetation and undergrowth; machine learning techniques, based on eigenfeature and support vector machine (SVM), will be used to perform this task.

THE UAV AND TLS DATASETS
The investigated area is part of the Tertiary Piedmont Basin of NW Italy (Figure 1). The survey of the outcrop was performed with laser and photogrammetric techniques using a Leica Viva MS60 Multistation and a HEXA-PRO UAV system operated by the Laboratory of Geomatics of the University of Pavia (Italy). An integrated topographic survey, with two GNSS receivers and a Total Station, was performed first. Seven benchmarks were materialized in the area and were measured in a redundant way ( Figure 2): all the vertices were connected by topographic observation whereas points #1 and #7, those located in most favourable places, were also measured with static GNSS technique. The precision after adjustment is less than 1.5 cm in both the planimetric and altimetric components. The network was then used both to measure the photogrammetric targets and to orient the scans, defining the same reference frame for all the performed measurements.

Figure 2.
The topographic network whereas the topographic benchmarks are reported with red triangles and GNSS points with light blue squares; green lines represent the topographic observations. Google Satellite Data (MapData@2021Google) is used as background.
The geological outcrop was then surveyed with the Leica Viva MS60 Multistation by a surveyor form Leica Geosystems Italia. This is a modern surveying device combining the functionality of a robotic total station and a laser scanner. The MS60 Multistation can measure 3D point clouds, including true colour, intensity, and signal-to-noise data, within a range of 1000 m and with a frequency of 30000 Hz (i.e., 30 thousand points per second). The total station functionalities allow to orient directly all the scans in the reference frame represented by our topographic network; besides, the topographic adjustment of the scan positions has a precision comparable to topographic network itself. The outcrop was acquired with four partially overlapped scans ( Figure 3) Unfortunately, most of the area was inaccessible so the instrument was able to move only along the main road acquiring just the lower part of the outcrop; besides the presence of vegetation caused the presence of some holes in the TLS datasets. Each scan has a number of points ranging between 100000 and 175000; the whole point cloud is about 600000 points. Topographic adjustment, scan orientations and point classifications were performed directly by Leica.   The UAV system was equipped with an optical Sony A6000 camera having 6000x4000 pixels resolution, 16 mm of focal length and pixel size of about 4 microns. The outcrop morphology required acquiring multiple photogrammetric blocks to achieve full image coverage and the desired detail. These included two nadiral (i.e., downward looking camera, Figure 4), three oblique (camera tilted by 45° from nadir, Figure 5) aerial blocks, and a terrestrial block (acquired moving along the main road), which collectively yielded more than 1000 images with an average ground sampling distance (GSD) less than 1 cm. Before image acquisition, 34 artificial markers were deployed on the outcrop to be used as ground control points (GCPs), and their coordinates were measured with topographic survey ( Figure 6). The photogrammetric project was carried out with Agisoft Metashape TM , following the usual pipeline : image alignment, tie point extraction, and bundle block adjustment (BBA). Seven GCPs were used for orientation while the other 27 for evaluating the quality of the aerial triangulation.
The obtained RMSE are about 2 cm and 3 cm, for the planimetric

m
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) and altimetric components, respectively. Dense point cloud was then generated using half image size resolution and obtaining a cloud of about 118 million points.

THE METODOLOGY
Only points belonging to the bare rock will be used for comparison so these points must the extracted from the original point clouds through a classification procedure. For TLS dataset this step was directly done by Leica, so the considered cloud was already classified (ground label was used in our analysis); for UAV dataset, the classification was performed using the wellknow SVM (Support Vector Machine) algorithm according to both radiometric and geometric information. Comparison was then performed in terms of density, mutual distance, and distance to GCPs; in particular, mutual distance was assessed following different approaches: point-to-point and point-to-plane. This section will illustrate the procedures implemented for classification and validation; all the steps were realised in Matlab, realise 2019b.

Radiometric and geometric information
Both radiometric and geometric information were used to classify the UAV-based point clouds. Three labels were considered: vegetation, meaning trees and shrubs, undergrowth, constituted by moss and dead leaves, and bare ground or bare rock. The utility of radiometric information is quite trivial since it is easy to understand as vegetation and bare rock can be distinguished by colour; nevertheless, also geometric characteristics can contribute to their clustering. Based on the spatial information of all 3D points within a local neighbourhood, invariant moments can represent the geometric properties (Maas and Vosselman, 1999). The eigenvalues can directly be used to describe the local 3D structure or, alternatively, further measures based on these eigenvalues can be derived which encapsulate special geometric properties such as linearity or planarity (Mallet et al., 2011;West et al., 2004). These geometric descriptors, called eigenfeatures, can then be used to identify points useful for point classification. This strategy has the advantage to be applicable to any type of clouds be it produced by photogrammetry or by laser systems. The adequate choice of a neighbourhood for determining the eigenfeature values of each point, depends on the characteristics of the cloud data especially to its points density and 3D shape.
The choice can be based on a-priori definition of the search area in terms of radius or number of points (Arya et al., 1998;Friedman et al., 1977), or adapting this parameter according with the local geometry of the point cloud (Farella et al., 2019;Martin Weinmann et al., 2015). While the former requires an empiric knowledge of the scene, the latter is more versatile because it is not restricted to a specific dataset. The procedure implemented in this paper follows the first strategy and fixes a constant search radius; this means that for each point belonging to the cloud, a list of neighbours, falling in search radius, can be associated. Then, for each 3D point and its neighbours, the derived normalized eigenvalues with = 1,2,3 can be extracted using the Principal Component Analysis (PCA). These values, obtained from the covariance matrix, represent the variation of the points distribution along the three principal orthogonal directions. Eigenvalues can be combined to obtain some shape descriptors called eigenfeatures (Farella et al., 2019;M. Weinmann et al., 2015a) which enclose: linearity , planarity , scattering , omnivariance , anisotropy , eigentropy , sum of eigenvalues and change of curvature ; Table 1 reports theirs mathematical formulation.

Eigenfeature
Formula Sum of eigenvalues = 1 + 2 + 3 Change of curvature = 3 1 + 2 + 3  Weinmann et al., 2015b). Even if all the eigenfeatures have been extracted, it must consider they may contain redundant of irrelevant information with respect to the classification task. As highlighted by some authors (Roffo, 2016;Weinmann et al., 2013), it is often desirable to select a compact subset of the most relevant features which allows for classification/clustering without significant loss of information.
As we had chosen a supervised approach, eigenfeatures selection can be guided by corresponding class labels, making the task easier. Among supervised selection method, we had adopted the Minimum Redundancy Maximum Relevance (MRMR) algorithm, proposed by (Ding and Peng, 2005) , which selects a subset of features having the most correlation with a class (relevance) and the least correlation between themselves (redundancy). In this algorithm, the features are ranked according to the minimal-redundancy-maximal-relevance criteria. Relevance and redundancy are both calculated by using the mutual information approach.

SVM classification
Support Vector Machine (SVM) is a supervised learning model that allow to perform classification and regression analysis The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) (Boser et al., 1992). As classifier, starting for a set of labelled training data, the method can categorize new one. SVM plots each data item as a point in n-dimensional space, where n is number of categories, with the value of each feature being the value of a particular coordinate, such as radiometric a geometric information. Then, it performs classification by finding the hyper-plane that differentiates the classes as well as possible. The kernel function is introduced into the SVM so that the original input space can be transformed non-linearly into a higher dimensional feature space where linear methods may be applied (Zhou et al., 2002). Since SVM is a supervised method, a pre-labelled information must be provided to the algorithms; for our datasets three categories were established: bare ground, undergrowth, and vegetation. Given the data complexity, the preliminarily classification was performed by a human operator via manual labelling performed with the opensource software package CloudCompare™.

Mutual distance
The main aim of the paper is the validation of UAV-based point cloud by comparison with TLS data. This validation was performed by evaluating the mutual distance between the two point clouds. As shown in Results (Section 4), the main issue is their disparity in terms of density and, this difference influences the analysis of mutual distance. For this reason, three approaches were tested: point-to-point, point-to-global surface, and point-tolocal surface. Point-to-point distance is a quite common method used in several application such as ICP registration ; it was implemented by making use of the Matlab KDTreeSearcher class. The matching function works by first creating a kD-tree with the reference data and then runs a standard nearest neighbour search to match each point from the tested point cloud to a candidate point in the reference data set. However, if the density of the clouds is poor, it can lead to unreliable results. Point-to-plane distance can then be employed through the construction of mesh structures. Delaunay triangulation can be used to reconstruct such a surface as it is based on the establishment of neighbourhood connections between the points (Boissonnat, 1984). The Delaunay triangulation is a cell complex that subdivides the convex hull of the sampling. If the sampling fulfils certain non-degeneracy conditions, then all faces in the Delaunay triangulation are simplices and the Delaunay triangulation is unique. The combinatorial and algorithmic complexity of the Delaunay triangulation grow exponentially with the dimension of the embedding space of the original surface (Cazals and Giesen, 2006). Delaunay triangulation was used for the reconstruction of geological objects by several authors (Mulchrone, n.d.; Wang et al., 2017;Xue et al., 2004); nevertheless the complexity of outcrops can introduce unsatisfying results. If the surface is improperly sampled, for instance, the reconstruction algorithm can produce artifacts such as the presence of spurious surface boundaries in the model (Gopi et al., 2000). In the studied geological outcrop, the complexity of the object in addition to the presence of rip-up clasts can introduced artifacts in the reconstructed mesh. For this reason, we tested two approaches: the former based on the reconstruction on the whole surface (global) while the latter on the iterative reconstruction of the surface just in the small area neighbour to the tested point (local). For the reconstruction of global surface the MyCrustOpen function was implemented (Giaccari, 2020) while for the local one the Matlab delaunayTriangulation method was applied imposing the radius of the reconstructed area in the neighbour of the tested point.

RESULTS
The section describes the results obtained using the proposed strategy for UAV-derived point cloud comparison with TLS data, in terms of density and mutual distance. Even if the UAV surveys were related to a larger area, only the outcrop zone was considered in this section. This choice has a double reason: considered only the part related to the outcrop that was effectively measured by the two technologies, UAV and TLS, and reduce the computational effort. This smaller cloud, visible in Figure 8, is constituted by more than 36 million of points anyway.

Eigenfeature extraction and selection
Since comparison was performed on the bare rock only, the UAV datasets was preliminary classified using eigenfeatures, as reported in Section 3.1. A constant search radius is fixed inside which neighbours are identified; radius is set equal to 25 cm, according to area characteristics and point clouds density. Covariance matrix and eigenvalues are then determined, and eigenfeatures are calculated using the formulas reported in Table  1. Eigenfeatures and colour information are then normalized to the interval [0,1] and stored in a matrix having as many rows as the number of points, and eleven columns. MRMR rank method (Ding and Peng, 2005) is then applied to select main relevant eigenfeatures. Figure 7 reports the ranking positions in the form of bars; the lower values indicate high positions in the meaningfulness rank. Starting from the first two relevant eigenfeatureanisotropy and omnivariancewe performed several classifications adding progressively a new characteristic to each new iteration (i.e., in the second iteration we added scattering, and so on). Best results, as described in the next section, were obtained using the first three rank positions, anisotropy, omnivariance, and scattering.

SVM classification
As already mention, the mutual distance analysis was performed only considering points belonging to bare rock (undergrowth and high vegetation must be excluded to avoid unreliable figures). Well-known SVM learning model was chosen as classification method. Since it follows a supervised approach a training dataset is needed to instruct the algorithm; in our experiment, the 60% of the data were used for training while the other 40% for test. The classification for the training/testing was conducted manually via the opensource software package CloudCompare™ by an expert operator. Figure 8 shows this preliminary classification for the UAV-based point cloud; the figure reports the three considered The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) categories: bare ground, in blue, undergrowth (composed mainly by moss), in green and high vegetation, in red. To evaluate the SVM results, the confusion matrix was calculated: each points of the manual classification (true value) is compared to the corresponding one of the automatic segmentation (predicted value). Figure 9 reports the confusion chart obtained for the test dataset of UAV point cloud; labels 1, 2, and 3 represent bare rock, undergrowth, and high vegetation, respectively. In the main diagonal of the matrix, it is possible to visualize how many points are identified in the same way by both the manual and automatic approach. There is a good accordance for the high vegetation label since almost the 80% of the points were correctly classify considering both user and product accuracy. The quality significantly decreases around considering the undergrowth and bare rock as they reached values ranging between 61% and 72%. However, this result is not surprising considering the nature of our dataset. The latter two labels have very similar shape characteristics especially in the areas covered by a thin layer of moss. In these cases, eigenfeatures had a limited usefulness and radiometric information was not always sufficient to correctly classify the points, especially where dark areas, caused by rock humidity, were present. Besides, this specific outcrop presents many singularities, constituted by rip-up clasts, that further complicate the classification. These concretions, highlighted in Figure 10 and Figure 11, were mainly classify as undergrowth rather than as bare ground due to their shape. However, these misclassifications were not considered disabling for the next comparison steps, in particular for moss areas: even if they belong to undergrowth label, since their thickness can be considered limited, should not introduce significantly errors in the mutual distance evaluation.  . Classify UAV point cloud displayed using typological labels (blue for the bare rock, green for undergrowth, and red for high vegetation. Due to their shape, rip-up clasts were classified as undergrowth instead as rock.

Density analysis
While UAV has a uniform distribution of points, TLS is known to be affected by a varying density according to the distance between the object and the instrument. This lack is usually compensated by a careful planning typically constituted by several surveying points. In our case, that was not possible since the road, that run around the geological outcrop, was the only safe location in which the instrument could be set up; this constrain had as main consequence an inhomogeneity in the produced point clouds. Figure 12 reports the linear spacing for TLS-derived point cloud: the value ranges between 1.5 cm in the lower part of the outcrop (and in correspondence to instrument locations) to almost 50 cm in the upper part, not considering the top where few points were measure). Figure 13 represents the equivalent for the UAVderived point cloud: density is substantially uniform on the whole outcrop (bare rock) having a linear spacing of about 1.6 cm.
The two clouds are fully comparable in favourable parts, near the road, while the density of TLS becomes progressively poor moving away from it. As will be seen in the next sections, this behaviour will influence the distance analysis.

Distance from the GCPs
Before evaluating the mutual distance between the two point clouds, they were assessed through the use of the photogrammetric markers presented in the area (those directly located on the outcrop). These points, used to orient UAV images, were distributed quite uniformly on the outcrop ( Figure  6) and were measured via integrated survey (Section 2); they can be also used to evaluate the accuracy of the produced point clouds.
Distance between markers and clouds were computed, using a point-to-point approach, and principal statistical figures are calculated ( Table 2). The location of photogrammetric targets was optimized for blocks orientation therefore they are well visible on the UAV-derived point cloud; indeed 24 points are available to evaluate its accuracy. Due to the limited line of sight of TLS, since the instrument could only be moved on the main road, only 10 markers can instead be used to test the corresponding point cloud. TLS performs slightly worse due to discretization; indeed, UAV cloud is denser than TLS one, in any parts of the outcrop, and distance computation suffers this disparity. Nevertheless, there is a good overall accuracy, and the two clouds can be considered completely comparable proving that the two techniques can be interchangeable or can be use in an integrated way.

Mutual distance analysis
The results of classification step were finally used for evaluating mutual distance between the two clouds since only points belonging to bare rock were considered in this analysis. Three strategies for distance calculation were taken into account: pointto-point, global and local surface. The spatial distribution of the two datasets density is significantly different; while UAV points are uniform distributed, TLS presents inhomogeneity especially between the lower and the upper parts of the outcrop, as reported in Section 4.3. This aspect has an influence on the distance analysis as the choice of the adopted strategy for its calculation can lead to different results. First, point-to-point distance was evaluated using the Matlab KDTreeSearcher class and the knnsearch method. The two clouds, UAV and TLS, were considered as reference and tested dataset, respectively. Even if, theoretically, the TLS cloud should The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) be considered the reference data, this choice was adopted to reduce computational effort. The quantity of points belonging to the UAV clouds is significantly larger than that referred to TLS; setting the first dataset as reference reduces the searching procedure. Figure 14 shows the histogram for point-to-point distances; most of the points are contained in the bar ranging between 0.8 and 1 cm; the 95% of the distance are within 6 cm while the 50-percentile is 1.3 cm (Table 3).  Delaunay triangulation was used to evaluate mutual distance in a point-to-surface approach; both global and local strategies were followed. In the former, a global mesh was generated for the whole outcrop exploiting the MyCrustOpen function developed by (Giaccari, 2020); in the latter, a local mesh was created in the surrounding area of each tested point, fixing a constant radius of 25 cm. Figure 15 shows the result for global surface strategy whereas, Figure 16, reports that for local one. Histogram shapes are similar even if, the latter presents a steeper curve. For both, the most populated bin is the first one having a range between 0 and 6 mm; the 95-percentile significantly differs as for the former is 17 cm while for the latter 8 cm and the 50-percentile is 4.1 cm and 1.9 cm, respectively (  Table 3. The 50-and 95-percentile for the three considered strategies.

CONCLUSIONS
A laser and a photogrammetric point clouds were compared in terms of density, mutual distance, and distance from the GCPs located on the outcrop. Analysis was performed only for the bare rock, as reported in the method section, to obtain most reliable results. Density represents the main issue as the two datasets show significantly differences especially in the upper part of the outcrop. From this point of view, UAV provides better results since the structure of block acquisition has followed the outcrop shape; Leica MS60 has instead a limited line of sight as the instrument could only be moved on the main road. This difference in the point clouds density also influences the distances analysis; to overcome this problem different strategies were adopted for their calculation using point-to-point, global surface, and local surface approach. Results show a good agreement between the two datasets and an overall good accuracy.