GEOMETRIC FEATURES ANALYSIS FOR THE CLASSIFICATION OF CULTURAL HERITAGE POINT CLOUDS

: In the last years, the application of artificial intelligence (Machine Learning and Deep Learning methods) for the classification of 3D point clouds has become an important task in modern 3D documentation and modelling applications. The identification of proper geometric and radiometric features becomes fundamental to classify 2D/3D data correctly. While many studies have been conducted in the geospatial field, the cultural heritage sector is still partly unexplored. In this paper we analyse the efficacy of the geometric covariance features as a support for the classification of Cultural Heritage point clouds. To analyse the impact of the different features calculated on spherical neighbourhoods at various radius sizes, we present results obtained on four different heritage case studies using different features configurations.


INTRODUCTION
The ever-growing use in the last years of 3D models in different applications has led 3D data classification to become a very active research topic.The possibility to automatically group big data into multiple homogeneous regions with similar properties (segmentation) and attribute labels to them (classification or semantic segmentation), have become of primary importance in various applications and fields such as robotics (Maturana et al., 2015), autonomous driving (Wang et al., 2017), urban planning (Xu et al., 2014), heritage (Grilli and Remondino, 2019), geospatial (Özdemir and Remondino, 2018), etc. Different approaches were proposed in the literature (Grilli et al., 2017), but only recently significant progress has come out in automatic procedures thanks to the advent of Machine Learning approaches (Hackel et al., 2017;Weinmann et al., 2017;Wang et al., 2019).Machine Learning (ML) is a method of statistical learning.ML classifiers (e.g., Support Vector Machine -SVM or Random Forest -RF) are trained by giving them a set of features (Figure 1b) and training data that contains associated label information (i.e.classes -Figure 1c).A feature is a geometric or radiometric attribute that is useful or meaningful to the classification task.It is an integral part of observation for learning about the structure of the problem that is being modelled.Based on the training phase, a prediction is given for a semantic segmentation of the entire dataset (Figure 1d).So the choice of the features directly influences the predictive model and the results you can achieve.Although the extraction and selection of such features are prerequisites for point cloud classification, they are still considered very challenging and discussed operations (Khoury et al., 2017;Hearst, 2018;Li et al., 2018).In case of 3D point cloud data, 3D features typically result from a specific geometric characteristic of the global or local distribution of the points and a fair amount of them have been proposed in the literature (Weinmann et al., 2014;Georganos et al., 2015;Guo et al., 2016).The most common 3D features used to describe the local geometric behaviour of the point cloud are derived from the covariance matrix of the 3D point coordinates in a given neighbourhood.The goal of this work is to perform heritage point cloud classification employing covariance features (Chehata et al., 2009;Mallet et al., 2011).The possible and interesting applications linked to heritage 3D data classification include: • semantic annotation of the models: it can be useful to deepen the analysis and interpretation of the architecture as much as producing an aware representation of the models (Poux et al., 2017;Grilli et al., 2018); • automatic recognition of similar architectural elements in vast datasets: the detection of similar geometric properties in the scene can help the identification of a prevalent architectural style or constructive technique and could be a requisite for HBIM applications; • identification and distinction of structural and decorative architectural elements, highlighting their spatial distribution and organization.But the classification of heritage 3D data is more challenging for various reasons: • complexity of heritage elements to be classified (e.g.columns can be divided in base, shaft, and capital; windows in decorative and structural parts), generally required for analysis purposes; The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain • difficulty in identifying the classes of analysis: while in natural or urban scenarios the definition of object classes and labels is almost defined (mainly ground, roads, trees, buildings), it is much more variegate the delineation of them in the architectural/archaeological field: several classes could be used to identify and describe the same building characteristics, based upon different purposes; • problems at referring each class to semantic categories/ontologies developed for cultural heritage documentation (Doerr, 2009;Messaoudi et al., 2019); • difficult replicability/applicability of a specific training set to different heritage scenarios: geometric exceptions and variations of the elements are frequent in each architectural style; • lack of benchmark labelled 3D heritage point clouds in the current literature on which users can train and test their algorithms.This paper aims to investigate the effectiveness of covariance features for point clouds classification, in particular identifying a relationship between the geometric features, the scale of extraction of the features, and the architectural elements.The analysis investigates the factors that make geometric features more or less relevant for the classification task.The experiments are conducted on four different case studies, of different epochs but with similar architectural elements.The paper is organized as follows.The applied methodology, the extracted and selected features, and the machine learning classifier chosen for the process are described in Section 2. Section 3 contains the different experiments and results, whereas Section 4 will close the paper.

METHODOLOGY
The typical learning processing workflow proposed by Weinmann et al. (2016) for 3D classification includes five steps: (i) neighbourhood selection, (ii) features extraction, (iii) features selection, (iv) manual annotation and (v) classification.Following this workflow, our work aims at identifying a subset of features that performs well with different 3D heritage datasets.Our framework can be summarized in Figure 2. At first, we extract various geometric features (Section 2.1) at different scales (Section 3.2).After running a multi-scale classification with a Random Forest classifier (Section 2.2), we iteratively take into consideration only the more relevant features, and we re-run the classification process.Finally, we compare the different results relying on the traditional confusion matrix scores (Section 2.3).

Feature extraction -covariance features
The features we tested are based on the covariance matrix (Chehata et al., 2009) computed within a local neighbourhood of a 3D point.These features are shape descriptors obtained as a combination of the eigenvalues (λ1 > λ2 > λ3) extracted from the covariance matrix (Blomley et al., 2014).Features values highlight the main linear (1D), planar (2D) or volumetric (3D) structure of the point cloud in the neighbourhood.In addition to these covariance features, we took into consideration the Verticality V and the height of the points in the cloud (Z coordinate).The definition of these features is presented in Table 1. (1) (2) (3) (5) Verticality V= 1 -nz ( 9) Table 1.Considered local 3D shape features/covariance features.
As stated by Weinmann et al. (2013), different strategies may be applied to recover local neighbourhoods for points belonging to a 3D point cloud.In our case, we opted for a multi-scale approach (Niemeyer et al., 2014).Specifically, the features selected were calculated on spherical neighbourhoods at various radius sizes, to explore different responses in function to different geometric property of the heritage monuments.

Classification -Random Forest
The classification experiments were carried out using a random forest (RF) classifier.RF is an algorithm for supervised classification developed by Leo Breiman (2001) that uses an ensemble of classification trees, gets a prediction from each tree, and selects the best solution by means of voting.Two parameters need to be set to produce the forest trees: the number of decision trees to be generated (N tree ) and the number of variables to be selected and tested for the best split when growing the trees (M try ) (Belgiu et al., 2016).We leverage on the RF implementation available in the Scikitlearn Python library (version 0.21.1).During the training process, the N tree and M try are tuned considering the best F1-score computed on the test set.The RF classifier was chosen mainly for three reasons: • RF is considered as a highly accurate and robust method because of the number of decision trees participating in the process.• RF does not suffer from the overfitting problem as it takes the average of all the predictions, which cancels out the biases.• RF offers a useful feature selection indicator.Specifically, it shows the relative importance or contribution of each feature in the prediction: it automatically computes the relevance score of each feature in the training phase, then it scales the relevance down so that the sum of all scores is 1.

Evaluation test -confusion matrix
For all the case studies, we took into consideration a small portion of the entire dataset that we call test set.On this test set, we compare the label predicted by the classifier with the same previously manually annotated label.The number of correct and incorrect predictions are summarized with count values and broken down by each class inside a confusion matrix, a specific table layout that allows the visualization of the performance of the algorithm.Each row of the matrix represents the instances in a predicted class, while each column represents the instances in an actual class.Starting from the confusion matrix, for each class, we rely on (i) precision which represents a measure of exactness or quality, (ii) recall which represents a measure of completeness or quality, and F1-score which combines the previous ones: . Where Tp = true positive, 75 = true negative, :8 = false positive, :5 = false negative.

Case studies
To evaluate the aforementioned method and the effectiveness of covariance features for point clouds classification, four different case studies (Figure 3) were selected, featuring recurrent architectural elements such as columns, architrave, frieze, etc.: • The Basilica in Paestum (Italy): it spans ca 24,5 x 54 m and includes 18 columns on the long side and 9 on the short one, while the interior part has two lines of 3 and 4 columns.It was surveyed with TOF laser scanners (Fiorillo et al., 2013).

Feature relevance assessment -Basilica in Paestum
For the classification aim, we first need to annotate a small portion of the dataset, knowing which classes we want to identify in the dataset.To assess the feature relevance, we consider a subsample portion of the Basilica point cloud (Figure 4).Then, we extracted the previously described features on the point cloud using different search radii r, which we denote as a subscript to the name of the feature set, as shown in Table 2.For example, Planarity (0.2) means that the feature Planarity Pλ was calculated for a search radius of 0.2m.The dimensionality features are computed for increasing radius values between r min = 0.2 m and r max = 3 m.Considering the influence of the radius r in the feature response, we aimed to identify the optimal r able to better discriminate our classes.
By observing the ranking of the features, we can say that: • the Verticality V is highly relevant, even when considering different neighbourhood radii; • the features Eigenentropy Eλ and Sum of eigenvalues Σλ, are always among the least relevant features; • among the covariance features the most relevant are Surface Variation C λ , Planarity P λ, and Sphericity S λ ; • there is an apparent relationship between the radii of the columns and the most relevant radii of extraction of the features: i.e. the diameter of the columns is 1.4m, and we can notice that there are peaks for Planarity and Surface Variation at r = 1.4m as for Surface Variation (0.6) and Sphericity (0.8).This correlation could be related to strict proportional rules and dimensions used in the construction of this kind of structures.In order to investigate the features importance and how the various combinations of features can affect classification results, we start from 135 features and iteratively discarded the least important ones, down to 7 features.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain Table 2. Features importance ranking for a multi-scale classification of the Basilica dataset.
Then, we ran the same classification also considering the Z coordinates (height) of the points as extra feature.Finally, we compare all the different classification results with respect to F1score and time for training (Table 3).As we can see, by iteratively discarding the least relevant features the F1-score increases at the first step then slowly decreases.In contrast, the performance of the classifier is considerably improved by selecting only a small subset of useful features.Besides, it saves processing time concerning feature extraction, training time and classification.Confusion matrixes and classification results obtained using respectively 8 features extracted ad hoc and the same 8 features with the Z coordinate of the points are reported in Table 4 and 5.    6. Visual comparison of some geometric features extracted ad hoc on the different case studies.

Feature selection
The aim of features selection is to identify a small subset of features that can perform good prediction in a shorter time with respect to using many features extracted in a multi-scale approach.Considering the classification results for the Basilica datasets after the columns dimension was valid on different datasets with similar properties (Table 6).We can see that the feature Planarity extracted with r = Ø columns results being useful to identify as the planar elements (e.g.facades, pavements) as the cylindrical ones (e.g.columns).The Surface Variation is able to emphasise the same architectural elements of the Planarity with r = r columns, while the Anisotropy is useful to identify the capitals.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Basilica in Paestum, Italy
After the manual annotation (5 min), the features extraction and selection, we could extend the classification to the entire datasets (500 mil points) in ca half a minute (Figure 5).Results in terms of Precision, Recall, and F1-score are summarised in Table 7.
Figure 5. Classification result of the Basilica point cloud.

Temple of Neptune in Paestum, Italy
The data of the Temple consist of some 2.8 million points (Figure 6a).Starting from the previous experiences, we analysed the structure and decided to extract the features as a function of the three different order of columns of the temple (diameters 2 m, 1.4 m, 0.8m).Combining the use of 20 different features chosen ad hoc and annotating 10 classes on well distributed parts of the dataset (15 min -Figure 6b), we were able to classify the entire temple in about 1 hour (Figure 6c).Table 7 summarize classification results in terms of Precision, Recall, and F1-score.The rendering of the classification results of the Temple is available at https://youtu.be/8-muH633ud8.

Porticoes in Bologna, Italy
The Bologna dataset (Figure 7a) consist of about 1.2 million points.This structure combines various geometric shapes, different materials and many architectural details such as mouldings and ornaments.For the classification aim, 13 different classes are identified and annotated (Figure 7b).Considering the complexity of the task, in addition to 11 geometric features extracted ad hoc, we also considered the RGB values as fundamental elements for a successful classification of the point cloud (Figure 7c).The accuracy results are reported in Table 7.

Mausoleum of Cesare Battisti in Trento, Italy
The Mausoleum of Cesare Battisti (1935 A.D.) represents a modern re-interpretation of the classical architecture.As for the Basilica and the Temple case studies, we extracted the covariance features on spherical neighbourhoods at a few radii sizes as function of the diameter size of the columns (1.7m).The dataset (Figure 8a) (about 750.000 points) was then annotated on a small portion with 6 different classes (Figure 8b) and classified (Figure 8c).The numerical results are stated in Table 7.The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain Table 7. Summary of the classification results after the extraction of a few features ad hoc.All the classification processes were run with and without the use of the height information (Z coordinate).

CONCLUSIONS
The paper provided a general and straightforward method to classify heritage point clouds composed of repetitive architectural elements.The choice of the optimal neighbourhood radius for covariance features extraction is based on the knowledge of a few essential measures.Radii values are derived from simple proportional and dimensional rules, typically used for the construction of classical architectures and in the different heritage building of the following centuries.
Achieved results indicate that to obtain correct classifications, it's not necessary to use a lot of features extracted at many different scales.Indeed, the adaptive size strategy allows the retrieval of better results in a shorter time.
The inclusion of the height information (Z coordinates) of the points as extra feature proved to deliver a general improvement of the accuracy for the considered case studies (Table 7).
Further tests are planned to verify the replicability of the developed methodology with more complex and not repetitive structures, as well as to identify and test more geometric rules for the classification of buildings with different architectural styles.

Figure 2 .
Figure 2. Classification workflow (left) and our extension (right) to evaluate features relevance for the classification process.

Figure 3 .
Figure 3.The case studies of the work: Basilica in Paestum, Italy (a), Temple of Neptune in Paestum, Italy (b), Porticos in Bologna, Italy (c), Mausoleum of Cesare Battisti in Trento, Italy (d).
The Temple of Neptune in Paestum (Italy): it measures ca 24,5 x 60 m and consists of 6 frontal and 14 lateral columns while in the interior area it has two rows of double ordered columns.The point cloud is the result of a combined UAV and terrestrial photogrammetric survey(Fiorillo et al., 2013).•Part of a renaissance building with porticos in Bologna (Italy): they were surveyed with photogrammetric techniques(Remondino et al., 2016); we consider a portion of ca 85x6m.• The Mausoleum of Cesare Battisti in Trento (Italy): it is a circular funeral monument, surveyed with GeoSlam handheld laser scanner.For the study, we consider just the upper part of the entire structure, which includes 16 columns connected by an annular trabeation.For each case study, the classification is aimed at a semantic annotation of the different architectural and decorative elements.Object classification is a fundamental task in archaeology and heritage architecture, although it is essential to have a clearly defined purpose when developing and applying a classification procedure.

Figure 4 .
Figure 4.A portion of the Basilica point cloud manually labelled with 8 classes.

The
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W15, 2019 27th CIPA International Symposium "Documenting the past for a better future", 1-5 September 2019, Ávila, Spain

Table 3
. Comparison of different features combination for the Basilica's classification, wrt F1-score and time of training.CLASS Ster.Stylob Pav.Shaft Echin.Abac.Archit.Frieze Precision Recall F1

Average 79.12% 82.99% 80.51%Table 4 .
Confusion matrix and results for the Basilica dataset, using 8 geometric features ad hoc.

Table 5 .
Confusion matrix for the Basilica dataset and results, using 8 geometric ad hoc features plus the height information (Z coordinate).