REVISITING EXISTING CLASSIFICATION APPROACHES FOR BUILDING MATERIALS BASED ON HYPERSPECTRAL DATA

Classification of materials found in urban areas using remote sensing, in particular with hyperspectral data, has in recent times increased in importance. This study is conducting classification of materials found on building using hyperspectral data, by using an existing spectral library and collected data acquired with a spectrometer. Two commonly used classification algorithms, Support Vector Machine and Random Forest, were used to classify the materials. In addition, dimensionality reduction and band selection were performed to determine if selected parts of the full spectral domain, such as the Short Wave Infra-Red domain, are sufficient to classify the different materials. We achieved the best classification results for the two datasets using dimensionality reduction based on a Principal Component Analysis in combination with a Random Forest classification. Classification using the full domain achieved the best results, followed by the Short Wave Infra-Red domain.


INTRODUCTION
Knowledge about the materials found in urban areas is valuable information for municipalities while dealing with urban and regional planning.Such information is crucial for city models when a high level of detail is needed, but also for estimates of the anthropocentric inventory and as input to models of the built environment.The urban climate is strongly influenced by the existing materials, hence an increased number of large-scale land cover databases have emerged.
Data about building materials, such as the materials on a facade, have received an increased interest in recent times.As cities are trying to reduce the building energy consumption, information about the facade structure, its components and the materials are of major interest.With the usage of thermal imaging, it is possible to estimate the heat transfer coefficient for buildings when the material and the structure are known (Fokaides and Kalogirou, 2011).Hence, prior knowledge of the facade structure and materials is necessary for a meaningful estimation of the parameter.
Material classification based on data collected with remote sensing is an efficient approach since materials have different spectral reflectance curves, which can be extracted from the pixel values.The three most commonly used bands (red, blue and green), which ranges in the visible spectrum, are often not sufficient for material classification due to the limited spectral information.Therefore, data acquired with hyperspectral sensors have received an increased importance for classifications of urban materials since the spectral range is larger.Spectral libraries based on hyperspectral data are therefore often used for training and accuracy assessment of classification algorithms.
Spectral libraries can be based on data captured from either spaceborne, airborne or ground-based sensors/platforms (Heiden et al., 2007;Kotthaus et al., 2014;Herold et al., 2003).With the increased usage of Unmanned Aerial Vehicles (UAVs), such spectral libraries can be created using data acquired at lower altitudes * Corresponding author with innovative compact and lightweight sensors.Due to the small size of UAVs, flight campaigns with the sensors can be easily performed since acquisition of data can be done on the spot.In addition, data acquired with a UAV is less influenced by the atmosphere in comparison to data acquired at higher altitudes.
Hyperspectral sensors available for UAVs have limited technical specifications compared to bigger sensors, such as a narrower spectral range.It is therefore of interest to determine if offered spectral ranges are enough to classify building materials correctly.Hence, a comprehensive approach for material classification using feature extraction and band selection can determine if all bands are necessary or not.
In short, this study aims to classify urban materials, mainly those found on building facades, by using one existing spectral library and collected data from a spectrometer.The data is classified into commonly encountered building material classes, such as glass and concrete.Since the main task is to distinguish the basic materials, a fine level of detail of the material classification is not performed.The main objectives of this study are: • Classification of common materials found on buildings • Evaluation of different classification approaches • Utilization of an existing spectral library and collected data • Assessment of the added value by using dimensionality reduction and band selection We briefly summarize the literature related to the classification of hyperspectral data and the usages of hyperspectral libraries in Section 2. The used methodology is presented in Section 3, which consists of feature extraction, involving dimensionality reduction and feature selection techniques, and supervised classification.The utilized datasets, the experimental set-up and the experimental results are described in Section 4. Subsequently, the results are discussed in Section 5, while the final remarks and suggestions for future work are provided in Section 6.

RELATED WORK
To provide an overview of related work, the common approaches for classifying hyperspectral data are presented in Section 2.1 while the various applications of spectral libraries are presented in Section 2.2.

Classification of Hyperspectral Data
In recent years, a lot of attention has been paid to the classification of hyperspectral imagery (Plaza et al., 2009;Camps-Valls et al., 2014) which is nowadays typically acquired from airborne or spaceborne sensor platforms.Focusing on the classification on a per-pixel basis, the straightforward approach is to consider the reflectance values acquired for all spectral bands, to concatenate these values to a feature vector and to use a standard classifier such as a Support Vector Machine (Melgani and Bruzzone, 2004;Chi et al., 2008) or a Random Forest (Ham et al., 2005;Joelsson et al., 2005).
However, for high-dimensional data like hyperspectral data, a high degree of redundancy can be expected as values corresponding to adjacent spectral bands tend to be strongly correlated.Consequently, the classification approach has to deal with more or less relevant features as well as with redundant and possibly even irrelevant features.This is quite important, since an increase of the number of considered features over a certain threshold typically results in a decrease in classification accuracy, given a constant number of training examples (Melgani and Bruzzone, 2004;Keller et al., 2016).This effect is commonly referred to as the Hughes phenomenon (Hughes, 1968), and it can be addressed by using either dimensionality reduction or feature selection techniques.Thereby, dimensionality reduction can for instance be achieved via a Principal Component Analysis or an Independent Component Analysis as proposed in (Licciardi et al., 2012;Wang and Chang, 2006).While such approaches focus on transforming the acquired data to a new space, feature selection techniques focus on retaining only a subset of relevant and informative features.In general, feature selection typically allows to gain predictive accuracy, to improve computational efficiency with respect to both time and memory consumption, and to retain meaningful features (Guyon and Elisseeff, 2003;Saeys et al., 2007;Zhao et al., 2010).This has also been demonstrated for the classification of hyperspectral data, e.g. in (Melgani and Bruzzone, 2004;Le Bris et al., 2014;Chehata et al., 2014;Keller et al., 2016;Keller et al., 2017).
For the classification of hyperspectral imagery, there are also approaches exploiting spatial-spectral features, i.e. relations within a local image neighborhood are taken into account in addition to the spectral information per pixel.However, in the scope of our work, we focus on a classification of common building materials on facades by only considering the acquired spectrum per measurement.

Usages of Spectral Libraries
The number of spectral libraries are increasing as more applications are being developed and hence, more types of spectral libraries are being built.The specific types of materials and the level of detail when defining a spectrum are chosen depending on the intended application.Therefore, there is a large variation of spectral libraries and if publicly available, it is possible to choose a suitable library for a specific application.
One could group the types of spectral libraries into two bigger groups, the first one being libraries based on spectra from natural materials, such as vegetation (Zomer et al., 2009) and the other one being a mixture between natural and man-made materials, such as urban surface materials (Heiden et al., 2007).The applications of these types of libraries are thus different, since they have been developed and built for different purposes.
The different types of spectral libraries which are based on spectra from natural materials, are often developed for classification of different types of vegetation on a high level of detail.Using such libraries, it is possible to perform classification of soil properties (Shepherd and Walsh, 2002), vegetation species (Cochrane, 2000), crops (Rao et al., 2007) and even corals (Kutser et al., 2006).Hence, to distinguish the differences a high level of detail describing each spectrum is necessary.
Spectral libraries based on a mixture of man-made and natural materials may not need the fine level of detail when it comes to distinguish different types of vegetation, since vegetation is often grouped into a small number of spectra.On the other hand, the man-made materials are in such libraries distinguished on a finer level of detail (Baldridge et al., 2009;Kotthaus et al., 2014;Heiden et al., 2007).
In our work, the type of spectral library we use is based on spectra from urban land cover materials and thus, the man-made materials are defined on a finer level of detail.

METHODOLOGY
To classify common building materials on facades, we utilize a basic framework consisting of two steps.The first step addresses the extraction of features from the acquired hyperspectral data (Section 3.1), whereas the second step focuses on a supervised classification based on the derived features (Section 3.2).

Feature Extraction
The straightforward approach to obtain features from the acquired hyperspectral data is to directly use the values measured for all spectral bands and to concatenate these values to corresponding feature vectors.Depending on the classifier that is later used, it can be favorable to additionally introduce a normalization which maps the range of occurring values to the interval [0, 1] and thus adapts the value for each spectral band.Accordingly, we use a standard representation in terms of feature vectors whose entries correspond to reflectance values between 0% and 100%.
However, as already mentioned in Section 2.1, the values corresponding to adjacent spectral bands often tend to be correlated.Consequently, the degree of redundancy might be considerable for the considered hyperspectral data which, in turn, typically reveals a negative impact on the classification results which has for instance been demonstrated in (Melgani and Bruzzone, 2004;Keller et al., 2016).To address this issue, we involve standard approaches for dimensionality reduction and feature selection as recently used for the classification of simulated hyperspectral En-MAP data (Keller et al., 2017).

Dimensionality Reduction
For dimensionality reduction, we use a standard Principal Component Analysis (PCA).
The PCA uses an orthogonal transformation to map the considered data from one space spanned by possibly correlated variables to another space spanned by linearly uncorrelated variables which are referred to as principal components.The components are organized in a way that the first principal component covers the highest variability of the data, and that each subsequent principal component covers the highest possible variability under the constraint that it is orthogonal with respect to all previous principal components.As a consequence, the most relevant information of the considered data is covered by the first few principal components.The PCA-based dimensionality reduction hence focuses on only considering the first few principal components, while assuming that there will not be a significant loss of information when discarding all other principal components.For our experiments, we use the first few principal components which cover 99.9% of the variability of the given training data.
3.1.2Feature Selection For feature selection, we use an approach that has recently been proven to work well for the analysis of relevant spectral bands in simulated hyperspectral EnMAP data (Keller et al., 2017) and is referred to as Correlation-based Feature Selection (CFS) (Hall, 1999).In general, CFS accounts for (1) the correlation between features and classes to identify relevant features and (2) the correlation among features to identify and discard redundant features.With the average correlation between features and classes denoted by ρfc and the average correlation between different features denoted by ρff , the relevance r of a feature subset F comprising N f features is defined by A suitable feature subset F * can thus be derived by maximizing r over the set of all possible feature subsets (Hall, 1999).To achieve this, an iterative scheme is exploited where per iteration either a feature is added to the feature subset (forward selection) or a feature is removed from the feature subset (backward elimination) until the relevance r converges to a stable value.

Supervised Classification
Focusing on standard classifiers for supervised classification, we use a Random Forest classifier (Section 3.2.1)and a Support Vector Machine classifier (Section 3.2.2) which represent the most popular approaches among a rich diversity of classification approaches.

Random Forest
The Random Forest (RF) classifier proposed in (Breiman, 2001) is composed of an ensemble of randomly trained decision trees.More specifically, each decision tree is trained on a random subset of the training data, resulting in a set of decision trees which are randomly different from each another.Consequently, the predictions of the decision trees for a new feature vector can be considered as being de-correlated and, thus, improved generalization and robustness can be expected when taking the respective majority vote over the predictions of all decision trees (Criminisi and Shotton, 2013).

Support Vector Machine
The Support Vector Machine (SVM) classifier (Cortes and Vapnik, 1995;Chang and Lin, 2011) has originally been presented as binary classifier, which is trained to linearly separate two classes of interest by constructing a hyperplane or a set of hyperplanes in a high-dimensional space.Thereby, a good separation is expected to be achieved if the hyperplane has the maximum margin (i.e.distance) to the classwise nearest feature vectors in the training data.However, a linear separation of different classes in the considered space is often not possible.To allow for separability in such cases, a kernel function such as a (Gaussian) radial basis function (RBF) is typically introduced to implicitly map the training data to a new space of higher dimensionality where the data is linearly separable.To allow for multi-class classification, an SVM classifier is composed of several binary SVMs.For the latter, we use a one-againstone approach which means that, for each pair of classes, a binary SVM is trained to distinguish samples of one class from samples of the other class.Such a strategy may improve the training process and the subsequent discrimination of classes which are closely located in the feature space.

EXPERIMENTAL RESULTS
In this section, we present details about the used datasets (Section 4.1), a description of the implementation (Section 4.2) and finally the conducted experiment with the corresponding results (Section 4.3).

Datasets
To evaluate the proposed methods for feature extraction, we choose two datasets: an open access spectral library dataset and own collected data.Both datasets are presented in the following subsections.

Spectral Library
A number of commonly used open spectral libraries are available with variable emphasis on either artificial urban materials or natural materials (Le Bris et al., 2016).
We choose the Santa Barbara library for urban classification that has been compiled to study the spectral resolution requirements for urban area mapping (Herold et al., 2003).The available part of the Santa Barbara dataset is a collection of nearly 1000 field spectra of 26 differently labeled materials and land cover types that have been acquired in the region of Santa Barbara and Goleta, CA, USA (Herold et al., 2004).The spectra were measured on site with an ASD Full Range spectrometer, comparable to the instrument used in this study.Among others, the Santa Barbara spectral library is our preferred choice as it can be easily accessed and contains a comparably large compilation of in situ field measurements from materials that are relevant for facade classification.It fits our database most.

Collected Data
We collected field spectra of urban materials found on building facades and the near surrounding by using the high-resolution spectrometer FieldSpec 4 Hi-Res, which is an identical but updated version of the spectrometer used to collect data for the Santa Barbara dataset.The FieldSpec 4 has a spectral range of 350 to 2500 nm with a spectral sampling of 1.4 nm in the range of 350 -1000 nm and 1.1 nm between 1001 -2500 nm.The spectrometer has 2151 channels and with a wavelength accuracy of 0.5 nm.
Materials found on and near selected buildings were collected during mostly sunny conditions and one building can be seen in Figure 1.The collected dataset will from here on be referred to URBAN-SAT and was collected in an area north of the city of Karlsruhe in Germany.Some of the samples we collected can be seen in Figure 2 and the materials that we collected at the study area were: The numbers in brackets are the corresponding classes according to the classification system in (Herold et al., 2004).

Implementation
All implementation and processing was done in Matlab.From external packages, we used the CFS implementation provided with (Zhao et al., 2010) and the RF implementation provided with (Liaw and Wiener, 2002).
To evaluate the performance of our framework, we consider several evaluation metrics.Besides the overall accuracy OA indicating the overall performance of an approach, we also provide the κ-value indicating how good classes can be separated from each other.Furthermore, we focus on the average recall R, the average precision P and the average F1-score F1 across all classes.

Experiment and Results
In addition to using the full spectrum, we performed sub-band selection to determine if selected parts of the full spectrum are enough to classify materials using the two classification approaches, SVM and RF.The chosen sub-bands are: • Visible: 400 -700 nm • SWIR: 1000 -2400 nm • VNIR: 450 -900 nm These three band selections represent the spectrum of firstly the visible domain, secondly the Short Wave Infra-Red domain (SWIR) and lastly the common spectral range of Visible and Near Infra-Red (VNIR) hyperspectral sensors.The first spectral range was chosen as it is claimed that this range is not sufficient to classify materials, hence hyperspectral sensors are preferred for material classification.The second spectral range, SWIR, was picked on the basis that it is claimed to have an important contribution dealing with classification of urban materials (Le Bris et al., 2016).
The third selection was chosen since it represents the spectral range of the hyperspectral VNIR sensor Cubert Firefly S185, that will be used in future work for UAV based data collection.Hence, it is of interest to determine if the selected range is sufficient to classify building materials.
Coefficient of variation is an indicator which one can use to calculate the distribution and is expressed as percentage.It is defined by dividing the standard deviation with the mean, as seen in This indicator is in this work used to display the distribution of the spectral reflectance of the classified materials across different wavelengths.Hence, it is possible to determine which parts of the spectral domain contain the most variation.The mean spectra and the coefficient of variation spectra for the classes of the URBAN-SAT dataset can be seen in Figures 3 and 5 and for those of the Santa Barbara dataset in Figures 4 and 6.In a first experiment, we consider the URBAN-SAT dataset where the reference labeling refers to the five semantic classes Glass, Concrete, Metal, Wood and Fibre Cement.We randomly select 10 samples per class to obtain a training set, while we use all remaining samples as test set.On the one hand, we use all available spectral bands except the spectral bands that are sensitive to noise (i.e. the spectral bands in the range of 1351-1440 nm, 1798-1957 nm and > 2400 nm).As a consequence, we consider spectral information corresponding to 1801 spectral bands.Using the reflectance values corresponding to these channels delivers the feature set S all,orig , applying a PCA yields the feature set S all,PCA , and only using the spectral bands selected via CFS results in the feature set S all,CFS .In addition to the original spectra, we separately consider the 301 spectral bands that correspond to the visible spectrum in the range of 400-700 nm, which delivers the feature sets S visible,orig , S visible,PCA and S visible,CFS , respectively.Furthermore, we separately consider the 501 spectral bands that correspond to the VNIR spectrum in the range of 450-950 nm, which yields the feature sets SVNIR,orig, SVNIR,PCA and SVNIR,CFS, respectively.Finally, we consider the 1151 spectral bands that correspond to the SWIR spectrum in the range of 1000-2400 nm when discarding the spectral bands in the range of 1351-1440 nm and 1798-1957 nm.This results in the feature sets SSWIR,orig, SSWIR,PCA and SSWIR,CFS.Relying on the 12 different feature sets, we perform a standard supervised classification.The results achieved with the RF classifier and the SVM classifier are provided in Table 1.
In a second experiment, we consider the Santa Barbara dataset where the reference labeling refers to eight semantic classes which were defined by us: Composite, Gravel, Metal, Asphalt, Tile, Tar, Wood and Concrete.On the one hand, we consider the use of all available spectral information (172 spectral bands) after removing the bands which are sensitive to noise (i.e. the spectral bands in the range of 1325-1473 nm and 1783-2008 nm).On the other hand, we consider the use of only the spectral bands that correspond to the visible spectrum in the range of 400-700 nm (29 spectral bands), the use of only the spectral bands that correspond to the VNIR spectrum in the range of 450-950 nm (53 spectral bands) and the use of the spectral bands that correspond to the SWIR spectrum in the range of 1000-2400 nm (105 spectral bands) after discarding the spectral bands which are sensitive to noise.We therefore obtain 12 different feature sets in analogy to the first experiment.For classification, we again use a RF classifier and an SVM classifier.The respectively achieved classification results are provided as well in Table 1.

DISCUSSION
The main objectives of this work were to classify common materials found on buildings, by using the datasets of Santa Barbara and URBAN-SAT and to evaluate the achieved results.The added value of using dimensionality reduction and feature/band selection as well as parts of the full spectra domain were evaluated.Hence, by comparing the classification approaches, we have concluded the following.RF compared to SVM achieves better classification results for both datasets, which can be seen with the better overall accuracy and higher κ-values.SVM does achieve classification results close to RF when the full spectra domain is used, but not the sub-bands are chosen.Overall, the classification of the materials in the Santa Barbara dataset is superior to the classification of the materials in the URBAN-SAT dataset.However, the URBAN-SAT dataset compared to the Santa Barbara dataset consists of fewer samples, hence it contains a larger variation of spectra for each material.This is noticeable while studying the provided tables, since the scores using the URBAN-SAT dataset are lower than the scores generated from the Santa Barbara dataset in addition to the generally higher coefficient of variation.
The PCA-based dimensionality reduction achieves in general a higher overall accuracy than CFS and when none of them is used.The scoring also indicates that it is better to use no feature selection than to use CFS, since CFS performs in general worse.Additionally, the scores seen in the tables when the full domain is used are the best, followed by using the sub-band of SWIR.The VNIR domain achieves better classification results than the SWIR domain with the URBAN-SAT dataset, with the opposite results for the Santa Barbara dataset.However, the URBAN-SAT dataset contains in comparison to the Santa Barbara dataset more noise in the SWIR domain, which can be seen in Figure 5. Hence, the SWIR domain loses values for classifying materials with the URBAN-SAT dataset.This could indicate that the VNIR domain can be sufficient for material classification.Three materials existed in both two datasets, Metal, Wood and Concrete and are represented as gray, orange and red in the plotted graphs.One can see that the mean spectra for these three materials (in Figures 3 and 4) are not the same in the two datasets.This could be due to a variation of how the materials are defined.While both datasets contain metal, there is a variation of the spectra between different types of metal in addition to the paint and coating of the metal.In addition, the URBAN-SAT dataset consists of fewer samples and is therefore more sensible to the variations caused by for example the weather conditions and the types of metals.This variation in the URBAN-SAT dataset can be noted in Figure 5, where the coefficient of variation spectra has for some materials a value close or above 100%.Different types of building materials are used in different countries which needs to be kept in mind.In our case, the two datasets have been collected in two different countries.The type of wood which might be use as building material in the Santa Barbara region (USA) may not be the same in the Karlsruhe region (Germany).Therefore, it might not be feasible to compare the spectral reflectance of the same material from the two datasets, since they might not be the same type of material.

CONCLUSION AND OUTLOOK
In this work, we have classified building materials using the Santa Barbara dataset and our own dataset URBAN-SAT.Two classification approaches were used, SVM and RF, in addition to PCAbased dimensionality reduction and CFS-based feature selection.
The classification results indicate that RF is in general better than SVM for the two used datasets, achieving a higher overall accuracy and higher κ-values.In addition, usage of the PCA-based dimensionality reduction contributes as well to those better classification results.While comparing the three sub-bands, one can tell that the full domain provides the better values, followed by the SWIR and the VNIR domain.However, the results indicate that the VNIR domain is sufficient for material classification.
As future work, we want to significantly extend the URBAN-SAT database by having a larger variation of building materials and increasing the number of acquired samples.A spectral variation caused by small differences in the solar radiation is noticeable, as seen in results in the SWIR domain.URBAN-SAT should therefore contain material samples exposed from different intensities of solar radiation (at both cloudy and sunny conditions).This would extend the usage of the library, since the spectral reflectance is strongly affected by intensity of the solar radiation.
In general, laboratory created libraries do not reflect the reality since the data is collected with a constant intensity of solar radiation.The level of detail that describes the materials, meaning differencing for example metals, should be on a finer level than the one used in this work for an increased usage.

FigureFigure 2 .
Figure 1.One building used for data collection

Table 1 .
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3/W3, 2017Frontiers in Spectral imaging and 3D Technologies for Geospatial Solutions, 25-27 October 2017, Jyväskylä, Finland Classification results achieved for the two datasets