TIME SERIES OF IMAGES TO IMPROVE TREE SPECIES CLASSIFICATION

Tree species classification provides valuable information to forest monitoring and management. The high floristic variation of the tree species appears as a challenging issue in the tree species classification because the vegetation characteristics changes according to the season. To help to monitor this complex environment, the imaging spectroscopy has been largely applied since the development of miniaturized sensors attached to Unmanned Aerial Vehicles (UAV). Considering the seasonal changes in forests and the higher spectral and spatial resolution acquired with sensors attached to UAV, we present the use of time series of images to classify four tree species. The study area is an Atlantic Forest area located in the western part of São Paulo State. Images were acquired in August 2015 and August 2016, generating three data sets of images: only with the image spectra of 2015; only with the image spectra of 2016; with the layer stacking of images from 2015 and 2016. Four tree species were classified using Spectral angle mapper (SAM), Spectral information divergence (SID) and Random Forest (RF). The results showed that SAM and SID caused an overfitting of the data whereas RF showed better results and the use of the layer stacking improved the classification achieving a kappa coefficient of 18.26%. * Corresponding author


INTRODUCTION
Forests cover almost one third of the terrestrial surface and play an important role in biodiversity conservation and mitigation of world's climate change (Romijn et al., 2015).The identification of species and successional stage of forests helps to monitor this type of vegetation and Remote Sensing measurements appear as a promising alternative, mainly with imaging spectroscopy using hyperspectral sensors (Féret and Asner, 2013;Baldeck et al., 2015;Näsi et al., 2016;Nevalainen et al., 2017).
Miniaturized sensors have been developed to be attached to Unmanned Aerial Vehicles (UAV) for mapping purposes.These systems can acquire images with high spectral and spatial resolution, being able to be applied in classification of tree species.However, it is known that forests have a high floristic diversity with heterogeneous structure, what makes the tree species classification difficult (Féret and Asner, 2013).
Tree species present different behaviour according to the season of the year due to the flowering and appearance of fruit.These different characteristics of trees appear as a promising concept to classify tree species.Key et al. (2001) used multispectral aerial photographs to discriminate four tree species in West Virginia, USA.Somers and Asner (2014) showed the use of four different epochs of Hyperion images to classify tree species in Hawaiian forests using a spectral unmixing strategy and a feature selection process.
Two challenges when working with tropical forests, especially Atlantic Forests, are the forest structure and the spectral variability among tree species, which influence the tree species classification (Ferreira et al., 2016).In this sense, this work has the objective of evaluating the improvement of tree species classification using hyperspectral images acquired over Atlantic Forest in two different years.

Study area
The study area is a fragment of Atlantic Forest located in the northwest part of the ecological reserve named Ponte Branca (Figure 1  Starting from the access road nearby, the study area is a transect with approximately 400 m length and 50 m width.It comprises more than 1,000 trees, with diameter at breast height (DBH) higher than 3.8 cm, divided into 27 different species (Berveglieri et al., 2016).

METHODOLOGY
To discriminate the tree species, several steps were performed as follows: i) Image acquisition; ii) Dark current correction and radiometric calibration; iii) Geometric processing with bundle block adjustment; iv) Radiometric block adjustment; v) Tree delimitation and species recognition; vi) Image classification; vii) Results analysis.

Image acquisition
The Rikola Hyperspectral Camera, a hyperspectral imagery sensor developed by Senop Ltd. (http://senop.fi/) was used for image acquisition.This camera has two complementary metal oxide semiconductor (CMOS) frame sensors based on the Fabry-Pérot Interferometer (FPI) (Oliveira et al., 2016).It is able to acquire images from the visible to the near-infrared (VIS-NIR) and up to two spectral bands simultaneously.In addition, the camera can be connected to a global positioning system (GPS) and to an irradiance sensor, which does not record data in W/sr², but relative values.
A characteristic that makes this camera interesting in research applications is its capacity of selecting and acquiring different spectral bands in a range between 500 nm to 900 nm.
Regarding the spectral bands acquisition, the FPI makes it possible because each airgap of the interferometer corresponds to one wavelength with specific full width of half maximum (FWHM).The spectral bands are acquired sequentially causing a slightly different spatial position among the spectral bands, which can be corrected applying geometric transformations (Honkavaara et al., 2013).
Two models of this camera were used in this study: the first one was the 2014 model and the second one was the 2015 model.Table 1 shows the spectral settings of the cameras.
Table 1.Spectral settings of the FPI cameras The camera and its accessories were installed in an UAV octocopter, model SX8, developed by Sensormap company (http://www.sensormap.com.br/).This UAV is controlled by the DJI Wookong-M autopilot and is able to fly up to 30 minutes, depending on payload weight and weather conditions.
A Sony NEX A6000 RGB camera, a global navigation satellite system (GNSS) receiver, an inertial navigation system (INS) Novatel SPAN-IGM S1 and a Raspberry Pi B+, which was used to record the raw INS data, compose the UAV payload together with the FPI camera.Figure 2 shows the SX8 and its payload.The weather was dry and sunny, without clouds, during both flights and without rain for at least two weeks before the flights.In the 2015 flight, the average Sun zenith and azimuthal angle were 45°45'18" and 322°51'54", respectively.

Image processing
After the image acquisition, it was performed the dark current correction using a dark image acquired before the flight, and the radiometric calibration using a calibration file provided by the manufacturer.The Hyperspectral Imager, software provided by Senop Ltd, was used for both procedures.
To reconstruct the camera geometry, the Interior Orientation Parameters (IOP) were estimated using the on-job calibration, performed with AgiSoft PhotoScan.This software was also used to refine the Exterior Orientation Parameters (EOP) of three reference bands, for image orientation.The reference bands were those centred in 564.71 nm, 680.06 nm and 770.46 nm for the 2014 camera and 565.10 nm, 679.84 nm and 769.89 nm for the 2015 camera.The initial EOP values were determined by the GNSS/INS integrated system and postprocessed with the Waypoint Inertial Explorer software (http://www.novatel.com/products/software/inertial-explorer/).
After the imaging processing, the dense matching method was used to generate the digital surface model (DSM) of the area with 10 cm GSD, with AgiSoft PhotoScan.To correct the Bidirectional Reflectance Distribution Function (BRDF) and illumination variation caused by differences in the geometry of illumination and viewing during the imaging acquisition, it was applied the method proposed and presented by Honkavaara et al. (2013), Hakala et al. (2013) and Näsi et al. (2016).In this process, it is important to highlight that the irradiance data from the camera sensor were not used because it gives only relative values of irradiance.
Finally, to acquire physical values in the images, the empirical line method (Smith and Milton, 1999) was applied.Targets measuring 1.40 m by 1.40 m in black, grey and white were placed in the study area to be used as radiometric reference.
The mean hemispherical directional reflectance factors (HDRF) of the targets were 72%, 17% and 2% for the white, grey and black targets, respectively.Figure 3 shows the targets in a RGB image composition of the FPI camera (R = 650.96nm; B = 550.39nm; B = 506.22nm).
Figure 3. Radiometric reference targets for the empirical line

Image classification
After the image processing, polygons of tree crowns were manually delimited to be recognized in field.A native inhabitant of the area identified the polygons, divided into ten different species.However, as the objective of this work is to verify the contribution of temporal image series for classification of tree species, only four species were used in the analysis, because more than one sample tree is available for these species.The species were Copaifera langsdorffii (Cl), Apuleia leiocarpa (Al), Hymenaea courbaril (Hc) and Helietta apiculate (Ha).Table 2 shows how many trees were identified for each species and the average number of pixels of each crown.
Part of the crowns identified in field was selected to compose the training set.The sample selection criteria was to use parts of the trees crowns, which presented colour variation due to shadows or due to leaves moisture.Table 2 shows the number of pixels and polygons identified and used as the training sets.The average spectral response was calculated with the pixels of each polygon, which make up the sample.These average values were used as reference spectra for classification.In addition, it was used polygons which were not used in the training set for the results evaluation.These polygons were considered as reference targets and were used to calculate the confusion matrix of each classification.Table 3. Variables used in each data set The classifiers methods applied were the Spectral Angle Mapper (SAM), the Spectral Information Divergence (SID) and the Random Forest (RF).The first two were applied with ENVI version 5.1 (Exelis Visual Information Solutions, Boulder, Colorado) and the RF were applied using the ImageRF (Waske et al., 2012).All the classification results were evaluated using the reference polygons not used as training set, in the EnMAP-Box (van der Linden et al., 2015) SAM is an algorithm that determines the spectral similarity, among the reference spectra and the spectra of each pixel, by calculating their angular difference in radians (Yuhas et al., 1992).The maximum angle of separation among spectra was empirically chosen as 0.1 radians, due to the similarity of spectral responses of species.
SID was the second classifier method applied, and it measures the discrepancy probabilities among the reference spectra and each pixel (Chang, 1999).The selected threshold for maximum divergence spectra was 0.01, to minimize false positives during the classification, since there are more than 20 species in the study area.
The last classifier applied was the RF, a non-parametric algorithm that uses multiples trees of random subsets of the training data to classify the image (Breiman, 2001).One hundred trees were created using the reference spectra and the Gini Coefficient were chosen as impurity function.
After the classifications, confusion matrixes were calculated to evaluate the results, accuracy of all classes, and the improvement of classification in each group.The kappa coefficients (Cohen, 1960;Viera and Garrett, 2005) were also calculated to verify the degree of agreement between the real classes and the mapped one of each classification.

RESULTS AND ANALYSIS
Table 4 shows the confusion matrixes for SAM results.For this classifier, the kappa coefficient varied from 1.78% to 14.52% where the highest value was achieved for the DS2 data set.The DS1 obtained a kappa coefficient of 1.78% and the DS3 obtained a kappa coefficient of 7.95%.According to Viera and Garrett (2005), this value of kappa index shows a slight agreement for the classification.These results show that using images from different years did not improve the classification using SAM.Confusion among classes occurred in all data sets, especially with Al and the other classes such as Cl and Hc.These results were expected because the images were acquired in the same month, August, with similar pre-flight weather conditions, causing data overfitting.Table 4. Confusion matrix for the data sets classified with SAM Similar results can be observed in the SID results.The confusion matrixes (Table 5) for all data sets classified with SID showed classes confusions especially for the Al specie with the Hc specie.The highest value of kappa coefficient occurred for the DS1, 10.59%.The kappa coefficient for DS2 was 3.79% and the coefficient for the DS3 was 6.89%.This kappa values can be considered in a slight agreement with the ground truth, following Viera and Garret (2005).In addition, the time series of images did not improve the classification, due to the overfitting of the data, because both images were acquired in August.Considering the performance of different classifiers, it was noticed that the best classifier for this type of target was the RF, which achieves higher kappa values for all data sets.The reason is that RF uses decision trees of random vectors and it might help the classification, which does not occur when using SAM and SID, which uses all the spectral bands to calculate the divergence between spectra.
In addition, even with low kappa coefficient values, it can be remembered that only four species were used in the classification.The use of a higher number of species might help the classification accuracy.Another issue is the number of individuals for each species used in the training set.Cl was the specie most identified in field work, being the specie with higher number of training samples and validation.The more the number of samples are, more accurate is the classification result.
Finally, it was observed that the use of pixel-based classification can affect the classification accuracy.Trees are composed of leaves and branches.In forests, the understory can affect the classification.The use of region-based classification should improve the species classification as well the use of spatial information.

CONCLUSION
In this study, we presented the use of time series of images to verify the improvement of tree species classification.The tree species belong to the fragment of Atlantic Forest named Ponte Branca.Hyperspectral images were acquired by using an UAV in 2015 and 2016.The classifiers applied showed the data overfitting, except for RF technique, which achieved an improved classification result.However, it is expected that the use of images acquired in different seasons of the year, such as March or April and November or December could improve the image classification.Images acquired in these seasons can show a higher variance in the spectral responses of trees because of the characteristics of the semi deciduous trees and because of the differences in flowering and appearance of fruits.

Figure 1 .
Figure 1.Study area located in the northwest part of São Paulo State, Brazil, belonging to the Ponte Branca forest fragment.

Figure 2 .
Figure 2. SX8 UAV and its payload Three data sets, named DS1, DS2 and DS3, were classified with three classifiers each.The first data set (DS1) consisted of the mosaic from images acquired in August 2015 and all their 25 spectral bands were used for each classification technique.The mosaic obtained from the images of August 2016 compose the second data set (DS2), being used all of their 25 spectral bands for image classification.The last data set (DS3) is a layer stack performed with the mosaics from August 2015 and August 2016, generating 50 spectral bands, used in each classifier.This data set allows the comparison of the classification performance using temporal information against single date.Table3shows the information used for each data set.

Table 2 .
Average number of pixels per tree species, number of trees recognized in field and used for the training set and, number of pixels used as training set

Table 5 .
Confusion matrix for the data sets classified with SIDThe RF presented highest values of kappa coefficient when compared to SID and SAM.The higher coefficient value was founded for the DS3 dataset.The values for DS1, DS2 and DS3 with RF were, respectively, 16.33%, 17.29% and 18.26%.These values still present a slight degree of agreement(Viera  and Garret 2005), but, even though, showed better results.Analysing the results it is possible to observe that the temporal information, layer stacking of images from 2015 and 2016, helped to improve the classification.Even being acquired in the same month, August, classification with these presented a higher value of kappa coefficient in the DS3.This result can be explained because the RF is insensitive to overfitting(Belgiu  and Dragut, 2016)and the classification was improved.In order to have a better understanding of the results, the confusion matrixes calculated for each data set classified with RF is presented in Table6.

Table 6 .
Confusion matrix for the data sets classified with RF In general, it was noticed confusion in most part of the species.