FUSION OF SENTINEL-2 AND PLANETSCOPE IMAGERY FOR VEGETATION DETECTION AND MONITORING

Different spatial resolutions satellite imagery with global almost daily revisit time provide valuable information about the earth surface in a short time. Based on the remote sensing methods satellite imagery can have different applications like environmental development, urban monitoring, etc. For accurate vegetation detection and monitoring, especially in urban areas, spectral characteristics, as well as the spatial resolution of satellite imagery is important. In this research, 10-m and 20-m Sentinel-2 and 3.7-m PlanetScope satellite imagery were used. Although in nowadays research Sentinel-2 satellite imagery is often used for land-cover classification or vegetation detection and monitoring, we decided to test a fusion of Sentinel-2 imagery with PlanetScope because of its higher spatial resolution. The main goal of this research is a new method for Sentinel-2 and PlanetScope imagery fusion. The fusion method validation was provided based on the land-cover classification accuracy. Three land-cover classifications were made based on the Sentinel-2, PlanetScope and fused imagery. As expected, results show better accuracy for PS and fused imagery than the Sentinel-2 imagery. PlanetScope and fused imagery have almost the same accuracy. For the vegetation monitoring testing, the Normalized Difference Vegetation Index (NDVI) from Sentinel-2 and fused imagery was calculated and mutually compared. In this research, all methods and tests, image fusion and satellite imagery classification were made in the free and open source programs. The method developed and presented in this paper can easily be applied to other sciences, such as urbanism, forestry, agronomy, ecology and geology.


INTRODUCTION
Europe is the highly urbanised continent with a slow but steady degradation of urban vegetation.More than two-thirds of the European population live in urban areas.Naumann et al. (2011) have shown that green infrastructure (GI) is a network of natural and semi-natural areas, features and green spaces in rural and urban areas.GI provides various benefits such as environmental (removal of air pollutants), social (better health and human wellbeing, enhanced tourism and recreation opportunities), adaptation and mitigation to climate change.Today, GI faces harsh growing conditions with heavy traffic patterns and pollution as well as a restriction to water due to increased urbanisation and poor drainage conditions.Konijnendijk et al. (2005) reported that the vitality of urban trees falls drastically during the last 30-40 years to an average lifespan of a newly planted tree as low as 7-15 years.
Different spatial resolutions satellite imagery with global almost daily revisit time provide valuable information about the earth surface in a short time.Based on the remote sensing methods (e.g.classification, fusion etc.) satellite imagery can have different applications like environmental development (Gašparović et al., 2018c), urban monitoring (Gašparović et al., 2017), forestry management (Hermosilla et al., 2015), hydrology (Donlon et al., 2012) etc.For accurate vegetation detection and monitoring, * Corresponding author especially in urban areas, spectral characteristics, as well as the spatial and temporal resolution of satellite imagery is important.
Planet, an aerospace company, builds and operates the largest constellation of small imaging satellites PlanetScope (PS), also named Cubesat (Houborg and McCabe, 2018a) or Dove (Asner et al. 2017).Planet operates with more than 175 PlanetScope and collects multispectral (MS) imagery in 4 bands with a spatial resolution of 3.7 m and a collection capacity of 300 million square km per day.PS imagery is used for many scientific purposes: McCabe et al. (2017) used PS for vegetation dynamics monitoring; Traganos et al. (2017) used PS for seagrass detection, Gašparović et al. (2018b) used PS for urban vegetation detection and Shi et al. (2018) used PS for mapping damage from rice diseases.Fusion of PS imagery with other satellite data was researched by Houborg and McCabe (2018b) and Kwan et al. (2018).Gašparović and Jogun (2018) researched the effect of fusing Sentinel-2 (S-2) imagery on land-cover classification accuracy.Previously mentioned authors purpose interesting fusion approach based on the synthesised band calculated by averaging of 10-m bands 4 and 8. Inspired by research Gašparović and Jogun (2018), a method for fusion of Sentinel-2 and PlanetScope imagery was developed and described in detail below.
The fusion method validation in this research was provided based on the land-cover classification accuracy and compared with land-cover classifications provided separately based on input Sentinel-2, as well as, PlanetScope imagery.All three land-cover classifications were made based on the same supervised classification method, a random forest (RF) classifier.RF is a very popular machine learning algorithm for image classification used in many research as Rodriguez-Galiano et al. (2012), Ahmed et al. (2015) and Gašparović et al. (2018b).

STUDY AREA AND DATA
This research was provided in the capital city of Croatia, Zagreb.Zagreb is at an elevation of 122 m above sea level with an area of 641 square km.The city of Zagreb consists of several protected green areas like Medvednica Nature Park, Park Maksimir, as well as a significant number of small parks and recreation zones with developed and cultivated GI like green areas around Jarun lake.For this research central urban, eastern and southern lowland parts of the city were taken into consideration with an area of 125 square km (11.2 km x 11.2 km).The study area is surrounded by a Medvednica mountain on the north and river Sava on the south (Figure 1).For easier results representation on a larger scale, the example subset is defined.Example subset is located in surrounding of Jarun lake.In that area are located a lot of cultivated GI with low and high urban vegetation, as well as build-up, water and bare land locations.

METHODS
This section explains all the methods used in the research.Figure 3 shows the research workflow.
Figure 3.The research workflow

Preprocessing of satellite images
Preprocessing of Sentinel-2 imagery was performed according to the Level-2A algorithm in Sen2Cor (version 2.2.1) with Sentinel Application Platform (SNAP, version 5.0.0).Because S-2 imagery was georeferenced in the WGS 84 UTM 33N coordinate system (EPSG code: 32633), PS imagery was transformed to the same coordinate system, and each band was extracted to separate file.Further, both imagery was clipped to the study area.

Image fusion
Although, in nowadays research Sentinel-2 satellite imagery is often used for land-cover classification or vegetation detection and monitoring in rural, as well as in urban areas, we decided to test a fusion of Sentinel-2 imagery with PS because of its higher spatial resolution.The main goal of this research is development of the new method for Sentinel-2 and PlanetScope imagery fusion.Based on the previous research (Gašparović and Jogun, 2018) it is decided to use variational (P + XS) fusion method.The P + XS method introduces the geometry information of the higher resolution image by aligning all edges of the higher resolution image with each lower resolution multispectral band.To obtain the spectral information for the fused image, the method assumes that images taken in different spectral bands share common geometric information and that the higher resolution image can be approximated as a linear combination of the high-resolution multispectral bands (Ballester et al., 2006;He et al., 2012, Gašparović andJogun, 2018).Based on the previous research (Gašparović and Jogun, 2018), each S-2 band, was fused with a high-resolution PS band with similar spectral characteristics (Figure 2).Therefore, 10-m S-2 bands 2, 3, 4, 8 are fused based on the high-resolution PS bands 1, 2, 3, 4, respectively.Further, 20-m S-2 band 8A are fused based on the PS band 4. Spatial emphasis was given to the fusion problem of 20-m S-2 bands 5, 6, 7. Accordingly, to Gašparović and Jogun (2018) S-2 bands 5, 6, 7 are fused base on the synthesised band (S) given by the equation: where B3 and B4 represent PS band 3 and 4, respectively.Image fusion process of was conducted with the use of open-source software Orfeo ToolBox (OTB) version 6.0.0.OTB algorithm for image fusion was accessed from Monteverdi.

Land-cover classification
The fusion method validation was provided based on the landcover classification accuracy and compared with land-cover classifications provided separately based on input Sentinel-2, as well as PlanetScope imagery.All three land-cover classifications (S-2, PS and fused imagery) were made based on the same supervised classification method, a random forest (RF) classifier.
The RF classifier is a combination of tree predictors where each tree depends on the values of a random vector sampled independently from the input vector and with the same distribution for all trees in the forest (Breiman, 2001).Training polygons for the classification were manually selected based on the satellite imagery, randomly and equally for each class.For this research land-cover was divided into five classes ( For land-cover classification accuracy assessment, the reference polygons were chosen.Reference polygons are manually selected without spatial overlapping with training polygons.Totally 450 polygons were collected as reference polygons, with a share of a ~0.5% of the total area of the study area.Accuracy assessment of land-cover classification was calculated based on the confusion matrix.Confusion matrix shows class types determined from reference source in columns, and class types determined from the classified map in rows.Diagonals represent elements classified correctly according to reference data, while off-diagonals were misclassified.Overall accuracy is defined as a sum of the diagonal elements divided by a total number of elements.Besides the overall accuracy, within the confusion matrix, the kappa coefficient can be analysed.The kappa coefficient is a measure of overall statistical agreement of an error matrix, which takes non-diagonal elements into account.Kappa analysis is recognised as a powerful method for comparing the differences between various error matrices (Gašparović et al., 2018b).

Urban vegetation monitoring
For the vegetation monitoring testing, the Normalized Difference Vegetation Index (NDVI) from Sentinel-2 and fused imagery was calculated and mutually compared.This is important for the further research on projects GEMINI (Geospatial monitoring of green infrastructure using terrestrial, airborne and satellite imagery) and 3D-FORINVENT (Retrieval of Information from Different Optical 3D Remote Sensing Sources for Use in Forest Inventory) founded by Croatian Science Foundation, especially for detection and monitoring of urban vegetation as one of the most important factors of life quality in cities. Land-cover classification and urban vegetation monitoring were made in the open source software Quantum GIS (version 2.18.16),GRASS GIS (version 7.2.1) and SAGA GIS (version 6.2.0).

RESULTS
As mentioned in previous section, three land-cover classifications (S-2, PS and fused imagery) were made based on the same supervised classification method (RF) and same 150 training polygons.For better visual analysis land-cover classification results was shown on 3700 m x 1850 m example subset located near Jarun lake. Figure 4 shows example subset "true-colour" composite of S-2 and fused imagery.Further, all three land-cover classifications are shown on figure 5.As expected, figure 5 shows huge improvement in land-cover classifications made by PS and fused imagery in comparison to the S-2 imagery.Further, a detailed visual analysis of land-cover classification made base PS, and fused imagery, as well as comparison with satellite imagery with higher spatial resolution (e.g.WorldView-2), was made.From that visual analysis, the boundary of the forest, as well as low vegetation, is better defined by land-cover classification based on the fused imagery than the PS.In the PS-based land-cover classification class forest overemphasised low vegetation in comparison to land-cover classification based on fused imagery.Furthermore, class boundaries in land-cover classification based on the fused imagery are more natural, realistic, and have better coincide with higher spatial resolution satellite imagery than PS based landcover classification.
As an objective quality measurement of the land-cover classifications, accuracy assessment based on the 450 reference polygons was calculated.Table 2, 3 and 4 show confusion matrices, user's and producer's accuracy, kappa coefficient (κ), overall accuracy (OA) and the sum of pixels in row and column of confusion matrices (Σ) for land-cover classifications.
Accuracy assessment was calculated for the entire study area with dimensions 11.2 km x 11.2 km (Figure 1).As expected, tables 2, 3, 4 show better accuracy for PS and fused imagery than S-2.If we compare in detail PS and fused imagery it is obvious that overall accuracy (OA) and kappa coefficient (κ) are slightly higher for the fused imagery than the PS imagery.Accuracy per classes are also slightly higher for the fused imagery than the PS, but in fact, that is almost negligible.
For our research important information is the area of a particular class.Figure 6 shows the pertinent share of a class in the entire study area based on the three land-cover classifications.Figure 7 shows that NDVI value S-2 and fused imagery is very similar but in huge difference in spatial and temporal resolution.
It should be noted that the S-2 NDVI has 10-m spatial resolution and about 6 days temporal resolution, while the fused imagery we can have almost every day (1-day temporal resolution) and in a 3.7-m spatial resolution.

CONCLUSIONS
The importance of protected GI areas especially in urban locations are continuously growing.In this research is presented a methodology for vegetation detection and monitoring in an urban location, in Zagreb.The focus of this research is the fusion of Sentinel-2 and PlanetScope satellite imagery.Three landcover classifications were made based on the S-2, PS and fused imagery.As expected, results show better accuracy for landcover classification based on the PS and fused imagery than the S-2 imagery.PS and fused imagery have almost the same accuracy, but it can be seen that fused imagery have slightly higher accuracy than the PS imagery.The share of the class in the entire study area based on three independent land-cover classifications are almost equal.NDVI was used for the vegetation monitoring in this research, and it should be noted that NDVI value for S-2 and fused imagery was very similar.Entire research was provided based on the open source software (SNAP, OTB, GRASS GIS, Quantum GIS, SAGA GIS).The method developed and presented in this paper can easily be applied to other sciences, such as forestry, agronomy, urbanism, ecology and geology.

Figure 2 .
Figure 2. Overview of the Sentinel-2 and PlanetScope spectral bands used in this research

Figure 6 .
Figure 6.The share of land-cover classes in the entire study area for the classification based on the S-2, PS and fused imagery For the vegetation monitoring process, NDVI indices for S-2 and fused imagery are calculated.On figure 7 vegetation indices are shown.

Figure 7 .
Figure 7. NDVI value of the example subset based on the: (a) S-2 and (b) fused imagery

Table 3 .
Confusion matrix for land-cover classification based on the PS imagery

Table 4 .
Confusion matrix for land-cover classification based on the fused imagery