ASSESSMENT OF COMBINING CONVOLUTIONAL NEURAL NETWORKS AND OBJECT BASED IMAGE ANALYSIS TO LAND COVER CLASSIFICATION USING SENTINEL 2 SATELLITE IMAGERY (TENES REGION, ALGERIA)

Land cover maps can provide valuable information for various applications, such as territorial monitoring, environmental protection, urban planning and climate change prevention. In this purpose, remote sensing based on image classification approaches undergoing a high revolution can be dedicated to land cover mapping tasks. Similarly, deep learning models are considerably applied in remote sensing applications; which can automatically learn features from large amounts of data. Prevalently, the Convolutional Neural Network (CNN), have been increasingly performed in image classification. The aim of this study is to apply a new approach to analyse land cover, and extract its features. Experiments carried out on a coastal town located in north-western Algeria (Ténès region). The study area is chosen because of its importance as a part of the national strategy to combat natural hazards, specifically floods. As well as, a simple CNN model with two hidden layers was constructed, combined with an Object-Based Image Analysis (OBIA). In this regard, a Sentinel-2 image was used, to perform the classification, using spectral index combinations. Furthermore, to compare the performance of the proposed approach, an OBIA based on machines learning algorithms mainly Random Forest (RF) and Support Vector Machine (SVM), was provided. Results of accuracy assessment of classification showed good values in terms of Overall accuracy and Kappa Index, which reach to 93.1% and 0.91, respectively. As a comparison, CNN-OBIA approach outperformed OBIA based on RF algorithm. Therefore, Final land cover maps can be used as a support tool in regional and national decisions.


INTRODUCTION
Land cover mapping plays a crucial role in various applications, such as territorial monitoring, environmental protection, urban planning and climate change prevention. In this purpose, Remote Sensing based on image classification undergoing high revolution with the appearance of high spatial and spectral resolution of satellites. Therefore, remotely sensed data based on machine learning algorithms of classification is considered as an efficient way to analyse land cover and extract its features. In this context, Sentinel-2 optical products have known by their spectral and spatial information is a source of data very useful in remote sensing addressed for several applications. Sentinel-2 is launched as a part of the Global Monitoring for Environment and Security Program supported by the European Space Agency (ESA). The high resolution of S2 satellite can be presented in three types: Spatial, Spectral and Temporal resolution, each have a specific characteristic. In fact, S2 sensors can potentially address to land cover analysis. In addition, it is reported that the use of spectral indexes of the sentinel band combination plays a key role in land cover analysis, and they can reach good classifications results (Deliry et al, 2021).
Furthermore, several scientific land cover studies used Sentinel-2 as input remotely sensed data, either to extract specific features (crops, trees, forest, urban), or to produce land cover maps. These maps can help in urban planning and land management or ecosystem monitoring (Zhang et al, 2021). For example, (Novelli et al, 2016) applied Sentinel-2 similarly with Landsat 8 data to extract greenhouses, which is a very important land cover unit at the agricultural scale. (Lebourgeois et al, 2017) combined Sentinel-2 time series with others data sources to map smallholder agriculture in Madagascar. (Mousavi et al, 2019) produced a land cover map, which is used in flood assessment. (Phan et Martin, 2018) tested several machines learning algorithms on sentinel data to determine suitable land cover results. In Central Europe, based on Sentinel-2 data, (Immitzer et al, 2016) applied machines learning classifiers to Tree Species Classification. Similarly, (Niculescu et al, 2018) tested a deep learning algorithm on multi-temporal sentinel-1 images for the classification of coastal vegetation of the Danube delta.
Moreover, classification tasks in remote sensing usually processed with traditional approaches mainly pixel-based image analysis and Object-Based Image Analysis approach (OBIA), Based on machine learning algorithms. In pixel-based analysis, the classification is processed at pixel level, when classifiers used spectral information existed in each pixel (Blaschke, 2010). However, OBIA classification is based on objects. These units are provided by segmentation step with considering spectral, textural and contextual information of similar pixels (Blaschke, 2010). Therefore, due to the uncertainty and "salt and pepper" effects of pixel based image analysis, usually in regions with high spectral heterogeneity (Shendryk et al, 2019), the OBIA classification approach provided best accuracies results compared to pixel based image analysis in several applications. In result, OBIA approach in recent years is largely used in remote sensing community. For example, (Belgiu et Cssillik, 2018) found that OBIA approach superposed pixel analysis in cropland mapping. In France study area, (Yang et al, 2020) applied OBIA approach to estimate surface water extent. Achieved accuracies in these and others studies showed the potential of the OBIA approach in similar with the robustness of machine learning algorithms addressed to classification, for instance, Random Forest (RF) (Breiman, 2001) and Support Vector Machine (SVM) (Vapnik, 1995).
Otherwise, Deep Learning algorithms include methods that automatically learn features from large amounts of data and perform with large numbers of samples (Liu et al, 2018). Deep learning models are considerably applied in remote sensing applications. Among these models, the convolutional neural network (CNN), have been increasingly performed in several image classification researches, in particular land cover (Zhang, 2020;Zhao, 2020). Recently, the combination of CNN model and object-based image analysis (OBIA) classification approach is applied in multiple land cover modelling tasks (Martins et al, 2020;Zhang et al, 2020).
This study presents a simple method CNN-OBIA, combined CNN model with an OBIA classification approach. The aim is to analyse land cover and extract its properties. In this regard, a free of charge Sentinel-2 single image was used, as input data set. Similarly, by making use of spectral index combinations of Sentinel-2. The region of interest is a coastal area located in north western Algeria (Ténès region). The study also aims to compare accuracies results of land cover classification of the proposed approach with an OBIA approach experiments using two Machine Learning algorithms mainly; RF and SVM.

STUDY AREA
The study area is located in the north of Algeria; on the Mediterranean coast. Ténès is a tourist and port city surrounded by high mountains and it is the second largest city in the Chlef wilaya. A Sentinel-2 image of 1720*1365 pixels encompasses the city was chosen to implement the proposed approach detailed in the following sections. As a part of the national strategy to combat natural hazards, specifically floods, Ténès region was chosen because of its regional importance given its geographical location and presence face to natural hazard, mainly floods risk. Figure 1 shows the location of the region of interest.

DATA SET
The Sentinel-2 image used in this study was acquired on 29 October 2020 at the 1C level. The data consists of 13 spectral bands, including four bands with 10 m spatial resolution, six bands with 20 m spatial resolution, and three bands with 60 m spatial resolution. An atmospheric correction was performed with Sen2cor algorithm implemented in SNAP software and a resampling into 10 m resolution was applied, before using the image in classification process.

METHODOLOGY
The workflow of the study consists to provide a CNN model applied on spectral features combination of Sentinel-2 data, followed by an OBIA classification approach. Furthermore, an accuracy assessment of classification was performed to evaluate classification results. Figure 2 shows methodology steps of the proposed approach. In addition, in order to evaluate the performance of the proposed approach compared to other methods, an OBIA approach was applied with two machine learning algorithms; SVM and RF. All steps carried out will be detailed in following sections.

Feature extraction
Based on S2 satellite image, spectral features were extracted in order to facilitate and improve classification task. These features are mainly, the Normalized Difference Vegetation Index (NDVI), the Normalized Difference Built-up Index (NDBI) and the Normalized Difference Water Index (NDWI). Then, a combination of all spectral features was lead.
The Normalized Difference Vegetation Index (NDVI) is an effective index to evaluate vegetation class of land cover. The NDVI is calculated from the channels, red (R) and near infrared (PIR) by the formula: The Normalized Difference Water Index (NDWI) (McFeeters, 1996) is calculated using the green and Near Infrared (NIR) bands of Sentinel-2 image, based on the principle of high water absorption capacity and low radiation in the visible to infrared  image The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2021 XXIV ISPRS Congress (2021 edition) wavelength range. In most cases, the NDWI can effectively enhance water information (Du et al, 2016). This index is given by the formula: The Normalized Difference Built-up Index (NDBI) was proposed by Zha et al. (2003). The aim is to map urban built-up area automatically. This index is provided by the mid-infrared (SWIR) and near infrared (NIR) band of Sentinel-2 where builtup areas have a high reflectance in the SWIR band and a low reflectance in the NIR band. THE NDBI is calculated by the formula:

Generation of training and control samples
To perform classification process, using interpretation of spectral combination satellite image, and a very high spatial resolution images, eight land cover classes were defined: The generation of samples was provided randomly into two categories, training samples and validation samples. A set of 2380 training data and 1800 control data were created for all classes of land cover, using visual interpretation of the input image and another satellite image with a very high spatial resolution. Given that, training and validation samples were located in different areas of the image. Thus, Training data were used to produce the CNN model and complete the classification process with OBIA approach. However, control data were used in validation step and the accuracy assessment of classification results.

CNN model generation
The CNN model was applied using Trimble's eCognition Developer 10. The advantage of this software that is giving the possibility to combine a CNN model with OBIA approach to perform the classification where complex problems of image objects recognition can be solved.
As well, CNN architecture characterised by many layers giving the main: hidden layers, convolution layers, max-pooling layers and fully connected layers. Figure 3 shows a simple CNN model architecture integrated in eCognition software. Based on training vector data represent classes, a 10000 samples patches were generated for each class with a size of 32*32. Hence, using sample patches, a CNN architecture was created and trained with two hidden layers, applied with a max pooling. The aim is to reduce the number of units by keeping only the maximum response of several units of the hidden layers. This operation was applied with a convolution for each layer. The Kernel parameter value for the first hidden layer was 7 and the value for the second layer was 3. Based on trained step, applying CNN model gives a fully connected layers or a heatmap. Eight heatmaps layers were created corresponding the eight land cover categories. Heatmaps have a unit for each class that the CNN predicts with two possibilities, values close to one indicate high likelihood of the class, and values close zero indicate a low likelihood of the class.

OBIA classification
An OBIA approach was provided using the results of the CNN model (heatmaps layers), to classify the image. The OBIA approach requires a segmentation step. The principle of this process is the object generation based on spectral, textural and contextual information of pixels. For this, at pixel level, the Sentinel-2 image was segmented into homogeneous objects using Multiresolution algorithm, one of the most popular algorithms of segmentation in OBIA (Baatz et Shape, 2000).
Multiresolution is a region-growing algorithm that aggregates pixels into objects by an iterative way, in accordance with homogeneity conditions imposed by the user (Belgiu et, Csillik, 2018). Hence, heatmaps classified which are the output results of the CNN architecture process were used as features in the final step of classification.

Applying OBIA approach with Machine learning algorithms
On the other hand, to compare results of the proposed approach. An OBIA approach was applied with two Machine Learning algorithms, Random Forest (RF) and Support Vector Machine (SVM). The RF classifier was proposed by (Breiman, 2001). The RF is simple to execute and only two parameters need to be set up: the number of trees (ntree) and the number of features in each split (ntry) Moreover, the classification using the SVM algorithm is based on the linear function kernel (Vapnik, 1998). The principle of this algorithm is to find a hyperplane that separates between two classes. The values closed to the hyperplane are the support vectors. The two essential parameters of this algorithm are parameter C that controls the complexity of the classifier, and the second parameter γ, which controls the number of carrier vectors to obtain the best hyperplane (Huang et al, 2002).

Accuracy assessment
The accuracies of classifications obtained were evaluated in terms of overall accuracy, producer's accuracy, user's accuracy metrics (Congalton, 1991), and kappa coefficient (Cohen, 1960). The control vector data were used in this step, which it performed in eCognition.

RESULTS
The results of the workflow applied in this study are a land cover map of the coastal area Ténès chosen for this study. The process of classification was performed on spectral features image extracted from the Sentinel-2 satellite image. The purpose is to test a simple and a new approach in the remote sensing community. The proposed method combined a deep The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII- B3-2021XXIV ISPRS Congress (2021 learning classifier presented by CNN model with OBIA approach. However, to compare classification results with another classification method, an OBIA approach was applied simply in eCognition software with two machine learning algorithms, which are; RF and SVM classifiers. A multiresolution algorithm for the two approaches was provided in segmentation step. Land cover categories carried out in classification process are mainly: water, build-up, cropland, forest, average vegetation, roads, uncultivated land and barren land. Furthermore, classification accuracy was assessed through the confusion matrix calculated based on control samples. As well, based on confusion matrix, an overall accuracy (OA) and kappa index were calculated.

Results of CNN model creation
The CNN model applied in this study was implemented Trimble's eCognition Developer 10.1. Three steps needed to create a model, which are; model creation, model training and model applying.
As well, for the classification process, CNN model was trained using patches of a 32×32 size. The patches were randomly sampled on target image. For each of the 8 land cover categories, 10,000 samples were selected. Consequently, 10,000 × 5 = 50,000 patches were generated. The CNN model was generated using two hidden layers, which convolve the input layers using different kernels with 7×7 and 3×3 of sizes respectively. The learning rate parameter was set to 0.0015 after trial-and-error tests. Then different feature maps were and generated. Results of training CNN model are 8 heatmaps layers corresponding to land cover categories. Figure 4 shows some examples of final produced heatmaps.

Results of land cover classification
Based on CNN heatmaps layers, an OBIA approach applied on the satellite image, through multiresolution segmentation step with following parameters values: Scale = 20 Shape =0.1 Compactness = 0.5 As a consequence, results of derived classification for each approach are reported in Figure 5. The visual analysis of land cover maps, reveal to a sharper representation of the investigated area in terms of land cover classification. Hence, all land cover categories are presented in final classification results. However, proposed approach, which combined a CNN model and OBIA classification, shows successful results, where the Overall Accuracy (OA) reaches to 93.1% and the Kappa value reaches to 0.91. Moreover, in terms accuracy per class, further overall accuracy values achieved good results , where, best accuracies per class are mainly; water ( 100%), cropland (100%) and forest (98%) .
By considering machine, learning classifiers processed in OBIA approach. Results show that RF algorithm outperform SVM algorithm, in terms of overall Accuracy and Kappa index. Thus, the OA value reaches to 92.1% for RF algorithm and 79.6% for SVM and a kappa value of 0.90 for RF and 0.74 for SVM. In terms of accuracy per class, water (100%), cropland (98.1%) and uncultivated land (97.4) are well classified.
Thus, The comparison our proposed method with Machine Learning classifiers results, it is essential to note that CNN-OBIA approach is more successful and outperforms the OBIA applied with RF classifier. Results of overall accuracy and Kappa index for CNN-OBIA, OBIASVM and OBIARF are reported in Table 1, which demonstrate the high quality performance of CNN-OBIA method.

DISCUSSION
In the last decade, and given its importance on a global scale, it has become essential to model, analyse and monitor land cover. The huge progress of remote sensing applications is carried on land cover mapping. Based on image classification approaches, land cover mapping tasks, generally have often solved exploited traditional approaches such as pixel based analysis and object based analysis. These approaches have essentially founded on machine learning algorithms (Khatami et al, 2016).
Despite the application of OBIA classification approach to land cover mapping achieves a satisfactory result demonstrated in several studies (Belgiu et Csillik, 2017;Georganos et al, 2018), although, the development increasing very recently of deep learning, mainly CNN model, is considered a major leap in remote sensing field applied on multiple satellites data. Indeed, the model, has demonstrated its ability to solve image recognition problems, in particular land cover tasks. Several architectures based on CNN models are developed to answer land cover features detection. (Pan et al, 2020) investigated Land-cover classification of multispectral LiDAR (Light Detection and Ranging) data using deep CNN architecture with seven fundamental functional layers that reaches to good OA results with a value of 96.99%. (Cao et al, 2019) evaluate the performance of CNN methods for land classification and land cover and land use detection. A database of orthophotos was provided to detect land cover changes. The results of the study indicate the performance of proposed CNN architecture. Hence, (Liu et al, 2020) applied CNN model with a particular architecture named relation-enhanced multiscale convolutional network. The purpose was to delineate urban land cover classification experimented on high resolution Aerial Imagery. The results demonstrate that the proposed method can effectively detect urban land cover categories and improve the accuracy of land cover classification. The result of OA value achieved in this method was 90.46%.
Otherwise, remotely sensed community has experienced a new challenge by integration of CNN models with OBIA classification approach applied to response land cover feature detection. In addition, multiple studies demonstrated the effeteness of this method.
Compared results of the proposed method of this paper with others researches, the approach combined CNN with OBIA classification has further achieved successful results with an OA value of 93.1% and improved its effectiveness to extract land cover information. Otherwise, as demonstrated in most of the studies above, this approach outperformed machine learning algorithms. Thus, in our case, The OBIA approach based on machine learning algorithms (RF and SVM), achieved an OA value of 92.2% for the RF classifier and 79.6 % for the SVM classifier. In addition, CNN-OBIA method outperformed the RF classifier (order to 0.9 % of OA) ( Figure 6).
Furthermore, the achieved results and the important land cover information extracted through this approach confirm the performance of integrating deep learning models with a traditional approach of classification Indeed. Results can be used in future applications as a supplement of this research.

COCLUSION
In this paper, Convolutional Neural Networks (CNN) architecture combined with Object based image analysis (OBIA) has been proposed, experimented on Sentinel-2 spectral features data set. The aim is to analyse, assess land cover, extract and discriminate among the land cover classes. Experiments carried out on a coastal town located in northwestern Algeria (Ténès region). The study area is chosen because of its importance as a part of national strategy to combat natural hazards, specifically floods. As well as, a simple CNN model with two hidden layers was constructed to solve the mentioned issue, combined with an object based image analysis (OBIA) classification, to classify the full input image. Results of accuracy assessment shows best values of overall accuracy (93%). Furthermore, the comparison of CNN-OBIA approach with OBIA approach based on machines learning algorithms (RF and SVM) refers that our proposed approach outperformed and worked better than OBIA based on machine learning classifiers. In addition, due to spectral and spatial information of Sentinel-2 sensors, the study demonstrates the potential of this data source that can be conceivably addressed to land cover analysis. This paper research is considered as the first in its field that applied on Ténès study area with its such importance at a national scale. Final land cover results can be used as a decision support tool and further help in regional and national territory planning.