FUSING OF OPTICAL AND SYNTHETIC APERTURE RADAR ( SAR ) REMOTE SENSING DATA : A SYSTEMATIC LITERATURE REVIEW ( SLR )

Remote sensing and image fusion have recognized many important improvements throughout the recent years, especially fusion of optical and synthetic aperture radar (SAR), there are so many published papers that worked on fusing optical and SAR data which used in many application fields in remote sensing such as Land use Mapping and monitoring. The goal of this survey paper is to summarize and synthesize the published articles from 2013 to 2018 which focused on the fusion of Optical and synthetic aperture radar (SAR) remote sensing data in a systematic literature review (SLR), based on the pre-published articles on indexed database related to this subject and outlining the latest techniques as well as the most used methods. In addition this paper highlights the most popular image fusion methods in this blending type. After conducting many researches in the indexed databases by using different key words related to the topic “fusion Optical and SAR in remote sensing”, among 705 articles, chosen 83 articles, which match our inclusion criteria and research questions as results ,all the systematic study ‘ questions have been answered and discussed . * Corresponding author


INTRODUCTION
The use of image fusion techniques has been increased recently in multiple and variant fields in remote sensing as a result of the continuous improvement of remote sensors.The researchers (Van Genderen and Pohl 1994)have defined image fusion as "Image fusion is the combination of two or more different images to create a new image by using a certain algorithm".Briefly, the main purpose of image fusion is to reach a better and accurate image (output), by integrating and combining different data, and this output is required in many remote sensing's applications.There are so many fusion images types in remote sensing such as; fusion multispectral MSI and panchromatic PAN which called Pan sharpening Fusion (Xing et al. 2018), fusion of Hyperspectral HS and multispectral MSI named Spatio-temporal Fusion (Wei et al. 2015), fusion of synthetic aperture radar (SAR) imagery and optical images (McNairn et al. 2009), and synthetic aperture radar SAR images Fusions such as multi-frequency, multi-polarization , Multi-resolution (Simone et al. 2001), (Turkar and Deo 2014), (Shimoni et al. 2009) , etc.In this survey paper, we only focused on fusion of optical and synthetic aperture radar (SAR), this blending can get advantageous information; by inheriting the both complementary characteristics from the different sensors and obtain a better description of the information and can lead to improved classifications accuracy.SAR images have several advantages like all-weather and day and night acquisition capabilities, sensitivity to dielectric properties surface roughness and it is capable to penetrate clouds.Optical images are rich in spectral and spatial information.After launch the first ERS-1 SAR sensor in July 1991, we noticed many papers that have been studied the fusion of optical and SAR data like Landsat TM/ETM+ and ERS-1/2, SPOT-4/5 and ERS-1/2, and Landsat TM/ETM+ and ALOS PALSAR data to improve land use/land cover classification (Zhang et al. 2018).The fusion of optical and SAR images carries out at three processing levels Depending on the processing phase at which the fusion happen: Pixel level, Feature level and Decision level (Pohl and Van Genderen 1998) , pixel level means all the fusion processing takes place by the blending the pixels of the images which don't need to feature extraction, it only require to geocoding and co-registration between the datasets before the fusion Although many studies pointed out that this level is unsuitable for fusion of optical and SAR images due to speckle noise of SAR images and the different electromagnetic spectrum between optical and SAR data (Zhang et al. 2010) (Zhang and Xu 2018).Feature level based on features extraction, such as the Grey level co-occurrence matrix (GLCM), textures measures and segmentation procedures, finally, Decision level or interpretation level is the highest level which based on voting the best results from more than one classifier and integrating this multiple findings to produce the final decision (Zhang et al. 2009).Numerous articles were related this topic by a group of respectful researchers in which they applied the fusion of optical and SAR in several applications; e.g.estimating forest biomass (Choi and Sarker 2013) (Zhao et al. 2016), road network extraction (Khesali et al. 2016), assessment of water body structure (Hunger et al. 2016) ,earthquake damage assessment (Tamkuan and Nagai 2017) , land use/ land cover classification (Sukawattanavijit et al. 2017), maritime monitoring (Liu et al. 2015), shoreline extraction (Abd Manaf et al. 2015) and Flood detection (Ward et al. 2014).This present research is arranged as follow: The methodology and the processing steps of the systematic review are presented in section 2. The results and findings of this topic are demonstrated in the section 3. Discussion and interpretation besides some recommendations that can help in improving the fusion images techniques moreover highlighting the best methods which were proven to be efficient to use in remote sensing are proposed in section 4. Finally, the conclusion is in the section 5.

Protocol and definition of systematic literature review
In this work we pursue a systematic literature review approaches in which we conducted the peer-published articles that investigate the objective behind the use of image fusion methods in remote sensing.This sort of study resembles the previously published studies related to remote sensing's topic (de Araujo Barbosa et al. 2015) (Hoque et al. 2017).The entitled book "Cochrane Handbook for Systematic Reviews of Interventions" (Higgins and Green 2005) has defined systematic review as follow "a systematic review is a review of a question that has been clearly formulated and uses systematic and clear methods to identify, select and critically evaluate relevant research, and collect and analyze data from the studies which are included in the review."In addition, in the selection of the articles, we have adopted and followed the guidelines of Preferred Reporting items for Systematic reviews and Meta-analyses: the PRISMA statement (Moher et al. 2009), to sum up briefly the methodology of this research composed by the following essential steps: formulating the research questions, setting up the standards and criteria in the selection of the articles as mentioned before we used PRISMA methods (Moher et al. 2009), and then study and present this finding data.

Formulating the research questions.
The definition of the research questions is the first step to conducting systematic literature review because they addressed the problem that we are looking forward to solve and assess it, and they must be answerable by selection of articles.The table below mentioned the questions of this study (Table 1).

Brief description of process and the finding data
The practical steps consist of this research were as follow: firstly, we searched into electronics indexed databases by using the defined key-words.These Quires words assemblage all articles that related to our topic 'Fusion of Optical and synthetic aperture radar SAR image'.Secondly, we arranged all the Citations articles in Reference Manager Endnote, we found 3495 articles after that we removed the duplicates articles and transfer them to excel with all article's details (Author, Year, title, abstract, etc) and then we screened them by using the researches questions and PRISMA method.As the diagram below shows all analysis steps (Figure 1).

RESULTS AND DISCUSSIONS
In this research, we are going to answers all the questions aforementioned in the last section: Q1: what are the research studies conducted on data fusion of optical and radar image in remote sensing during this period?
Through study duration (2013 -2018), we found 705 papers, 83 were only include which that underwent to the inclusion criteria.The articles excluded were papers related only to pre-fusion process as Co-Registration, those focused on altimetry for instance Multi-angular fusion, and also papers were not clear.

Q.2: what are the main objectives of remote sensing data fusion?
The main objectives of remote sensing data fusion (our case optical image and microwave radar images) is tow benefits; on one hand, using the rich information content optical images and on the other hand, using the all advantages and capabilities of the synthetic aperture radar SAR images such as penetrate the cloud and the full-time acquisition data.

Q.3: In which fields or applications the fusion of optical and SAR was used?
The table below shows in details the application fields of the optical and SAR fusion selected articles, 31 papers studied the Land Cover and Land Use classification and mapping which are certainly including; forest and vegetation monitoring ,urban ecosystem service mapping (Haas and Ban 2017), wetland vegetation mapping (Fu et al. 2017), protected area mapping (Otukei et al. 2015) and landscape diversity assessment (Kuchma 2016).In spite of the majority of fields such as change detection, Forest biomass estimation, mangrove communities and differentiate grassland area belongs to Land Cover and Land Use class; we separated them for more clarity and detail.Five papers investigate the fusion of SAR and optical for change detection application like (Yousif and Ban 2017).Then, two papers used the fusion for estimate earthquake damage assessment (Piscini et al. 2017), two papers used the blending for assessment of water body structure (Hunger et al. 2016), three papers worked on the Forest aboveground biomass estimation (Zhao et al. 2016), three articles proposed a new blending method for Road network extraction (Perciano et al. 2016), two papers for flood detection (Ward et al. 2014), two papers for hotspot detection (Ahmed et al. 2013),four articles investigated the improving in impervious surface estimation (Zhang et al. 2014), 21 articles were not Classified which could either be comparative studies or development papers and 8 papers distributed on different aspects, (Table 3) and (Figure 2) those including ; mangrove communities (Kumar et al. 2017a), Paleochannel delineation (Islam et al. 2016), lithological and structural mapping (Pour and Hashim 2015), detection of environmental hazards (Errico et al. 2015)  The pre-processing steps are depended on choice of a suitable fusion level.In case of pixel level image fusion, it consists of the following steps: Firstly, rectification the SAR and optical images to map coordinates by using (GCP), ground control points, and the root-mean-square error must be ± 1 pixel.Then, hence, the filtering the SAR image to reduce the speckles noise by using either a suitable window & kernel size of Gamma-MAP filter or Enhanced Lee Filter, those filters which are the most commonly-used, in addition to the choice of kernel sizes values should be based on the resolution and landscape of the study area, secondly, the geo-coding step of SAR image by using the digital elevation model after that, the SAR image should be registered to the optical image with less than ±1 pixel RMS accuracy (Zhang and Xu 2018)and (Pohl and Van Genderen 1998).In case of the Feature and decision levels, each type of image takes place process steps individually until feature extraction.The most popular combinations sensors used in these studies are Landsat with ALOS PALSAR sensors, 11 papers which were used this combination then, all of the following combinations had 3 papers; Landsat with RADARSAT-2 , Landsat with Envisat-Asar, Landsat with Terrasar-X, IKonos with Terrasar-X, Quickbord with Terrasar-X, Sentinel-2 with Sentinel-1, Sentinel-2 and ALSO AVNIR -2 with Sentinel-1 and ALOS PALSAR, all the rest of studies has either one or two combination sensors.The resolution sensors of the selected papers are widely varied according to the using of optical and SAR image fusion for instance the study of the earthquake damage assessment and road network extraction needed very high-resolution image (Table . 4) and (Fig. 3).In contrast, forest aboveground biomass estimation and other applications required less resolution than the previous.The table above shows the resolutions that which has been used in this study.For the radar images 23 papers have used very high resolution -(between 4 and 0), followed by high resolution-(between 4 and 15), and then, both medium resolution and studies have different multi resolution.For the optical 32 papers have used Medium Resolution, followed by very high resolution then, high resolution, in addition to 14 papers have different multi resolution.The majority of studies (52 of 84) were static (Table 6) and have been worked in one date for several applications such as land use/ land cover mapping, road network extraction, and forest aboveground biomass estimation.On the other hand, 10 studies were multi-temporal studies worked for the following applications; change detection, flood detection and earthquake damage assessment in addition to 21 studies were not classified such as a comparative study (Table 5).As aforementioned, the methods of image fusion in remote sensing data can be categorized into three levels: Pixel level, feature level and decision level (Table 7) (Pohl and Van Genderen 1998).In this surveys paper, we arranged the methods of fusion for the selected researches into the four prior levels: 49, 19, 8, and 7 papers, which are pixel level, feature level, decision level and hybrid level respectively.
Figure 4: Types of combinations of satellite images used in fusion The Figure 5 shows the numbers of studies with each level from 2013 to 2018.Until 2016 the pixel level was the most commonly used, but after that, in 2017 and 2018 was reduced while the feature and decision methods have been increased, despite the fact that many studies have been used pixel level that considered as an unsuitable for fusing optical data and synthetic aperture radar (SAR) because of speckle noise in radar data.Hence, feature and decision level methods are suitable and more profitable for fusing radar and optical data.That's what has been achieved in the last two years (Karimi et al. 2017).2016).Among 49 publications of pixel level, we found 32 studies involved these four families of the traditional method.So, in the following section, we will present the most popular techniques of fusion optical and radar data belongs to each type of traditional method families.Firstly, for Component Substitution (CS) family; the most commonly techniques which have been employed for fusion of optical and SAR images are Intensity-Hue-Saturation (IHS),Gram-Schmidt (GS), Principal Component Analysis (PCA), and Brovey Transform (BT).Secondly, Multi-resolution analysis (MRA) includes: High-Pass Filter (HPF), a trous wavelet and wavelet-based Discrete Wavelet Transformation (DWT).For the third type (hybrid method), there is Elhers while the last traditional family (model) was present through two models; Hierarchical Markov Random Fields and Markov random field.Numerous papers have been proposed these families and compare them efficiency for fusing of optical and synthetic aperture radar (SAR) images with a view to improving remote sensing application accuracy.The table 6 shows a few comparison papers for the traditional methods and their conclusions.According to the previous comparison studies, we conclude that PCA is the most preferred method of traditional methods for many researchers' papers, but there are some problems with these methods such as the foreshortening, shadowing and layover.(Otukei et al. 2015) reported that it is better to explore object based analysis methods to fuse optical and SAR images for avoiding the effects of relief on the classification accuracy of image fusion of traditional methods that was caused by the foreshortening, shadowing and layover (Otukei et al. 2015).Beside to the efficiency of the traditional methods is varied from study to another due to several reasons: the type of application field, the quality of the experiment data; the shape of the study area (Pal and Mather 2006), (Foody and Mathur 2006) and (Joshi et al. 2016).The second part of the pixel level is Fusion Indices.Six papers have been achieved depending on the previous types (Kim et al. 2017) treated to fusing hyperspectral vegetation index (HVI) and radar vegetation index (RVI) to support vegetation monitoring.(Qin et al. 2016) fused NDVI with (HV, Difference HH-HV, and Ratio HH/HV ) radar data to generate annual forest distribution map, (Havivi et al. 2016) merged the Coherence Change Detection (CCD) of the SAR images and Covariance Equalization (CE) Change detection for multispectral for emergency response estimation.(Ward et al. 2014) used the normalized difference infrared index (NDII), the normalized difference water index (NDWI) and its modifications (MNDWI), and Scan SAR HH to flood detection and mapping.(Mishra and Susaki 2014) have fused normalized difference ratio (NDR) from SAR images and difference vegetation index difference (NDVI) from optical images for land use/land cover change detection mapping.(Ahmed et al. 2013) used Vegetation indices (optical),Global Environment Monitoring Index GEMI ,Purified Adjusted Vegetation Index PAVI and polarimetric indices SAR (CPR, HV/HH and HV/VV) to detect the subsurface hotspots.The third part of pixel level is two articles based on Hierarchical Markov Random Fields models (Hedhli et al. 2015(Hedhli et al. , 2017)), and finally nine papers applied others different methods including layer stacking (Sameen et al. 2016), Genetic algorithm image fusion technique (Ahmed et al. 2016), multi-scale decomposition and sparse representation (Zhouping 2015), the combination method band 3, band 7 of Landsat ETM+ with a modified HH polarization of SAR image (Xiao et al. 2014) , Closest Spectral Fit (CSF) algorithm with the synergistic application of multi-spectral satellite images and multi-frequency Synthetic Aperture Radar (SAR) data.(Eckardt et al. 2013), applied learning Artificial Neural Network at pixel level ANN (Piscini et al. 2017), these three typical manifold learning ; ISOMAP, Local Linear Embedding (LLE), principle component analysis (PCA) (Zhang et al. 2018) and two papers the first are not clear and the last without fusion method.

CONCLUSIONS
Image fusion was become very important to improve the resolution of the satellite images throughout the recent years, especially fusion of optical and synthetic aperture radar (SAR).This study mainly provides a systematic literature review (SLR) of the image fusion in remote sensing based on the pre-published articles on indexed database related to this subject and outlining the latest techniques as well as the most used methods.Among 705 articles chosen, 83 have been undergone to inclusion criteria of this systematic review.In this surveys paper, we arranged the methods of fusion for the selected researches into the four prior levels: 49, 19, 8, and 7 papers, which are pixel level, feature level, decision level, and hybrid level respectively.This study shows that PCA is the most preferred method of traditional methods for many researchers.Also, the suitable levels for fusion optical and SAR images are Feature level and Decision level because of the effects of the speckle noise of the radar image which leads to a lot of issues such as foreshortening, shadowing and layover as for pixel level can beneficial in one case when we use Machine learning based methods.

Figure 1 :
Figure 1: Methodology of work (the PRISMA diagram)

Q. 5 :
what are the pairs of combination sensors that have been used in these studies?what are the most popular combinations?and what are their spatial resolutions?

Fig. 2 :
Fig. 2: Diagram of the resolution types Q.6: Did this integration of studies static or multitemporal?And what applications which have worked for them?

Fig. 3 :
Fig. 3: Classification of studies Q.7: what is the suitable fusion level in this type of data fusion?

Figure 5 :
Figure 5: The numbers of studies with each level from 2013 to 2018

Table 3 :
Applications fields Q.4: Are there pre-processing steps required for data fusion?And what are these steps?

Table 6 :
Conclusion and comparison of processing levels