THE RESEARCH ON DRYLAND CROP CLASSIFICATION BASED ON THE FUSION OF SENTINEL-1 A SAR AND OPTICAL IMAGES

In recent years, the quick upgrading and improvement of SAR sensors provide beneficial complements for the traditional optical remote sensing in the aspects of theory, technology and data. In this paper, Sentinel-1A SAR data and GF-1 optical data were selected for image fusion, and more emphases were put on the dryland crop classification under a complex crop planting structure, regarding corn and cotton as the research objects. Considering the differences among various data fusion methods, the principal component analysis (PCA), Gram-Schmidt (GS), Brovey and wavelet transform (WT) methods were compared with each other, and the GS and Brovey methods were proved to be more applicable in the study area. Then, the classification was conducted based on the object-oriented technique process. And for the GS, Brovey fusion images and GF-1 optical image, the nearest neighbour algorithm was adopted to realize the supervised classification with the same training samples. Based on the sample plots in the study area, the accuracy assessment was conducted subsequently. The values of overall accuracy and kappa coefficient of fusion images were all higher than those of GF-1 optical image, and GS method performed better than Brovey method. In particular, the overall accuracy of GS fusion image was 79.8%, and the Kappa coefficient was 0.644. Thus, the results showed that GS and Brovey fusion images were superior to optical images for dryland crop classification. This study suggests that the fusion of SAR and optical images is reliable for dryland crop classification under a complex crop planting structure. * Corresponding author


INTRODUCTION
The classification of crops is generally considered as the initial phase and crucial link of the agricultural condition monitoring system (Jia et al., 2012).Moreover, the accurate classification information is one of elementary scientific bases for estimating crop sown acreage and yield, meanwhile providing a reference for the related agricultural policies and economic plans.Previously, optical remote sensing images are widely used in agricultural condition monitoring, showing the advantages of high temporal resolution, wide coverage and low cost.However, optical images are always affected by the cloudy and rainy weather, inevitably inducing the difficulties in the obtaining process during the key growth periods of crops.Hence, the insufficient monitoring capabilities of optical remote sensing images have negative effects on the accuracy and timeliness of crop classification.In recent years, as the beneficial complement for traditional optical remote sensing, the synthetic aperture radar (SAR) performs stronger support capabilities for crop classification with the advantages of all-weather, all-time, high resolution and wide coverage (McNairn et al., 2014;Silva et al., 2009;Stankiewicz, 2006).As the updating and improvement of radar sensors, more and more researchers put emphasis on the combination of SAR and optical data in the aspects of theory and technology, intending to the improve the accuracy and efficiency of crop classification (Blaes et al., 2005;Dong et al., 2013).SAR data have different characteristics from optical remote sensing data in both physical properties and imaging mechanism.Compared with the character-level and decision-level image fusion, data-level fusion methods used to be conducted to reserve more original information, contributing to the visual interpretation and information extraction.However, the data-level methods call for a high-precision calibration and registration between multi-source images, and the huge amounts of data in the fusion process also induce a high cost in performance.Hence, the contrastive analysis should be conducted on various data-level methods from the aspects of accuracy and efficiency, intending to obtain greater quality of crop classification information from the fusion images.Previously, most researches put emphasis on the acreage estimation and growth monitoring of rice, owing to the relatively apparent difference of backward scattering coefficients between rice and its symbiotic crops (Bouvet et al., 2009;Haldar and Patnaik, 2010).In contrast, it's difficult for the classification of dryland crops with SAR data to satisfy the application requirements.The dryland crops used to grow under a complex planting structure, and the same growing season among various species brings more difficulties to the issue of classification.
In this study, considering the differences among various data fusion methods, the contrastive analysis was conducted among principal component analysis (PCA), Gram-Schmidt (GS), Brovey and wavelet transform (WT) methods.According to the results of performance evaluation, the applicable fusion method for dryland study area was proposed, intending to realize the fusion of Sentinel-1A SAR and GF-1 optical images.Moreover, the classification of corn and cotton in dryland area was regarded as the research object, which was also used to verify and compare the classification accuracy between fusion image and original optimal image.In order to eliminate the interference factors, the widely used nearest neighbour supervised classification was adopted to realize the classification with the same training samples.This study intends to provide a reference for improving the accuracy and timeliness of crop classification with fusion images.

Study Area
Nangong County is selected as the study area, which belongs to Xingtai City (113°52′E~115°49′E and 36°50′N~37°47′N).The county lies in the main grain producing area of Hebei Province, China.Nangong County covers 863.3 km 2 within the autumn harvesting corn and cotton growing region in the North China Plain.As described above, the classification of corn and cotton in dryland area was regarded as the research object.In particular, the growth period of corn begins from June to October, while the cotton grows from April to October.Hence, the two species are considered to have the same growing season in the study area.
Figure 1.Location map for study area in Xingtai City, Hebei Province, China

Data Preparation
In this study, Sentinel-1A SAR data (at August 29 th , 2016) and GF-1 optical data (at August 27 th , 2016) were selected for image fusion.Sentinel-1A is the first environmental satellite of the Copernicus global monitoring programme and was launched at April 3rd, 2014 in French Guiana.Sentinel-1A carries a C-band full polarimetry SAR sensor, and provides 4 kinds of operational modes, including the wave mode (WM), strip map mode (SM), interferometric wide swath mode (IW) and extra wide swath mode (EW).The revisit period of Sentinel-1A for anywhere in the world is 12 days.We adopted the IW SLC image, which belongs to the default mode over land.This mode has a swath width of 250 km and a ground resolution of 5 × 20 m, and the radar beam scans back and forth three times within a single swath (called sub-swaths), resulting in a higher quality and homogeneous image throughout the swath.The operations of multilooking, filtering, geocoding and radiometric calibration were then carried out.
GF-1 satellite was launched at April 26th, 2013, carrying 2 kinds of optical sensors, PMS (2m resolution panchromatic and 8m resolution multi-spectral image) and WFV (16m resolution multi-spectral image).We adopted one image of WFV sensor, covering the entire Nangong County.With the calibration coefficient of each band provided by China Centre for Resources Satellite Data and Application, radiometric calibration of WFV data was carried out.Then, the atmospheric correction based on FLAASH model and further geometric rectification were also conducted.Furthermore, Sentinel-1A SAR image was registrated to match GF-1 image before the image fusion operation.
Besides, the sample plots were evenly distributed in the study area, intending to provide a reference for the accuracy assessments of dryland crop classification.The measurements in the sample plots clearly define the landcover types of corn, cotton, other crops and non-crop.

Satellite
Sentinel

The Image Fusion of SAR and Optical Remote Sensing Data
Prior to the image fusion, Sentinel-1A SAR and GF-1 images were selected as the experimental datum, and the conversion of DN value to backward scattering coefficient and reflectance were conducted, respectively.According to the references, PCA, GS, Brovey and WT fusion methods were widely used in previous studies (Hu et al., 2002;Wu et al., 2005;Yang et al., 2001).
PCA method is also called K-L (Karhunen-Loeve) method in mathematics, intending to convert multiple components to a few aggregative ones with the statistical analysis algorithm.The basic idea of PCA method focus on the acquisition of new principal components sorted by their importance, from most to least significant, and the components are uncorrelated from each other.Moreover, by replacing the first principal component with SAR data, the fusion image is reconstructed with sufficient texture information from SAR. PCA method is appropriate for the image fusion with any number of bands, inevitably encountering the large amounts of computation and resource consumption.GS method belongs to the orthogonalization algorithm, and the application is now in underway stage for the fusion of remote sensing images.Also, compared with PCA method, there is no apparent difference among the information contents of principal components with GS method.Brovey method is known as the colour standardization transform method, putting emphasis on the normalization of red, green and blue bands of optical images.And then, SAR data is multiplied by each band to accomplish the image fusion.However, the method results in a compression of histogram and darkened fusion image.WT method intends to realize the wavelet decomposition of SAR and optical images, obtaining the corresponding low frequency component and high frequency component.Then the wavelet reconstruction is conducted with the high frequency component of SAR image and the low frequency component of optical image.And to the extent, the reconstruction results in relative spectrum distortion.
The image fusion experiments are evaluated with the qualitative and quantitative analysis.The qualitative evaluation is conducted according to the image resolution and spectrum maintenance, putting emphasis on the visual effect.In contrast, the quantitative evaluation is carried out based on objective indicators.

Indicator Formula Illustration
Mean value 1 1 1 (i, j) A smaller change of the mean value corresponds to a better performance of the spectrum maintenance


A larger value of the standard deviation corresponds to an apparent contrast within the image A larger value of the comentropy corresponds to more abundant information and a better performance of image fusion Average gradient


Average gradient reflects the image resolution, details of the contrast, and the changes of the textural features Correlation coefficient reflects the degree of correlation between the original image and the fusion image Table 2.The evaluation indicators for the fusion performance As shown in Table 2, M and N are the row number and column number of the image, respectively; (i, j) Z represents the gray value of each pixel, Z represents the average of the gray values within an image; i P represents the frequency of occurrence for the i th gray value; (i, j) A and (i, j) F are the gray values of the pixel before and after fusion, respectively; A and F are the averages of the gray values within an image before and after fusion, respectively.

The Nearest Neighbour Algorithm for Supervised Classification
The nearest neighbour algorithm belongs to the supervised classification method and was conducted based on the objectoriented technique process in this study.The object-oriented method intends to assemble adjacent pixels into image objects, making full use of the spatial, spectral and textural characteristics of the extraction objects.In particular, the operation of multi-resolution segmentation was first conducted to the images, and then, the training samples of the supervised classification were selected form the image objects (Ferro-Famil et al., 2001) (Figure 2).
The categories of the image objects should be defined at first, and the training samples of each category were then selected accordingly.Moreover, in order to distinguish the targets from backgrounds, at least two or more features were adopted to establish the multi-dimensional feature space.As discussed above, the image objects were turned into information carriers with the operation of multi-resolution segmentation, and the segmentation results preliminarily reflected the differences among the varieties of ground objects.Hence, the apparent characteristics such as the spectrum and texture were analysed and selected to realize the classification of corn and cotton, intending to get rid of the confusion from each other.The nearest neighbour classifier was used to calculate the distance l from the unclassified image object to each category, and the membership assignments to the categories were conducted according to the values of distance l .The value of membership (ranges from 0 to 1) is inversely proportional to the distance l , and the distance is calculated as follows: 2 ( ) Furthermore, the multi-dimensional membership function was calculated based on the distance l , presenting with the exponential form.

( )
kl z l e   (2) in which, the value of k determines the rate of reduction of (l) z , and different value of distance l corresponds to different membership values.In the multi-dimensional feature space, the image object gets closer to the training sample with the decrease of the membership function slope.

The Evaluation Indicators for Results Accuracy Assessments
For the fusion image and original GF-1 image, the nearest neighbour algorithm was adopted to realize the supervised classification of corn and cotton with the same training samples.Furthermore, the accuracy assessments for the classification results were carried out based on the sample plots in the study area (Chen and Wei, 2009;Kussul et al., 2013;Stehman, 2009).
The error matrix, also called as confusion matrix, is widely used as the standard format for accuracy assessments (Foody, 2005).The calculated indicators from the error matrix generally includes the user accuracy, producer accuracy, overall accuracy and the Kappa coefficient.In particular, the user accuracy Pui provides an estimation of the probability that a classified pixel correctly represents the features on the ground, it also reflects the errors of commission (inclusion).And for the potential user of the map, user accuracy indicates the probability of correctly finding that class on the ground.Producer accuracy PAj provides an estimation of the probability that a reference pixel is correctly labelled in the classified data, it reflects the errors of omission (exclusion).And for the producer of the map, producer accuracy indicates the probability that features of interest are omitted from the classified data.Overall accuracy PC is calculated from the diagonal and off diagonal elements in the confusion matrix.User and producer accuracies are calculated using the row and column totals, describing the errors related to individual classes.In contrast, overall accuracy describes the proportion of the total number of pixels that have the same class in the reference and classified data for all classes.Besides the user accuracy, producer accuracy and overall accuracy, the kappa coefficient was introduced by Cohen (1960) as a reliability statistic when two judges are classifying targets into categories on a nominal variable.It is most commonly used to estimate interrater reliability.Kappa coefficient measures the percentage of data values in the main diagonal of the table and then adjusts these values for agreement.
in which, r represents the number of columns in the error matrix; xii represents the number of correctly classified objects on the i th row and i th column; xi+ and x+i are the total numbers of the objects on the i th row and i th column, respectively; N represents the total number of the objects used for the accuracy assessments.

The Qualitative and Quantitative Evaluations of the Fusion Methods
The image fusion was conducted with PCA, GS, Brovey and WT methods, respectively.And the experiments were then evaluated with the qualitative and quantitative analysis.The qualitative evaluation is conducted according to the image resolution and spectrum maintenance, putting emphasis on the visual effects (Figure 3).In GF-1 optical image, the croplands of corn and cotton were relatively discrepant with different shades of green, corresponding to the differences of gray features in SAR image.In particular, the spectrum maintenances of PCA and WT fusion images were closer to the GF-1 optical images, basically reflecting the actual colours of construction land, crop land, water body and other surface features.The textural features of PCA, GS and WT fusion images were almost the same, while the granular sensation of PCA fusion image was more apparent.In contrast, the colour distortions were more apparent in Brovey fusion image, in which the colours of construction land and water body reflected larger differences with the actual conditions.However, the textural features of Brovey fusion image were the most apparent among the four methods, showing a better performance on the textural information of different land cover types.In Brovey fusion image, the boundary of construction land and roads were clearly distinguished from the cropland, and the textural features of corn and cotton were also discrepant from each other.Hence, the four methods reflected different fusion performances in view of the image resolution and spectrum maintenance, and the best selection for the fusion of SAR and optical image was not determinate.
Figure 3.The comparison of fusion images with different methods (displayed in partial study area)  3. The evaluation indicators for the fusion performance Besides the evaluation with the qualitative methods, the quantitative analysis was also conducted based on the objective indicators.For the indicator of Z , GS method performed to be almost unanimity with the GF-1 optical image, while Brovey method represented the biggest difference.A smaller change of the mean value corresponded to a better performance of the spectrum maintenance, so the GS and Brovey method corresponded to the best and the worst performance of spectrum maintenance, respectively.In contrast, the largest value of the standard deviation of Brovey method corresponded to the most apparent contrast within the image, contributing to effectively distinguish the various crops.For the indicator of E , GS method represented the largest values, reflecting the abundant information in the fusion image.Just as the indicator of standard deviation, the largest value of G of Brovey method corresponded to the best performance in the details of the contrast and the changes of the textural features.For the indicator of correlation coefficient, WT method represented the largest values, reflecting a stronger correlation between the original GF-1 image and the fusion image.
In conclusion, the fusion methods of SAR and optical image were compared in view of the qualitative and quantitative analysis.Focusing on the actual requirement for dryland crop classification, the abundant information and apparent contrast within the image were taken into consideration with high priority.Hence, the GS method and Brovey method were proved to be more applicable for the fusion of Sentinel-1A SAR and GF-1 optical images, contributing to the dryland crop classification in the study area.

The Supervised Classification Results for Fusion and Optical Images
The nearest neighbour algorithm belongs to the supervised classification method and was carried out according to the object-oriented technique process in this study.In particular, the operation of multi-resolution segmentation was first conducted to the images, intending to assemble adjacent pixels into image objects.Based on the segmentation results, the training samples of the supervised classification were selected form the image objects.
Usually, a larger value of the segmentation scale resulted in the poor homogeneity of image objects, meanwhile inducing the fuzzy cropland boundary.The mixed image objects brought difficulties for the subsequent extraction process.A smaller value of the segmentation scale corresponded to the preferable homogeneity of image objects and clear cropland boundary, but inevitably resulted in a extraction efficiency in the subsequent process.Finally, the segmentation scale was set as 15 (shape parameter 0.1, compactness parameter 0.5) after the repeated experiments, intending to assemble the pixels with similar spectral and textural characteristics as one image object (Figure 4).Under the same segmentation scale, the image objects in fusion images were presented to be more fragmented than those in GF-1 optical image.
The training samples of the supervised classification were selected form the image objects, and the same training samples were applied to GS fusion image, Brovey fusion image and original GF-1 optical image.The categories of the image objects were defined as corn, cotton, other crops and non-crop, and then the training samples of each category were selected accordingly.Moreover, in order to distinguish the targets from backgrounds, apparent characteristics such as the spectrum and texture were analysed and adopted to establish the multidimensional feature space.Based on the characteristics of fusion and optical images, the corresponding multi-dimensional feature space (including Brightness, Mean Layer, Standard Deviation, Ratio Layer, GLCM Texture Layer, Max. diff. et al.) for each image was established for dryland crop classification in the study area, focusing on the identification of corn and cotton.

The Accuracy Assessments for Dryland Crop Classification
The nearest neighbour algorithm was adopted to realize the supervised classification of corn and cotton with the same training samples.Moreover, the accuracy assessments for the classification results were carried out based on the sample plots in the study area.The values of user accuracy, producer accuracy, overall accuracy and Kappa coefficient were calculated with the error matrix.As shown in Table 4, the values of overall accuracy and Kappa coefficient of fusion images were all higher than those of GF-1 optical image.In particular, the values of user accuracy and producer accuracy of GS and Brovey fusion images were basically higher than those of optical image, except the user accuracy of cotton.Moreover, GS method performed better than the Brovey method, and all the values of user accuracy and producer accuracy were above 80%.In the classification results with GS and Brovey fusion images, the distributions of corn and cotton maintained the overall consistency (Figure 5).The accuracy assessments reflected the advantages of fusion images for dryland crop classification under a complex crop planting structure.However, for the Sentinel-1A SAR image, there was no spatial resolution advantages than GF-1 optical image, and the introduction of SAR data did not provide enough beneficial complements of spatial information.Moreover, GLCM texture in the multi-dimensional feature space also resulted in a longer calculation process.In further study, in order to make full use of the SAR data, more emphasis should be put on the adoption of high spatial resolution SAR image and the improvements of computation efficiency of supervised classification.

CONCLUSION AND DISCUSSION
The accurate classification of various crops contributes to a good estimation for the crop planting structure, sown acreage and spatial distribution, meanwhile providing a reference for the related agricultural policies and economic plans.Compared with the optical remote sensing images, SAR images perform stronger support capabilities for crop classification with the advantages of all-weather, all-time, high resolution and wide coverage.Moreover, the fusion of optical and SAR images provides beneficial complements for the traditional remote sensing techniques in the aspects of agricultural condition monitoring.In this study, PCA, GS, Brovey and WT fusion methods were evaluated with the qualitative and quantitative analysis, taking the visual effects and objective indicators into consideration.Then, with the applicable methods, the fusion images were applied for the dryland crop classification.The nearest neighbour algorithm was adopted for supervised classification, and the same training samples were applied to all images.Furthermore, the accuracy assessments for the classification results were conducted, and the fusion images were proved to be superior to optical image.Hence, the fusion methods allowed for the dryland crop classification under a complex crop planting structure, also providing a reference to improve the accuracy and timeliness of crop classification.
In this study, the fusion methods of SAR and optical image focused on the actual requirements for dryland crop classification.Considering the abundant information and apparent contrast within the fusion images, GS method and Brovey method were proved to be more applicable for the fusion of Sentinel-1A SAR and GF-1 optical images.However, the various remote sensing sensors correspond to different adaptations to the fusion methods.Thus, the selection of best fusion method calls for the analysis on the characteristics of sensors and the actual application requirements.More emphasis should also be put on the reduction of influences from the speckle noise of SAR images, intending to realizing better fusion performances.
With the object-oriented technique process, the nearest neighbour algorithm was adopted to realize the supervised classification.However, more classification algorithms should be taken into consideration in further study, which has a large space for the possibility of improvements in classification application.Also, the classification accuracy depends largely on the selection of training samples.So, the training samples of the supervised classification need to be adjusted when the method is applied to other typical croplands, which calls for more supports from measurements in the study areas.

Figure 2 .
Figure 2. Flow chart of the nearest neighbour classification based on the object-oriented technique


represents the value of Feature f of the training sample; o f  represents the value of Feature f of the unclassified image object; f  represents the standard deviation of Feature f.
i th class of the classified data and the j th class of the reference data.

Figure 4 .
Figure 4.The segmentation results of the optical and fusion images (displayed in partial study area)

Figure 5 .
Figure 5.The dryland crop classification results with GS and Brovey fusion images