A PAN-SHARPENING METHOD BASED ON GUIDED IMAGE FILTERING : A CASE STUDY OVER GF-2 IMAGERY

The GaoFen-2 satellite (GF-2) is a self-developed civil optical remote sensing satellite of China, which is also the first satellite with the resolution of being superior to 1 meter in China. In this paper, we propose a pan-sharpening method based on guided image filtering, apply it to the GF-2 images and compare the performance to state-of-the-art methods. Firstly, a simulated low-resolution panchromatic band is yielded; thereafter, the resampled multispectral image is taken as the guidance image to filter the simulated low resolution panchromatic Pan image, and extracting the spatial information from the original Pan image; finally, the pan-sharpened result is synthesized by injecting the spatial details into each band of the resampled MS image according to proper weights. Three groups of GF-2 images acquired from water body, urban and cropland areas have been selected for assessments. Four evaluation metrics are employed for quantitative assessment. The experimental results show that, for GF-2 imagery acquired over different scenes, the proposed method can not only achieve high spectral fidelity, but also enhance the spatial details.


INTRODUCTION
With the rapid development of remote sensors, a great deal of optical earth observation satellites and digital aerial cameras can simultaneously obtain high spectral resolution multispectral (MS) and high spatial resolution panchromatic (Pan) images (Yun, 2012).However, due to the physical constraints, the spectral information is only rich in MS images, and it is difficult to acquire the images with both high spatial and spectral resolution.
The images obtained from a single sensor often cannot meet applications, such as visual interpretation, change detection and detailed land cover classification, etc.Therefore, it is increasingly important to combine the strengths of the MS and Pan images (Dong, 2009) and (Ehlers, 2010).
To date, a large number of pan-sharpening methods have been  Corresponding author: Leiguang Wang; Email:wlgbain@126.com proposed.Among them, component substitution (CS) (Qizhi, 2014) methods are more practical and widely used because of its fast speed in calculation and convenient implementation.The representative CS methods include principal component analysis (PCA), Gram-Schmidt transformation (GS), Intensity-Hue-Saturation (IHS) and University of New Brunswick (UNB) method (Zhang, 2004), etc.However, with more and more sensors with different spectral and spatial properties were launched, these existing methods show various limitations, and have not fully assessed on data sets captured by the new sensors (Zhang, 2004).GF-2 satellite was launched on August, 2014.It is a selfdeveloped civilian optical remote sensing satellite of China, which is also the first satellite with the resolution of being superior to 1meter in China.It can achieve a spatial resolution of 0.8 meter with a swath of 48 kilometers in panchromatic mode, in contrast, 3.2 meter and 4 spectral bands in multispectral mode.
In this context, a pan-sharpening method based on guided image filtering is proposed and applies to GF-2 images.Experimental results show that the proposed method can achieve a better effectiveness on spectral information preservation and spatial detail enhancement.

Guided Image Filtering
The guided image filtering is firstly proposed by He et al. (He, 2013) in 2010.It is quite popular due to its edge-preserving property and is widely used for combining features from two different source images, such as image matting/feathering (Levin, 2006), flash/no-flash de-noising (Petschnigg, 2004), HDR compression (Durand, 2002) and haze removal (He, 2011), etc.By using the guidance image to affect the process of filtering, the original image can be smoothed, meanwhile, the gradient information of the guidance image can also be retained.
The guided image filter (He, 2013) assumes that the filtering output is a local linear model between the filter output Q and the guidance image I in a local window k  centered at pixel k . , where k a and k b are the linear coefficients considered to be constant in a small square image window k  .The local linear model guarantees Q a I    , that is, the filter output Q has an edge only if the guidance image I has an edge.Here, the coefficients k a and k b are computed by minimizing the following cost function: where  is a regularization parameter that set up by users and prevents k a from being too large.The linear coefficients are directly resolved by the linear ridge regression (Draper, 1981) as where k  and 2 k  are the mean and variance of is the number of pixels in k  , and k p is the mean of p in k  .However, all windows that contains i will involve the pixel i , so different windows will have different values of i Q .Then one effective method to resolve this problem is to average all the possible values of i Q to obtain the filtering output image Q .
Therefore, after calculating     As shown in Figure 1, this proposed pan-sharpening method consists of the following four procedures:

Proposed Pan-Sharpening Method
(1) The original multispectral image is registered and resampled as the same size as the Pan image P .
Thereafter, by introducing ( 5) into ( 6), a synthetic lowresolution panchromatic image P can be obtained. 4 where P is the simulated low resolution panchromatic image and i w is the weight for the i-th band ( , which is constant for the given band. (3) Take each where otherwise, the weight should be large.

Experimental Setting
For analysis and comparison of the proposed and other pansharpening methods, three pairs of Gaofen-2 imagery were selected for test in this paper.Table 1 shows  Table 1.Characteristics of the employed GF-2 datasets In order to verify the effectiveness of the proposed approach, three state-of-the-art fusion methods, including the GS transformation (Laben, 2000) and NND method (Sun, 2014) in ENVI software, and UNB method (Zhang, 2004) in PCI Geomatica software, were employed in the experiments for comparison.

Assessment Metrics
Four widely used metrics are selected for quantitative assessment, they are the entropy, the correlation coefficient (CC) (Klonus, 2007), the universal image quality index (UIQI) (Wang, 2002) and the relative dimensionless global error in synthesis (ERGAS) (Ranchin, 2000).The resampled MS image is taken as the reference image.
1) The entropy can be used to measure how many spatial information that the fused image contains.The higher the entropy is, the better the quality of the fused image will be.Its

 
Fi is the probability of pixel value i in the image.
2) CC (Klonus, 2007) measures the correlation between the MS image and the fused image.The higher correlation value indicates a better correspondence between the MS image and the fused image.It is expressed as: M and F stand for the mean values of the original MS and fused image, and CC is calculated globally for the entire image.
3) UIQI (Wang, 2002) models any distortion as a combination of three different factors: loss of correlation, luminance distortion and contrast distortion.Its definition is given by: 4) ERGAS (Ranchin, 2000) evaluates the overall spectral distortion of the pan-sharpened image.It is defined as: where is the ratio between pixel sizes of the Pan and MS images, K is the number of bands,

 
MEAN i is the mean of the i-th band, whereas   RMSE i is the root-mean-square error between the i-th band of the reference image and the i-th band of the fused image.

Results and Performance Comparison
As shown in Fig. 2 to Fig. 4, local patches with size of 400×600 pixels over different land cover types are clipped from the fused results and displayed in true color with the same stretching mode.After the visual comparison, the fused images yielded by the NND method have obvious spectral distortion on the green vegetation regions, especially the Figure 3 and Figure 4, not match the original deep green color.This may be due to the NND method is more suitable for fusing low resolution images, like Landsat 7 data, etc.While UNB and GS methods achieve excellent performance on spatial quality.Moreover, as it can be seen from all these figures, the proposed method has better effectiveness especially on spectral fidelity.2).The best performance of each metric is in bold.
For quantitative assessments, all of the metric scores of the proposed method are the best in Table 2 and Table 4   Table 3. Quality evaluation of fused images: Urban (corresponding to Figure 3).The best performance of each metric is in bold.Table 4. Quality evaluation of fused images: Cropland (corresponding to Figure 4).The best performance of each metric is in bold.

CONCLUSION
In this research, a pan-sharpening method based on guided image filtering is proposed, and applying it to GF-2 images.A great number of experimental results and quality assessments have demonstrated that, for GF-2 imagery acquired over different scenes, the proposed method can consistently achieve high spectral fidelity and enhance the spatial details independent of the content of the images.Furthermore, how to adaptively select the window size of weight calculation and estimate the parameters of guided filtering will be future researches.
all windows k  in the image, the filter result is computed by:

Figure 1 .
Figure 1.The processing flowchart of the proposed pansharpening method.
guidance image to guide the filtering process of low resolution Pan image P , and then obtain the filter outputi

i
is the weight corresponding to i-th MS band at the position   is the band number of MS image and 1, 2, 3, 4 n  is the total band number of the MS image.It is obvious that the greater the distance, the smaller the weight; the characteristics of this dataset.The test images were acquired over Guangzhou, China, three scenes including urban, water body and the cropland areas.The MS image consists of four bands and the spectral range of the MS bands is exactly covered by the range of the Pan band.The size of each image is 1000  1000.


are the standard deviation of the fused and original images respectively.

Figure 2 .
Figure 2. The fused images with different methods of GF-2 image over the water body.From left to right, up and down: MS; PAN; GS; NND; UNB and the proposed method.
preserving and spatial enhancement than other methods.It can attribute to the edge-preserving feature of the guided filtering, on the other hand, it is also because the proposed method takes advantage of the proper weights to inject the spatial details into each band of the resampled MS image.Furthermore, the results of quality assessment also agree with the visual comparison.

Figure 3 .
Figure 3.The fused images with different methods of GF-2 image over the urban.From left to right, up and down: MS; PAN; GS; NND; UNB and the proposed method.

Figure 4 .
Figure 4.The fused images with different methods of GF-2 image over the cropland.From left to right, up and down: MS; PAN; GS; NND; UNB and the proposed method.

Table 2 to
Table 4 correspond to these quantitative evaluation results.The best performance of each metric is in bold.

Table 2 .
Quality evaluation of fused images: Water body (corresponding to Figure . In Table3, the ERGAS value of the proposed method is the second best, but other metrics are all the best.This demonstrates that the proposed method achieves better performance on spectral