REMOTE SENSING IMAGE FUSION USING ICA AND OPTIMIZED WAVELET TRANSFORM

In remote-sensing image processing, fusion (pan-sharpening) is a process of merging high-resolution panchromatic and lower resolution multispectral (MS) imagery to create a single high-resolution color image. Many methods exist to produce data fusion results with the best possible spatial and spectral characteristics, and a number have been commercially implemented. However, the pan-sharpening image produced by these methods gets the high color distortion of spectral information. In this paper, to minimize the spectral distortion we propose a remote sensing image fusion method which combines the Independent Component Analysis (ICA) and optimization wavelet transform. The proposed method is based on selection of multiscale components obtained after the ICA of images on the base of their wavelet decomposition and formation of linear forms detailing coefficients of the wavelet decomposition of images brightness distributions by spectral channels with iteratively adjusted weights. These coefficients are determined as a result of solving an optimization problem for the criterion of maximization of information entropy of the synthesized images formed by means of wavelet reconstruction. Further, reconstruction of the images of spectral channels is done by the reverse wavelet transform and formation of the resulting image by superposition of the obtained images. To verify the validity, the new proposed method is compared with several techniques using WorldView-2 satellite data in subjective and objective aspects. In experiments we demonstrated that our scheme provides good spectral quality and efficiency. Spectral and spatial quality metrics in terms of RASE, RMSE, CC, ERGAS and SSIM are used in our experiments. These synthesized MS images differ by showing a better contrast and clarity on the boundaries of the “object of interest the background”. The results show that the proposed approach performs better than some compared methods according to the performance metrics. * Corresponding author


INTRODUCTION
Images of modern remote-sensing systems allow us to solve various problems such as carrying out operational monitoring of land resources, city building, monitoring state of the environment and influence of anthropogenic factors, detecting contaminated territories, unauthorized buildings, estimating the state of forest plantation and other (Pandit, 2015, Hnatushenko, 2016).These images may be captured from different sensors, acquired at different times, or having different spatial and spectral characteristics.Many remote-sensing satellites can obtain images in multispectral and panchromatic bands (Gnatushenko, 2003).Color is often the most significant component of the majority of remote sensing images.While panchromatic images can be very informative, the number of shades of grey distinguished by the human eye is limited to 20 to 30 on the scale of contrasts.As a contrary to this, the eye can distinguish 20,000 or more color shades, which allows us to detect small, but often significant, details of images.However, multispectral images have relatively low spatial resolution, which stipulates importance of development of methods for automated increase of the information value of source data, in particular, by combining high spatial and spectral resolution in one image.The lower spatial resolution multispectral images are fused with higher spatial resolution panchromatic images.Fusion or pansharpening is the processing that combines the high spatial and multispectral information of the same scene to obtain a fused multispectral image that retains the spatial information from the high resolution panchromatic image, as well as the spectral characteristics of the lower resolution multispectral image (Han, 2008, Pohl, 1998, Zhang, 2012).To receive a synthesized image with the best value of information content it is necessary to choose the most effective fusion method because it affects the thematic problem solving.Spatial resolution of multispectral images increases, as a rule, in consequence of implementation of the following basic steps (Amro, 2011, Chaudhari, 2014, Nikolakopoulos, 2008): 1. Converting a multispectral image with low spatial resolution from the red-green-blue (RGB) basis to any three coordinate basis, in which one of the coordinates is equivalent to brightness distribution.2. Increasing the sample rate of the converted image up to sample rate of a panchromatic image and further interpolation (linear, bicubic, nearest neighbor rule, etc.).3. Replacement of the brightness component of the converted image with a panchromatic image of high spatial resolution.4. Inverse transformation into the RGB basis.The quality of the fusion method is determined by the increase of spatial resolution and the existence (absence) of color distortion.Numerous works related to preprocessing of multichannel digital images focus on improving their visual quality, while excluding physical mechanisms of fixing information, especially, correlation between channels, making it impossible to determine the image information content from the standpoint of analysis and interpretation (method "Brovey").Other works (Palsson, 2015, Shahdoosti, 2016, Wang, 2008) based on calculation of statistic parameters of digital images ("principal component analysis", (PCA) and "independent component analysis", (ICA)), determination of which is difficult due to large volume of primary data.Some works are based on the transition to color-difference metrics of computer graphics ("color-metric methods"), where the question about decorrelation of primary data is solved (Ling, 2007, Rahmani, 2010, Thomas, 2008).The modification to the IHS method, called the IHS-SA, proposed the incorporation of weighted coefficients on the green and blue bands so as to reduce the difference between the intensity and the panchromatic bands (Tu et al., 2005).However, these methods allow us to take into account only spectral components of primary grayscale image (Vivone, 2015).Like IHS method, Gram-Schmidt method also requires forward and backward transformation of MS image.But, this method lacks a clear and standard procedure to calculate MS to PAN weights.In the past years, new approaches or improvements of the existing approaches are regularly being proposed to overcome the problems in the standard techniques (Manu, 2015).One of the most modern methods is the wavelet transform.It has been found that wavelet-based fusion techniques outperform the standard fusion techniques in spatial and spectral quality, especially in minimizing color distortion (Amolins, 2007, Cheng, 2015).Schemes that combine the standard methods (IHS or PCA) with wavelet transforms produce superior results than either standard methods or simple wavelet-based methods alone (González-Audícana, 2004, Hnatushenko, 2015a, Manu, 2015).However, the tradeoff is higher complexity and cost.Many fusion algorithms (Tu, 2005, Zhang, 2005, Zhang, 2010) were developed and tested on different commercial satellite data such as Ikonos and Quickbird, however, the efficacy of these algorithms is poorly assessed on the satellite data of Worldview-2 (Padwick, 2010).The panchromatic band of Ikonos and Quickbird ranges between 0.45 and 0.90 micrometers; the Worldview panchromatic band is narrower and ranges between 0.45 and 0.8 micrometers.In addition, modern scanner systems are more advanced and have more than four channels (such as WorldView-2eight channels).Therefore, there is a necessity in a new technology, which will increase spatial resolution of satellite images considering physical mechanisms of fixing species of information and conducting further research of fusion efficiency with obtaining quantitative estimates (criteria) of information content (quality) of synthesized images.

INPUT DATA
The inputs of the proposed algorithm are 8-band multispectral (MUL) and 16 bit panchromatic (PAN) images provided by DigitalGlobe.The panchromatic band has a spatial resolution of 0.46 m.The multispectral bands, each with a spatial resolution of 1.84 m, are coastal blue (400-450 nm), blue (450-510 nm), green (510-581 nm), yellow (585-625 nm), red (630-690 nm), red edge (705-745 nm), NIR1 (770-895 nm), and NIR2 (860-1040 nm).In the proposed fusion method, when converting into the HSV color model we only consider the three bands R=5, G=3 and B=2.The imagery was obtained over Dnepropetrovsk.Limited to the pages, we only show some parts of the images.The fragments of original images are shown in Figure 1 and Figure 2. The MUL images are of 512*512 pixels size whereas the PAN image is of 2048*2048 pixels size.The images should be registered to the same size before applying the fusion technique.For this, the MUL image, which is of 512*512 pixels size is upsampled by a factor of 4 to the size of 2048*2048 pixels by using bicubic interpolation.Generally, radiometric correction is also required before data fusion operations.Investigation of the effect of the choice of interpolation methods and computational procedures for geometric and radiometric corrections on the performance of fusion algorithm is a separate complex problem and is not the subject of this paper.

PROPOSED FUSION METHOD
In this paper new information technology based on ICA-and wavelet transforms is proposed, which can significantly increase the information content of the primary data and does not lead to color distortion.The algorithm scheme is shown in Figure 3.

ICA Transform
The use of Independent Component Analysis for image fusion was proposed in (Mitianoudis et al., 2006).Here, the authors proposed the use of bases trained using ICA on similar-content images as the analysis tools for image fusion.The bases were selected in such a way that they fit arbitrarily on the object types which were to be fused.This framework was found to outperform generic analysis tools like wavelet analysis.The independent component analysis is considered as an extension of the principal components analysis to the problem of blind separation of independent sources from their linear mixtures.The independent component analysis is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible.In addition, the axis does not need to be orthogonal.The model used in the analysis of independent components can be represented as: where y = m-dimensional random vector x = n-dimensional random vector with independent components H = some unknown transformation Rn → Rm, m ≥ n.

HSV Color System
This one of the most widely applied fusion procedure for merging panchromatic imagery with three-color multispectral imagery (Zhang, 2010).HSV method consists on transforming the R, G and B bands of the multispectral image into HSV components (Hue, Saturation, Value), replacing the value component by the panchromatic image, and performing the inverse transformation to obtain a high spatial resolution multispectral image.The problem with this approach is that it introduces color distortion result of the mismatches; that is, the Pan and V are spectrally dissimilar.To solve this problem, we adopt new technique: replace the "Value" image by the high resolution image after ICA-transform.

Wavelet Transform
One of the most advanced and powerful mathematical tools for pansharpening aerospace images is wavelet transform (Amolins, 2007).The general principle of building the basis of a wavelet transform lies in the use of a system of scale transformations and shifts.This is owing to the scales that wavelets are capable of revealing differences in characteristics on various scales, and, by the use of the shift, of analyzing properties of the signal at various points over the whole interval under consideration.The element of the wavelet transform basis is a well localized function quickly tending to zero outside of a small interval, which allows one to carry out a "localized spectral analysis".In other words, the wavelet transform automatically possesses a moving time/frequency window, narrow on small scales and wide on big ones.Initially, the images are subject to decomposition of the first level, as a result of which, for each of the images, matrixes of approximating and detailing coefficients are obtained.But the separate application of the wavelet transform often leads to artifacts in the synthesized image.

Methodology
On the base of packet wavelet transforms, technology is built, in which contribution of corresponding low frequency components of the primary multispectral and panchromatic images is taken into account with the use of linear forms, determination of coefficients of which is formulated as an optimization problem with the criterion of maximization of the information characteristic (entropy) at the optimal level of decomposition and comprises the following stages: 1. Resampling of the multispectral image with the purpose of bringing its dimension to the dimension of the panchromatic image.
2. Geometric and radiometric correction of the multispectral and panchromatic images.
(3) 10.A pyramidal decomposition with the given wavelet basis to the given decomposition level (L) of the panchromatic image:  12. Wavelet synthesis and transition to the RGB color metrics: Presence of the stage of formation of new components of decomposition according to the chosen rule on the base of wavelet transforms leads to the necessity of choosing the most optimal rule of decomposition.The input data for the implementation of wavelet reconstruction is the approximating coefficients obtained after wavelet decomposition of the synthesized color images, which are taken without changes, and, as the detailing coefficients, the obtained linear combinations are used.Then follows the search of arguments, which are the coefficients of the linear combination of forms, in which the entropy for the new synthesized image reaches maximum values.Thus, as a result of wavelet reconstruction, we obtained multichannel images with high information content, synthesized with the use of wavelets, by maximizing the entropy function.

Experimental Results
Research of influence of fusion methods was performed on the primary scanner images by WorldView-2 satellite.The synthesized image of the new information technology based on ICA-and wavelet-transformation is shown in Figure 4. Figure 5(a) is the fragment of input color composite image of band 5, 3 and 2; figure 5 (b) shows the fusion result of applying the proposed method.The figure 6 shows the results of different classic fusion methods (fragments of RGB multispectral images).After image fusion by methods discussed above, images that even visually are more precise than primary multispectral image were obtained, but with significant color distortion (Figure 6 a-f).The synthesized multispectral image by proposed method differs by showing a better contrast and clarity on the boundaries of the "object of interest -the background".We cannot easily develop conclusions regarding which method is better in terms of the clarity of the fused images based on the above experiment.A more accurate assessment is conducted in the quantitative analysis in Section 3.6.

Visual and Quantitative Evalutions
The evaluation of the employed fusion technique is also an important step in the fusion process (Alparone, 2004, Hnatushenko, 2015b).Various quality metrics have been used in the literature to study, compare and assess implementation of the fusion technique (Ehlers, 2008, Karathanassi, 2007, Wald, 2000, Zhang, 2008).In order to determine the capabilities of each method as for the quality of multispectral images, the quantitative estimates of the information content of initial and synthesized multispectral images were obtained for classic fusion methods and for the newly developed technology based on ICA-and wavelet transformations, such as information entropy, SSIM, CC, ERGAS, etc.
Entropy is a measure of image information richness is an important indicator of the size of the entropy reflects the amount of information contained in the image number, under normal circumstances, the greater the entropy of fusion images, the amount of information it contains, the more the better integration.Image entropy H is defined as Often, in general, the estimation is based on subjective visual perception of the synthesized image by a particular user.In (Wald, 2002) an index of error was proposed, which reflects the complete picture of the quality of the synthesized image.The author named it relative global dimensional synthesis error (ERGAS): where where M = the mean radiance of the K spectral bands (B i ) of the original multispectral bands.The correlation coefficient (CC) is a similarity metric wherein a value closer to one represents good similarity between the compared images: where A and B stand for the mean values of the corresponding data set, and CC is calculated globally for the entire image.The result of this equation shows similarity in the small structures between the original and fused images.In this paper, we use the SSIM to calculate the similarity of corresponding regions in two images.It is defined as follows (Wang, 2002(Wang, , 2004)) The tables 1, 2 show the values of information entropy, RASE, RMSE, CC, ERGAS and SSIM obtained for multispectral and panchromatic primary images and also for the synthesized images by all specified methods.From Table 1 it is clear that entropy of the image merged by proposed technology is higher than others.The results show that the proposed approach performs better than some compared methods according to the performance metrics.

CONCLUSIONS
In this paper, to minimize the spectral distortion we propose a remote sensing image fusion method which combines the ICA and optimized wavelet transform.The proposed method is based on selection of multiscale components obtained after the ICA of images on the base of their wavelet decomposition and formation of linear forms detailing coefficients of the wavelet decomposition of images brightness distributions by spectral channels with iteratively adjusted weights.These coefficients are determined as a result of solving an optimization problem for the criterion of maximization of information entropy of the synthesized images formed by means of wavelet reconstruction.Further, reconstruction of the images of spectral channels is done by the reverse wavelet transform and formation of the resulting image by superposition of the obtained images.To verify the validity, the new proposed method is compared with several techniques using WorldView-2 satellite data in subjective and objective aspects.The merging of the ICA, HSV and optimized wavelet transforms improves the spectral quality of the results while keeping its spatial qualities.The performance evaluation metrics confirmed the capacity of the proposed method.While the information capacity of multispectral images is increased with the use of the proposed method, the contrast and the number of gradations of brightness increase, the correlation decreases, which indicates the increase of the information entropy of the images; the maximum difference increases, and the errors decrease.

Figure 1 .
Figure 1.Input panchromatic image 3. Contrast and outline correction of the multispectral and panchromatic images.4. ICA fusion.5. Converting of the obtained multispectral image (MUL MOD ) into the HSV color model (HSV MOD ). 6. Transforming the R, G and B bands of the multispectral image into HSV components (HSV MUL ). 7. The next step of formation of a new image is replacement the brightness component V of HSV MUL image with the brightness component of HSV MOD and converting the result from the HSV color model to the RGB color model.8. Decorrelation of the new multispectral image in accordance with the chosen algorithm: pyramidal decomposition with the given wavelet basis to the given decomposition level (L) of the brightness channel (X): of new components of the decomposition according to the following rules of unification of coefficients: fusion algorithm in more detail.After the ICA converting, it is proposed to replace the first component of multispectral image with panchromatic image.The next step after replacing is the inverse ICA-transform and converting of the obtained image into the HSV color model (HSV MOD ).Another step of the algorithm is transformation of original multispectral images into HSV color space (HSV MUL ).The next step of formation of a new image with high information content is replacement the brightness component V of HSV MUL image with the brightness component of HSV MOD and converting the result from the HSV color model to the RGB color model.The final step of the algorithm is fusion of the Mul ICA image with PAN using wavelets and obtaining of the output image.One of the most important results is determination of basic wavelets and the level of decomposition.It was defined that the best results of panchromatic and multispectral image fusion gives Daubechies orthogonal wavelet by the 4th decomposition level.

Figure 4 .Figure 6 .
Figure 4.The fusion result of applying the proposed method h = the resolution of the high spatial resolution image l = the resolution of the low spatial resolution image K = the number of spectral bands involved in fusion ) (k  = the mean value of band k of the reference image RMSE = the root mean square error computed between the degraded fused image and the original image for band k: m,n), F(m,n) = reference and fused images, respectively, M, N = image dimensions.The ERGAS index equals zero when the degraded fused image is equal to the original image.Therefore, low ERGAS values indicate high image fusion quality.The relative average spectral error (RASE) index expressed as a percentage.It characterizes the average performance of the method in the spectral bands considered: of A and B, respectively AB  = covariance of A and B.

Table 2 .
Comparison of fusion methods