PAN SHARPENING USING RELATIVE SPECTRAL RESPONSE OF SENSOR FOR CARTOSAT-1 PAN AND RESOURCESAT LISS-4 MX DATA

Most of the Indian remote sensing systems, provide sensors with one high spatial resolution panchromatic (PAN) and several multispectral (MS) bands. An increasing number of applications, such as feature detection, change monitoring, and land cover classification, often demand the use of images with both high spatial and high spectral resolution. Image fusion or pan sharpening, is a technique to enhance the spatial resolution. The most significant problem in the traditional fusion methods is spectral distortion of fused images. The main reason for this being, the physical spectral characteristic of the sensors are not considered during the fusion process, resulting in undesirable effects such as modified spectral signatures resulting in classification errors and resolution over injection. For most earth resource satellites which provide both PAN and MS bands, in ideal condition, all MS bands would be well separated and would cover exactly the same wavelengths as the PAN band. Theoretically, the measured energy in the PAN band can be obtained with the summation of corresponding MS bands. As the measured energy in an individual channel is the sum of incoming radiation and relative spectral response: Lk = L(λ) Rk(λ); where λ is the wavelength, the in-band radiance, L(λ) at aperture spectral radiance and Rk(λ) the peak-normalized spectral response. Therefore, the energy in PAN band can be estimated by defining weights as follows: Pan = wR R + wG G + wNIR NIR + other; where Pan, G, R, NIR represent the radiance of individual spectral bands wG, wR, wNIR are the weights of corresponding MS bands and other for the influence of the spectral range which is missing from MS bands but still covered with the PAN band. In this paper, a novel spectral preservation fusion method for remotely sensed images using Cartosat-1 PAN and ResourcesatLiss4 Mx data is presented by considering the physical characteristics of the sensors. It is based on the curvelet transform using relative spectral response (RSR) values of the sensor, improved in two parts: 1) the construction of PAN image using RSR values and the curvelet components, 2) the injection method of detail information. The performance and efficiency of the proposed method is compared with traditional IHS, wavelet based methods both visually and quantitatively. The results show that the proposed method preserves spatial details and minimize spectral distortion.


INTRODUCTION 1.1 Need for Image Fusion
Earth observation satellites provides images at different spatial, spectral and temporal resolutions (Ranchin et al., 1996).In order to be able to effectively use the available data from various remote sensing sensors, advanced methods of data fusion are being developed.These methods integrate data from different sensors in order to obtain additional information.During the last decades, data fusion has been a rapidly developing area of research in remote sensing.Several authors have contributed in developing new and improved algorithms and their applications, with emphasis on preserving the spatial and spectral charactersitics of the input images used in fusion to generate a high fidelity fused output product.
Most of the Latest remote sensing systems, such as Pleiades, Ikonos, QuickBird, IRS, SPOT and Landsat 8, provide sensors with one high spatial resolution panchromatic (PAN) and several multispectral bands (MS).The most important reason for not acquiring the remote sensing images merely in high resolution is the incoming radiation energy and the data volume collected by the sensor (Zhang, 2004).PAN images cover a broader wavelength range and in order to receive the same amount * Corresponding author of incoming energy, the size of a PAN detector can be smaller than that of a MS detector.Therefore, on the same satellite or aerial platform, the resolution of the PAN sensor can be higher than that of the MS sensor.On the other hand, the data volume of a high-resolution MS image would be significantly larger and could mitigate the problems of limited on-board storage capacity and limited data transmission rates between the platform and ground (Zhang, 2004).
Many Remote sensing applications like cartography, cadastre mapping, vegetation studies, Geo morphological studies etc., need both high spectral and high spatial resolution data.This is achieved by using image fusion or more precisely, PAN sharpening or resolution merge.Image fusion is a method, which is used to increase the spatial resolution of multispectral images (ideally without the loss of spectral information), through the combination of low spatial but high spectral resolution multispectral data and higher spatial but low spectral resolution panchromatic data.

Types of Image fusion strategies
The image fusion techniques can be broadly classified as spectral preserving techniques, and spatial preserving.Based on the principle of fusion the methods can be further classified into component substitution methods and Multi resolution Analysis (MRA) based methods.Several component-substitution (CS) methods exist for merging (fusing) the data sets, including: 1. Simple band-substitution methods; 2. Color space transformation (Pohl and Genderen, 1998)  The other class of pan-sharpening algorithms are multiresolution analysis (MRA) based on the use of wavelets or Laplacian pyramids and Ehler's fusion method based on an IHS transform and Fourier filtering.

Spectral Response of Sensors in Fusion
The main objective of all image fusion methods is extraction of information by using the high spatial resolution panchromatic data, which in turn will improve the spatial resolution of the multispectral image by preserving its spectral characteristics.The panchromatic image is closely linked to the intensity component of multispectral image and therefore, the preservation of spectral characteristics is possible only in the event of a spectral equality of these two.However, since the panchromatic image has a higher spatial resolution, exact color preservation is theoretically not possible.The investigations of the correlation between the intensity data acquired from multispectral images and the panchromatic data suggests requirement of additional preprocessing (Wald et al., 1997)  The panchromatic band covers most of the visible and a significant part of the near infrared wavelengths.The measured energy in an individual channel is the integral sum of incoming radiation and relative spectral sensitivity.Therefore, it is possible to obtain the values of the panchromatic band with the summation of respective spectral bands.In ideal conditions all spectral bands would be well separated and would cover exactly the same wavelengths as the panchromatic band.The panchromatic band can be obtained using the individual multi spectral bands as: where Pan, G, R, NIR represent the radiance of individual spectral bands, wG, wR, wNIR are the weights of corresponding MS bands and other for the influence of the spectral range which is missing from MS bands but still covered within the PAN band.
With Equation-1 and an appropriate combination of the respective multi spectral bands, panchromatic band digital values can be computed.Since this is as acquired by the sensor, the relation enables matching by preprocessing of individual bands prior to image fusion.The main reason for changing the panchromatic band is to obtain a strong spectral resemblance to the intensity image, which will be replaced during the fusion process.
Since the simulated PAN band is obtained using Low resolution multispectral bands there is a need to sharpen the spatial details of this band for further use in Fusion process.Hence, a curvelet based fusion is adopted to sharpen the synthesised PAN band with the original PAN band acquired from the sensor.

Curvelet based Image Fusion
Recent studies showed that a wavelet-based image fusion method provides high quality spectral content in fused images.However, most wavelet-based methods yield fused results with spatial resolution less than that can be obtained using the Brovey, IHS, and PCA fusion methods.
In this paper, we use curvelet based fusion to sharpen synthesised intensity image using original PAN band, which represents edges better than wavelets (Verma et al.,2012).Since edges play a main role in image representation, one effective way to enhance spatial resolution is to enhance the edges.The curvelet-based image fusion method provides richer information in the spatial and spectral domains simultaneously.
Curvelets and ridgelets take the form of basis elements, which exhibit very high directional sensitivity and are highly anisotropic.Therefore, the curvelet transform represents edges better than wavelets, and is well-suited for multiscale edge enhancement (Choi et al., 2004).The proposed technique uses a new image fusion method based on RSR curves and curvelet transform.The fused image using these yields almost the same detail as the original panchromatic image, because curvelets represent edges better than wavelets, it also gives the same spectral details as the original multispectral images, using RSR based correlated PAN image.The new method is an optimum method for image fusion.
The remainder of this paper is organized as follows.The next section-2 describes the theoretical basis of the curvelets.RSR and curvelets based image fusion approach for Cartosat-1 panchromatic and Resourcesat-2 Liss4 multispectral images is presented in section-3 and section-4 presents the Data sets used with details of processing, section-5 presents results and analysis and the section-6 gives final conclusions of the proposed algorithm.

CURVELET TRANSFORM
Curvelet transform which is a multiscale geometric transforms was developed in the last few years in an attempt to overcome inherent limitations of traditional multiscale representations such as wavelets.Conceptually, the curvelet transform is a multiscale pyramid with many directions and positions at each length scale, and needle-shaped elements at fine scales.Curvelets have useful geometric features that set them apart from wavelets and the likes (Choi et al., 2004).

Spectral sensitivity of sensors
Spectral sensitivity curves are available from appropriate data providing agencies for all the remote sensing datasets, which includes data sets from Ikonos [Error!
In the equation λ is the wavelength, k the band number, the in-band radiance, L(λ) the spectral radiance at the sensor aperture, and the peak-normalized spectral response.The weights are obtained by comparing the response of individual bands of the sensor and the response of the panchromatic band.
If the respective components of the original multispectral images are combined according to Equation-1, image which closely resemble the corresponding panchromatic image is obtained.Hence, the panchromatic image, which will replace the intensity image, will be spectrally similar to the intensity image if bands not included in the intensity are subtracted from it.
The spectral band to be subtracted from the highresolution panchromatic image has a lower spatial resolution than the original panchromatic image, leading to a deterioration in spatial resolution.If the spectral characteristics are well preserved, the spatial quality is lost.Considering the spectral sensitivity and width of individual bands, the obtained intensity image from the multispectral bands is sharpened using curvelet fusion to preserve the spatial details which will be used for generating the fused output product.

Fusion algorithm using RSR and Curvelets
The   The PAN thus obtained used for PAN sharpening of Liss4 Mx image using synthetic variable ratio fusion.

DATA SETS USED AND PROCESSING
The details of the sample data sets used are Hyderabad tank bund (Latitude=17.413932N,Longitude=78.508638E)surroundings and Karanataka, Kudala sangama surrounding (Latitude=16.209845N,Logitude=76.070546"E)areas acquired by Cartosat-1 and Resourcesat-2 Liss4 Mx.The chosen data sets cover the various thematic features for verifying the proposed algorithm and found that the results are spatial and spectral details preserving.The meta information of the data sets used are given in the Table -2.Table 2: Details of the Data sets used

RESULTS AND ANALYSIS
In order to evaluate the quality of the fused image, many quality metrics were being used in terms of both qualitative and quantitative analyses.Qualitative analysis determines the performance of the fused image by visual comparison between the fused image and raw input images.On the other hand, quantitative analysis determines the performance of the fused image by two variants such as with reference image and without reference image.When the reference image is available, the performance of fused image is evaluated using the metrics such as root mean square error, mean bias, mutual information, etc.When the reference image is not available the performance of fused image is evaluated using the metrics such as standard deviation, mean, etc.
(Error! Reference source not found.)

Visual Analysis
Since the wavelet transform preserves the spectral information of the original multispectral images, it has the high spectral resolution in contrast with the IHS-based fusion result, which has some colour distortion.But the wavelet-based fusion result has much less spatial information than that of the IHS-based fusion result.
To overcome this problem, we used the RSR based curvelet transform in image fusion.Since the curvelet transform is well adapted to represent panchromatic image containing edges, the RSR and curvelet based fusion result has both high spatial and spectral resolution.From the output images of IRS satellite data fusion presented in Figure-4 to 7, it can be noted that both the spatial and the spectral resolutions have been enhanced, in comparison with the original multispectral image.The fused result contains both the structural details of the high spatial resolution panchromatic image and the rich spectral information from the multispectral images.Moreover, compared with the fused results obtained using the wavelet and IHS, the RSR based curvelet fusion result has a better visual effect, such as contrast enhancement.

Quantitative Analysis
In addition to the visual analysis, we conducted a quantitative analysis.The experimental results were analysed based on the mean, standard deviation, β-factor, Q-avg and ERGAS spatial and spectral.Table-3 presents a comparison of the experimental results of image fusion using the RSR and curvelet based image fusion method, the wavelet-based image fusion method, and the IHS and Brovey methods in terms of the above metrics.The metrics β-factor, Q-avg and ERGAS spatial and spectral are generated using fused and input data sets.

3 .
Curvelets efficiently address very important problems where wavelets fail (E.J. Candes, L. Demanet, D. L. Donoho, L. Ying, 2005), three such examples are 1.Optimally sparse representation of objects with edges: Curvelets provide optimally sparse representations of objects which display curvepunctuated smoothness except for discontinuity along a general curve with bounded curvature.Such representations are nearly as sparse as if the object were not singular and turn out to be far more sparse than the wavelet decomposition of the object.2. Optimally sparse representation of wave propagators: Curvelets may also be a very significant tool for the analysis and the computation of partial differential equations.Optimal image reconstruction in severely illposed problems.Curvelets also have special micro local features which make them especially adapted to certain reconstruction problems with missing data.The digital implementation of the second generation curvelet transform in two and three dimensions provided by curvelab-2.1.3has been used in the present work.The first generation digital transformation is based on unequally-spaced fast Fourier transforms (USFFT) while the second is based on the wrapping of specially selected Fourier samples[Error!Reference source not found.].The two implementations essentially differ by the choice of spatial grid used to translate curvelets at each scale and angle.Both digital transformations return a Reference source not found.],QuickBird, and Landsat ETM+ and IRS sensors.Prelaunch Normalised RSR lab measurements for all invidual PAN and Mx bands were used in this paper.The spectral sensitity curves of IRS Cartosat-1 PAN FORE,AFT and Resourcesat-2 Liss4 Mx Band2, Band3, Band4 are shown in Figure-1 below.

Figure 1 :
Figure 1: Spectral response of C1 PAN and RS-2 Liss4 Mx Required weights were calculated by considering the inband radiance at the sensor aperture (Svab Lenarcic and Otir, 2006): block diagram showing the fusion process is shown in Figure-2 below.

Figure 2 :
Figure 2: RSR and Curvelet based Optical Image Fusion As shown in the diagram, using relative spectral response values of the sensors, Cartosat-1 PAN and Resourcesat-2 Liss4 Mx, Equation-1 is solved by multiple linear regression to obtain the band weights.The band weights are in turn used to synthesise PAN band using the Liss4 Mx multispectral bands.The obtained synthesised PAN band is highly correlated with the multispectral intensity image but lacks in spatial details from the original PAN image.The synthesised PAN band is sharpened with

Figure 3 :
Figure 3: Curvelet based Fusion For improving spatial details

Figure 5 :Figure 7 :
Figure 4: Fused output results of Hyderabad scene

Figure 8 :
Figure 8: Spectral Response of radiometric targets after fusion

Table 3 :
Quantitative results of Fused product6.CONCLUSIONWe have presented a new fusion method based on spectral response of the sensor and the curvelet transform for fusing IRS Cartosat-1 PAN and Resourcesat Liss4 Mx images.In this paper, an experimental study was conducted by applying the proposed method, as well as other image fusion methods, for the fusion of Hyderabad and Kudala sangama, Karnataka images.A comparison of the fused images from the wavelet and IHS, Brovey methods was made.Based on experimental results pertaining to the quality metrics-the proposed method provides better results, both visually and quantitatively, for remote sensing optical image fusion.Since the curvelet transform is well adapted to represent panchromatic images containing edges and the spectral response of the sensor preserves the spectral information of the original multispectral image, the fused image has both high spatial and spectral details.As can be seen from the results presented in Table-3, β-factor and Qavg is highest for proposed method, and the ERGAS scores are the lowest.Hence, both visually and quantitatively, the proposed method has the best spatial and spectral preserving characteristics than the methods under comparison.