A NOVEL FUSION-BASED UNSUPERVISED APPROACH FOR MULTISPECTRAL IMAGE CHANGE DETECTION WITH SALIENCY MAPS

To fully utilize the spectral information and remove noise in multispectral image change detection, A fusion-based unsupervised approach, which exploits NSCT (Nonsubsampled Contourlet Transform) and multi-scale saliency maps for detecting changed areas by using multispectral images is presented in this paper. Firstly, aiming at make full use of multispectral information, each band of the multitemporal images is applied to get an initial difference image set (IDIS), which is then decomposed into several low-pass approximation and high-pass directional sub bands by NSCT; In order to remove most of the noise, saliency maps of each sub bands and each scales are obtained by processing only the low-frequency sub-band coefficients of the decomposed image; Finally the binary change map is extracted by using a novel inter-scale and inter-band fusion method. Experimental results validate the superior performance of the proposed approach with respect to several state-of-the-art change detection techniques. * Corresponding author


INTRODUCTION
Remote sensing image change detection is the technique of qualitatively or quantitatively analyzing and determining the characteristics of surface change in multi-temporal remote sensing images of the same geographical area but from different times (Singh, 1989).In recent years, with the development of aerospace and imaging technology, remote sensing image change detection technology has played an active and important role in many fields such as land dynamic detection, forest vegetation change, urban expansion and disaster monitoring (Radke and Andra, 2005).However, since the quality of remote sensing images is susceptible to seasonal change, atmospheric radiation and other factors, how to accurately determine the changed and unchanged areas from the image pairs still remains a difficult problem.
Change detection algorithms can be divided into two categories on the basis of whether priori information is used during the process, supervised and unsupervised method.Unsupervised change detection is mainly implemented by using clustering algorithms (Celik 2009;Singh 2014;Ding et al. 2015) or statistical models (Bruzzone 2000;Bazi et al. 2005;Celik et al. 2011;Zanetti et al. 2016) based on the difference images.The advantage of unsupervised method is that there is no need for a prior knowledge of the ground truth information about the study area but it is hard to get an accurate difference map.Some fusion-based approaches on the ground of statistical models like DeNSCT (Li 2012) has also been proposed but it fails to balance the approximate part and detailed part of the difference image.The supervised change detection method mainly includes two aspects: post-classification algorithms (René 2012;Seebach 2013) and classifier-based algorithms such as Support Vector Machine (SVM) (Volpi 2010), neural network (Wang 2015) and random forest (Wessels 2016).This method achieves higher accuracy and identify the type of land cover but proper training set is needed, which may cause a lot of inconveniences.So unsupervised methods are still widely used and studied.
To address the problems discussed above, in this paper, we proposed a fusion-based unsupervised change detection method with saliency maps for multispectral images.As is illustrated in Figure 1, firstly each band of the multitemporal images is applied to get an IDIS, which is decomposed into several lowpass approximation and high-pass directional sub bands by NSCT; Then saliency maps of each sub bands and each scales are obtained by processing only the low-frequency sub-band coefficients of the decomposed image; The final change map is extracted by a novel inter-scale and inter-band fusion method.Experimental results validate the effectiveness and feasibility of the proposed method.

Initial Difference Image Set
In the first chapter, let us consider a pair of two registered multispectral images with the size of I J  and N bands, acquired at the same location but different time distances.The initial difference image set (IDIS) is expressed as follows: where n D is the initial difference image set and n is the number of the band.

Nonsubsampled Contourlet Transform(NSCT)
The NSCT (Da 2006), based on the theory of contourlet transform (Do 2005), is a flexible multiscale, multi-direction, and shift-invariant image analysis method, which is able to sufficiently capture the geometrical details of the source image (Li 2012).
The main structure of NSCT is composed of two filter banks, Non-Subsampled Pyramid Filter Banks (NSPFB) and Non-Subsampled Directional Filter Banks (NSDFB) .The NSPFB is a multi-scale decomposed structure developed from átrous algorithm.Images or signals can be decomposed into a low-pass sub band (also called approximate sub band) and a bandpass one without subsampled processes.The sequent scale is repeated by the former low-pass sub band iteratively.As a result, a tree-like structure that enables multiscale decomposition is built.The NSDFB, constructed by combining the directional fan filter banks, is used to decompose the bandpass sub-band into several directional sub-bands without subsampled process, too.If L levels of directional decomposition on a sub-band that decomposed by NSPFB in a certain scale is conducted, then 2 L number of band-pass subimages, the same size to original one will be produced.
Compared with other multiscale methods (wavelet transform, contourlet transform et al.), NSCT is multi-directional, anisotropic, translation invariant, and capable of accessing the accurate edge, contour, and details information of an image.A brief structure of NSCT with two levels is depicted in Figure 2. while neglecting high-frequency information.Suppose the IDIS is decomposed into K levels, the approximated coefficient set is given by: where k is the scale level and n is the number of band.

Saliency Maps Extraction
Visual saliency is the perceptual quality that makes an object, person, or pixel stand out relative to its neighbors and thus capture our attention (Achanta 2009).It presents a hotspot in the area of image segmentation, video monitoring, target detection and tracking and so on.
Recently, several researchers have applied visual saliency theories into remote sensing change detection due to the local contrast between changed and unchanged areas in difference image (Tian 2007;Hou 2016;Zheng 2016).In this paper, we introduce a frequency-tuned saliency detection (Achanta 2009) method (FT method for precision) to guide the change detection result.It can output full resolution saliency maps with welldefined boundaries of salient objects by retaining substantially more frequency content from the original image.FT method mainly consists of two parts: (1) DoG band pass filtering and (2) Saliency computation.Following are the details.
(1) DoG band pass filtering is used to the initial image to avoid the impact of noise, coding artifacts, and texture patterns.The equation is defined as formula (3)    is the union operator.

Dataset
In order to validate the effectiveness of the proposed method, a pair of real bi-temporal images over Beijing urban area (Hou 2016) acquired on 28 June 2009 and 19 September 2015 respectively, is used in our experiment.This dataset contains 3 bands in visible spectrum with a size of 540×537 pixels and spatial resolution is 4 m.For the sake of achieving higher detection accuracy, typical pre-processing steps like image registration, radiation correction are executed in the first place.The pre-processed image is shown in Figure 3, from which we can clearly see that several buildings have changed greatly during this period of time.

Results and Analysis
To evaluate the performance of proposed method, we compared four existing change detection algorithms, PCAKmeans (Celik 2009), DeNSCT (Li 2012), FKLICM (Singh 2014), SHC (Ding 2015).The qualitative results are displayed in Figure 4 ((f) is the ground truth acquired by visual interpretation) while the quantitative results are presented in Table 1.From Figure 4 and Table 1 we can see that our method contains less false alarmed pixels, which can be seen clearly in the red rectangle, but the cost is the increase of the missed alarmed pixels (as shown in yellow rectangle).The PCAKmeans method uses block-based data analysis in order to reflect the contextual information, but if the false alarmed pixels cover a large region, it may fail to judge whether they change or not.DeNSCT employs both the high-pass and low-pass sub band for change detection, it can fully expresses the details information of the changed object, but it's hard to decide the portion of the high-pass sub band.FKLICM exploits both the Kohonen network and FLICM to detect changed area of remote sensing images, the obvious problem is it fails to consider the relationship of different band of multi-spectral images.SHC method generates massive changed areas to avoid making fragmentary patches, which causes too much false alarmed pixels in the resultant mask.In our experiment, decomposed scale was set as 4. In order to verify the impact the scale k would have on the final result, we set k as 1,2,3,4,5,6 respectively.The below histogram gives a general overview of the influence of different scales on the result.False alarm decreases and missed alarm increases when k changes from 1 to 6 and when k equals to 4, the total error reaches a minimum value, see Figure 5.

CONCLUSION
In this paper, an unsupervised approach with Nonsubsampled Contourlet Transform and multi-scale saliency maps change detection method is proposed for multispectral images .By decomposing the initial difference image set into several scales, most of the noise could be removed.Then a frequency-tuned saliency detection method is introduced to obtain the saliency maps in which the changed and unchanged area is clearly discrepant.Next EM algorithm is applied to these saliency maps to acquire the initial change mask set.In order to comprehensively utilize the information of each band and each scale, the final binary change mask is extracted by a novel interscale and inter-band fusion procedure.Given our experimental results, we find that this method can accurately detect the changed area for multi-spectral images.

Figure 1 .
Figure 1.Framework of the proposed method

Figure 2 .
Figure 2. Brief structure of NSCT with two levels Since the directional sub bands contain the most part of the noise, which may cause high false alarm rate in change detection result, in our method, only the approximate sub band is considered.These approximated images on different scales are acquired only by reconstructing the low-frequency image


is set to infinity.So formula (3) is just equal to a simple Gaussian filtering.The Gaussian blurred image is calculated as is the Gaussian blurred image,   , I x y is the initial image and  stands for convolution operation.

I
is the mean image feature vector, and () ( , )L g xy I is the corresponding pixel vector value of the Gaussian blurred image.In our case, since the single band is used for saliency detection, this process just is conducted on the original colour space and the formulation is as follows: value of pixel at the location of   , xy .The resultant saliency maps set of the approximated coefficient collection is expressed in Equation (7). in (7), the change detection mask at scale k from n th band (denoted as n k SaDM ) is acquired by EM algorithm.Considering that the details would decrease and the approximated image would become vaguer and vaguer as the scale k increases, the communal part of the masks within a band may have the highest possibility to change.On the basis of this tuition, an inter-scale fusion is adopted to produce a series of change masks.Let n SaDM denote the inter-scale fusion result, the fusion method is shown in Equation (8).

where
AND operator.Since each band of the multispectral image contributes to the change detection result, the final change detection mask can be generated by conducting an inter-band fusion operation, which is written as 1 SaDM represents for the final binary change mask and Experimental images.(a) image in 2009 (b) image in 2015 Ours (f) Ground truth Figure 4. Results of other method and ours and the ground truth

Figure 5 .
Figure 5.The influence of different scales

Table 1
Quantitative results of five methods (false alarm, missed alarm and total error stand for the number of pixels)