CHANGE DETECTION VIA SELECTIVE GUIDED CONTRASTING FILTERS

Change detection scheme based on guided contrasting was previously proposed. Guided contrasting filter takes two images (test and sample) as input and forms the output as filtered version of test image. Such filter preserves the similar details and smooths the nonsimilar details of test image with respect to sample image. Due to this the difference between test image and its filtered version (difference map) could be a basis for robust change detection. Guided contrasting is performed in two steps: at the first step some smoothing operator (SO) is applied for elimination of test image details; at the second step all matched details are restored with local contrast proportional to the value of some local similarity coefficient (LSC). The guided contrasting filter was proposed based on local average smoothing as SO and local linear correlation as LSC. In this paper we propose and implement new set of selective guided contrasting filters based on different combinations of various SO and thresholded LSC. Linear average and Gaussian smoothing, nonlinear median filtering, morphological opening and closing are considered as SO. Local linear correlation coefficient, morphological correlation coefficient (MCC), mutual information, mean square MCC and geometrical correlation coefficients are applied as LSC. Thresholding of LSC allows operating with non-normalized LSC and enhancing the selective properties of guided contrasting filters: details are either totally recovered or not recovered at all after the smoothing. These different guided contrasting filters are tested as a part of previously proposed change detection pipeline, which contains following stages: guided contrasting filtering on image pyramid, calculation of difference map, binarization, extraction of change proposals and testing change proposals using local MCC. Experiments on real and simulated image bases demonstrate the applicability of all proposed selective guided contrasting filters. All implemented filters provide the robustness relative to weak geometrical discrepancy of compared images. Selective guided contrasting based on morphological opening/closing and thresholded morphological correlation demonstrates the best change detection result.


INTRODUCTION
Change detection problem means detecting new or disappeared objects on images registered at different moments of time and possibly in various lighting, weather and season conditions.A lot of change detection techniques are developed for remote sensing applications (Singh et al., 1989;Hussain et al., 2013).Two main categories of change detection techniques are pointed: pixel-level and object-level.Pixel-based methods provide better computational efficiency.Object-based techniques provide the high detection quality.
In this paper we propose new morphological filters, which improve the previously proposed change detection technique based on generalized ideas of Morphological Image Analysis (MIA) (Pyt'ev, 1993, Vizilter et al., 2016).Such morphological mid-level change detection provides some compromise between the computational efficiency of pixel-based methods and detection quality of object-based techniques.
The practical contribution of this paper is a new set of selective guided contrasting filters based on different combinations of various smoothing operators (SO) and binary local similarity coefficients (BLSC).The theoretical contribution of this paper is the proof that selective guided contrasting satisfies the conditions of comparative filter stated in (Vizilter et al., 2016).

RELATED WORKS
There are some well-known reviews of change detection approaches both classical and modern enough (Singh et al., 1989;Hussain et al., 2013).In (Hussain et al., 2013) two main categories of methods are pointed: pixel-based change detection (PBCD) and object-based change detection (OBCD) techniques.
The PBCD category of change detection methods contains the direct, transform-based and classification-based comparison of images at the pixel level.Some machine learning techniques are applied at the pixel level too.The OBCD category contains direct, classified and composite change detection at the object level.We start our brief overview from pixel-level techniques and then go to object-level comparison.
The simplest direct image comparison technique is an image difference calculation from intensity values of original or transformed images (Lu et al., 2005).Since relative changes occur in both images, then the direction of image comparison should be selected (Gao, 2009).Image rationing forms regions that are not changed with ratio value approximately equal to 1 (Howarth, Wickware, 1981).Image regression represents second image as a linear function of first one (Ludeke et al., 1990).A regression analysis, such as least-squares regression, is used for identification of regression parameters (Lunetta, 1999).Changes are detected by subtracting regressed image from the original one.
Transform-based imaged comparison presumes the analysis of transformed images.Change vector analysis (CVA) was developed for change detection in multiple image bands (Bayarjargal et al., 2006).Change vectors (CV) are calculated by subtracting pixel vectors of co-registered different-time dates.The direction and magnitude of CV correspond to the type and power of change.Principal component analysis (PCA) is applied for change detection in two main ways.The first one is to apply PCA to images separately and then compare them using differencing or rationing (Richards, 1984).The second way is to merge the compared images into one set and then apply the PCA transform.Principal components with negative correlation should correspond to changes in compared images (Deng et al., 2008).Tasselled cap transformation (KT) is a particular case of spectral transform presented in (Kauth, Thomas, 1976).It produces stable spectral components which allows developing baseline spectral information for long-term studies of forest disturbances (Jin, Sader, 2005) or vegetation change (Rogan et al., 2002).Different texture-based transforms are developed and used, for example, for urban disaster analysis (Tomowski et al., 2011) and land use change detection (Erbek et al., 2004).
Classification-based change detection contains the postclassification comparison techniques and composite classification methods.Post-classification comparison presumes that images are first rectified and classified (Bouziani et al., 2010).The supervised (Ji et al., 2006) or unsupervised classification (Ghosh et al., 2011) can be of use.Then the classified images are compared to measure changes.Unfortunately, the errors from individual image classification are propagated into the final change map, reducing the accuracy of change detection (Lillesand et al., 2008).In the composite or direct multidate classification (Lunetta, 1999), (Lunetta et al., 2006) the rectified multispectral images are stacked together and PCA technique is often applied to reduce the number of spectral components to a fewer principal components (Mas, 1999), (Singh, 1989).The minor components in PCA should represent changes (Collins et al., 1996).But due to the fact that temporal and spectral features are fused in the combined dataset, it is difficult to separate spectral changes from temporal changes in the classification (Schowengerdt, 1983).
Machine Learning algorithms are extensively utilized in change detection techniques.Artificial Neural Networks (ANN) are usually trained by supervised learning on a large training dataset for generating the complex non-linear regression between input pair of images and output change map (Dal, Khorram, 1999).ANN approach was applied for land-cover change detection (Dal, Khorram, 1999), (Abuelgasim et al., 1999), forest change detection (Woodcock et al.) and urban change detection (Pijanowski et al, 2005).The Support Vector Machine (SVM) approach based on well-known SVM technique (Vapnik, 2000) considers the finding change and no-change regions as a binary classification problem (Huang et al., 2008).The algorithm learns from training data and automatically finds the binary classifier parameters in a space of spectral features (Bovolo et al, 2008).SVM approach is used for land cover change detection (Nemmour, Chibani, 2006) and forest cover change analysis (Huang et al., 2008).Some other machine learning techniques are applied for change detection via learning to change and non-change separation: decision tree (Im, Jensen, 2005), genetic programming (Makkeasorn et al., 2009), random forest (Smith, 2008) and cellular automata (Yang et al., 2008).Object-based techniques operate with objects instead of pixels.The Direct Object change detection (DOCD) is based on the comparison of objects extracted form compared images.Changes are detected by comparing either geometrical properties (Lefebvre et al., 2008) or spectral information (Miller et al., 2005) or extracted features of the image objects (Lefebvre et al., 2008).In Classified Objects change detection (COCD) approach the extracted objects are compared based on information about both the geometry and the class membership (Chan, Kelly, 2009).OBCD framework based on postclassification comparison was proposed in (Blaschke, 2005).Different algorithms like decision-tree and nearest neighbor classifier (Im, Jensen, 2005), fuzzy classification (Durieux et al., 2008), and maximum likelihood classification (MLC), are used for extracting objects and independently classifying them.Some applications of COCD is updating maps or GIS layers.
COCD is applied for forest change detection (Hansen, Loveland, 2012), land cover and land use change analysis (Gamanya et al., 2009) and so on.Multitemporal-object change detection presumes that the joint segmentation is performed once for stacked (composite) images.In (Stow et al., 2008) the multi-temporal composite images are used both at segmentation and classification stages for map vegetation change objects.Clustering on multi-date objects for deforestation analysis if proposed in (Duveiller et al., 2008).
There are some combined approaches those utilize different combinations of described ideas.In (Al-Khudhairy et al.,2005) particular, change detection is performed via differencing after PCA.In (Niemeyer, Nussbaum, 2006) the pixel-based information is combined with object-based information via pixel labeling based on statistical and semantical models.
This work presents a new morphological filters, which improve the previously proposed change detection technique based on generalized ideas of Morphological Image Analysis (MIA) (Pyt'ev, 1993, Vizilter et al., 2016).Let's note that terms "morphology", "morphological filter" and "morphological analysis" refer to Mathematical Morphology (MM) proposed by Serra (Serra, 1982) as well as to MIA.These theories of shape have a common algebraic basis (lattice theory), but different tasks and tools.The overview of MIA and its relation to MM is given in (Vizilter et al., 2015).Morphological change detection approach is based on the analysis of morphological difference map formed as a difference between test image and its morphological projection to the shape of sample image.In our generalized approach the role of morphological projector is played by comparative morphological filter with weaker properties, which transforms the test image guided by the shape of sample image.The shape of sample image is described by mosaic segmentation or by local texture features of objects (regions).So, such morphological approach implements some important properties of object-level image comparison immediately in the pixel-level image filtering.Due to this, we can speak about the morphological mid-level change detection procedure.It should provide the desired compromise between the computational efficiency of pixel-based methods and detection quality of object-based techniques.

METHODOLOGY
This section describes the previous and proposed methodology.

Guided contrasting filters
The scheme Fig. 1 demonstrates the main idea of this approach.Let two images are given: test image g(x,y) and sample (or reference) image f(x,y).The guided contrasting filter of g with respect to f can be formally described in the following form where g S =Sg is a result of filtering of g by some smoothing operator S; o(x,y)  constany constant-valued (flat) image; w(x,y) is a sliding window at position (x,y); a(f,g w(x,y) ) is a local similarity coefficient (LSC) of test image fragment g w(x,y) with sample f.In order to provide the robustness relative to weak geometrical discrepancy of compared images practice we proceed to guided contrasting filter with local search (in some search zone p(x,y)).
Figure 1.The scheme of local guided contrasting.

New set of selective guided contrasting filters
We propose and implement new types of guided contrasting filters based on different combinations of various SO and LSC.
Following "smoothing" operators are considered: -linear mean and Gaussian smoothing; -nonlinear rank filtering (min, max and median); -filters of mathematical morphology (opening and closing based on structuring elements) (Serra, 1982).
In selective guided contrasting filters we use binary local similarity coefficient (BLSC) a t (f,g w(x,y) ){0,1} instead of a(f,g w(x,y) ) in (1).It is formed via binarization of LSC with some fixed threshold t: Thresholding of LSC allows operating with non-normalized LSC, in particular, MI.Additionally, this thresholding of LSC enhance the selective properties of guided contrasting filter: details are either totally recovered or not recovered after the smoothing.Due to this such filters called "selective".
For all proposed selective guided contrasting filters we prove that some threshold t exists such that these filters satisfy the conditions of morphological comparative filter: stated in (Vizilter et al., 2016): .

Mutual information
Mutual information I(A,B) (Maes, 1997)  Different successful application were created based on this MI approach in recent years (Goebel, 2005).

Morphological image analysis and geometrical correlation
Morphological Image Analysis (MIA) proposed by Pytiev is based on geometrical and algebraic reasoning (Pyt'ev, 1993).In the framework of this approach images are considered as piecewise-constant 2D functions where nnumber of non-intersected connected regions of tessellation F of the frame , F={F 1 ,…,F n }; f=(f 1 ,…,f n )corresponding vector of real-valued region intensities; Set of images with the same tessellation F is a convex and close subspace FL 2 () called shape-tessellation, mosaic shape or simply shape: For any image g(x,y)L 2 () the projection onto the shape F is determined as In (Vizilter, Zheltov, 2012)

EXPERIMENTS
The results of experimental exploration of both comparative filtering and proposed change detection pipeline are reported in this section.In the first part of section some examples of guided contrasting and corresponding morphological difference map forming are demonstrated applying to real images for different scene types and change detection cases.In the second part the results of change detection experiments on the public benchmark containing simulated aerial images are described.Interesting results are obtain with thresholded MI combinations (Fig. 3) where the difference maps contains only pronounced changes.Also, it should be noted that filtering with MSMCC as LCC is not robust due to the segmentation step to forming the mosaic shapes (4) of etalon and test images.
As noted in (Vizilter et al., 2016), some additional analysis of formed morphological difference map is needed for final testing of the formed change proposals based on other type of taskspecific information and guided contrasting filtering and corresponding morphological difference maps can be useful as parts of different task-oriented change detection pipelines.

Quantitative change detection experiments
All proposed guided contrasting filters are tested as a part of previously proposed change detection pipeline (Vizilter et al., 2016).It contains the following steps: 1. Guided contrasting (1) using the image pyramid; 2. Calculation of morphological difference map (2); 3. Binarization and filtering of MDM; 4. Forming change proposals; 5. Testing change proposals using local MCC; 6. Forming the output binary map of changes.
In our experiments with proposed change detection pipeline for long-range remote sensing we use the public Change Detection dataset introduced in (Bourdis et al., 2011) (Fig. 5).This dataset contains 1000 pairs of 800x600 simulated aerial images and 1000 corresponding 800x600 ground truth masks.Each pair consists of one reference and one test image.Some of image pairs contain scene changes and illumination differences.The dataset consists of 100 different scenes with moderate surface relief and several objects (trees, buildings etc.).Each scene is rendered with various viewpoints.The cameras are distributed at steps of 10 degrees on a circle of radius 100 meters at approximately 250 meters high, and with a fixed tilt of about 70 degrees.All images are modelled with a ground resolution of about 50cm per pixel.
The methodology of our experiments is the following.We select a subset of 100 reference and test image pairs for 50 different scenes with 0 degrees relative camera angle.As proposed in (Bourdis et al., 2011) we compare the detection results with respect to the ground truth at pixel level, but calculate the precision and recall values at the object (region) level.In order to do this, we from the list of ground truth objects and list of detected objects (accepted regions of filtered binarized morphological difference map).Then we perform the object-toobject comparison via computing of object intersection area.If the intersection area is more than 50% then we decide that objects match each other.The numbers of true and false object detections determine the corresponding precision and recall values.
We implement and test our pipeline with following parameters: guided contrasting window size is 77 pixels; number of pyramid levels is 3; the size of disk structuring element in MM opening and closing is 5 pixels, the threshold value for morphological correlation coefficient at the final testing step is 0.5.(Vizilter et al, 2015), (Bourdis et al.,2011), excepting filter with MSMCC as LSC.As noted above (sect.4.1) we observe low robustness because there is the segmentation procedure for forming the mosaic shapes (4) of input images in the pipeline.Combinations with LCC and nonlinear smooth operators of median and Serra's opening and closing gives the best results in the experiments.
estimates the dependence of two random variables A and B by measuring the distance between the joint distribution p AB (a,b) and the distribution of complete independence p A (a)p B (b): A) is an entropy of A, H(B) is an entropy of B, and H(A,B) is their joint entropy.For two image intensity values a and b of a pair of corresponding pixels in the two images, required empirical estimations for the joint and marginal distributions can be obtained by normalization of the joint (2D) and marginal (1D) histograms of compared image fragments.


Pytiev morphological comparison of images f(x,y) and g(x,y) is performed using the normalized morphological correlation coefficients of the following form The first formula estimates the closeness of image g to the "shape" of image f.Second formula measures the closeness of image f to the "shape" of image f.For elimination of constant non-informative part of image brightness following image normalization is usually performed: fprojection of image f onto the "empty" shape O with one flat zone.This projection is a constant-valued image filled by mean value of projected image.
the geometrical shape comparison approach was developed based on Pytiev's morphological image analysis.Let f(x,y) from F is a piecewise-constant 2D function described above and image g(x,y) from G is an analogous 2D function with m as a number of tessellation regions G={G 1 ,…,G m }; g=(g 1 ,…,g m )vector of intensity values;  Gj (x,y){0,1}support function of j-th region.Let's introduce following additional set of "S-variables": Sarea of the whole frame ; intersection F i G j .Mean square effective morphological correlation coefficient (MSEMCC) for shapes F and G is determined as , (F i ,G j ) = S ij / Snormalized influence coefficient for pair of regions F i and G j ; K M 2 (G j ,F i ) = S ij / S jsquare of normalized morphological correlation for pair of regions.

4. 1 Figure 2 .Figure 3 .Figure 4 .
Figure 2. Example of morphological difference maps based on guided contrasting with various combinations of LSC and SO (outdoor video surveillance)

Figure 5 .
Figure 5. Example of simulated data from benchmark: a) reference image; b) test image; c) ground truth mask.

Table 1 .
Results of quantitative experimentsResults of quantitative experiments demonstrates that comparative filtering based on guided contrasting with considered variations of thresholded LSC and SO parameters is generally better, than approaches propose and implement new set of selective guided contrasting filters based on different combinations of various smoothing filters and thresholded local similarity coefficients.Qualitative experiments demonstrate their applicability and robustness relative to lighting changes and weak geometrical discrepancy of compared images.Quantitative experiments on the benchmark containing simulated aerial images demonstrate that the best change detection rate is provided by selective guided contrasting based on non-linear smoothing operators (median and Serra's morphological opening and closing) and thresholded normalized linear and morphological correlation for detail recovery.