A JOINT PIXEL AND REGION BASED MULTISCALE MARKOV RANDOM FIELD FOR IMAGE CLASSIFICATION

MRF model is recognized as one of efficient tools for image classification. However, traditional MRF model prove to be limited for high resolution image classification. This paper presents a joint pixel and region based multi-scale MRF model for high resolution image classification. Based on initial image segmentation, the region shape information is integrated into MRF model to consider the pixel and region information simultaneously. The region shaped information is used to complement spectral signature for alleviating spectral signature ambiguity of different classes. The paper describes the unified multi-scale MRF model and classification algorithm. The qualitative and quantitative comparison with traditional MRF model demonstrates that the proposed method can improve the classification performance for regular shaped objects in high resolution image. * Corresponding author.


INTRODUCTION
Image classification is a process to categorize all pixels in an image into one of several predefined classes.Each pixel is assigned a class label based on the spectral signature of the pixel and the relationship with its neighbors.The performance of image classification is of paramount importance for many subsequent image based applications.Information extraction based on remote sensing image is an important application of image classification.Once the classification result is obtained, the categorized data can be used to produce thematic map of land cover present in the image.With the availability of high resolution remote sensing image, image classification has been applied in several diversity applications, such as road network extraction, urban planning and natural disasters prevention.The classification of moderate spatial resolution remote sensing image is mostly carried out based on the statistical separability of each classes (Melgani F., et al, 2000).The performance of pixel level classification is deteriorated when high resolution image is under processed.The improvement of spatial resolution increases the internal spectral variability within the same land cover classes and decreases the spectral variability between different classes.As a result, pixels will be misclassified due to the ambiguity between different classes.Despite the characteristic of high resolution image increase the complexity of classification problem, it provides geometrical information to be considered in the process of classification.So, the critical part of high resolution image classification is how to combine the spatial and spectral information to mitigate the spectral ambiguity between different classes.Previous work on high resolution image classification has been done by several authors.Unsalan (Cen Uslan, and Kim L Boyer, 2004) proposed a technique to identify different classes by using statistical property of extracted straight lines and spatial coherence constraint.Lee (Sanghoon Lee, Melba M. Crawford, 2005)presents a multi-stage image classification method.This method first segments image by making use of spatial contextual information that characterizes the geophysical connectedness of image structure.Then the segment results are classified into distinct states by sequential region merging.Bruzzone (L.Bruzzone, and L.Carlin, 2006)proposes a supervised method to classify high resolution image.Adaptive multilevel spatial context driven feature extraction is first performed, and then the SVM classifier is used to obtain classification results.In (L.Zhang, et al, 2006), a pixel shape index is introduced to represent geometrical information and SVM is used to classify the image.Z. Lei(Z.Lei, et al, 2011) introduce the conditional texton forest to take use of spatial contextual information to perform the land cover classification for very high resolution image.Bellens(Rik Bellens, et al, 2008) uses morphological profile to extract the geometrical information as complement to spectral information.Bouziani(Mourad Bouziani, et al, 2010)proposes a rule based method for high resolution image classification by using spectral, geometric, and contextual information.The aforementioned previous work on high resolution image classification shows that the combination of geometrical information and spectral information are critical part of high resolution image classification technique.Although great efforts have been taken on this topic, few of the proposed methods can meet the end-user requirement on accuracy and efficiency.For this reason, there still remain many open issues that allow further investigation.Since the complexity of high resolution image classification, besides the geometrical information, it is necessary to take into account the contextual information about the image under analysis.The Markov random field(MRF) is a powerful tool to model the contextual information about the image.Bayesian approach is a simple, yet effective tool to integrate both the contextual information and statistical model of image in the classification process.In fact, the Bayesian approaches have been widely applied to perform image classification.At the early stage of MRF used for image classification, it is defined on the pixel level (Besag. J, 1986), because the neighboring relationship among pixels is regular on the 2D lattice, the pixel-based MRF methods could conveniently model the spatial contextual information.However, pixel level MRF can not consider information, such as shape, texture and spatial relation of land cover classes for classification.Hence, many researchers have extended the MRF model from the pixel level to the region level (Zhang, L., Q. Ji, 2010).The region-based MRF methods usually divide an image into over segmented regions firstly(Antonis Katartzis, et al, 2005).Then, the regionbased MRF model is defined on these initial regions to obtain the finally classification results.Although the MRF at the region level overcomes some shortcomings of the pixel-based MRF, it still suffers the inaccuracy of the over segmented regions and the irregular spatial contextual relationship.In order to improve the ability to describe large scale behaviors, both pixel and region based MRF model can be extended to multiscale MRF(MSRF).The inter-scale dependencies of multi-scale class labels across scales can be captured with MSRF structures, and a non-iterative algorithm can be developed to speed up classification.
In this paper, we propose a new classification method that unifies the pixel level and region level MRF in multi-scale space(UMSRF).This method attempts to improve the MSRF(C.Bouman, M. Shapiro, 1994)model by taking advantages of both pixel and region based MRF to get better classification result.The classification method is carried out on multi-scale region adjacency graph(RAG), which can utilize information about region shape, texture.Specifically, we focused on how to introduce shape information into MSRF model to mitigate appearance ambiguity between different land cover classes.This is motivated by the fact that most man made objects in remote sensing image can be modeled by simple mathematical function, and thus can be easily integrated into MRF model.In order to take into account the pixel and region features, the likelihood function of UMSRF is decomposed into the product of the pixel likelihood function and the region likelihood function.Region-based likelihood function is based on the introduced region feature, which captures the interaction between regions and characteristics within a region.The UMSRF based method consists of two modules, multiscale image segmentation and inference of land cover classes label of each pixel.The first module is about to build image pyramid and partition input image at each scale.Then region feature is extracted to describe region shape and contextual information.The hierarchical segmentation is carried out in wavelet domain, taking wavelet coefficients as image feature.The watershed transform is used to partition the image at each scale.The second module is to assign a class label to each pixel.The standard two sweep forward-backward algorithm is extended to integrate the pixel and region information.The upward sweep starts at the finest scale to compute the likelihood which takes into account the interaction across scales.The upward procession repeated until reaches the coarsest scale.Then the downward sweep starts at coarsest scale to get the label of each pixel.In this sweep, the label of each pixel is obtained by maximizing the posteriori probability.When computing the posteriori probability, the likelihood is decomposed into pixel likelihood and region likelihood.The process repeated until reaches the finest scale.The UMSRF model parameters were estimated by EM algorithm This rest of the paper is organized as follows.After brief review of MRF model based image classification in section 2, the multi-scale image segmentation, region shape feature extraction and UMSRF model are discussed in section 3 in detail.In Section 4, we illustrate classification results on high resolution image and perform a comparison of our method with pixelbased classification approaches that follow the Bayesian inference.Finally, conclusions and directions for future research work are given in Section 5.

FRAMEWORK OF MRF MODEL BASED CLASSIFICATION
This section briefly presents the framework of MRF based image classification.Let S denote a set of sites, } is the observed random field defined on S which represent the spectral statistical property at each site and is denoted as the occurrence of Y. MRF model assumes that the behavior of Y is dependent on a corresponding unobserved label field.The unobserved label field is denoted as and take their value in a discrete set L={1…M}, where M is the total number denote a realization of X.The image classification is to estimate the that maximizes the posterior probability x PX y , given the observed image y.Under the Bayesian law, the that maximizes the x x y is equal to maximize .
( | ) ( ) The joint probability models the spatial context of different land cover object.The label random field X is assumed to possess the Markovianity property, and then it follows Gibbs distribution.The multilevel logistic model is often used to model the spatial contextual relationship.The MLL model favors smooth classification result.This could make the MRF model resist noise and reduce the impact of intra-class variation.

( ) P X
The likelihood function ( | )  P y X is used to model the statistical characteristic of observed image given the label field.The Gaussian distribution is usually employed to model ( | )  P y X for simplicity.MRF models can be defined both on pixel level and region level after initial segment.For the Pixel-based MRF model, each element


are the observed image data and label for each pixel, respectively.Due to the regular spatial context, one can conveniently define the neighborhood system for the MRF, such as the 4-neighborhood system and the 8-neighborhood system.However, the local pixel-based neighborhood relationship is limited to describe large range interaction of image data and limit the classification accuracy. are the region feature and label of the region s, respectively.The region level observation field could enhance the ability of the MRF for describing the region geometrical information, which would improve the classification accuracy.However, it also brings disadvantages at the same time.There are mainly two kinds of disadvantages.First, the initially over segmentation may be imprecise.As mentioned in (Kuo. W.F. and Y.N. Sun, 2010), the approaches used for initially segmented, such as watershed, has some imprecise segments that can't be redressed in the following processes.Second, the spatial context relationship is irregular.Both the pixel and region level MRF can be defined in multiscale space to model large range of interaction.The MSRF will be discussed in section III.

UNIFIED REGION BASED MULTISCALE MRF MODEL
This section discusses the proposed unified multi-scale region based MRF model.It is an extension of the multi-scale random field model(MSRF) proposed by Bouman(C. Bouman, M. Shapiro, 1994).We first briefly present the MSRF model, and then discuss the proposed unified multi-scale region based MRF model.

MSRF Model and Statistical Inference
3.1.1MSRF model Defined on Quadtree: Multi-scale random model composed of a series of random field at different scale, which aims at to capture the interaction between neighbored scales of random field.The random field which represents the spectral properties at each site at given scale is the observations field that needs to be classified into different groups of distinct statistical behavior.At given scale, we can define restriction of label random field X to level S n : { , } n s , S n is the set of sites at scale n.Similar notations holds for occurrence of X: x s S  .In the same way, observation field Y and its occurrence y can be partitioned as Two assumptions are made about the label field X.First, the element in will be conditionally independent, given elements in .Second, each element n s X will be only dependent on a local neighbourhood of elements at the next coarse scale.Based on these two assumptions, the transition probabilities must have the form.The MSRF model of observation and label are define on quadtree.Each node at coarse scale corresponds to four node at next fine scale.Given the label of parent node, the probability of the child node take the value of given label is given as follows:  is the probability that the labeling remain unchanged from scale n+1 to n.This model favors identity between the parent and children, all other transitions being equally likely.

Statistical Inference on Quadtree:
In most case, the MAP criterion is too conservative to get meaningful segmentation since it does not consider how much difference these configurations are.Bouman[3] introduced a new estimator to take into account the location of estimation errors in the hierarchical structure, which has been named "sequential MAP".The higher a node on the tree, the more error estimation cost is set by this estimator.By using Bayesian rule, the Markov properties of X, and assuming that labeling field at the coarsest scale is uniformly distributed, the inference of X is performed with following topdown recursion: (4) The conditional distribution of Y can be computed in preliminary bottom-up sweep: .Therefore, both the multi-scale pixel and the region information could be considered in the UMSRF model.In this case, the best realization given by ( 4) is reformulated as:

Image Segmentation and Region Feature extraction
We need to first segment the input image in multi-scale space to get initial segment results.After multi-scale segmentation is performed, region features are extracted to calculate region likelihood ( | ) 1  is the probability density for s x given its parent s x  at the next coarse scale.For the observation field, the conditional distribution of observation Y at the finest scale has the following form: is the conditional density for an individual element s y given the class label.
likelihood and transition probability is estimated by using the method proposed in[].Region level likelihood is assumed to be Gaussian distribution and is determined based on region feature extraction results.
at each scale.International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australiathe influence of region likelihood would decrease if the region shape is irregular.This work will be done in future.