GAUSSIAN MIXTURE MODEL AND RJMCMC BASED RS IMAGE SEGMENTATION

ABSTRACT: For the image segmentation method based on Gaussian Mixture Model (GMM), there are some problems: 1) The number of component was usually a fixed number, i.e., fixed class and 2) GMM is sensitive to image noise. This paper proposed a RS image segmentation method that combining GMM with reversible jump Markov Chain Monte Carlo (RJMCMC). In proposed algorithm, GMM was designed to model the distribution of pixel intensity in RS image. Assume that the number of component was a random variable. Respectively build the prior distribution of each parameter. In order to improve noise resistance, used Gibbs function to model the prior distribution of GMM weight coefficient. According to Bayes' theorem, build posterior distribution. RJMCMC was used to simulate the posterior distribution and estimate its parameters. Finally, an optimal segmentation is obtained on RS image. Experimental results show that the proposed algorithm can converge to the optimal number of class and get an ideal segmentation results.


INTRODUCTION
Image segmentation is one of the important steps in the image processing.The good segmentation result has an important influence on other works in image processing (Drǎgut, 2010).With the increasing improvement of the remote sensing image resolution, that poses the challenge to the image segmentation method (Meinel, 2004).
Gaussian Mixture Model (GMM) (McLachlan, 2000;Blake, 2004), that Gaussian distribution is used to describe the distribution of pixel intensity of homogeneous area, is widely used in image segmentation.Because of traditional GMM only uses pixel gray information, and without the pixel space location information (Blekas, 2005).Therefore, this segmentation method is extremely sensitive to image noise.To overcome the shortcoming, spatial neighborhood is imposed.Recently, Markov random field (MRF) (Pal, 1993;Hou, 2011) is well-known method to reduce noise influence in image segmentation.Many MRF variants functions are proposed.Such as, Sanjay-Gopal (1998) proposed spatially variant finite mixture model, called SVFMM.Nikou (2007) proposed directional class adaptive spatially variant finite mixture model, called DCA-SVFMM.
Estimating parameters method is usually EM algorithm (Ji, 2012;Zhang, 2001.) in image segmentation based GMM.But, due to impose spatial neighborhood by MRF into the segmentation model, EM algorithm becomes more difficult and complicated.At the same time, the number of component is fixed in EM algorithm.The correct number of class would get well segmentation results.Therefore, estimating the number of classes is a difficult and important task.So it receives great attention (Zhao, 2016).Reversible jump Markov Chain Monte Carlo (RJMCMC) method (Kato, 2006;Zhang, 2004.) is widely used to image segmentation to estimate the number of classes.
This paper proposed a RS image segmentation method that combining GMM with RJMCMC.In the proposed algorithm, GMM was designed to model the distribution of pixel intensity in RS image.Assume that the number of component was a random variable.Respectively build the prior distribution of each parameter.In order to improve noise resistance, used Gibbs function to model the prior distribution of GMM weight coefficient.According to Bayes' theorem, build posterior distribution.RJMCMC was used to simulate the posterior distribution and estimate its parameters.In order to verify the feasibility and effectiveness of proposed algorithm, use the real RS image to experiment.Experimental results show that, the proposed algorithm can converge to the optimal number of class and get an ideal segmentation results.

GMM
An observed image x ={xi, i =1, …, n}, where i is the index of pixels, zi is the intensity of pixel i, n is the number of pixels of x.GMM is used to model the image x, and assumes that each observation xi is considered independent.The density function at an observation xi is expressed as Where j is the index of class, k is the number of component; w ={wij; i =1, …, n, j =1, …, k} is the weight coefficient of Gaussian distribution, and satisfies the constraints 0< wij <1 and the sum for class is 1.And N(xi |θj) is the Gaussian distribution.Each Gaussian distribution can be written as Where θj ={μj, σj 2 ; j =1, …, k}.μj is the mean, σj 2 is the variance.Note that the observation xi is independent, the joint conditional density of the data X can be written as

Prior distribution
From (3), the parameters of the model is {w, θ}.In order to build the posterior distribution of the parameters, define prior distribution of the parameter.Suppose each parameter is independent, and satisfies a certain distribution.Therefore, define the following prior distribution.
To reduce the noise influence, MRF is introduced to model the weight coefficient of GMM.It imposes the spatial neighbor relationship into segmentation model,it can be written as Where A is a normalizing constant, T is a temperature constant, and β is a constant value that controls the smoothness.Ni is the neighbor number of the pixel i, and m is the index of neighbor pixels.A square window of size 3*3 is used in this paper.
Assume the prior distribution of mean and variance obey the Gaussian distribution with specified mean and variance.The prior of mean and variance can be written as Where the μμ (σμ) is the mean, and σμ (σσ) is variance.And they are constant value.
Suppose the number of component obeys the Poisson distribution, and it can be written as Where λ is the parameter of the Poisson distribution, and it is the constant value.

Posterior distribution
Using Bayesian paradigm, the posterior distribution can be written as ) is defined as the objective function, and uses maximum a posterior (MAP) to estimate parameters.

Simulation
To segment a RS image, it is necessary to simulate from the posterior distribution defined in (8) and estimates its parameters.Let's Ω =(k, w, θ) be the parameter vector of the posterior distribution.When k is a variable,the dimension of the parameter vector Ω varies.In this paper, RJMCMC algorithm is used to simulate samples from the posterior distribution of Ω.In each iteration, a new candidate Ω * for Ω is drawn from a proposal distribution.Calculate the acceptance probability to accept or reject the candidate.
The move types designed in this paper include the following: 1) Update Gaussian distribution parameters.The parameter vector of Gaussian distribution can be written as θ ={θj; j =1, …, k}, where θj =(μj, σj 2 ).Assume that the candidate parameters μj * and σj 2* are Gaussian distribution with mean μj and σj 2 and variance σμ 2 and σσ 2 ,respectively.The accepting rate of distribution parameters can be obtained as 2) Update the weight coefficient of Gaussian distribution.A weight coefficient wij is randomly drawn.To update it, a new weight coefficient wij * is randomly drawn from (0, 1).In order to satisfy the constraint conditions that sum is 1, change other weight coefficient of this class j.The candidate weight coefficient is 3) Birth or death the number of class.Assume that the current number of class is k.Consider a birth operation that increases the number of component from k to k+1.Let the parameter of Gaussian distribution (μk+1 * , σk+1 * ) be Gaussian distribution with mean μμ and μσ and variance σμ 2 and σσ 2 respectively and the weight coefficient wk+1 * is randomly drawn from (0, 1), where μμ, μσ, σμ 2 and σσ 2 are all specified.In order to satisfy the constraint conditions that sum is 1, change other weight coefficient.
The candidate weight coefficient is . The acceptance rate can be obtained as The steps of implementing the proposed algorithm is Step 1 Set the thresholds include: the initial number of class, the temperature value, the total iterations, the parameter iterations, the specify mean and variance.
Step 2 Initial the parameters: weight coefficient, mean, and variance.
Step 3 Perform RJMCMC operations.Update Gaussian distribution parameters, update the weight coefficient of Gaussian distribution, birth or death the number of class for some times iterations.
Step 4 Repeat step 3 for the total iterations.
Step 5 Output final result by maximum a posterior probability

EXPERMENTS
In this section, the proposed algorithm is tested with real RS images.The compared algorithm is spatially variant finite mixture model (SVFMM).The two algorithms all combine GMM with MRF to model the image, but SVFMM algorithm estimate parameters by EM and proposed algorithm by RJMCMC.

Segmentation of RS images
The real RS images from the worldview1 satellite are shown in Figure 1, its resolution is 0.5 m, and the size of image is 128*128 pixels.SVFMM algorithm is used to compare with proposed algorithm.
The image in Figure 2 is the segmentation results of SVFMM algorithm.The image in Fig. 3 is the segmentation results of proposed algorithm, The parameters of the proposed algorithm is set up as follows, the temperature value β is set a value of 0.2, the total iterations is 4000, the parameter iterations is 200.In SVFMM, the temperature value β is set a value of 4 based on the experience, the total iterations is 4000, the fix number of class is 3, 3, and 4. As can be seen from the segmentation results, SVFMM algorithm is not well fitting the boundary.And there are some segmentation error in Fig. 2 For accurately evaluating proposed algorithm, make a standard image, shown in Figure 4.In table 1, we list some computed common precision measures to quantitatively assess the accuracy of the segmentation results shown in Figure 2(a) and Figure 3(a).From table 1, it can be seen that the accuracy of proposed algorithm is higher than SVFMM algorithm.

Figure 4 Standard image
In order to verify the convergence of algorithm, plot the convergence image for the number of class, shown in Fig. 5.The horizontal axis is the number of iterations and the vertical axis is the number of class.In order to clearly seen the effect of the algorithm convergence, respectively taken 100 times, 2000 times and 100 times iteration.The image shown that proposed algorithm is convergence from 10 times, 700 times, 10 times.The final number of class is 3, 3, and 4.After that the number of class is remain the same.This suggests that proposed algorithm can converge to the optimal class.

CONCLUSION
This paper proposes an image segmentation method that combining GMM and RJMCMC.In the proposed algorithm, GMM is used to model the pixel spectrum measurement distribution of the image.The number of component is a random variable.In order to decrease the affection of image noise, use Gibbs function to model the prior distribution of GMM weight coefficient.According to Bayes' theorem to build posterior probability.For realizing that automatically determine the number of class, RJMCMC was adopted in proposed algorithm.The segmentation shown that the proposed algorithm not only automatically determine the number of class, but also segment images accurately.
coefficient can be obtained as reduce the number of component from k to k-1.Delete mean μj, variance σj 2 and weight coefficient wj.In order to satisfy the constraint conditions of weight coefficient that sum is 1, change the other weight coefficient value.The new weight coefficient is (w1/(1-wj),…, wj-1/(1-wj), wj+1/(1wj),…, wk/(1-wj)).The acceptance rate can be written as Figure1 Real RS images image for the number of class