ACCURACY IMPROVEMENT OF CHANGE DETECTION BASED ON COLOR ANALYSIS

Change detection is one of the most important topics in the application field of aerial images and satellite images. In this paper, a novel framework for accuracy improvement on change detection is proposed. The proposed framework consists of three parts. Firstly, the location difference between the two orthoimages for change detection is rectified globally. Secondly, the illumination change of the orthoimages from two different times is adjusted to produce the orthoimages with more unified illumination condition. Thirdly, the location difference between the two orthoimages is adjusted in more precise level of local regions. Experimental results show that the detection accuracy of change detection is greatly improved through the pre-processing of the proposed framework.


INTRODUCTION
As one important application field of aerial images and satellite images, change detection is attracting more and more attentions in recent years.Due to the fast development of cities especially in the developing countries, there are new reconstructions, demolished old buildings, vacant land changing into grassland and so on, nearly happening everyday.This leads to dramatic land change in a short time.In order to timely update the map of these changing places in wide area, change detection with high accuracy on aerial images or satellite images is quite necessary.Before, this updating job is realized by manual check of human operators, but it turns out to take very long working time and there also exist changing regions missed by human operators.Under this background, several change detection systems have been developed to automatically carry out the change detection on aerial images and satellite images.For example, we can extract the color change and the height change based on orthoimages and Digital Surface Model(DSM), which are generated based on stereo images and aerial triangulation data.The real application projects prove that automatic change detection systems help to greatly reduce the overall processing time, and further the overall cost.At the same time, automatic systems also improve the overall detection rate, by extracting the regions apt to be missed by human operators.To assure that no changing region is missed in the final result, automatic change detection systems tend to firstly extract much larger number of candidate regions far more than the number of actually changing regions, which needs further processing by the operators to remove false detections.Though the overall processing time is already greatly reduced, operators still have to spend long time to remove wrong detections.To solve this problem, we propose a framework to reduce most of the wrong detections and at the same time keep the correct detections.The experimental results show that through the processing steps in the proposed framework, the number of wrong detections is greatly reduced and the location and range of detected regions are also improved.
The rest of the paper is organized as follows.Section 2 explains the research background for the improvement of change detection.In section 3, the details of the whole methodology in three steps are stated.Experimental results are shown and analyzed in section 4. Finally, we draw a conclusion and also introduce future prospects in section 5.

RESEARCH BACKGROUND
Due to its wide application, many change detection methods [1,2,3] have been proposed.In comparison, there is no systematic discussion on how to improve the accuracy of change detection by resolving some common problems in almost all change detection methods.In this paper, we concentrate on analyzing the problems that affect the accuracy of change detection.Firstly, the necessary requirements for a change detection system are discussed as follows.(1) Completeness.No changing region is missed no matter of its scale, shape and changing pattern.(2) Correctness.No unchanging regions are extracted.
(3) Preciseness.The location and the range of the extracted changing regions should conform to the truth.According to these requirements, current automatic change detection systems have relatively good completeness.But to assure the completeness, some suspectable but actually unchanging regions are also detected.Through the analysis of several different projects, we find the following two main reasons for the wrongly detected regions: illumination change between two orthoimages and location difference between two datasets.At the same time, there exist incomplete candidate regions because of illumination change and location difference.We propose a new framework to solve the above two main problems, and finally find that the preciseness of the detected regions is also improved at the same time.

METHODOLOGY
The proposed framework for accuracy improvement of change detection aims at solving the problems of illumination change and location difference between orthoimages.After analyzing the mutual influence of the above problems, we design the following framework.Firstly, the global location difference between two orthoimages for change detection is rectified, which should be implemented at first especially when there exists overall large location difference.Secondly, the illumination change adjustment is carried out to make two orthoimages have more unified illumination condition.Thirdly, more precise rectification of location difference in local regions is performed, aiming at making the same object appear in the same location.This framework for accuracy improvement is implemented before the change detection as pre-processing, since the above three steps improve the quality of original input data for change detection.By implementing these steps to get more accurate input data, more accurate detection results and less processing time of change detection can be expected.

Global Location Difference Rectification
As it is stated in the above, change detection is carried out on both orthoimages and DSMs from different times.For both orthoimages and DSMs, the location difference between two datasets may interfere with the detection accuracy of final results.The location difference between two datasets happens due to various different factors.For example, the aerial triangulation data used for generating DSM from stereo images may have different accuracy levels for two datasets.Or there are different systematic errors coming from camera, aerial triangulation calculation, and stereo matching.Rather than analyze the above reasons of the location difference, we decide to directly analyze the two datasets to find out the location error between them.For the two datasets for change detection, in the wide area shown in the images, most of the parts are not changing.For example, according to [4], the percentage of changing buildings annually in one photograph is only 3% to 5% of the overall number of buildings in most cases.Based on this, from another point of view, it is possible to carry out matching on the unchanging parts in two datasets from different time.Compared with DSM data including only height information, orthoimages have more information with three color channels and more characteristics are able to be extracted.Therefore it is easier to carry out matching between two orthoimages than DSM data.Through image matching, we attempt to find the location difference between the parts in respective orthoimage that describe the same area in the real world.Basically, orthoimages are ortho-rectified results of original aerial images based on 3D information of DSM.Strictly speaking, after ortho-rectification, each pixel in the orthoimage corresponds to only one point in the real world with unique latitude and longitude.In this sense, in the two orthoimages of different time, two corresponding points should be in the same location.But due to the errors stated in the beginning of this section, there still exists some small difference between corresponding points.At the same time, the orthoimages input for change detection are already sampled to the same resolution to facilitate change detection.Based on the above analysis, we conclude on the rotation invariance and scale invariance for the two orthoimages.The location difference between them can be simply described as the shift in X and Y direction.
We utilize a global matching method to obtain the overall location difference for the whole orthoimage.The computing scheme is described as follows.For two orthoimages  Experimental results of the global matching method show that compared with original datasets the location difference over the whole image is reduced.Especially for the orthoimage showing relatively flat land, there is almost no location difference after the global rectification.In comparison, for the orthoimage including height changing landform, we find that for the regions with various altitudes, there are still remaining location errors, respectively different in each region.This phenomenon happens because for different altitude levels, the ortho-rectification amount on the original images is different, which results in different location difference.To solve this problem, a local matching method is further proposed later.

Illumination Change Adjustment
For change detection, the original images are taken under different conditions like the season, the weather and the shooting time in a day, which result in different illuminations in the images.Even for the same rooftop, its color may appear in quite different ways in the orthoimages of two different times.
And this phenomenon leads to many wrong detections of color change.In order to solve this problem, we decide to unify the illumination of the two orthoimages for change detection.
There have been many methods to analyze the illumination model of the sun for the aerial images according to the shooting season, the shooting time in the day and sometimes the characteristics of the rooftop material for light.To save the efficiency of the whole processing, rather than such complex model analysis, we decide to undertake the color transfer, to only adjust the color tone of one orthoimage to make it look like another orthoimage.
Here we utilize the method of histogram matching [5], which adjusts each color channel based on the global image statistics.
In details, for each channel, the following function is designed to transfer each intensity value in the source image to the target image. 255 where s g = intensity mean of the source image t g = intensity mean of the target image s σ = standard deviation of the source image t σ = standard deviation of the target image Since the main transfer function for intensity value between 0 and 255 is linear, we call it linear transfer function here.The above method is designed to assure the following rules.We show the experimental results of illumination change adjustment through linear transfer function in Figure 1.From it, we find that after adjustment, the color tone in orthoimages becomes more similar than the original case, i.e.(a) and (c), also (b) and (d) are more similar in illumination than (a) and (b).A piecewise cubic spline transfer function is also proposed in [5], to overcome the fast saturation near very low and very high intensity values.According to our experimental results on several datasets, the illumination change adjustment under this function results in fewer wrong detections, but also fewer correct detections.To assure no correct detections are missed, we decide to take the linear transfer function in this framework.Furthermore, when it comes to select which orthoimage as the source image, we make further experiments on comparing the change detection results from two possible cases.The experimental results show that it is better to choose the darker orthoimage as the source image and the brighter one as the target image, based on the rule of fewer wrong detections and no influence on correct detections.To summarize the above analysis, we undertake the linear color transfer on the darker orthoimage to make it have similar color tone to the brighter orthoimage.That is to say, in the case of Figure 1, after illumination change adjustment, (a) and (c) are taken as the input data for the next step.

Precise Location Difference Rectification
To solve the remaining location errors after global location difference rectification by global matching method, we further propose the following local matching method.In this method, the rectification amount of location difference is computed for each local region.It mainly consists of three steps.Firstly, key points are extracted over the whole image by Harris corner extraction [6].Note the threshold to select the key points is set relatively high to assure that only reliable corners are selected.We perform Harris corner extraction respectively in two orthoimages and select the one with more reliable corners for example 2 I , as the benchmark image.Secondly, for each key point in the benchmark image 2 I , we try to search for its matching point in 1 I by template matching.Since there is only small local difference after global rectification, the searching for the matching point is only carried out in the neighbourhood of each key point.The computing is similar to Function (1) of the global matching, only with the difference that the matching cost is computed in a template surrounding the key point, and the template is shifted in the neighbourhood of the key point.We further filter the pair of matched points with high matching cost.In this case, wrong matching pairs are removed, including cases like the pair including the corners from some noise in the benchmark orthoimage, or the corners from the moving object in the benchmark image, and so on.Through the experimental results, we find that after filtering, only the matching pairs of unchanging object corners are mainly left, like the corners of unchanging buildings.Thirdly, in each local block, the rectification amount is obtained by a voting scheme.All the remaining matched pairs are used for voting by the shift information between the two corresponding pixels.For each possible shift position in the shift range, the one with the highest votes is taken as the final rectification amount of this block, as described in Function (7).Here the whole image is segmented into several same-sized non-overlapping rectangle blocks.The size of each block is also carefully set.Because if it is too small, the shift amount for rectification may be easily affected by the image details, while if it is too large, there is not quite much difference from the global matching method.Based on the experience, for the experimental image of 3000*2500, we set the block size as 500*500.In Figure 2, the distribution of voting counts in the shift range of [-5, 5] both in X and Y direction is shown.From it, we find that around the bin with the highest value, the counts in the surrounding bins are also relatively high.This shows that in this local block, most of the key points have the location difference similar to the selected shift position.This shows that it is reasonable to select the highest voting one as the final shift amount for the whole block.This phenomenon is also witnessed in other local blocks and other test orthoimages.What's more, we find that the final rectification shift amount for each local block has similar value to that of its surrounding blocks, though each block has different shift amount values.This shows that the landform is always gradually changing in the altitude.Based on the above analysis, we conclude that with local matching method, the location difference is more precisely rectified.Furthermore, both the shift amount from the global matching method and that from the local matching method are also reflected to the DSM data, so as to improve the detection accuracy of height change simultaneously.

EXPERIMENTAL ANALYSIS
In this section, various experimental results are shown and discussed.To illustrate the effectiveness of the proposed framework, we select one example data including two datasets from different times with relatively large location difference and also quite different illumination condition.To show the accuracy improvement of each step in the proposed framework, we compare the change detection result on the original dataset with the result from the processed dataset after each step.The test othoimages are with the size of 3000*2500.There are 46 changing spots between the two orthoimages based on human check result.
In this example, even for the original datasets, there are 6 changing regions missed.And after the 1 st and 2 nd step, the number of wrong detections is reduced in large numbers.At the same time there are 2 more changing regions newly extracted because of the "global location difference rectification" and the "illumination change adjustment".While after the 3 rd step of "precise location difference rectification", not only the number of wrong detections is further greatly reduced, but also all the changing regions are correctly detected.By greatly reducing the number of wrong detections, it is expected that the checking time of operators is reduced a lot.In this way, the overall cost and processing time is also greatly improved.We analyze that the precise rectification of location difference locally makes each object appear almost the same position in the two orthoimages, and thus makes it possible to correctly extract even small changing regions and also to extract each region more precisely in its range and location.By human check, it is found that from the result of the 2 nd step to the 3 rd step, except for the 4 newly detected changing regions, the range and location accuracy of all 42 regions are improved.We further analyze the left wrongly detected regions in the result after the 3 rd step processing.The analysis result shows that among them, 65% are due to ortho-rectification noise, 13% are from moving cars in the street or changing cars in the parking lot, 10% are from great change of color due to shadow or other illumination changes, 9% are because of the growing trees, 3% are from the remaining location difference.According to this, it is clear that ortho-rectification noise is the main reason of the remaining wrongly detected regions.

CONCLUSION
In this paper, we present a novel framework to improve the accuracy of change detection by three processing steps, which are global location difference rectification, illumination change adjustment, and precise location difference rectification.Experimental results show that the proposed framework can not only greatly remove wrong detections while extracting all the changing regions, but also improve the range and location accuracy of correctly detected regions.In details, the correct detection rate is improved from 15.7% to 41.4%, and the completeness rate improved from 87.0% to 100%.For future work, we firstly want to remove or reduce the orthorectification noise since it is the main reason of wrongly detected regions after the processing of the proposed framework.On the other hand, we are also considering implementing some post-processing such as removing the wrongly detected regions with weird shape like elongated one, and the regions detected because of moving car in the street or changing cars in the parking lot, and so on.

1 I and 2 I 2 I . Then initially we put 1 I and 2 I
with the same size, any of them can be selected as the comparison target image, for example totally overlapped with each other, and then shift 1 I in both X and Y directions in a defined certain range, [-r, r].For each shift position the matching cost is computed and the position with the minimum matching cost is decided as the global location difference between two orthoimages.
, = count on the pair of corresponding pixels in shift position of n m, n m, = shift position of 1 I ] , [ r r − = shift range of m and n

Figure 1 .
these rules, the designed transfer function is able to map all the possible intensity values still to the range of [0,255], and also map the intensity mean from s g to t g , and the standard deviation from s σ to t σ .Illumination change adjustment in one example (a) Original old year orthoimage (b) Original current year orthoimage (c) Adjusted current year orthoimage according to old year orthoimage by linear transfer function (d) Adjusted old year orthoimage according to current year orthoimage by linear transfer function.
number of votes for the shift position n m,

Figure 2 .
Figure 2. Voting count distribution in the shift range for one example local block.

Figure 3 .Figure 4 .
Comparison of original orthoimages and rectified results using global matching method and local matching method (a) Original old year orthoimage (b) Original current year image (c) Shifted current year image by global matching method (d) Shifted current year by local matching method ((c) and (d) are already after illumination change adjustment).To illustrate the preciseness of the local matching method compared with global matching method, we show two examples in Figure 3 and 4. For each example, the cross point of the blue lines in (a)~(d) shows the same location, basically the location of one corner point in (a).In (b)~(d), by comparing the location of the cross point and that of the corner point in the new image, we may find the location difference.From both figures, it is easy to find out the following things.The existing location difference shown in (b) is relatively reduced in (c), but more precisely rectified in (d).What's more, by comparing Figure 3(b) and 4(b), we find that the two corner points have different location difference.Therefore, the same shift rectification for the whole image is not appropriate, which is proved by the remaining error shown in Figure 3(c) and 4(c).In comparison, by rectification of local shift amount there is almost no remaining error in Figure 3(d) and 4(d).Another example part in the same orthoimage as the one in Figure 3. (a)~(d) has the same description as Figure 3(a)~(d).

Table 1 .
Comparison of the change detection result on the original dataset and the dataset after the each processing stepIn Table1, the change detection result on both the original dataset and the dataset after each processing step is shown.By comparing the number of wrong detections, correct detections, and the newly defined correct detection rate and completeness rate, it is easy to find that the detection accuracy is improved step by step.Since the number of detected regions under each dataset is different from each other, it is unreasonable to simply use the "ROC (receiver operating characteristic) curve" to analyze the performance.Instead we define two new rates to explain it.