EFFICIENT EXTRACTION METHOD OF THE CHANGE OF BUILDINGS FOR FIXED PROPERTY INVESTIGATION

In order to appraise fixed property, local governments in Japan annually investigate taxable buildings on January 1, but the efficiency of the investigation is an issue. With this context, change extraction methods of buildings have been studied using various data. However, labor and time optimization for change extraction methods remains an issue. With this in mind, the authors have concentrated on developing a change extraction method for buildings that uses only photomaps of two separate times. This paper describes the extraction method using color and texture, which can be obtained from photomaps. In order to evaluate effectiveness, the proposed method was investigated using photomaps of two separate times on various locations. It was confirmed that the proposed method extracted the changed buildings. It was also verified that the change of buildings cannot be extracted by color information only; rather, the change extraction of buildings was accomplished by using both color and texture. It should be noted that all changed buildings include extracted regions so that the change extraction of buildings is effectively performed without oversights. The proposed method could extract all changed buildings. In conclusion, the proposed method can reduce overextraction of changed buildings by intensively visually checking extracted regions that have changes in color or texture rather than by visually checking all regions of photomaps. It is concluded that the proposed method has the ability to effectively extract changed buildings by using only photomaps. * Corresponding author.


INTRODUCTION
Fixed property tax constitutes about 40% of the municipal tax revenue for local government in Japan.Local governments annually impose the fixed property tax for taxable lands, buildings, and depreciable assets (fixed property) on owners on January 1. Investigations that identify newly erected buildings, extension and reconstruction buildings, demolished buildings, and so on, are important sources of fixed property tax for buildings.However, to do this, workload such as labor, time, and so on for the investigations remain unsolved.With this context, various methods have been attempted to reduce this workload.For example, one method visually checks aerial photographs of the investigation field taken at two separate times.In addition, some papers have reported change extraction methods for buildings using various data.These include using aerial television images captured by helicopter (Mitomi et al., 2002), using satellite synthetic aperture radar (SAR) intensity images (Matsuoka et al., 2002), using highresolution satellite imaging (Miura et al., 2004), and so on.However, to use these methods, it is necessary to obtain data and create new data, both of which increase workload.With this in mind, the authors have concentrated on developing a change extraction method for buildings using photomaps of two separate times (Tanauchi et al., 2011).That is to say, in order to avoid the creation of new data or the collection of existing data, we proposed a method that uses photomaps of two separate times drawn from the aerial photographs that are constantly captured by the local governments.In addition, the proposed method involves a low risk of missing changed buildings in order to extract changes in terms of three metrics, color, shape, and texture obtained from the photomaps.However, certain shortcomings of this method, such as overextraction of changed buildings and considering about automatic thresholding of the extraction algorithm.Therefore, in this study, we developed a change extraction method for buildings, which includes automatic thresholding and reduces over-extraction of changed buildings using only photomaps of two separate times.The overview of this paper is as follows: Section 2 explains the proposed change extraction method for buildings.Section 3 specifies the photomap data for accurate verification of change extraction.Section 4 gives the change extraction results and their analysis.Finally, Section 5 concludes this paper.

Methodology
The proposed method utilizes photomaps in order to equalize positions and inclinations of the features in photographs taken at two separate periods.The method attempts change extraction for buildings by using color and texture obtained from the photomaps.Incidentally, a previous method used not only color and texture but also edges as shape information (Tanauchi et al., 2011).The risk of missing changed buildings was reduced by using shape information.However, over-extraction of changed buildings increases when edges are not extracted perfectly.Accordingly, in this study, in order to reduce over-extraction of changed buildings, the change extraction method for buildings limited the information to color and texture.

Color change extraction
In order to extract color changes in photomaps of two separate times, we used the hue, saturation, and intensity (HSI) color model shown in Fig. 1.
Finally, H is defined by equation (4) as when but when H < 0, we add 2π to H.
In order to extract complex HSI changes, the color changed value (d HSI ) was defined by equation ( 5) as the distance between the HSI color pixel values of two separate times, as shown in Fig. 1.The color changed values were calculated each same pixel coordinate on photomaps of two separate times.For example, the red arrow in Fig. 1 shows d HSI when the color changes from blue to red.
where H 1 , S 1 , and I 1 are the hue, saturation, and intensity pixel values for the former photomap H 2 , S 2 , and I 2 are the hue, saturation, and intensity pixel values for the latter photomap x 1 , y 1 , and z 1 are the coordinates of the HSI color pixel value in the HSI color model for the former photomap x 2 , y 2 , and z 2 are the coordinates of the HSI color pixel value in the HSI color model for the latter photomap.

Texture change extraction
In order to extract pattern changes for buildings such as a change in the buildings material, we extracted texture changes calculating the standard deviation from a grayscale histogram.
That is to say, the difference between two standard deviations for two different periods can be used to extract changes in building textures because this difference indicates differences in pattern fineness.
Standard deviations for the two periods were calculated next two steps.First, RGB photomaps were converted into grayscale photomaps using equation ( 6) that uses NTSC coefficients defined by ITU-R BT.601 (ITU, 2011).Second, the standard deviation values were calculated for each mesh (= 5 × 5 pixels) in grayscale photomaps.
where Y is a grayscale pixel value The texture changed value (d T ) was defined by equation ( 7) as a difference between the standard deviation values for the two periods.In addition, d T was calculated at the same mesh position on photomaps of two separate times.
where σ 1 is the standard deviation for the former photomap σ 2 is the standard deviation for the latter photomap.

Change extraction
To extract changed buildings with a low risk of missing changes in buildings, we perform change extraction using both color and texture changes.Fig. 2 shows the change extraction concept.The numbers in Fig. 2 show the changed values of color and texture.First, we temporarily set thresholds for change values for color and texture at 5, and then we execute a labeling process towards the target pixels having four neighbors and change value than the threshold.Second, we extract the region with color and texture changes in which the sum of adjacent pixels is more than 8 pixels.In Fig. 2, yellow pixels show a region with color change and red pixels show a region with changed texture.
Finally, the union of the regions having color and texture changes is the changed region in this study.These are the purple pixels in Fig. 2. Incidentally, the sum of the labeled pixels falls under the area in actuality; details of thresholds decision for the changed values and area are expressed in the next subsection.

Thresholding
The thresholds for the changed values of the method used in the past was decided that detailed changed buildings can be able to extract.However, visually searching detailed changed buildings on photomaps before the extraction process as well as overextraction of changed buildings have been unsolved issues.Therefore, for automatic thresholding and reduction of overextraction of changed buildings, the thresholds for changed values in this study were divided into two categories.A region is shaded region in both time whose changes such as color or texture is subtle, another region is other regions (shaded region in old time and not shaded region in new time; not shaded region in old time and shaded region in new time; not shaded region in both time) whose changes are clear.To explain, first, we extracted a shaded region in each photomap for the two periods using Suzuki's method (Suzuki et al., 2011) and divided the shaded region in both times and other regions.Second, we created four histograms (color changed value for shaded region in both times; color changed value for other regions; texture changed value for shaded region in both times; and texture changed value for other regions).Last, the thresholds were decided automatically by Otsu's (1980) from each histogram, which was segmented into two parts.For extraction of detailed changed buildings such as extended and reconstructed buildings, the threshold for the area was 10 m 2 .

TEST DATA
The test data are three pairs of photomaps taken at two separate times, as shown in Fig. 3. Table 1 gives the information contained in photomaps.There were differences in color tone on photomaps for each site.Therefore, the photomaps were corrected by converting the color tone in the photomap in the former time to the color tone on photomap in the latter time.As a result, color tones on photomaps were similar at both sites.Changed buildings are shown by a red circle on the photomap of the former time.A visual check confirmed that there were 61 changed buildings (of 1034 buildings) in Site1; there were 5 changed buildings (of 808 buildings) in Site2; and 38 changed buildings (of 395 buildings) in Site3.

Shaded region extraction
A part of the shaded region extraction in Site1 is shown in Fig. 4. Fig. 4 (a) and (b) show photomaps in old and new time.Fig. 4 (c) and (d) show extracted regions as shaded regions (blue).It was confirmed that the shaded region and a few black roofs were extracted.Table 2 shows the ratios of shaded region in both times and other regions compared to the whole area used in the photomaps.The area in the shaded region in both times was less than in other regions in all sites.

Thresholds for changed values
Histograms for the color changed values in each site are shown Fig. 5.Because of the areas in other regions in all sites are large and color changes are clear, the appearance pixels in other regions were larger than in shaded region in both time.Table 3 shows the thresholds for the color changed values in each site calculated from the histograms for the color changed values by Otsu's method.Because of the histogram shape for the color changed values, the thresholds for color changed values in other regions were large in all sites.Histograms for the texture changed value in each site are shown in Fig. 6.For the same reasons as for color change, it was confirmed that the appearance pixels in the other regions was large.The thresholds for the texture changed values are shown in Table 4; the thresholds for the texture in other regions were large in all sites.Table 5 shows the extraction results of changed buildings in Site1.The results of color change extraction only were 617 extracted regions, 60 extracted changed buildings, and 1 not extracted changed building.The result of texture change extraction only was 962 extracted regions, 56 extracted changed buildings, and 5 not extracted changed buildings.However, the results obtained by using the proposed method, which uses both color and texture, was no extracted changed buildings though 1062 extracted regions were large.Fig. 8 and Fig. 9 show the changed buildings not extracted by each change extraction.The reason the method using color or texture individually could not extract all the changed buildings is that color or texture changes were subtle.For extractions of all these changed buildings, we can consider that we change the each threshold for changed value to less value.However, changing less thresholds value is not relevant because of extractions for many subtle feature changes on all regions of the photomaps.Therefore, it is possible to extract changed buildings effectively by the proposed method whose methodology is that color change extraction extracts changed buildings not extracted by texture, and texture change extraction extracts changed buildings not extracted by color.In order to compare the methods between the proposed method and the past method, Table 6 shows the extraction results by the past method in Site1.Compared to Table 5, the proposed method extracted all changed buildings though extracted regions were less than those extracted by the past method.In order to appraise with the proposed method on various locations, Table 7 shows the extraction results for changed buildings in Site2 and Site3.All extracted changed buildings were extracted by the proposed method.From these results, it was determined that the proposed method can extract changed buildings automatically and efficiently on various locations.

CONCLUSIONS
In this study, we proposed an efficient and automatic extraction method for changed buildings using only photomaps of two separate times.As a result, we reached the following two conclusions.
① Although all changed buildings were not extracted by using color or texture information individually, using both color and texture information makes it possible to effectively extract changed buildings.
② Considering automatic thresholding and limiting information gained from photomaps for extraction, we can extract changed buildings more efficiently than we could with previous methods.
The proposed method can reduce oversight of changed buildings by intensively visually examining extracted regions that have color or texture changes rather than visually examining all regions of photomaps.Therefore, the proposed method can contribute to making building investigations more efficient.It is necessary to consider further reduction of overextraction of changed buildings in the future.

Fig. 1 .S
Fig.1.HSI color modelEach red, green, blue (RGB) color pixel value in the photomaps of two separate times was converted to an HSI color pixel value by using equations (1) to (4).The values of R, G, B, S, and I range from 0 to 1, while H ranges from 0 to 2π.I is defined by equation (1) as Photomaps and changed buildings (red circle) (a) Photomap in old time (b) Photomap in new time (c) Shaded region in old time (d) Shaded region in new time Fig.4.Result part of shaded region extraction in Site1 the extraction region in Site1 as the extraction results of changed buildings are shown in Fig.7, which is the same location as that shown in Fig.4.It was confirmed that the changed building was extracted by each of the three change extraction methods.(a) Color change extraction (b) Texture change extraction (c) Extraction by the proposed method Fig.7.Result part of extraction regions in Site1 time Fig.8.Changed buildings not extracted by color extraction (a) Old time (b) New time Fig.9.Changed buildings not extracted by texture extraction

Table 2 .
Ratios of shaded region in both times and other regions

Table 5 .
Extraction results of changed buildings in Site1

Table 6 .
Comparison results between the proposed method and the past method in Site1

Table 7 .
Extraction results of changed buildings in Site2 and Site3