EXTRACTION OF HUMAN SETTLEMENTS FROM HIGH RESOLUTION REMOTE SENSING IMAGERY BY FUSING BOTH RIGHT-ANGLE CORNERS AND RIGHT-ANGLE SIDES

A method for human settlements extraction from high resolution remote sensing imagery using feature-level-based fusion of rightangle-corners and right-angle-sides is proposed in this paper. First, the corners and line segments are detected, the right-anglecorners and right-angle-sides are determined by cross verification of the detected corners and line segments, and these two types of features are rasterized. Second, a human settlement index image is built based on the density and distance of the right-angle-corners and right-angle-sides in a local region. Finally, the polygons of human settlements are generated through binary thresholding of the index image, conversion from raster format to vector format, and sieving. Three images are used for testing the proposed method. The experimental results suggest that our proposed method has higher accuracy than the existed method. Specifically, the correctness, completeness, and quality of our method are 6.76%, 10.12%, 12.14% respectively higher than the existed method. *Corresponding author.


INTRODUCTION
Very high spatial resolution (VHSR) aerial and satellite images provide valuable information (Sirmacek and Unsalan, 2011) in diverse fields such as geography, cartography, surveillance, city planning, surveying and mapping, etc.One of the most important information is about the built-up regions.Because the information about the area, shape, location, distribution, growth, and characteristics of built-up regions can greatly help government agencies and urban region planners in updating land-use maps, forming long-term plans and monitoring urbanization.As a result, monitoring the built-up area receives an increasing attention (Tao et al., 2013).In this kind of monitoring, the first step is to detect built-up areas from the VHSR remote sensing images (Sirmacek and Unsalan, 2010).Generally, a built-up area represents a vital and highly dynamic environment which is mainly composed of both man-made and natural objects (Tao et al., 2013).For low and medium spatial resolution remote sensing images, the researchers have proposed many built-up area indices such as Normalized Difference Built-up Index (Xu, 2007), Enhanced Built-Up and Bareness Index (Assyakur et al., 2012), Combinational Build-Up Index (Sun et al., 2015) etc.On the other hand, many scholars have considered automatic methods for built-up area detection from VHSR remotely sensed images.Depending on whether the training samples are used, these existing methods can be roughly divided into two groups (Liu et al., 2013): The first group relies on supervised classification.They often need a large number of representative training samples to learn the patterns of built-up areas for detection.For example, Benediktsson et al. investigated classification of panchromatic high-resolution data from urban areas using morphological and neural approaches (Benediktsson et al., 2003).Zhong and Wang presented an ensemble model of multiple conditional random fields to incorporate multiple features and learn their contextual information for urban detection (Zhong and Wang, 2007).Pesaresi et al. used a novel image classification method, called Symbolic Machine Learning (SML), for detailed urban land cover mapping (Pesaresi et al., 2016).Hu et al. presented a novel approach for built-up area detection from high spatial resolution remote sensing images, using a block-based multi-scale feature representation framework (Hu et al., 2016).However, different images or scenes may need different training samples and different classifiers.


The second group directly detects built-up areas without using any training data.Furthermore, these methods can be divided into four subcategories based on the employed features:  Texture-based approaches.PanTex (Pesaresi et al., 2008;Pesaresi et al., 2011), being contrast measurement of texture features using the gray-level co-occurrence matrix, has been widely used for global human settlement extraction.However, the forested areas with high contrast due to the tree shadows also have high PanTex values, and they are subject to be taken as built-up areas. Building-density-based approaches.Huang and Zhang propose a building detection method using the difference of morphological profiles (Huang and Zhang, 2012), and the corresponding buildingdensity-based feature is employed to extract the built-up areas in (Liu et al., 2013).However, the building extraction is a still difficult problem and faced with great challenges.

Commission III, WG III/6
 Corner-density-based approaches.The local key point features such as (scale invariant feature transform) SIFT (Sirmacek and Unsalan, 2009), local feature point extraction using Gabor filters (Sirmacek and Unsalan, 2010), junctions (Liu et al., 2013), and Harris corners (Li et al., 2015) are widely employed for built-up area detection.Furthermore, some improved methods such as improved Harris (Tao et al., 2013), modified Harris for edges and corners (Kovács and Szirányi, 2013), are presented.However, corners are frequently existed in farmland areas and highways, which leads to the farmlands and highways may be mistaken as human settlements. Edge-density-based approaches.Edge is an importance feature for image understanding.For example, Gong and Howarth incorporate the edgedensity feature into the image classification to increase the accuracy by approximately 10% (Gong and Howarth, 1990).Ü nsalan and Boyer introduce a set of measures based on straight lines to assess land development levels in high-resolution panchromatic satellite images (Unsalan and Boyer, 2002).
Recently, Chen et al. realized extraction of built-up areas from high-resolution remote-sensing images using edge density features (Chen et al., 2014).However, edges also densely exist in the images of natural scenes.In this paper, we focus on the extraction of built-up areas from VHSR satellite images without using any training samples.A major issue of the existing methods belonging to the above second group is that the employed clues of built-up areas may be similar to clues of non-built-up areas.Thus, development and selection of the unique clues of the built-up areas is a critical problem.In VSHR satellite images, the built-up areas have two dominant classes of man-made objects, namely buildings and roads.Moreover, most of the building roofs are with rectangular shapes, and they may derive a large number of right angle corners and right angle sides.Furthermore, the road marks may derive many line segments which are significantly brighter than the backgrounds.In this sense, both of the corner points and the line segments (including the right-angle-sides and road mark line segments) can be used as the unique features for built-up area detection.Thus, an Index using Joint Density of Corners and Line Segments (IJDCLS) is proposed to extract the built-up areas from VHSR satellite images.

THE PROPOSED FRAMEWORK FOR BUILT-UP AREA EXTRACTION
Our proposed method is composed of six main steps, as shown in Figure 1. 1) Detection of Line Segments.The LSD (Gioi et al., 2012) is adopted to detect the line segments in VSHR images in this article.Moreover, among the detected line segments, a line segment is deleted if it is shorter than or longer than .For example, Figure 3  3) Verification of Corners by Line Segments.In this article, a Harris corner with supporting line segments is defined as a Harris corner with two orthogonal line segments around it, and the both line segments have medium length.As shown in Figure 2, for each Harris corner , let and be the two nearest line segments around , and be the angle between the line segments and .Moreover, the length of the line segments and is and , respectively; the distance between the Harris corner and the line segments and is and , respectively.Additionally, the distance between the Harris corner and a line segment is calculated as follow.Draw a line perpendicular to , and the perpendicular line is through .If the foot of the perpendicular was fallen between the two end points of , the is equal to the distance between the foot and the .Otherwise, is equal to the distance between the and its nearest end point of .Thus, a Harris corner with supporting line segments should meet the three criteria: and (1) (2) and (3) where , , , = predefined thresholds.A Harris corner will be pruned if it does not satisfy the above three criteria.4) Construction of Built-up Area Index.Generally, the right angle corners, the right angle sides and the road mark line segments in the built-up areas tend to closely locate in the neighboring spatial domain with high density.On the other hand, the right angle corners, the right angle sides and the road mark line segments in a non-built-up area are more likely to be sparsely distributed.This means that, if an image pixel ( , ) belongs to a built-up area, we expect that there are more corners and line segments in its neighborhood.Hence, the density map of the corners and line segments, which could be computed by spatial voting, can be utilized to identify built-up areas.The spatial voting is performed as follows.For an image pixel ( , ), suppose there are Harris corner pixels and line segment pixels within a predefined distance threshold . is a predefined threshold.We define the following likelihood function to measure the possibility that a pixel ( , ) belongs to a built-up area: (4) where ( , ) represents the spatial coordinate of a corner, for = 1, . . ., ; ( , ) represents the spatial coordinate of a line segment pixel, for = 1, . . ., .The likelihood function highlights the built-up region in the pixel neighborhood.If it is a good candidate for the built-up region, a high value is expected.The likelihood function highlights the built-up region the pixel neighborhood.If it is a good candidate for the built-up region, a high value is expected.5) Thresholding of Human-Settlement Index.Once the index image of IJDCLS is constructed, a corresponding frequency histogram of IJDCLS is formed.The human operator will select an experienced value to threshold the IJDCLS.The built-up regions are the pixels with the larger than .Finally, if a detected region with its area smaller than , it is deleted.

EXPERIMENTS AND ANALYSIS
A prototype system for built-up area extraction based on our proposed method is developed by C++ language in Microsoft Visual Studio 2010.The method in (Liu et al., 2013) is used to perform compariion.Additionally, the hardware we used is a ThinkPad W520 laptop with Intel Pentium 2.40 GHz CPU and 2.98 GB RAM.
Three VHSR images are used for the testing.The first one is an airborne image with a size of 26590  49654 pixels.The ground sampling distance (GSD) is 0.10m, and the covered area is located at the suburban of Yishui, Shandong Province, China, as shown in Figure 3  As far as our proposed method is concerned, values of the seven parameters of our method for the three images are listed in Table 1.Additionally, manual digitization is adopted to create reference data to verify the performances of the above two methods.In the process of full manual annotation, manual clicks should be as accurate as possible.P and quality q P , are used to evaluate the performance, and they are borrowed from (Tao et al., 2013).They are defined as follows:  2. The statistics in Table 2 suggest that the correctness, completeness, and quality of our method are on average 6.76%, 10.12%, 12.14% respectively higher than the existed method of Liu et al. (2013).

CONCLUSIONS
In VHSR satellite images, there are more right angle corners, right angle sides and road mark lines in the built-up areas than in the natural environments, which is used by our method as a unique clue for the extraction of built-up areas.We name the proposed method IJDCLS.IJDCLS needs seven parameters, but most of the parameters can be determined by prior knowledge.Three VHSR satellite images are used to test the proposed method and the method in (Liu et al., 2013).The experimental results suggest that our proposed method has higher accuracy than the existed method.Specifically, the correctness, completeness, and quality of our method are 6.76%, 10.12%, 12.14% respectively higher than the existed method.
(a) displayed the original VHSR image, and Figure 3(b) displayed the retained line segments after deleting the short or long line segments when pixels and pixels.2) Detection of Harris Corners.We still use the classic Harris corner detector to extract low-level corners in the VSHR images for the indication of the built-up areas.

Figure 2 .
Figure 1.Flow chart of the proposed method (a).The second is a QuickBird panchromatic image with a size of 6904  6905 pixels.The GSD is 0.61m, and the covered area is located at central urban area of Linzhi City, Tibet Autonomous Region, China, as shown in Figure 3(b).The third is a GeoEye-1 image with a size of 3000  2000 pixels.The GSD is 0.50m, and the covered area is located at Inakadate, Janpan, as shown in Figure 3(c).

Figure 3 .
Figure 3.The extracted human settlments by our proposed method, including the first image and its result (a), the second image and its result (b), and the third image and its result (c)

S
is the area of automatic extracted result, manual S is the area of the manually extracted reference result, the manual & auto S is area of the intersection of the automatic extracted result and manually extracted reference result, and manual || auto S is the area of the union of the automatic extracted result and manually extracted reference result.The values of the above three indicators of the two methods on the three testing datasets are listed in Table

Table 1 .
The values of the input parameters of our proposed method for the three testing datasets c P (%) e P (%) q P (%)

Table 2 .
Statistics about the accuracy of human settlements extractionIn this article, three indicators, including completeness c The above experiments suggest that our proposed method has potential applications.The future work will include: (1) fusion of big/open data and VHSR images for accurate detection of built-up areas; (2) the optimization of the algorithms and adoption of the high performance computation to speed up the efficiency; (3) vegetation index is used to promote the separation of farmland from the built-up areas.