AERIAL IMAGERY AND LIDAR DATA FUSION FOR UNAMBIGUOUS EXTRACTION OF ADJACENT LEVEL-BUILDINGS ’ FOOTPRINTS

Our paper aims to present a new approach to identify and extract building footprints using aerial images and LiDAR data. Employing an edge detector algorithm, our method first extracts the outer boundary of buildings, and then by taking advantage of Hough transform and extracting the boundary of connected buildings in a building block, it extracts building footprints located in each block. The proposed method first recognizes the predominant leading orientation of a building block using Hough transform, and then rotates the block according to the inverted complement of the dominant line’s angle. Therefore the block poses horizontally. Afterwards, by use of another Hough transform, vertical lines, which might be the building boundaries of interest, are extracted and the final building footprints within a block are obtained. The proposed algorithm is implemented and tested on the urban area of Zeebruges, Belgium(IEEE Contest,2015). The areas of extracted footprints are compared to the corresponding areas in the reference data and mean error is equal to 7.43 m2. Besides, qualitative and quantitative evaluations suggest that the proposed algorithm leads to acceptable results in automated precise extraction of building footprints.


INTRODUCTION
As a consequence of rapid urbanization and population growth, having comprehensive and accurate information on urban structures and particularly buildings, is becoming essential for a proficient urban management.Such information is also a prerequisite for many other applications, such as urban risk management, disaster monitoring, real estate, and national security.One of the most important features of buildings is their footprint that can be extracted directly using remote sensing data or field survey.Even though such information can be obtained using field survey which is costly, time consuming, and it cannot be updated automatically.In this regard, in order to expedite the extraction and update the process using remote sensing data would be evidently more effective.Building footprints can be extracted using remote sensing data manually or automatically.Since data updating is important in urban management, automatic methods are absolutely preferable.This paper aims to present a valid and tractable method to extract building footprint using remote sensing optical image and LiDAR data.The proposed method is after automated building footprints' extraction, using Hough transformation in combination with optical imagery and LiDAR data.Footprint extraction for individual building within a building block in addition to isolated buildings is also considered in this paper.There are two main remote sensing data sources available for building's attribute extraction including optical imagery (with three visible bands) and LiDAR data.Due to varying input data sources, there exist different techniques for building footprint extraction using remote sensing data.In the approach presented by d.San Koc (D. Koc San et al, 2010), buildings were extracted from high-resolution satellite images using SVM (Support Vector Machine) classifier and Canny edge detector.In another work, Yangeng Wei and his colleagues (Yangeng Wei et al 2004) * School of Surveying and Geospatial Engineering, University of Tehran, Tehran, Iran -Hossein.arefi@ut.ac.ir employed an unsupervised clustering method on panchromatic images to separate buildings from background and also take out the shadow class.Krishnamachari (Krishnamachari and Chellappa, 1994) used edge detector operator in order to extract edges in an image.Then the Markov chain model was utilized on extracted edges with a proper neighbourhood.Since there is no guarantee to always extract a complete polygon from the model, active curve model is subsequently implemented for this purpose.In another research area-based active contour model was further used to detect homogenous area (Nosrati and Saeedi, 2009).However, in their method a threshold were suggested for separating homogenous areas related to ground and vegetation from buildings.An approach proposed by Demir et al, incorporated four methods to extract buildings by combination of optical imagery and LiDAR data.In the first method, the buildings were extracted using DTM/DSM comparison as well as NDVI index analysis.The second method made use of a supervised classification on multispectral image that was refined by height information from LiDAR data.Last two methods were also founded upon LiDAR data.The former used voids in LiDAR DTM and NDVI classification, while the latter was based on the analysis of the density of raw LiDAR DSM data (Demir et al,2009).Siddiqui et al. also proposed a robust gradient-based method for building extraction from LiDAR and photogrammetric imagery.By transforming LiDAR information to intensity image without interpolation of point heights, and then analyzing the gradient information in the obtained image, their proposed algorithm could extract small single buildings with transparent roofs (Siddiqui et al,2016).Aforementioned studies have proposed methods to extract footprint of small and large isolated buildings.In other words, these studies have not presented a way to extract buildings within a building block.Since building blocks contain several connected buildings with almost constant heights, edge analysis based on height or non-ground area classification in tree and building class, cannot be considered as a proper way to extracts buildings within a block.Hence in this study, we propose a new method to extract buildings located in a building block as well as isolated buildings.The proposed method first recognizes the predominant leading orientation of a building block using Hough transform, and then rotates the block according to the inverted complement of the dominant line's angle.Therefore the block poses horizontally.Afterwards, by use of another Hough transform, vertical lines, which might be the building boundaries of interest, are extracted and the final building footprints within a block are obtained.In the following sections, details on the proposed method are described and results obtained through its implementation are presented and discussed.

PROPOSED ALGORITHM
As stated, this paper aims to extract building footprints automatically, using Hough transformation in combination with optical RGB image and LiDAR data.Figure ( 1) illustrates a workflow of our proposed method to this end.It should be noted that a first assumption to this method is that the RGB image and LiDAR data are quite well geo-referenced.In order to extract buildings, an initial step is to first distinguish buildings from other objects.Since the elevation information is one of the most important components in building detection, LiDAR image contains this source of useful information.In the first step of processing chain nDSM is created from DSM.The nDSM is selected so that includes objects that are high-rise but not vegetation.This (masked) nDSM can be considered as building map.Thus, buildings are obtained from LiDAR data, and each building whether isolated or in a building block, are projected on the optic image assuming that the optic image and LiDAR image are properly co-registered.Afterwards, if the candidate building is actually a building block, the outer and the inner boundaries are extracted.The following sections describe the proposed algorithm in detail.

nDSM Generation
Laser pulses emitted from a LiDAR system reflect from objects both on and above the ground surface.Objects above the ground surface can be extracted by subtracting bare ground or DTM (which is calculated using the geodesic morphological reconstruction algorithm (Arefi et al,2005)) form above ground or DSM as equation ( 1): nDSM = DSM − DTM (1)

Building Extraction
In this step, non-ground points must be additionally classified into buildings and trees.First, buildings whose boundaries have high local variance and low vegetation index, are extracted from LiDAR data (Arefi et al,2003).Besides, there exists a big gap in the reflected pulses of LiDAR data (Arefi,2009), which can be considered as an index for distinguishing vegetation from buildings and hence is looked at as a novel index for vegetation discovery as equation ( 2), (3).
In our data set, vegetation was already eliminated, therefore only a threshold was adopted to remove garage and car segments.The output is a binary image showing building segments.
()  ℎ = FP − LP (2) Where FP=First Reflected Pulse in LiDAR data LP = last reflected pulses in LiDAR data Each region with the value of "1" in the output image is assumed as a candidate building.Candidate buildings are individually taken into account and projected to the geo-referenced optical image.Each candidate building in LiDAR image can be used as a mask in which building regions have a value of "1" and background and other buildings have a value equal to "0".This mask is projected to the optical image and as a result the image of candidate building is obtained.The resulting image is a colored image where buildings' pixels have the same value as pixel values of building image, and other pixels have the value of 0.

Buildings' Outline Detection
In order to determine building's footprint it is required to locate the outer boundary of the building.To this end, first the optic image of candidate building is transferred to gray value image, then an edge detector is used to extract patent changes throw boundary and the dark background.In other words, the goal of this step is to make the overall changes in the building surrounding against the black background more clear.Besides, it is essential to use a point reduction algorithm for simplification and elimination of bothersome points around the building of interest.Douglas-Pocker algorithm (Dilip K et al,2012) is used for this sake.

Inner Boundaries Extraction
As mentioned, for each candidate building, the dominant line must be extracted using Hough transformation.If the dominant line has a length more than a threshold (which is determined empirically by considering the length-to-width ratio of the smallest building in dataset), the so-called building might probably be a block and thus the inner boundaries must be detected.For each building that now has a dominant line, building block is rotated according to the inverted complement of the dominant line's angle, to have a horizontal orientation.Then, another Hough transformation is used to extract shorter length lines on the edge map extracted from RGB and LiDAR images.By block rotation, inner boundaries can be detected as vertical lines and thus the following computations will be easier.
The lines with a slope equal to 10 degrees from vertical aspect are selected and considered as inner boundaries.Shorter lines, which belong to chimney edges, are eliminated by comparing the proportion of the vertical line's length to the distance from top and down boundaries along the so-called vertical line.If there exists more than one edge as inner boundary, a line should be fitted to them to get rid of the extra lines.Finally to ignore the ridge lines, local maxima are extracted.
Figure 2. Workflow of Extraction of adjacent footprints in building block based on Hough transformation

Computing the Footprint of Each Building
The points of the outer boundary are categorized based on the inner boundary, and the intersections of the block outer boundary with the final segregating lines are proliferated at one pixel rate, and as the result all the points belonging to each footprint is designated to a specific group.As a final step, an inverse rotation is to each footprint in order to have the extracted buildings in the right orientation.

IMPLEMENTATION AND RESULTS
The following subsections introduce case study and cover the evaluation results.

Case Study
The data we used in this study is consisted of an orthophoto RGB aerial image with 5cm spatial resolution captured from an urban area with 127.5*125 squared meters extent in Zeebruges, Belgium, as well as the LiDAR grid data of the same spot acquired in 10cm capture rate )IEEE data contest 2015(.This data is considered as a suitable evaluation case for building extraction algorithms due to the variety of building blocks in shape and size (fig 3).

Results
The aforementioned steps for extracting building footprints are graphically illustrated in following figures.The same procedure is applied for all other building blocks in the study area.
The RGB image of the case area is used for the results evaluation.

EVALUATION AND DISCUSSION
One straightforward strategy to assess the results of the proposed algorithm is to compare the building areas with their corresponding areas from the buildings extracted by humanoperated digitization.To this end, we used ArcGIS software for building digitization (fig 12).
Figure 12 .Digitized footprints in ArcGIS (blue) and extracted footprints by algorithm (yellow) Boundaries of the adjacent buildings in the eastern and western blocks are prone to less error due to closeness to optic nadir.
Besides, due to a small difference in color and height of adjacent buildings in southern block, one of the inner boundaries is not detected.
Building footprints or what is called as cadastral parcels are considered as the foundation for all land administration systems (LIS).The area of a parcel is apparently one of the most important features, which can be calculated from those vertices.Besides, by having the length of the edges and the area of the polygon of interest, one can mathematically define the shape of the polygon(T.Xiaohua et al, 2001).Therefore, we can relate and estimate the errors of the proposed algorithm by RMSE index calculated for building areas(X.Tong et al, 2005).In the case dataset, we had 4 building blocks and a total number of 26 buildings for which we had their digitized footprints extracted.The areas of those footprints are compared to the areas calculated with the automated algorithm proposed in this paper. No.
Area As demonstrated in the table, the minimum difference between the manually-digitized area and the automatically-calculated area is 0.5 m 2 which is related to the building number 8. The maximum error of 54.96 m 2 is due to the building for which the proposed algorithm could not distinguish its boundary from the neighboring building.Regardless of specific cases, the average error and the RMSE of the proposed algorithm is equal to 7.43 and 14.45 m 2 respectively.

CONCLUDING REMARKS
The proposed method revealed a great potential for inner boundary distinction from the ridges.However, in the future studies it is better to free the algorithm from the restrictions of threshold settings.Since the core of this algorithm works by edge detection and is based on some assumptions about roof color and height, in cases where buildings have the exact same height and roof color, the algorithm cannot provide proper results.To solve this problem, we are working on neighborhood and adjacency information as extra sources of information to distinguish those buildings as well.

Figure 1 .
Figure 1.Workflow for building footprints using optical imagery and LiDAR data

Figure 3 .
Figure 3. Test data: (a) optic image of test data , (b) DSM of test data

Figure 4 .
Figure 4 .Building area extracted from DSM The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W4, 2017Tehran's Joint ISPRS Conferences of GI Research, SMPR and EOEC 2017, 7-10 October 2017, Tehran, Iran extraction phase.The following subsections describe each step in details.
Figure (2) illustrates the flowchart of inner boundary calculated by ArcGIS( 2 )