Extraction of building roof edges from LiDAR data to optimize the digital surface model for true orthophoto generation

: Various kinds of urban applications require true orthophotos. True orthophoto generation requires a DSM (Digital Surface Model) to project the photo orthogonally and minimize geometric distortion due to topographic variance. DSMs are often generated from airborne laser scan data. In urban scenes, DSM data may fail to deliver sharp and straight building roof edges. This will affect the quality of the resulting orthophotos. Therefore, it is necessary to incorporate good quality building outlines as breaklines during DSM interpolation. This study proposes a data-driven approach to construct building roof outlines from LiDAR point clouds by a workflow consisting of the following steps: given roof segments, roof boundary points are extracted using a concave hull algorithm. Straight edges may be difficult to find in complex roof configurations. Therefore, two ingredients are combined. First, RanSAC corner point preselection, and second, DBSCAN-based clustering of edge points. The method is demonstrated on an area of ±1.2 km 2 containing 42 buildings of different characteristics. A quality assessment shows that the proposed method is able to deliver 92% of building lines with acceptable geometric accuracy in comparison to a building line in the base map.


Background
Many urban applications require true orthophotos as a basic geospatial data source to construct real-world visualizations with accurate positions. Buildings are one important objects to be presented on the map. High-elevated buildings may suffer from positional distortion due to relief displacement caused by the aerial photo acquisition geometry. True Orthophoto generation is one of the most reliable methods to remove distortion by projecting back image pixels to the correct location using a DSM. However, because of DSM gridding, some buildings may lose their sharp edges. This will introduce building boundary errors in the true orthophotos (the building edges looks crooked or bent in the image). A possible solution is to add building outlines as breaklines to the DSM to preserves sharp building edges. The suitability of true orthophotos to define object edges relies on the quality of the DSM. The sharper the building edges are represented in the DSM, the better the building edge representation will be in the resulting true orthophotos.
In the absence of base maps or updated maps, it is required to extract building edges. LiDAR data is known for its high 3D positional accuracy and high point density for mapping. In the case of boundary detection, LiDAR may have a limitation in providing sharp edges, as LiDAR points, in general, do not always hit the exact building edges. This paper proposes and implements a semi-automated workflow to extract building roof edges from LiDAR data by combining an adapted version of RanSAC (RANdom SAmple Consensus) line fitting with DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering of point-wise line parameters.

Related Work
Some true orthophoto refinement efforts that use breaklines for DSM improvement have already been conducted (Wang et al., 2018, Hu et al., 2016, Wu et al., 2011, Kato et al., 2010. Wu et al. (2011) improved building boundaries by fusing LiDAR and aerial photos to get 3D lines. The 3D building boundaries were extracted from segmented LiDAR points using a convex hull algorithm followed by a back-projection of 3D to 2D image space. However, many buildings boundaries are not as straight as expected. Hu et al. (2016) worked on orthophotos with occluded areas detected by elevation-buffer visibility from DSM and building roof model. Then, occluded areas are filled with data from overlapped orthophotos. This method still has limitations since the results may be not good along building boundaries in case of inaccurate building roof models. Wang et al. (2018) proposed to match 2D line segments to reconstruct 3D lines. The 3D end points of each line are then incorporated with dense image matching points clouds in a TIN (Triangular Irregular Network) processing.
Automatic building roof outline extraction is still a challenging problem, especially when straight and accurate edges are required. Several types of research on automatic building edge extraction from either LiDAR data and images have been conducted (Awrangjeb, 2016, Zhao et al., 2016, Siddiqui, et al., 2016, Yang et al., 2013, Wu et al., 2011, Awrangjeb, 2010. Wu et al. (2011) fused rough building lines from a LiDAR DSM and 2D lines from oriented aerial images to extract straight 3D building lines. These straight building lines are then used to produce true orthophotos. However, some resulting building boundaries were still not straight. Zhao et al. (2016) and Awrangjeb (2016) determined building outlines from point cloud data by boundary tracing and regularization. Somehow, errors occur due to missing low buildings and vegetation points near to the buildings Zhao et al. (2016), or failures to estimate a dominant/principal direction Awrangjeb (2016). Siddiqui et al. (2016) proposed a gradient-based technique to extract building lines from LiDAR and images. Roof edges correspond to big height changes that are revealed by large gradients in a height intensity image. Straight lines are then extracted from the image using Canny and Gaussian functions. The buildings outlines were confirmed after color matching, shadow detection, and noise filtering. The proposed method is only designed for buildings with a flat or sloped roof. However, the paper did not address the possible relief displacement on the building roof may occur in the image space. A common problem of model-driven approaches is that some important lines that are not dominant may not be found. Moreover, the quality of geometric accuracy of these methods is not reported and results often seem incomplete.

STUDY AREA AND DATA DESCRIPTION
The chosen suburban study area is located on Maros, Sulawesi island of Indonesia. The total size of the study area is ±1.2 km². There are two main components of data used in this study: a. Main input data LiDAR data was acquired by a Leica ALS70 instrument. The average point density is 10 -12 points/m. A Leica RCD30 instrument captured the aerial photos with 9cm spatial resolution. The data use the WGS84 reference system and is projected in UTM Zone 50S. Aerial photos and LiDAR data are acquired at the same time from the same platform in 2012. The specification of these main input data is designed and intended for base mapping at a scale of 1:10.000.
b. Reference data This study uses 1;10.000 Topographic Base Maps as a reference to validate the accuracy of the resulted lines and the true orthophotos. The topographic base map has a geometric accuracy better than 5 meters. The reference data consists of 3D building rooflines that are manually delineated from stereo-photos acquired at the same time as the LiDAR data. .

METHODOLOGY
The research workflow, as illustrated in Figure 1., is designed to extract building roof outlines to improve the quality of DSM for true ortho-rectification without any presence of base maps.

Figure. 1. Methodological workflow
This study aims to extract building roof outlines that meet specifications and assumptions as specified below: -The building is represented by the size and shape of a 2D representation of the roof. That is, the size and shape of the buildings are similar to the roof as seen from the top. -Small details or interior parts inside the building roof will not be considered. For example, in extracting the outlines, a building with a gable roof will be treated as a flat roof. -The expected result should meet the requirements of 1:5.000 map scale. A building of an area of 2.5 m x 2.5 m or larger should be present in the map. -The minimum positional accuracy of building edges vectors is 2.5 meters.

LiDAR Point Cloud Filtering
LiDAR point cloud filtering is the first important step to guarantee the quality of the extracted buildings edges. This study uses a set of building points extracted by a rule-based approach in the TerraScan software as a starting point. In rule-based filtering, the first step is to separate non-ground from ground points. The roof points are then selected from the non-ground points based on their planarity and connectivity. The result building filtering of the study area is shown in Figure 2.
(a) full points (b) filtered building points

Building Points Clustering and Edge Point Selection
In order to find roof edge points, roofs are identified by segmentation. This study uses DBSCAN, introduced by Ester et al. (1996), to cluster points into single buildings. Ghosh and Lohani (2013) performed a DBSCAN on a small subset of LiDAR data (100 m by 100m) with a high success rate w.r.t. computation time and accuracy. After clustering some noise or outliers (including tree points) may remain that will finally affect the roof edge extraction. DBSCAN is chosen because of its capability to detect outliers in the data. To find a cluster, DBSCAN starts with an arbitrary seed point p and retrieves all neighbouring points (density-reachable) from p that are located within a given radius distance (eps) and contains a given minimum number of points (minPts). Outliers are defined once minPts cannot be achieved within the given eps. DBSCAN uses region-growing to grow the cluster. The cluster will grow as long as nearby points within the eps distance from seed p fulfill the minPts threshold. In case minPts within distance eps is not fulfilled, a point or group of points is considered as outliers.
During the cluster growing, outliers may change into a member of the cluster once they are in the eps distance from the respective seed. To grow the next cluster, the next seed is chosen that does not belong to any cluster. The clustering stops once all points are assigned.
In this study, we implement DBSCAN clustering from 3D and 2D point data. By applying a similar parameter threshold, we found that clustering 3D point data discovers more buildings than 2D points as discussed in Figure 3. Therefore, we used 3D points as input for the edge line extraction.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy From one building segment, ideally corresponding to a roof, a concave-hull algorithm is used to find the outermost points of that segment. This study uses a generalization of the convex hull, named alpha-concave hull modified by Asaeedi et al. (2003). It uses alpha to define the smoothness threshold. In order to preserve the building shape, the alpha threshold parameter of the concave hull may need to be adapted especially for noisy data. Even though the building outlines obtained by the concave hull method somehow fit the roof lines, they still do not meet the requirements of producing a straight line.

Building lines extraction
The workflow design assumes that most buildings roofs consist of straight lines instead of curved lines. RanSAC has the ability to fit a mathematical primitive to set of data containing noise. This algorithm was introduced by Fischler and Bolles in 1981.
In this study, RanSAC is employed to find the initial line models through all the boundary points. RanSAC typically finds the edge sampled by most points first. Since the RanSAC line model is influenced by the selection of random points, we guide RanSAC better to find suitable lines. Instead of using two random points, we use one initial point and one random point. To find the given initial points, we compute the mean position (centroid) of all points and calculate the Euclidean distance of each point to the centroid. The farthest point from the centroid is typically one of the corner points of the cluster. This is then used as one of the two points for line fitting in each RanSAC run.  # line model1 is found Given optimum distance threshold (dopt) for any p in D: Do step 2 to25 for the D2 dataset (to find edge-2) In case of noisy edge points, RanSAC may still fail to find enough potential points as inliers. Therefore, as an additional step the distance threshold is relaxed if needed. The resulting new inlier points are then added to the existing inliers, as already found by RanSAC. On the final set of inliers, we apply leastsquare line fitting to get a robust first building edge. For the second edge, we remove all inliers points belonging to the first line from the dataset. Then, for the remaining points, we apply the same adaptive RanSAC procedure. This procedure is repeated until all building edge lines are found (Figure 5.). Given the building lines, corner points are computed based on the line intersections.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy Any edge extraction method should accommodate variations of building size and shape. Based on our experiments, the proposed adaptive RanSAC approach perfectly works for simple buildings (rectangular shape) with less noise but may fail to find lines for more complex buildings (multi-square shape) that have more noisy edge points also. To accommodate complex roofs, we further adapted RanSAC by combining it with DBSCAN clustering. DBSCAN is used to cluster 2D edge points to give direction to RanSAC to find inliers and create suitable lines. The second procedure is started by line fitting using adaptive RanSAC as described in procedure 1. In most cases, RanSAC will be able to find the longest edge since it has most points. Once the corresponding line is found, we remove its supporting inliers from the dataset. Then, we apply DBSCAN to cluster the remaining points. The biggest cluster is selected first. The RanSAC procedure is then applied to this cluster to find the second line. Again, the iteration stops once all lines are found.

DSM and true orthophotos generation
The extracted building lines are then reprojected into 3D lines by interpolation using the original LiDAR DSM. These 3D lines are then used as hard breaklines for DSM optimization. The breaklines are expected to represent surface discontinuities and enhance the height jump at building roof edges. Figure 7. shows the difference of a DSM without breaklines (left image) and an optimized DSM with breaklines (right image). The building edge in the right image has indeed more straight representation. Based on height information, high-rise objects like buildings, trees, or bridges are set back into their true location in the true orthophotos. Therefore, a sharp edge will preserve a correct and straight building representation in true orthophotos.

Validation
The quality of the extracted building roof edges is validated based on the number of extracted buildings and the positional accuracy. It uses buildings from the base map as the reference data. Completeness of the extracted buildings resulting from our method is assessed by using the number of buildings, edges and corner points that are correctly detected. The positional accuracy is measured based on the corner points position. The squared root of the average of the squared differences of X and Y directions is calculated to estimate the RMSE (Root Mean Square Error). We use US-NMAS (United States National Map Accuracy Standards) to determine the circular accuracy. It specifies that 90% of the well-defined points that are tested must fall within a specified tolerance. If an error is normally distributed in each the x-and y-component and error for the x-component is equal to and independent of error for the y-component, the factor 2.146 is applied to compute circular error at the 90% confidence level (Greenwalt and Schultz, 1968). Therefore, the circular map accuracy is: Circular Map Accuracy = 1,5175 x .

RESULTS AND DISCUSSION
This study focuses on the extraction of building edge extraction and particularly, aiming to improve the DSM. However, the results and discussion will focus on the results of the methodology for roof line extraction.
The line fitting procedure is successfully implemented especially for simple roof buildings. Our method is also able to produce accurate building edges in case of many outliers due to filtering errors as illustrated in Figure 8. Adaptive RanSAC succeed to find accurate lines in a simple building roof with no outliers (left), a simple building roof with many outliers (centre), and a rectangular building with outliers (right).
There are some buildings are not or incompletely detected (showed by the red arrow) as shown in Figure 9. This can be attributed to the failure of filtering in detecting building roof points. Another issue is that reason is that we found one building which has two clusters (showed by a red ellipse). The possible cause is that DBSCAN is sensitive to its input parameters, which makes it difficult to segment a point cloud of varying point density correctly. Figure 9. Base map (orange) and filtered building points (blue) Figure 10. presents the comparison of building roof lines from the base map and roof lines resulting from the proposed method. From 42 buildings in the base map, our workflow is able to find 39 buildings. It means that the implemented workflow is able to detect more than 92% of the buildings in the study area.
Nevertheless, around 18% of the result (7 buildings) have big geometric differences: -Five buildings are under-segmented but have a similar shape. Our roof lines result has a smaller size than the reference (showed by black arrows in Figure 10.) -One building has a different building shape. Our line result is pentagonal while the reference has a heptagonal shape (showed by the red arrow in Figure 10.) Figure 10. Base map (orange) and buildings from adaptive RanSAC (green) We provide complete results of our building roof line extraction in Table 1. Regarding positional accuracy, the average of geometric accuracy for a simple building roof is 0.576 meters. While for a successfully extracted complex building roof, it has less geometric accuracy, 0.819 meters. Still, in both cases, it has a sub-meter difference to the base map.
(a) Concave hull (green) and simplified hull lines (orange) (b) Building line result (red) of the proposed method Figure 11. Comparison of the concave hull, a simplified concave hull, and adaptive RanSAC line We made a comparison between building lines resulting from a concave hull line simplification and our lines. The experiment shows that a concave hull simplification may still create corners and lines that deviate from the original position in some particular part in producing a straight line, while our result is able to fit to the corner points and preserve the roof shape as shown in Figure  11a.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2, 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020", 4-7 June 2018, Riva del Garda, Italy Due to a lack of points on the edge segments, the outlines of small buildings may deviate from its real shape or even fail to be extracted as shown in Figure 12. Another cause of errors is the presence of outliers on a small building roof. In the study area, a small balcony roof is assigned to a different segment than the main building roof. This small balcony has a size of 1m x 6 m, and it has two short edges. The boundary point distribution with very short edges (that only have one or two points) makes it impossible to find correct lines.

CONCLUSION
Our experiment shows how straight and accurate lines are extracted using the adaptive RanSAC technique. It outperforms existing methods in case of noisy edge points. The positional accuracy of the resulting lines is good, 92% buildings are detected with sub-meter accuracy, compared to the buildings in a reference base map. This workflow can be implemented for mapping activities.
However, the adapted RanSAC algorithm has difficulties on short and noisy edges. Thus, the result depends on the quality of point clouds filtering and segmentation. Rather than working persingle building, our future work will focus on how to automate the building line extraction for a complete area at once. Combining RanSAC lines from LiDAR and color information from aerial photos may give better performance and automation for building edge detection. Solutions for non-straight building edges (such as a dome roof) and buildings with sharedboundaries also need to be addressed.