EXTRACTION OF ROOF LINES FROM HIGH-RESOLUTION IMAGES BY A GROUPING METHOD

This paper proposes a method for extracting groups of straight lines that represent roof boundaries and roof ridgelines from highresolution aerial images using corresponding Airborne Laser Scanner (ALS) roof polyhedrons as initial approximations. The proposed method is based on two main steps. First, straight lines that are candidates to represent roof ridgelines and roof boundaries of a building are extracted from the aerial image. Second, a group of straight lines that represent roof boundaries and roof ridgelines of a selected building is obtained through the optimization of a Markov Random Field (MRF)-based energy function using the genetic algorithm optimization method. The formulation of this energy function considers several attributes, such as the proximity of the extracted straight lines to the corresponding projected ALS-derived roof polyhedron and the rectangularity (extracted straight lines that intersect at nearly 90). Experimental results are presented and discussed in this paper.


INTRODUCTION
Building extraction methods are important in the context of capturing and updating spatial data for applications that involve urban areas.For example, accurate and reliable building roof boundaries are useful in applications that involve real estate, large-scale mapping, and risk management.However, automated building roof extraction has remained a challenging task, mainly due to the varying building roof configurations, the varying neighborhood contexts, and noise in the input data.Gilani et al. (2015) presented a commonly used criterion for classifying building extraction methods that is based on the input data, such as methods that use ALS data, image data, and combinations of ALS and image data.Image-based building extraction methods exploit two-dimensional information that is derived from a single image or three-dimensional information that is derived from stereoscopic images.Examples of these methods are found in Fua and Hanson (1987), Müller and Zaum (2005), Akçay and Aksoy (2008), Ferraioli (2010), andSırmaçek andÜnsalan (2011).
Methods that are based on the combination of photogrammetric and ALS data seek to take advantage of the synergy between both data sources.ALS data are superior in terms of height accuracy, and image data are superior in terms of boundary definition.Many methods of this category are found in the literature, as e.g. in Haala and Brenner (1999), Sohn and Dowman (2003), Jaw and Cheng et al. (2008), Kim and Habib (2009), Chen and Zhao (2012), Awrangjeb et al. (2013), andGilani et al. (2015).This paper proposes a method for extracting groups of straight lines that represent roof boundaries and roof ridgelines from high-resolution aerial images using corresponding ALS-derived roof polyhedrons as initial approximations.The remainder of this paper is organized as follows.Section 2 describes the proposed method, the results are presented and discussed in Section 3, and Section 4 presents the main conclusions.

PROPOSED METHOD FOR EXTRACTION OF STRAIGHT LINE GROUPS
The proposed method is based on two main steps.First, straight lines that are candidates to represent roof ridgelines and roof boundaries of a building are extracted from the aerial image.A group of straight lines that represent roof boundaries and ridgelines of a selected building is then obtained through the optimization of a Markov Random Field (MRF)-based energy function using the genetic algorithm optimization method.

Extraction of straight lines
The input data include ALS-derived roof polyhedrons and an aerial image (along with orientation parameters).The first step consists of transforming the vertices of a roof polyhedron into the image space.Mathematically, this requires standard photogrammetric treatments using the camera model.Bounding boxes are then generated around the projected straight lines, which are defined by two adjacent projected roof polyhedron vertices.
The next step consists of applying the Canny edge detector (Canny, 1986) and the Steger line detector (Steger, 1996) within each bounding box followed by an edge-linking algorithm (Jain et. al, 1995) and the Douglas-Peucker polygonization algorithm (Douglas and Peucker, 1973).As a result, the following basic algorithm for extracting straight lines that represent roof boundaries and ridgelines with a minimum number of false positives is proposed: 1) apply the Canny and Steger detectors within all of the bounding boxes; 2) organize the edge points into straight lines using the edgelinking and Douglas-Peucker algorithms; and 3) check the pairs of Canny-derived straight lines for parallelism and proximity and eliminate the pairs that match this criterion and that have a Steger-derived straight line between them.
This algorithm allows for the extraction of n sets of straight lines, each of which is a candidate to represent a roof boundary or ridgeline.

Identification of roof lines using a MRF-based grouping method
The MRF model has the advantage of characterizing the contextual knowledge by modeling the spatial relationships between the primitives (e.g., segments of lines, contours) that represent real world objects.
Let X= {X 1 , X 2 , …, X n } be a family of random variables that are defined over the set of n primitives R= {R 1 , R 2 , ..., R n }, where each X i corresponds to R i .According to the Hammersley-Clifford theorem (Kopparapu and Desai, 2001), an MRF can also be characterized by a Gibbs distribution, i.e., Where: x is a realization of X; Z is a normalization constant; and U(x) is the Gibbs energy function.
The straight lines that are extracted from the image around the edges of the projected roof polyhedron are used to build a MRF model that expresses specific forms of building roofs with reference to the projected roof polyhedron.The energy function is defined so that each straight line is associated with a random variable (x i ), which takes binary values according to the following rule, This rule (Equation 2) results in an n-dimensional random vector, where n is the number of straight lines.This random vector is the unknown that is determined with an optimization process.
Figure 1.Geometric elements for defining the relative length, proximity, and orientation energy terms Figure 1 shows a hypothetical example of a building roof that is projected onto the image space.The projected straight lines are f 1 , …, f n .We assume that n k straight lines were extracted around the projected straight line f k .Figure 1 shows only the ith straight line (F i ) that was extracted from the f k bounding box and the jth straight line (F j ) that was extracted from the f k+2 bounding box.We say that the projected straight line f k (or f k+2 ) is the nearest line to the extracted straight line F i (or F j ).These elements, and others that are explained just below, are sufficient to understand the formulations of the relative length, proximity, and orientation energy terms.
The energy function U(x) is composed of five energy terms: 1) The first term (Equation 3) favors long straight lines with reference to the nearest projected straight line.It is called the relative length energy term.
Where: L ! ! is the length of the projected straight line (f k ) that is nearest the ith straight line that was extracted from the image (F i ); and L ! ! is the length of the extracted straight line (F i ).
2) The second term of the energy function (Equation 4) is called the proximity term and is defined for two extracted straight lines (e.g., F i and F j ) that are in the neighborhood of two different projected straight lines (f k and f k+2 considering F i and F j , respectively).This term favors a pair of straight lines that is closer to the projected straight lines.
Where:  ,  = 0.5 the distances between the endpoints of the extracted straight line F i and the nearest projected straight line (f k );  !! and  !! are the distances between the endpoints of the extracted straight line F j and the nearest projected straight line (f k+2 ); and N i is the set of straight lines that is in the neighborhood of the extracted straight line F j .
3) The third term is called the orientation term (Equation 5) and favors a pair of extracted straight lines (e.g., F i and F j ) with similar orientations to the nearest projected straight lines (f k and f k+2 considering F i and F j , respectively).

∑ ∑ =
Where: ! is the angle between the extracted straight line F i and the nearest projected straight (f k );  ! is the angle between the extracted straight line F j and the nearest projected straight line (f k+2 );  is a positive constant; and  ! is the optimal value (0° or 180°) of the parameter .
4) The fourth term is called the rectangularity term and favors configurations of extracted straight lines that intersect at approximately right angles.This occurs with rectilinear roof boundaries.This property is usually not valid for straight lines that represent roof ridgelines.The term is as follows, Where:  !" is the angle between the extracted straight lines F i and F j , each of which neighbors two adjacent projected straight lines.
5) The fifth term is called the corner term and should favor extracted straight lines that intersect near a corner in the image where two roof edges intersect each other at approximately a right angle.This principle can be mathematically represented as Where: D ij is the Euclidian distance between the intersection point of two extracted straight lines (F i and F j ) that neighbor two adjacent projected straight lines and the nearest corner (q ij ) detected in the image; and λ !" is the angle between the edges that define the corner q ij .It is worth noting that the fourth and fifth energy terms should only be applied for straight-line candidates to represent roof boundaries.Roof ridgelines usually do not intersect at right angles.Finally, the energy equation is formulated as follows, Where: k 1 , …, k 5 are positive constants, with k 1 + … + k 5 = 1.
The optimal configuration x is obtained by minimizing the energy function U(x); i.e., x= argmin(U(x)).We employed the Genetic Algorithm (GA).
GAs are based on evolutionary ideas of natural selection.They exploit a random search to solve optimization problems.The search space comprises a population of individuals (chromosomes) that represent possible solutions to a problem.A random vector of components represents each individual.Usually, each component is coded to a binary value (i.e., 0 or 1).After randomly selecting an initial population, the algorithm proceeds by employing the following three operators: 1) selection, which equates to survival of the fittest; 2) crossover, which represents the coupling between individuals; and 3) mutation, which introduces random modifications.The GA optimization converges when the fittest individual is good enough according to a fitness measurement.For more details, please refer to the relevant literature, such as Goldberg (1989).

EXPERIMENTAL RESULTS
The data set that was used in our experiments includes ALSderived polyhedrons that represent building roofs, which were extracted from an ALS point cloud with a density of approximately 2 points/m 2 , and 20-cm digital aerial images along with interior and exterior orientation parameters.
The following parameters and thresholds were used: constant that controls the shape of the sigmoid function: = 20; weights that control the relative importance of the energy terms of the energy function: k 2 = 0.1; k 4 = 0.3; k 1 = k 3 = k 5 = 0.2. Figure 2 shows a building roof that is defined by four sides and five ridgelines.Figure 2(a) shows the projected ALS-derived roof polyhedron overlaid on the image.Note that the projected straight lines 3, 4, and 8 coarsely match the corresponding roof boundaries or roof ridgelines.Moreover, registration errors are visible along the other roof boundaries and ridgelines.Figure 2(b) shows nineteen straight lines that were extracted using the pre-processing techniques that were proposed in Subsection 2.1.They are candidates to represent roof boundaries and roof ridgelines.Roof ridgelines 6 and 8 each have only one candidate, roof boundaries 1, 2, and 4 each have two candidates, roof boundary 3 and roof ridgelines 5 and 9 each have three candidates, and ridgeline 7 has two candidates.Moreover, roof boundaries 1, 2, and 3 have candidates that belong to the ground or to a neighboring building's roof.The larger the registration error, the larger the area within the bounding box and, as a result, the higher the probability of extracting straight lines from adjacent objects.Figure 2(c) shows the result of the optimization process.Although most of the candidate sets involve two or more nearly parallel straight lines, the optimization process was able to find all of the correct correspondences.Moreover, roof boundaries 1 and 2 have candidate straight lines that belong to a neighboring building.In all of these cases, the correct candidates are the longest ones and are also nearer to the corresponding projected straight lines, which favors the minimization of the relative length and proximity energy terms.The extraction of the correct corner also helps in the global minimization of the energy function.For example, the two corrected candidate straight lines that were extracted from around projected straight lines 2 and 3 have the highest probability of intercepting near a right-angle roof corner; in this example, this is the image corner that is defined by roof edges 2 and 3.

CONCLUSIONS AND FUTURE WORK
This paper proposes a method for extracting groups of straight lines that represent roof boundaries and roof ridgelines from high-resolution aerial images using corresponding ALS-derived roof polyhedrons as initial approximations.First, a strategy for extracting straight lines around roof boundaries and ridgelines was presented.The best group of straight lines that represent roof boundaries and ridgelines were then identified using the GA optimization method.
To preliminarily demonstrate the performance of the proposed method, two experiments were presented, and their results were analyzed.These experiments were performed with two relatively simple hip roofs.The main difficulty of these examples is the relatively greater registration errors because it results in a higher chance of extracting candidates that belong to neighboring objects.Additionally, the worse the image orientation parameters, the poorer the projected ALS-based roof polyhedron, which will provide a weak reference for establishing the MRF building model.The examples showed that the proposed method can identify straight-line segments that represent roof features even when the registration error is large.All of the correspondences were correctly found for both test buildings.
Future work includes the use of straight lines that represent roof boundary sides, which were precisely extracted by using the proposed approach, to improve the accuracy of ALS-derived building polyhedrons.This can be accomplished by properly back-projecting the straight lines extracted by the proposed method onto the ALS-derived building polyhedrons.
Test 1.(a) Projected ALS-derived roof polyhedron; (b) straight lines extracted from the image; and (c) group of straight lines obtained by the GA optimization Figure3shows the second test building.Figure3(a) shows the projected ALS-derived roof polyhedron superposed on the image.It is worth noting that a large registration error is clearly observed because the projected ALS-derived roof polyhedron is displaced to the right.Eighteen straight lines were extracted around the roof boundaries and ridgelines (Figure3(b)).As in the previous example, the pre-processing step retains few candidates that represent every roof boundary and ridgeline.All of the boundaries have two candidates, four ridgelines have one or two candidates, and one ridgeline has four candidates.However, few straight lines are extracted from adjacent objects due to the large registration error.Figure3(c)shows the result of optimizing the MRF-based energy function using the GA.The method correctly matches the candidate straight lines to the corresponding projected ALS-derived straight lines; in other words, no false negatives or positives occur in this example.
The application of the Canny and Steger detectors within the bounding boxes instead of applying them to the entire image is intended to avoid the extraction of straight lines that have no chance of representing a boundary or ridgeline of a roof.The rationality of combining both detectors is that roof boundaries and ridgelines in very-high-resolution aerial images exhibit different image profiles.Most roof boundaries have step-edge profiles, while most roof ridgelines have linear profiles.This means that the Canny edge detector usually detects only one edge per roof boundary and two near and parallel edges per roof ridgeline.Conversely, the Steger detector usually detects only one line per roof ridgeline.Moreover, ridgelines that are detected by the Steger detector approximately coincide with the axes of symmetry of the corresponding double edges that are detected by the Canny detector.