BUILDING FACADE AND ROOFTOP SEGMENTATION BY NORMAL ESTIMATION FROM UAV DERIVED RGB POINT CLOUD

Point cloud segmentation is a significant process to organise an unstructured point cloud. In this study, RGB point cloud was generated with the help of images acquired from an Unmanned Aerial Vehicle (UAV). A dense urban area was considered with varying planar features in the built-up environment along with buildings with different floors. Initially, using Cloth Simulation Filter (CSF) filter, the ground and the non-ground features in the Point Cloud Data (PCD) were segmented, with non-ground features comprising trees and buildings and ground features comprising roads, ground vegetation, and open land. Subsequently, using CANUPO classifier the trees and building points were classified. Noise filtering removed the points which have less density in clusters. Point cloud normals were generated for the building points. For segmentation building elements, normal vector components in different directions (X component, Y component and Z component) were used to segment out the facade, and the roof points of the buildings as the surface normals corresponding to the roof will have a higher contribution in the z component of the normal vector. The validation of the segmentation is done by comparing the results with manually identified roof points and façade points in the point cloud. Overall accuracies obtained for building roof and building facade segmentation are 90.86 % and 84.83 % respectively.


INTRODUCTION
Identification of building rooftops is primarily useful for understanding the photo-voltaic (PV) potential of urban areas (Wiginton et al., 2010). There are also other areas of study that require this information such as the study of urbanisation and related phenomenon (Zhao et al., 2015), BIM (Dawood et al., 2017;Pirotti et al., 2019) and municipal management. With ever-growing urban areas and the need for sustainable energy practices, reliable and cost-effective techniques to identify building rooftops are, therefore, a useful requirement.
Airborne LiDAR has been used for identification of various rooftops owing to the geometric details present in the 3D point clouds acquired using this platform (Hujebri et al., 2019;Jung et al., 2017;Kushwaha et al., 2019;Nguyen et al., 2012;Pirotti et al., 2019;Zhao et al., 2015). Although several studies have implemented different methods to extract rooftops and facades from LiDAR data, the data is generally very costly, and there are some limitations on the flexibility of data acquisition. Unmanned Aerial Vehicles (UAV) platforms are relatively more flexible, and the costs are not high. However, there is a trade-off between these platforms in terms of the coverage. Besides, UAV-based LiDAR is also a recent development. On the other hand, UAV-based photogrammetry has been extensively applied to several urban-related studies in the recent years (Fernandez Galarreta et al., 2015;Palanirajan et al., 2019;Vetrivel et al., 2018). The costs involved with this technology are significantly lower. The optical sensors that are mounted on these systems are relatively inexpensive, and they also provide textural information which can significantly aid in segmentation results (Rouhani et al., 2017;Vetrivel et al., 2015).
As mentioned previously, there have been significant advancements in the algorithms for extraction of rooftops. This study focused on utilising some simpler methods and techniques for identification of rooftops using a low-cost UAV based photogrammetry approach that would possibly enable an enduser to obtain preliminary information about rooftops in a given area with relative ease. In this study, UAV based photogrammetric point clouds were processed to extract the roof and facade features of built-up structures. Surface normals are very useful to characterise the underlying surface of an object. They were used to identify roof elements and separate them from facade elements. As buildings generally comprise standard geometric elements such as planes, curves etc., the normals of these elements tend to have regular 'patterns'. These normals on buildings were analysed to understand the differences in normals for facades and rooftops.

STUDY AREA AND DATASET USED
The UAV dataset was acquired in a dense urban area located at Khanjarpur area, Uttarakhand, India. The location of the study area is depicted in Figure 1. The study area comprises densely packed buildings, which is a common feature in urban settlements in India. Characteristics of rooftop areas in such urban structures include parapet walls, staircase openings etc.
Parameter  The dataset used for this study was acquired using a DJI Phantom 4 Pro (UAV) with an optical sensor mounted on it.
The dataset parameters are tabulated and shown in Table 1. Flight parameters were decided prior to the acquisition so as to ensure the acquisition of images that were ideal for a dense point cloud generation. A total of 102 images were acquired. Pix4D was used for generation of a dense point cloud, and CloudCompare was used for processing the point cloud.
The subset that was chosen for this study is shown in Figure 3. This area was considered as there is a presence of several urban elements such as buildings with roofs at different levels, roads and some trees. The data acquisition was made by keeping the camera angle nadir which resulted in less dense points in the facade regions.

METHODOLOGY
Pix4D was used to reconstruct the 3D point cloud, and a subset of the larger area was used for further processing and analysis. The point cloud was cleaned for any noise by manual segmentation. It was then classified to obtain ground points using the CSF filter (Zhang et al., 2016). The study area is a flat terrain. A cloth resolution of 0.5 and a classification threshold of 0.5 were considered. The non-ground points contained objects such as trees, buildings, cars, etc. The CANUPO (Brodu and Lague, 2012) classifier was then trained to filter the buildings and trees from the remaining non-ground points.
A normal vector has three components in x-direction, ydirection and z-direction respectively. The contribution of the normal vector to its component depends on the direction of the normal at the given point. For example, if the normal is predominantly oriented upwards, then the contribution of the normal vector in z-component will much greater than the xcomponent and y-component. Normals were estimated for the filtered dataset, and the components of the normals were extracted. It was observed that, for the z-component of the normal vector, the components representing the roof portions had a range of 0.97 to 1. Subsequently, points with zcomponents in this range were classified as roof elements. The z-components less than 0.97 were classified as facade points. The methodology is shown in Figure 2. The subset of the photogrammetric point cloud generated from the UAV raw images used for the research is shown in Figure 3.   The non-ground points consist of trees and buildings. The classifier was trained in such a way that the building edges were also considered in the same class as trees due to the presence of shadows. Similarly, shadows were also present at the staircases to the roofs and between buildings where sufficient illumination is generally absent. Such characteristics were common in this area. As the focus was on the extraction of the planar regions representing the roof and facade, the elimination of the other 'noisy' features was useful. The classified building (roof) points are shown in blue and trees, staircase, shadow, and building boundary points are shown in red in Figure 6.

Accuracy Assessment
Three buildings (Figure 8) were considered to assess the results of the methodology . The equation mentioned below was used for this purpose.
Percentage accuracy = (No. of points segmented by the proposed method/ No. of points segmented manually) x100 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W17, 2019 6th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 2-3 December 2019, Strasbourg, France   There is a difference in the point density between the proposed segmentation method and manual segmentation method. After the buildings were segmented out as non-ground points, noise filtering was done in which the points not close enough to the underlying planar surface (roof or facade) were removed as noise points. The regions where point's density was less were also removed as noise points. Furthermore, the points generated for a plane by photogrammetric reconstruction are comparatively 'less planar' in comparison to those acquired using LiDAR. This could affect the normal estimation. In order to avoid this, noise removal was carried out. These could perhaps be a contributing factor in the reduced point densities after the noise removal stage. However, the general area representing the feature of interest was identified with reasonable accuracy. In the near future, the algorithm will be tested in different study areas with different features.

CONCLUSIONS
The proposed methodology is simplistic yet efficient to obtain a preliminary classification of building points into roof and facade points of the buildings. UAV photogrammetry can be implemented easily in comparison to other 3D technologies such as LiDAR and SAR as it is a cost-effective technique.
Overall accuracies obtained for segmentation of building roofs and façades was 90.86 % and 84.83 %, respectively which potentially formal useful inputs for studying photo-voltaic (PV) potential of urban areas, BIM etc. The resulting segmentation was successful in identifying the flat surfaces which are necessary for the installation of solar panels while avoiding other miscellaneous elements. Although the study area is fairly representative of the various kinds of built-up structures, this study is planned to test the efficacy of the proposed methodology, and suitable modifications, on diverse structures.