INFERRING ROOF SEMANTICS FOR MORE ACCURATE SOLAR POTENTIAL ASSESSMENT

In guiding the energy transition efforts towards renewable energy sources, 3D city models were shown to be useful tools when assessing the annual solar energy generation potential of urban landscapes. However, the simplified roof geometry included in these 3D city models and the lack of additional semantic information about the buildings’ roof often yield less accurate solar potential evaluations than desirable. In this paper we propose three different methods to infer and store additional information into 3D city models, namely on physical obstacles present on the roof and existing solar panels. Both can be used to increase the accuracy of roof solar panel retrofit potential. These methods are developed and tested on the open datasets available in the Netherlands, specifically AHN3 lidar point-cloud and PDOK aerial photography. However, we believe they can be adapted to different environments as well, based on the available datasets and their precision locally available. * Corresponding author


INTRODUCTION
The share of renewable energy consumption in the Netherlands is currently at 7.4%. Comparatively small, both in respect to the other energy sources and compared to the International Energy Agency members median of 15.5%. With the goal to raise the share of solar in energy production to 27% by 2030, solar energy plays an important role in the transition towards renewable energy sources and future development of Dutch energy network (Dutch New Energy Research, 2021). To maintain this development, tools to evaluate solar potential are valuable resources for determining the suitability of mounting solar panels on buildings.
The solar potential depends, next to atmospheric conditions, and the geometrical layout of the building and its surroundingsnamely the aspect, material and tilt of the roof -on the size of installation area and shaded areas caused by any obstacles (Machete et al., 2018). Small-scale obstacles on the roof, which also influence the available area and shading, are often not or only insufficiently regarded in the solar potential models and analysis process (Rodríguez et al., 2017).
Our research, focusing on buildings' roofs, aims to detect any physical obstacles and existing solar panels from various types of open datasets available in the Netherlands and integrate them as additional attributes into existing 3D city models (CityJSON). The information can then be integrated into solar potential models and ultimately allow for more accurate assessment of solar panel retrofit potential.

LITERATURE REVIEW
The available roof area for the installation of PV panels is one of the crucial factors for an individual building's suitability of solar energy production. Roof obstacles, such as dormers, chimneys, and roof windows play an important role in this sense but are often not sufficiently regarded. Aarsen et al (2015) worked on creating a database of currently installed PV panels in a given research area, including already registered ones, to detected them through an algorithm. This algorithm uses aerial images and is based on the shape, the contrast, and the color of solar collectors (Aarsen et al., 2015). In the city of Amsterdam, research has been conducted to detect and model square chimneys to monitor their compliance with chimney height regulations. For this, points in a point cloud above a modelled roof structure are filtered out and subsequently analyzed with a rule-based approach (Alexandridis et al., 2020).
Similar to these projects, most of the research found focuses on a specific type of obstacle only. In that sense, Satari (2012) proposes a way to recognize dormers based on lidar data and an LoD2 model without detailed roof structures through support vector machine, a supervised machine learning model. The approach is based on the different normal direction of dormer points to the main roof points.

METHODOLOGY
Three different approaches based on different types of open datasets are explored and implemented to produce the information and to meet the objectives of the project. The Dutch datasets were used to test the methods, but the latter can be applied to different country depending on their data available. However, the precision and resolution of the data used would alter the result's quality.

Geometry-based classification
The first method uses AHN3 Dutch airborne lidar point cloud and the derived 3DBAG LoD2.2 city model of all buildings in the Netherlands as the input. The aim is to couple and compare these two to identify their differences which would correspond to some installations' relief present on the roofs.
First, the detection of obstacle points is achieved by computing the distance between the mesh and the points. The points below the 3D mesh are discarded as well as the one belonging to a vertical surface, that is, if the angle between the vertical axis and its normal is greater than 50 degrees. For the remaining points, when its distance to the mesh is greater than 40 cm, it is considered as an obstacle. This parameter is experimentally determined and gives the best results. Additionally, isolated points are discarded, using a radius search method to find the number of neighbouring points.
Next, the points are considered in 2D. Since we did not obtain satisfying results by clustering the points into obstacles, we form the obstacle shapes by offsetting the points as polygons and merging them together (Figure 1). The buffer offset size is based on the point cloud density, which is around 30cm in the case of AHN3.

Unsupervised image classification
The second method relies on an image classification model to identify roof obstacles. For this, the openly available aerial images from the Netherlands (25cm resolution) are used (PDOK Luchtfoto), along with the footprints from the Key Register Addresses and Buildings (BAG).
Since the aerial photography used is not a true ortho-photo, the building footprint polygons are aligned with the roofs in the aerial imagery using the Canny edge detection algorithm and edge matching.
Ultimately, a k-means clustering algorithm is used on the aligned image to detect two classes: the main roof surface, and the obstacles on it. Additionally we add an additional class for pixels out of building bounds ( Figure 2).

Supervised image classification
The third method explores utilizing supervised image classifiers to identify and classify existing objects on the roof, namely solar panels. Since we failed to find any open datasets with individual panels labelled, we resorted to labelling the solar panels manually, focusing on an area of Delfgauw located nearby Delft. A convolutional neural network (CNN) was trained on this data, aiming at detecting existing solar panels in aerial imagery and therefore, identifying the buildings already equipped and having less priority for panel retrofitting.
The output from the network is a per pixel map of probabilities ranging from 0 to 1 for a solar panel being located at a given pixel. Since the probability range is continuous while we prefer a fully binary result, we implement a chain of gaussian blur filters and thresholding operations to identify the blobs / areas of high solar panel probability and filter out the noise. ( Figure  3).

Combining the three detection methods
The three approaches are integrated into a common pipeline. The obstacle polygons derived from first two methods are rasterized and an obstacle probability value is computed per pixel, assessing the chance for it to be an obstacle (Figure 4, right). Our experiments show that obstacles detected by the first method are usually correct but incomplete: higher obstacles such as chimneys are detected whereas lower ones, like solar panels, are missing due to not crossing the obstacle distance parameter. The second method tends to detect more obstacle surfaces than the first one, especially due to misclassifying thrown shadows. Conclusively, we assign a 100% probability to the output from the first method if they are also represented in the second method. We give a 50% chance to the other pixels, and to the ones obtained by image classification. The others are assigned an obstacle probability of 0%.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-4/W4-2021 16th 3D GeoInfo Conference 2021, 11-14 October 2021, New York City, USA Furthermore, the 2D area of the obstacle can be computed. To retrieve the area in 3D, the point cloud dataset is used: the normal angle value of each point is projected in 2D and rasterized to obtain a value per pixel. An Inverse Distance Weighting (IDW) interpolation method is used to obtain a value for each pixel (Arroyo Ohori et al., 2020) (Figure 4, left). These angle values evaluate the tilt of the roof per pixel and consequently allows us to compute the corresponding 3D area.
Additionally, the solar panel detection model infers whether the building is already equipped with solar panels. The computed obstacle surface areas and detected solar panels are outputted and integrated back into the CityJSON model as new semantic attributes, which can be used to improve the accuracy of 3D city models-based solar potential assessment such as Solar3DCity (Biljecki at al., 2015).

Obstacle detection validation
We test the methods on a 50 buildings dataset located in Eindhoven. In order to quantify the quality of our results, we use a contingency matrix for pixel classification as explained by Janssen (2004). A comparison between a manually drawn ground truth and our classification result is obtained by building an error matrix based on the comparison of their respective pixel values ( Figure 5). This is done twice with two sets of results applied to the 50 buildings dataset: once, with a manual alignment of the footprints to the aerial images and once with the automatic alignment method we developed. The validation method is adapted to our case: the pixels which obtained a probability value of 50% in the classification process are used both as main roof area and roof obstacle area but weighted by 0.5 in the error matrix. Finally, the error matrix allows us to compute some objective accuracy parameters: on the one hand, the so-called "producer accuracy" and "user accuracy", and on the other hand an "overall accuracy" (Janssen, 2004).
The overall accuracy of the obstacle detection method when compared to the ground truth for a validation sample set was 76%. 78% if the orthophoto was manually aligned, instead of using our automated orthophoto alignment pipeline (Table 1,2). Table 1. Contingency matrix for a sample building, using the automated ortho-photo alignment method Table 2. Contingency matrix for a sample building, using manual ortho-photo alignment

Solar panel inference
The accuracy of the solar panel detection model was 70% for a validation dataset, but with very low accuracy of only 25% in positive cases (see Table 3). We believe the results could be further improved with more diverse training dataset, since the validation dataset included high number of building typologies not encountered in the training data.

Impacts on the combined results
Regarding the detection of any installation on the roof, the major aspects impacting the results were identified as the followings: first of all, the rasterization process involved when merging the methods provokes some loss of accuracy. Secondly, we noticed the impact of the correct alignment between the BAG and the aerial images datasets, which was an issue not entirely solved. This aspect might differ from an area of study to another, depending on the open data available. Furthermore, the obstacle-detection accuracy is highly impacted by the different types of roof structures and materials. It is thus highly dependent on the sample data used. We could observe a high variability of buildings' type in our datasets: housing in rows such as bigger isolated buildings, with various roof materials/colours. Finally, the validation method we chose impact the quality assessment. Indeed, we consider the results' quality by considering the pixels one by one, in a more accurate way than our methods were themselves developed.

Limitations of the respective methods
The combined results are primarily dependent on the three methods used and their respective limitations.
The limitation of the supervised image classification method relies mostly in the amount and diversity of training data that we could generate. The trained models are too context sensitive (e.g., model trained in a small neighborhood of Delft not being able to generalize to typologies from different locations), or on the other hand training sets are so broad, that even with larger number of images, the model struggles with delivering reliable results.
The obstacle-detection method based on geometry remains very dependent on the input 3D model and the way the latter was generated. In our case, we use the 3D-BAG dataset as main input, but the latter does not represent the buildings in a consistent manner regarding the high-level semantics: e.g., sometimes modelling dormers, sometimes not. The 3D-BAG algorithm is indeed dependent on numeric parameters and aims at keeping low-complexity. Also, this method is dependent on the point-cloud acquisition-pattern (parallel lines) and accuracy. Both aspects influence the obstacle point retrieved and the obstacle's shape.
The unsupervised image classification often detects the shadows of an obstacle rather than the obstacle itself. This misclassification yields bigger surface areas than reality and non-overlapping results with the previous method. Additionally, the resolution of the aerial images used impacts highly the results. In our study-case, the resolution of 25 cm is not enough to detect small obstacles and to perform a better automated alignment. Finally, labelling correctly and automatically the outputted classes is not straightforward, since the clusterization.

CONCLUSIONS
The objective of this paper is to present a method to infer semantic properties of roofs by focusing on roof obstacles such as dormers, chimneys, and solar panels. Inspired by the existing strategies to detect roof obstacles, three methods have been developed. The first one depends on a geometry-based classification in which the distance between the AHN3 point cloud and the 3D BAG LoD2 model determines the obstacle points, which are then offset and merged to form the obstacle geometry. The second one is based on an unsupervised image classification where the first step aligns the aerial image with the 3D BAG footprints, the second one uses K-means clustering to create two clusters with obstacle and non-obstacle features. Finally, the third one is based on a supervised image classification to detect solar panels on roofs, in which we first create the training data by manually labelling over a thousand individual solar panels and generating XYZ map tiles with the input data and output labels to create the training pairs, and then use a simple CNN that outputs a probability value of a solar panel located at a given pixel.
Considering the low accuracy of individual methods, the three methods are combined. The first two methods are combined first by rasterizing the polygons resulting from the first method and overlaying them with the output of the second one to use a pixel-based approach and to decide if a pixel contains an obstacle. The obstacle areas are also calculated in 3D with each point's normal indicating the roof's tilt and added to the 3D BAG LoD2 model in CityJSON format as an attribute. Moreover, another attribute is added for the information of having a solar panel or not as a Boolean value.
To validate and assess the results, we use an error matrix that compares the ground truth with our classification. The overall accuracy of obstacle detection changes between 2% and 60% while the detection of main-roof area presents an accuracy of 70% to 90%. This high difference can be caused by the different number of pixels concerned by each class: since much less pixels depict obstacles, retrieving the same ones through different methods is less probable. Moreover, limitations coming from the geometry-based and unsupervised image classifications such as the quality of the 3D BAG dataset and the resolution of aerial images affect the results. As for the supervised prediction model, we achieve 70% accuracy for the test dataset, although the results are better for other validation datasets typologically closer to the training data. Additionally, we achieved interesting results with predicting the solar radiation values at the roof surface using picture-to-picture generative adversarial networks.
The described methods are an effective way to increase the accuracy of a solar potential assessment performed on 3D city models when both dense point-cloud and aerial imagery are available. In areas without frequently updated point-cloud data, the image-based methods might still be useful if adapted by compensating for the tendency to overestimate the obstacle surface areas.
Future work should include adapting the algorithm for other datasets to see how the density of point clouds and the resolution of aerial images change the results. Moreover, instead of merging the three approaches, an opposite approach could be explored where the data sources are integrated at input instead to create a dense coloured point cloud so that only one processing method is used at the end.