TLS AND SHORT-RANGE PHOTOGRAMMETRIC DATA FUSION FOR BUILDINGS 3D MODELING

The adoption of 3D survey techniques is essential to promote efficient and timely information acquisition on constructed buildings. This article addresses terrestrial LiDAR (TLS) and close-range photogrammetric data fusion for the 3D modeling of a building in Level of Detail (LoD) 3. The selected building presents challenging elements for modeling, such as extended curved slabs, external glass walls, recessed facades and diverse roof pitches. It is located on the campus of the Federal University of Paraná (UFPR) in Curitiba, Brazil. The accuracy of the data integration was obtained through the analysis of deviations between the clouds of primary points. The accuracy of the point cloud model was verified by comparing its dimensions with the real dimensions of the building, obtained by means of a handheld laser distance meter (EDM). The results demonstrate that there was a correspondence between the EDM measures and the model, with a satisfactory statistical agreement between the estimated and reference values and a general maximum absolute error of 4.5 cm. The article focuses on the accuracy of point cloud models for the cadastral updating of buildings, providing information for decision making in projects documentation and interventions.


INTRODUCTION
Since the Industrial Revolution, in the 18th and 19th centuries, urbanization has become a large-scale phenomenon worldwide, transferring large numbers of rural populations to urban centers. This phenomenon was accentuated in the twentieth century and continues to steadily increase in the first decades of the current century. The global urban population, which is 54% in 2020, is expected to amount to 60% by 2030, reaching rates close to 70% in 2050, as indicated by UN projections (Knorr et al., 2018). In this sense, there is also an increase in megacities, which have more than ten million inhabitants, from 10 in 1990 to 37 in 2015, corresponding to almost half a billion residents, that is, 12% of the population (United Nations, 2018). Therefore, considering that cities shelter the greatest share of the world population, it is undeniable that they should receive important investments in favor of their inhabitants´ quality of life.
The urban population growth impacts the civil construction sector, which experiences a greater demand in view of the need for renovations and new constructions. This increases the need for efforts to design, register and document existing, planned and under construction buildings. In urban centers in developed countries, there is a tendency to decrease the construction of new homes, and hence, increase the number of renovation and maintenance projects in existing buildings (Kim et al., 2020). Developing countries, such as Brazil, have a high housing deficit, increased by real estate speculation, besides vacant, abandoned or unfinished properties. A direct consequence of such facts is the increase in slums and the occupation of precarious buildings, which become the temporary home for thousands of needy people (Ramírez-Orozco, 2019). In this context, it is necessary to elaborate diagnoses and proper documentation for public and private buildings that are unfinished, abandoned, or vacant, so as to improve their performance, functionality and environmental quality, aiming to meet the current sustainability standards.
In many cases, registering or revitalizing old buildings becomes difficult due to the lack of documentation (Tran, Khoshelham, 2019). This task can be accomplished by three-dimensional urban modeling, which plays an important role in the integration of economic and social data, favoring private companies and the public sector in decision-making processes. Currently, 3D models of buildings are widely used in various fields, such as post-occupation evaluation (POE) and construction pathology studies, design of smart buildings, protection of architectural, landscape and archaeological assets, urban management and in many other areas in the public and private spheres. The rapid development of the various economic and industrial sectors also demands a growing advance in technologies for the acquisition and processing of data related to the urban built environment.
In this scenario, remote sensing can offer techniques to automate the process of capturing and constructing 3D models of buildings. The use of three-dimensional point clouds, obtained through photogrammetry or laser scanning, offers a representation of the current state of buildings, being especially useful, among other purposes, for old buildings (Lu, Lee, 2017), including historical buildings (Carnevali et al., 2019).
To standardize the building modeling and documentation process, the US General Service Administration − GSA (GSA, 2020) defined four levels of details (LoDs), from a simple twodimensional projection to detailed 3D representations. In view of the current technological development, the literature shows that, to achieve greater levels of detail, it is necessary to integrate data from different acquisition platforms (Wen et al., 2019). In order to scan the facades and roofs of buildings, photogrammetric and LiDAR data obtained from different platforms can be used. Facades and interior environments point clouds can be acquired by terrestrial LiDAR platforms, either fixed or portable, called Terrestrial Laser Scanning (TLS). Aerial platforms, called Aerial Laser Scanning (ALS), are commonly used to obtain roof data (Teo, Huang, 2014). Likewise, photogrammetric data can be obtained from terrestrial or aerial positions, depending on the platform used. Thus, data from different platforms can be integrated to model from roofs to internal details of buildings, reaching the highest possible level of detail.
Despite the development of aerial platforms, such as Remotely Piloted Aircraft (RPAs), capable of obtaining facade and roof data, advancements of techniques and analyses on data integration are still necessary, as it is not yet possible to obtain comprehensive data from buildings through a single platform. In recent years, several works have coped with the different types and methods of data integration for building modeling, from ALS and TLS to LiDAR and photogrammetry fusion (Cogima et al., 2020).
Aiming to collaborate with the challenges of buildings 3D modeling by integrating data from different platforms, this article presents a quality assessment of a 3D building model constructed by means of TLS and short-range photogrammetric data fusion. The main objective is to promote point cloud modeling proving the accuracy and efficiency of 3D buildings models in level of detail LoD3, which includes external elements of buildings. The case study is a five-storey building on the campus of the Federal University of Paraná (UFPR), located in the city of Curitiba, south of Brazil. The choice regarding the sensors and data fusion methods was based on the amount of the building architectural details, quality of the collected data, efforts to acquire and process data, and field work accessibility. Considering the obtained results, an evaluation on the processed data and their acquisition in terms of scale, complexity and reach of the sensor devices is presented, serving as a basis for future alike initiatives for 3D buildings modeling.

Study Area
The studied building, located on the UFPR campus in the city of Curitiba-PR, Brazil, and shelters the Occupational Therapy School (Figure 1). The building has modern architecture with rectilinear shape, containing extended curved slabs, external glass walls, recessed facades and diverse roof pitches. The building was chosen in face of its challenging architectural style for its remote sensing-based 3D modeling.

Input Data
In this work, terrestrial LiDAR (TLS) and short-range photogrammetric data were used. The terrestrial data were obtained from the Faro LASER scanner Focus 3D 120 platform.
In turn, the photogrammetric data were obtained using a Remotely Piloted Aircraft (RPA) Phantom 3 Standard. For the acquisition of TLS data, the method of multiple sequential scans was applied, aiming at capturing greater details of the building facades. To this end, a previous analysis of the building's surroundings was carried out, in order to inspect the terrain unevenness and nearby trees. The platform was placed in 21 positions around the building, aiming to eliminate the interferences of the trees nearby. In order to ensure data registration, artificial targets were used in the field. The inclusion of these targets, which are considered as control points within the studied area, minimizes uncertainty by optimizing the orientation in the recording of scans (Fassi et al., 2011). These targets were made up of circular boards (spheres) and checkerboards, which were distributed in the field around the scanning platform in each one of the scans.
Roof photogrammetric data were obtained by automatic 50 m height flights. According to this setting, 89 photographs were obtained with frontal overlap of 80% and side overlap of 75%, with a resolution of 1.1 cm/px.

Facade Extraction
Firstly, multiple scans were recorded. Each point cloud was loaded into the Faro® Scene and the registration process started, which basically consisted of two steps. The first one refers to the alignment of the scanning positions through the automatic identification of artificial targets. In the next step, it was necessary to manually check the targets for the exclusion of those incorrectly marked. This allows the user to confirm that the artificial target extraction is correct and, if necessary, add natural targets from the scan data to assist in the registration process. After checking the artificial targets in each point cloud, the registration tool was used within the software, merging the scans into a single point cloud. The software's default registration settings were used.
After the scans registration process, the facades of the Occupational Therapy School building were delimited, excluding points related to other targets in the surroundings. This process was manually performed in the CloudCompare software using its segmentation tool.

Roof Extraction
The processing of photogrammetric data can be done using various softwares, relying on SfM algorithms that have an intuitive and automated workflow for the user. The photographs are obtained from different angles, distances and overlapping perspectives to recreate the 3D geometry of a given object. In other words, SfM algorithms are a photogrammetric approach that computationally reproduces the human capacity to understand the three-dimensional structure of a moving object through a collection of photographs (Szeliski, 2010). To achieve this goal, it comprises a series of algorithms to create clouds of three-dimensional points, which can create digital elevation models (DEM), terrain models and orthorectified mosaic images (Green et al., 2014).
In this work, the software Agisoft PhotoScan® Pro v1.4.3 was used, which has an integrated processing device developed and marketed by the company Agisoft® (Agisoft, 2020). All photographs were added to the software. Next, the features and their correspondences were extracted to promote the grouping of the photographs, a process that is called alignment. Thus, it was possible to generate sparse and dense point clouds. The building's roof was thereof rebuilt, recreating its surface in 3D.

Data Fusion
Data fusion or data integration is performed by combining data from different platforms. Initially, it was necessary to register the data sets, as they were obtained in equipments with different levels of precision, especially horizontal (Abdullah et al., 2017). The registration procedure was performed in CloudCompare. The first analysis of the data sets revealed large displacement and low overlap, which led to a preliminary semiautomatic alignment step.
After the initial alignment, the automatic registration Iterative Closest Point (ICP) was applied. The TLS point cloud was defined as a reference for its better accuracy. The standard stopping criterion, defined by the minimum root mean square error (RMSE), was used. Finally, a 10% final overlap was defined, as these are data obtained from different perspectives.

RESULTS AND DISCUSSION
First, the building's primary clouds were generated; the roof with photogrammetric data, and the facade by means of TLS. From them, the integration was carried out resulting in a single point cloud of the building, shown in Figure 2. Table 1 presents details of the point cloud model. It is possible to observe through the photographs several trees close to the building, which jeopardized the acquisition of the facades data. In addition, the unevenness of the terrain on the south side compromised the TLS scanning, as the platform was placed very close to the facade, causing occlusions due to the high inclination of the laser beam. The main areas with no data are indicated by the red arrow. The model has shortcomings in the upper areas of the walls, especially in the roof parapets.
As the data integration is restricted to specific intersection areas between the roof and the facade and involves a low number of points, the integration was analysed over a shorter range of distances. For this, distances of up to 50 cm were separated and the Gaussian function was adjusted for this interval (Figure 3),  observing an average and standard deviation of 17 cm and 13 cm, respectively. The distribution of distances within this tolerance interval occurs in the greatest overlap of point clouds (Figure 4).  Deviation analysis was used to assess the data fusion accuracy. For this purpose, the distance calculation between the integrated point clouds was performed. The CloudCompare "Cloud/Cloud Distance" tool calculates the distance from the nearest neighbor between the reference cloud or mesh and the compared cloud. For each point in the compared cloud, its nearest neighbor in the reference is searched and its Euclidean distance is calculated (Ahmad Fuad et al., 2018).
The accuracy of the point cloud model was assessed by comparing the field measurements, performed by means of a handheld laser distance meter, also called electronic distance measurer (EDM), with those presented by the point cloud model. Measurements were made of several elements of the building using a Leica DISTO ™ Classic. The accuracy of this equipment was verified through calibrations performed at the Mechanical Metrology Laboratory belonging to the Laboratory for Satellites´ Assembling, Integration and Test (LIT) of the Brazilian National Institute for Space Research (INPE).
A previous analysis was carried out in the point cloud to select areas that could be measured in CloudCompare, selecting representative elements, such as bathrooms and classrooms windows, glass walls and walls of various sizes. Thus, 79 measurements of building elements were carried out, with dimensions ranging from 0.695 m to 23.82 m. Due to a wide variation in dimensions and for better analysis and understanding, the measures were divided into classes, described in Table 2.
The dimensions obtained by the handheld laser distance meter were compared with the corresponding dimensions extracted from the point cloud. Firstly, the absolute error was calculated, which corresponds to the Leica DISTO ™ Classic measure difference module and the corresponding one obtained in the models, leading to the relative percentage errors, which are considered as model accuracy. To analyze the variation in dimensions, the Square Root of the Mean Square Error (RMSE) was calculated, which can indicate the magnitude of the error indicating the accuracy of data adjustment. The analysis of agreement between the laser distance meter and model dimensions was verified by means of simple linear correlation, which measures the degree of relationship between the variables, describing it using a regression equation. The degree of linear relationship between the variables is measured by Pearson's coefficient of determination, known as R².
The accuracy presented by the point cloud model is verified by the mean and percentage absolute error ( Figure 5). It can be observed that the average absolute error increases with the increase in the dimensions of the distance classes, varying between 1 cm and 17.6 cm. The general mean absolute error was 4.5 cm. The absolute percentage error was less than 1% in all of the classes and in general. The sizing accuracy of the model, indicated by the RMSE, was 8.8 cm.   With the aid of resources employed in the field, it was possible to obtain a very precise 3D building model. TLS positions around the building associated with artificial targets minimized interference from nearby trees and buildings. Due to the terrain unevenness, the TLS was placed just less than 4 m away from the wall on the south side, producing a lack of data in the last floors of the building due to an obstruction of the laser beam, as shown in Figure 7. Through the integration of point clouds, it was possible to generate a realistic model capable of representing the building's external geometry, where facade and roof data were complemented by means of different data collection platforms. Data registration was performed using the ICP method, widely used and available in CloudCompare. The integration accuracy showed a centimeter deviation, in accordance with the literature, as highlighted by Cheng et al. (2018), who reported that for small-scale digital archiving of architectural heritage, the accuracy of the registration must be within the range of centimeters and, whenever possible, millimeters.
The limited overlap between the facade and roof data made the registration process difficult in view of their different acquisition perspectives. In this case, it is worth stressing the proposals for optimizing registration methods that use characteristics of objects based on filtering (Pomerleau, 2015) or on the outline of buildings (Cheng et al., 2018). Also, recent studies targeted to develop automatic ALS and TLS registration methods stand out, as proposed by Liang et al. (2020), who conceived an approach to optimize the alignment through the horizon context, identifying potential overlapping areas of these point clouds.
The quality presented by the herein presented point cloud model corroborates the high accuracy attained in LiDAR data-based buildings reconstruction works, as described by Kedzierski and Fryskowska (2015), Cheng et al. (2018), Wen et al. (2019). Considering only distances classes 1, 2 and 3, the accuracy of the model is compatible with the results of 0.5 to 3 cm achieved by Sepasgozar et al. (2014) and Kim et al. (2020).

CONCLUSION
This work presented the results of a TLS and short-range photogrammetric data fusion for the 3D modeling of a fivestorey building. The data processing resulted in the generation of a 3D volumetric model in the form of point clouds. The conditions of the terrestrial laser scanning as well as the building surrounding trees imposed additional challenges to the data collection, resulting in meaningful shortage of data in the model. On the other hand, this absence of data was not observed in the roof points cloud, due to their different perspectives. Data integration proved to be very dependent on the data acquisition process. Considering this, it is important to use advanced capturing methods to overcome the problems of lack of overlap and differences in acquisition perspectives. The results demonstrate that there is a match between the building real dimensions and those obtained from the point clouds model. There is a satisfactory statistical agreement between such dimensions, and a low absolute error was identified, what is compatible with the literature.