AUTOMATIC SEGMENTATION OF POINT CLOUDS IN THE ARCHITECTURE ENVIRONMENT

Correct and reliable identification and classification of different structures and infrastructures that make up a city (e.g. residential buildings, school buildings, hospitals, power stations, routes of communication, etc.) are of great importance for the AEC/FM (Architecture, Engineering, Construction, and Facilities Management) domain and for seismic risk assessments, among others. For decades, the method of collecting buildings information has been through field campaigns. This practice requires significant resources in terms of qualified engineers or architects to identify the geometry of the different elements that constitute the structure, building materials and construction processes. Nowadays, there are different geospatial techniques that allow data acquisition on a massive scale in a short period of time. In particular, by means of laser measurements, it is possible to have clouds of millions of points with geometric and radiometric information in a matter of seconds. In this article, we present ABM-indoor, a LIDAR-based approach that automatically provides a three-dimensional models of buildings in vector format. Models include floors, ceilings, walls (up to five dominant directions), columns, elements located on floors and elements hanging from ceilings. Efforts are underway to transfer this model to a Building Information Model (BIM).


INTRODUCTION
Over the last decade, the demand for digital twins has increased considerably in different research fields, such as AEC/FM, earthquake engineering, cadastre and urban planning, among others. In order to carry out a successful assessment of the seismic performance of a building, it is necessary to have reliable digital twin models. Creating a model that contains 100% of the elements and their characteristics is not an easy task. Furthermore, it requires extensive information about the geometry, structural characteristics and materials of the building.
The digital representation of buildings started with drawings, through three-dimensional models to Building Information Models (BIM). The information from a building might be readily available for parts of the building stock through cadastral data. However, the cadastral data is often not complete and falls short of the minimum required information to create reliable structural models. This is especially true for older buildings in the developed parts of the world. On the other hand, the cadastral data in developing countries often fail to provide any reliable information about a vast majority of the structures rendering reliable seismic risk assessment virtually impossible. In order to overcome these shortcomings, the experts rely on extensive field campaigns, that require significant resources in terms of trained personnel, hardware and software to manage all collected data, etc. Furthermore, in most cases, non-structural and content elements, which are a major source of economic losses in case of strong earthquakes, particularly in the low seismic hazard regions (Torres et al., 2019), are omitted. This leads to uncertainties in the results obtained.
Nowadays there are several geospatial techniques that make it possible to obtain massive and accurate building information in a matter of seconds. However, the processing of this information is a major challenge. In order to provide an automated and consistent approach to information processing and to eliminate the shortcomings of extensive field campaigns, we have developed a method that complements existing approaches for obtaining digital indoor and outdoor models of buildings, and Building Information Models (BIMs).
As authors has shown in Romero-Jarén & Arranz (2021), ABMindoor, is an approach that focuses on the automatic segmentation, classification and modelling of point clouds. The approach is an algorithm that automatically identifies architectural elements and creates 3D models of the indoor building elements: floors, ceilings, walls columns, and content. ABM-indoor is a module of the software MDTopX (2021), which is used by many users related with the processing of point clouds and digital models.
Up until now, ABM-indoor generates accurate 3D surfaces of building elements. Planar elements are created as a vector format and non-planar elements as TIN format. Nevertheless, we are currently working on transferring the 3D surfaces into Building Information Model (BIM) elements.
The structure of this manuscript is as follows. Section 2 includes a review of the state of the art related with BIM applications and point cloud processing. Section 3 presents the proposed approach and the experimental results obtained. Section 4 includes the development of manual creation of structural models. Finally, Section 5 shares our conclusions and objectives for further research.

BIM applications
The term "Building Description Systems" has been used in the field of civil engineering and architecture since 1976 (Eastman). In 1986, Rufle first used the word "Building Model" to refer to the digital representation of a building. Nowadays, BIM models are one of the products generated within the BIM methodology. These models are three-dimensional representations of a building with all its elements and at different levels of detail (AIA, 2013). The use of BIM methodology and models is widespread in a variety of research fields.
Our work focuses on the use of BIM models in the field of seismic engineering. Previous studies used this models to performed seismic risk analysis in school buildings (Biagini et al, 2019) where the model is the results of a manual design in Revit from 2D drawings and point cloud data. Authors in (Perrone et al., 2017) used BIM models to design seismic sway braces for pressurized fire suppressant sprinkler piping systems. Vitiello et al. (2019) use a BIM model to evaluate the economic performance and losses of a building expose to seismic risk.
A different but commonly application of BIM models is in the field of quality control and monitoring during construction. In Pučko et al. (2018), authors present semiautomatic algorithms to track the status of a steel structure under construction. The reconstruction of historical heritage can be also developed through use of BIM. Authors in (Angelini, 2017) manually construct a BIM model from multisource data: topographic total stations, Terrestrial Laser Scanning (TLS) and photogrammetry.

Point cloud processing
BIM models are a very powerful tool, as they are reliable representations of reality. In addition, it is possible to include a large number of elements at the level of detail desired by the user. However, the creation of the models is not simple. Different sources of information are used to create the models: 2D drawings, information obtained from total stations or massive data from point clouds or virtual reality devices, among others. Point clouds provide accurate geometric and radiometric information in a given space. The main disadvantage of these massive data is in terms of storing, managing and processing the information. With millions of points, we are referring to dozens of GB of information and many hours of work to manually identify real elements associated with the structures. Nowadays, there is no single tool that allows the automatic creation of these models from the aforementioned information. Tang et al. (2010) and Pətrəucean et al. (2015) present a review of the state of the art of different techniques for automatically processing point clouds.
Scientists are actively working on the development of methods and approaches for automatic point clouds processing using renowned algorithms such as the Random Sample Consensus (RANSAC) algorithm. It was developed as a robust detection of outliers and noise that involves random sampling with the implementation of constraints (Fischler & Bolles, 1981). Authors in Jung et al. (2014), present a semiautomatic approach to model indoor point clouds. They used RANSAC algorithm to automatically segments the point cloud for plane extraction. Another example of previous publications that use RANSAC algorithm is presented in Ochmann et al (2016). The approach creates 3D indoor models of building from static point clouds.
The models include floors, ceilings, walls, and wall openings, such as doors and windows.
Authors in Wang et al. (2017) use different algorithms and techniques to process the point cloud: graph-cut and hierarchical clustering. The approach models indoor unorganised point clouds The approach identifies planar elements (walls, floors, and ceilings).
Other important and widely used algorithms among researchers are: Hough Transform, Principal Component Analysis (PCA), Fast Point Feature Histograms (FPFH), Region Growing and Connected Components and Supervoxelization (Che et al., 2019).
For the development of our study we also distinguish between the different types of data sources. Authors in Zolanvari et al. (2018) segment facades of complex urban structures (Rubrics building in Dublin, Ireland) using a combination of airborne and terrestrial data. Another example of previous publication which use a combination of data is presented in Chen et al. (2018). Authors use topographic maps to obtain the footprint of the buildings and the elevation values, 2) airborne LiDAR, and 3) architectural conventions to impose some restrictions, to create City Information Models (CIMs).
One of the novelties of our approach compared to previous publications is that it works both with point clouds from static and dynamic sensors. Additionally, we have worked on modelling envelope elements (e.g. floors, ceilings and walls) and content elements, which can be structural elements (columns) or non-structural elements (furniture).

ABM-INDOOR METHODOLOGY
The proposed approach has two main objectives: 1) automatic classification of the elements of unorganized point clouds (floors, ceilings, walls, columns, etc.) and 2) three-dimensional modelling of the classified elements according to a LOD 300 model (AIA, 2013).
Our approach focuses on the geometric information of point clouds. We used two data sets to test ABM-indoor. One of the point clouds was acquired in a car park with a dynamic TLS, NavVis M6 (2020), an indoor Mobile Mapping System (MMS) with six cameras that obtain 360° imageries, four laser scanners at various heights and 6D Simultaneous Localisation And Mapping (SLAM) system (Surmann et al., 2004). In this case, the point cloud covers approximately 1.000 m 2 and has14 million points (Figure 1.a). The second point cloud represents an office space, covers approximately 800 m 2 and includes 3 million points (Figure 1.b). It was obtained from a static TLS. In this case, the data acquisition performs multiple scans where the position of the sensor is known so the point cloud is organized.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) Figure 1. Samples point cloud of the car park (a) and the office space (b). Figure 2 shows that ABM-indoor workflow has two main steps: 1) segmenting and labelling the point cloud using automatic clustering and 2) creating 3D surfaces of each classified element.

Figure 2. AMB-indoor workflow
Floors, ceilings and walls are considered planar elements. On one hand, we use a slicing method to identify the plane that best fits a cluster of points. Firstly, the algorithm identifies floors and ceilings, as horizontal elements by using the height value. Secondly, ABM-indoor creates 3D surfaces of floors and ceilings using Delaunay Triangulation (Isenburg et al., 2006). Finally, the algorithm applies minimum area constraints to eliminate irregular and useless surfaces and performs automatic edge fine-tuning to smooth surfaces boundaries. Before classifying the rest of the point cloud, the algorithm defines the principle direction of the building using edges longer than four meters.
On the other hand, ABM-indoor uses the principle direction of the building and the cluster direction value to identify walls. To create surfaces from wall clusters, the algorithm first projects the boundaries horizontally using Helmert 3D transformation and, finally applies polygon filtering and edge fine-tuning.
The algorithm also identifies non-planar elements, such as columns. Columns are vertical elements connected to the floor slab and to the ceiling of the building. ABM-indoor uses existing gaps in the floor and ceiling to identify points that are on the vertical between the two gaps ( Figure 3). The algorithm processes the remaining points to identify nonplanar elements once the floors, ceilings, walls and columns have been classified and modelled. The approach identifies content elements, such as cars, motorbikes, tables, chairs among other. To identify and model the aforementioned irregular elements, ABM-indoor uses a tetrahedrization algorithm ( Figure  4). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition)

Experimental results
Two indoor data sets were used to test the algorithm. In one hand, Figure 5.a. shows the created surfaces for the car park, where two different wall directions where identified: Wall 1 (orange) and Wall 2 (green). On the other hand, Figure 5. b shows the created surfaces for the office space. Four different walls directions were identified: Wall 1 (orange), Wall 2 (green), Wall 3 (pink), Wall 4 (blue).
After processing the point cloud, the approach developed a precision analysis using confusion matrix. From the matrix results, we calculated the false positive and true positive rates and plotted them in a Receiver Operating Characteristics (ROC) graph (Fawcett, 2006). Figure 6 shows the ROC graph for the car park. For this point cloud, the best classified object classes are floors and ceilings. Figure 7 shows the ROC graph for the office space point cloud classification. In this case, the best classified object classes are floors, ceilings and walls. The accuracy analysis of the car park point cloud, shows that all the classified elements, with the exception of "Other objects" and "Objects on ceiling", were identified with a precision over than 80% (Table 1). In contrast, the office space accuracy analysis shows that "Other objects", "Objects on floor", "Objects on ceiling" and "Columns" were identified with a precision below than 80% (Table 2). Nevertheless, both point clouds, where classified with a global precision of more than 90% (correctly classified points vs. total points) Non-planar elements are located on floors or hanging on ceilings. As mentioned above, ABM-indoor uses a tetrahedrization algorithm to classify the points belonging to these elements in order to create 3D surfaces. For instance, Figure 8 shows the 3D model of a car located in one of the point clouds. Content elements are identified and modelled as individual elements.    a) The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition)

MODELS COMPARAHASION AND MODAL ANALYSIS
We intend to compare the results obtained in a seismic assessment of a building using both a theoretical model, named "Traditional Model" (TM) and the As-Built Model (ABM), generated with AMB-indoor. We used the term "Traditional Model" to refer to the structural model created from drawings and field campaigns. This is a commonly used method to define structural analysis models, which does not consider information from any geospatial technology. For the current study, we have selected an educational building built in Madrid (Spain), "Escuela Técnica Superior de Ingenieros en Topografía, Geodesia y Cartografía" of the "Universidad Politécnica de Madrid". It is a steel-braced frame building from 1977. It has five floors and a total constructed area of 9.645 m 2 . The building has superficial concrete slab foundations under columns, joint extensions and braces in two of the facades.
On the one hand, we have created the TM using the software SAP2000 ( Figure 9). To construct the TM, we have used both the 1977 available drawings and the information gathered from inspections that we have carried out. Columns and beams are HE180B and IPE450 sections respectively and the building has one-way floor slabs of concrete (20 cm thickness) and brick partitioning walls (15 cm thickness). We have considered plastic hinges located in columns (P-M3), beams (M3) and steel braces (axial). On the other hand, we have an outdoor point cloud of the educational building ( Figure 10). This point cloud was acquired with the static TLS HDS7000 (Leica Geosystems, 2011) and the Airborne Laser Scanner ALS60 (Leica Geosystems, 2008), both from Leica Geosystems. The point cloud has seven million points with a resolution of 5 cm.

Modal analysis results
We have carried out a modal analysis of the educational building on the TM and we have computed the fundamental periods of the structure along with the mode shapes. We have included the results obtained for the four modes of vibration with the highest mass contribution in the two fundamental directions, X and Y. Table 3 shows the results obtained for the X direction. For this direction, the highest mass contribution, 24%, is associated to the fifth vibration mode (period of 0.88 seconds). The results associated with the Y direction are included in Table 4. For this direction, mode 11 has the highest mass contribution, 24%, with a period of 0.60 seconds.

X direction Mode
Period (

CONCLUSIONS
We present ABM-indoor, an approach to automatically classify and model unorganized and organized point clouds. The algorithm has an iterative workflow, as shown in Figure 2, which starts classifying and modelling floors and ceilings, continues with walls, columns and content elements. We only considered the geometric information of the point cloud. To create 3D surfaces of the elements, we used triangulation algorithms. We tested the algorithm with two indoor data sets and obtained as a main result that the classification results over 90% of global accuracy (number of correctly classified points to number of total points ratio).
AMB-indoor provides files in vector format for planar elements and in TIN format for non-planar elements. The algorithm will be used to create ABMs of buildings in order to perform the seismic assessment of singular buildings. We are currently working to improve the algorithm. Up until now, ABM-indoor is limited to modelling visible elements and we need to complement the ABM with data from the TM to consider hidden elements, such as beams. For this reason, we started working on a singular building.
As a preview of the study, we have carried out a modal analysis of the TM. After the modal analysis, we would expect to obtain vibration modes that reach 90% mass contribution. However, in our work, the maximum mass contribution obtained is 24%. Due to the existence of an expansion joint, it is possible to say that the building is divided into four sections. Each of these sections behaves independently for each mode of vibration. Therefore, the mass moving for each section represents a certain percentage of the total mass of the building and is higher if we only focus on the structure of that section.
We have included only four modes of the building for each façade direction. These modes correspond to the first vibration mode of each of the four individual building sections. However, we have not included the first four modes of each section because this would be a total of 16 modes and is beyond the length of this manuscript.
At present, we are working to obtain the indoor point cloud of the educational building with the aim of merging the indoor and outdoor point clouds to generate the ABM of the building. Finally, we would like to compare the results obtained for the modal analysis of the TM and the ABM.