TOPOLOGY-AWARE 3D MODELLING OF INDOOR SPACES FROM POINT CLOUDS

In this paper we introduce a topology-aware data-driven approach for 3D reconstruction of indoor spaces, which is an active research topic with several practical applications. After separating floor and ceiling, segmentation is followed by computing the α-shapes of the segment. The adjacency graph of all α-shapes is used to find the intersecting planes. By employing a B-rep approach, an initial 3D model is computed. Afterwards, adjacency graph of the intersected planes which constitute the initial model is analyzed in order to refine the 3D model. This leads to a water-tight and topologically correct 3D model. The performance of our proposed approach is qualitatively and quantitatively evaluated on an ISPRS benchmark data set. On this dataset, we achieved 77% completeness, 53% correctness and 1.7-5 cm accuracy with comparison of the final 3D model to the ground truth.


INTRODUCTION
Wide range of applications of 3D building models such as Building Information Modelling (BIM), facility management, Indoor Location Based Services (ILBSs), virtual and augmented reality, and emergency response motivates several researchers in photogrammetry, computer vision, computer graphics and other relevant disciplines to increase the level of automation of the reconstruction process (Mura et al., 2016;Ochmann et al., 2015;Pintore et al., 2020;Sahebdivani et al., 2020;Sanchez & Zakhor, 2012). Meanwhile, with the increasing availability of various types of the sensors for generating point clouds such as terrestrial laser scanners (Jung et al., 2018) as well as mobile laser scanners (Karam et al., 2018;Maboudi et al., 2018), and HoloLens (Hübner et al., 2020) point clouds are ubiquitous data source for 3D modelling of indoor (and outdoor) spaces. Low-cost and mobile system offer huge datasets, but most users prefer to work with vector representation which is much lighter and more manageable. High demand for 3D models and availability of data capturing devices with various prices and quality ranges, motivates ongoing research on automated 3D reconstruction approaches. since main structural elements of the indoor space of the buildings could` be generated directly or from extrusion of a 2D layout of the building, both 3D and 2D approaches are investigated by researchers for indoor 3D modelling (Bassier et al., 2020;Mura et al., 2016).
Several problems arise in real world 3D building reconstruction. One of the challenges is that the indoor spaces contain clutters (mostly due to furniture) that should be separated from permanent components of the buildings (Mura et al., 2016). Occlusion and incompleteness of the data are another important challenge which diminishes the completeness of the raw dataset and leads to missing elements in the reconstructed mode, misinterpretation of the structures and also can decrease the accuracy of the model. Complex layout of indoor spaces (Jung et al., 2018), and adding semantic information like doors and windows to the 3D model are the other challenges which attracted the attentions.
Floor, ceiling, and walls are main structural elements of the buildings. Moreover, in most of the buildings, floor and ceiling are parallel and horizontal. They also serve as lower and upper bounds of height of the points in each floor. Separating floor and ceiling is one of the first steps in many 3D reconstruction approaches which are mostly based on height histogram (Jung et al., 2018; or normal vector analysis (Sanchez & Zakhor, 2012). There are some other approaches like directly applying RANSAC for separating planes with most inliers (floor and ceiling) which are less efficient. (Wu et al., 2020) used RANSAC followed by a grid-based outlier removal algorithm to separate the floor and ceiling and eliminating the points on the ceiling height (for example in the staircases) which do not belong to the ceiling part. While most of the proposed approaches assume a flat ceiling, some research like (Mura et al., 2016) also allow having slanted ceiling.
Walls of the buildings are most important parts of the 3D models. After separating the floor and ceiling, different methods could be used to detect and model the walls. Here again the trivial assumption for the walls of the most buildings is to be vertical. Moreover, in a typical building, it is generally expected that wall structures cover a vertical extent that connect floor to the ceiling, while doors, most windows and almost all clutters do not. While most of the existing methods assume Manhattan-world layout of the walls, curved walls are also considered in some investigations (Turner, 2015;Yang et al., 2019) In more recent works an attention shift towards semantics to 3D model is also notable (Mura et al., 2016;Ochmann et al., 2019). Adding windows and doors to the 3D geometric model which already consists of floor, ceiling and walls will result in LOD3 (Jung et al., 2018;Ochmann et al., 2015). Nevertheless, modelling the wall openings like door and window is still challenging for many studies (Jung et al., 2018). Various approaches are introduced in (L. Díaz-Vilariño et al., 2017), (Assi et al., 2019) and (Jung et al., 2018) to detect door openings.  employed a density-based filtering approach which exploits perceptual organization for window detection from outdoor photogrammetric point cloud.
Separating the rooms is another step which is demanded by many applications. Recently, the trajectory of the mobile scanners could also be delivered (Karam et al., 2018;Maboudi et al., 2017) which provides valuable information for 3D reconstruction and separating the rooms. Some researchers use scanner locations or sensor trajectory for initial room segmentation (Jung et al., 2018;Ochmann et al., 2015). Nonetheless, this information is not always available and it decrease the generalization of the approaches which rely on sensor position and trajectory. This paper is organized as follows: after this introduction, literature review of the recent state-of-the-art on indoor 3D building reconstruction approaches is presented in section 2. Section 3 introduces our topology-aware 3D reconstruction method. Experimental results and evaluation of the reconstructed models obtained from our approach are reported in section 4.

LITERATURE REVIEW
3D reconstruction of indoor and outdoor spaces of the buildings has been a topic of high interest in the last two decades (Ikehata & Yan, 2015;Ochmann et al., 2019;Sanchez & Zakhor, 2012). Although indoor and outdoor building reconstruction are closely related (Mura et al., 2016) and the ultimate aim of a complete reconstruction process is the seamless reconstruction of inner and outer shell of the buildings, some challenges are different.
Modelling and reconstruction approaches could be clustered into two general categories, namely model-riven and data-driven methods. Model driven approaches are limited by the assumptions in the utilized model. For example, grammar-based approaches only work reasonable for grid-like architectural designs. In the other hand, data driven approaches are more flexible but more sensitive to noise and completeness of the dataset. Some hybrid methods like (Wu et al., 2020) also try to have the best of both worlds. However, the proposed hybrid approaches are still far from being a reliable solution for versatile data and applications.
Projecting the point cloud on 2D and point cloud slicing are two common approaches which are used by many researchers (Jung et al., 2018;Xie et al., 2019) for generating a 2D layout of the interior spaces. Usually these 2D maps are extruded to the height of the ceiling to generate a 3D model (more exactly 2.5D model) of the buildings. Some researchers used one or several slices at different heights between the height of the top of the doors and ceiling to minimize the effect of common clutters and occlusions of indoor spaces (Previtali et al., 2018;Yang et al., 2019). (Jung et al., 2018) proposed an approach which is based on 2D processing of the wall candidates and assumes a Manhattan structure for the building. After separating walls from ceiling and floor, 2D boundaries of each room and corridors are extracted using some morphological operations and skeleton extraction. The opening of each room is also closed and regularized using Douglas-Peucker algorithm. Then, parallel lines are detected and used to form the volumetric cuboid structure of the walls. Finally, doors and windows are added to the model. A volumetric, parametric 3D reconstruction approach is introduced in (Ochmann et al., 2015) which assumes that each room is scanned from one position (or a few positions) and utilizes this prior information for separating the rooms. In this approach 3D buildings model are considered as a planar graph in which edges are walls and nodes are the intersection of the walls. Each edge is attributed with its thickness. Furthermore, faces of this planar graph represents the layout of the rooms. Considering the walls as piece-wise linear entities, vertical wall candidates are projected in 2D and extended to build a set of possible wall surfaces. Then, an energy minimization strategy is employed to labels the wall candidates and edges separating the regions with different labels are extruded to build the 3D model which is enriched by doors and windows.
A shape grammar based approach is introduced in (Tran, . Based on the assumption that most indoor environments belong to a regular Manhattan world, a shape grammar for the generation of navigable spaces and building elements is proposed which can manage the incompleteness of the data. One important step for completing the topology (connectivity) is detecting the doors which is done manually. (Mura et al., 2016) employ a set of point clouds with known view-points (with at least one location per room) and use visibility clustering method to estimate the room shapes. Detection of structural elements (floor, ceiling, walls) and separating them from furniture is performed by analyzing the adjacency graph of co-planar points. This approach needs an interactive refinement step to correct the wrong results which are mainly originated from missing data. An indoor 3D modelling framework is presented in (Ikehata & Yan, 2015) which reconstructs indoor scenes using a structured model from panorama RGBD images. The data is captured with a camera and a depth sensor mounted on a motorized sensor. The geometry of structural elements is presented via the scene graph. For manipulating the scene graph a new grammar is defined. The grammar transformation rules are sequentially applied to recover the indoor geometry. Geometric representations and reconstruction algorithms are chosen based on various applications like scene visualization, automated floor plan generation, tunable reconstruction, and scan to BIM.

METHODOLOGY
In this paper, we propose a data-driven approach to reconstruct a 3D model of indoor space of the buildings using point clouds. The proposed approach works directly on point clouds and no assumption about the sensor is considered. Moreover, the position of the sensor or trajectory of the mobile laser scanner are not used in any step. Furthermore, our approach does not rely on Manhattan-world assumptions. The proposed approach is appropriate for typical buildings that ceilings and floors are planar intersecting the vertical walls. The main steps of the proposed approach are illustrated in Figure 1. We generated a simple and clutter-free synthetic dataset for calibration of the proposed approach and will use it in this section for visualizing some steps of the proposed approach. This dataset consists of 3 rooms (Figure 2a) with overall dimension of 6m in width, 15.4m in length and 3 m in height.
For the sake of better visualization, we do not show the ceiling in ( Figure 2).
Firstly, normal vectors of the points are computed based on a local surface fitting approach. For normal estimation we use a quite large radius to have less noisy normal on the flat surfaces at the cost of noisy normal vectors close to the borders of the surfaces. However, after initial plane fitting, we will analyze these points again to assign the most of them to the correct segments and will update the planes. Next, by considering the conceptual knowledge about the building that floor and ceiling contain a lot of points, we filter the points which correspond to the picks of Z histogram of the point clouds as ceiling and floor points. Due to high redundancy of point clouds, the correctness of this assignment is not critical, while wrongly assigned points will be later ignored using robust estimation of the floor and ceiling planes using RANSAC. After separating floor and ceiling, a clustering is employed based on the absolute values of the normal vectors in order to assign each point to a segment (Figure 2b).
In the initial modelling phase, a generalized RANSAC estimator called MLESAC algorithm (Torr & Zisserman, 2000) is employed to eliminate the effect of possible noise in the points assigned to each planar segment. Moreover, in order to avoid missing any relevant plane, outliers of the MLESAC are checked again, iteratively. If enough number of the outliers constitute a plane which span the whole story height, another plane will be assigned to them.
After segmentation, points on planar surfaces are mostly separated from non-planar objects and clutters. The next step is constructing an adjacency graph of the segments, to find the neighboring segments. In order to decrease the computational cost of the adjacency graph construction, we do not utilize the point clouds directly. Instead, using -shape method from computational geometry just the points which are close to the boundaries of segments are considered. -shape is a generalization of the convex hull and a sub-graph of the Delaunay triangulation which is applied in various fields. It is a set of piecewise linear simple curves which is used for shape reconstruction from a dense unorganized set of data points. This method depends on the parameter α, which defines a generalized disk of radius √ . This disk is used to carve out the object, which is formed from points to end up with the boundary points (Asaeedi et al., 2017;Edelsbrunner, 2010). The -shape of the synthetic dataset is depicted in (Figure 2c). After constructing the adjacency graph of the segments, all triple neighboring planes are intersected in order to generate the corners. Topological connections of the corners, edges, and faces are considered to shape the 3D model. Analytical intersection of the planes which are robustly estimated using MLESAC provides quite accurate corner points.
Various methods are employed in the literature for representation of 3D models, each offering specific advantages and suffering from some disadvantages. The most commonly used approaches include Boolean models, a.k.a Constructive Solid Geometry ( In order to find some of the topological problems of the initial 3D model, we use adjacency graph of the reconstructed planes ( Figure 3a). Each node of the graph correspond to a plane in the 3D model and each edge shows the connection between the planes that means those nodes are neighbors and we used their intersection to compute the nodes of the B-rep model. This graph provides valuable information which can be used to check the consistency of the reconstructed planes and also finding the possible problems in the 3D model. If the 3D reconstruction is perfect, we should have some closed parts in the graph which represent the rooms (Figure 3b) and corridors.
If the doors are open, all nodes should be connected to the floor node and removing floor plane node ideally results in some closed loops (Figure 3b). If by any reason (e.g. clutter, occlusion, etc.) one of the planes is missing, it could be inferred by analyzing the adjacency graph, in most cases. We illustrated this situation in (Figure 3c) where from the graph we can understand that nodes A and B (which are connected just to 2 neighboring nodes) have a problem and need more analysis to find the missing plane. Also closed loop which correspond to small spaces could be used to generalize the model to the specific level of details. It should be accentuated that this adjacency graph in the current form does not provide geometric information. Our synthetic dataset was clutter-free and all planes are detected ( Figure 3). However, we use the adjacency graph of the reconstructed structural elements of the building in our experiment on real dataset in the next section to refine the initial 3D model which is affected by clutters.

EXPERIMENTS
In order to evaluate the effectiveness of the proposed approach, we use one of the datasets from ISPRS benchmark on indoor modelling (Khoshelham et al., 2020) which is explained in the following sections.

ISPRS UoM (University of Melbourne) dataset
The proposed approach is applied on UoM dataset from ISPRS benchmark dataset (Khoshelham et al., 2020) which contains the point cloud of a building of the University of Melbourne, Australia (Figure 4).
This dataset with partial deviation from Manhattan-World configuration comprises six rooms and one corridor. The ceiling has different heights in three parts (see Figure 5). This point cloud is captured using Zeb-1 handheld laser scanner. The dataset also contains stairs. There are 13.9 × 10 6 points in this point cloud with the average points spacing of 0.6 cm. and there are clutters mostly due to the presence of several pieces of furniture.
2D View 3D view

Point cloud
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2021 XXIV ISPRS Congress (2021 edition) As described in section 1, the first step is to separate the ceiling and floor points from the rest of the point cloud. As it is illustrated in Figure 5, this dataset has multi-level ceiling and in order to reconstruct a correct 3D model, these parts should be considered separately. The result of the segmentation of the point cloud is depicted in Figure 6c. After finding the neighbouring segment, MLESAC is utilized for robust plane-fitting. The intersection of each pair of these planes and the plane of the floor and ceilings provide an initial 3D model which is illustrated in Figure 6b.
After constructing the initial 3D model, we generate the adjacency graph of the intersecting planes. In Figure 6c, the 2D view of the initial model is represented. Each plane is attributed with an ID which is used in generating and analyzing the adjacency graph of the intersecting planes ( Figure 6d). As it could be seen in Figure 6, there are some imperfection in the model which we try to resolve them using the graph. As described earlier , each node of the graph correspond to a plane in the 3D model and each edge shows the connection between the planes that means those nodes are neighbors and we used their intersection to compute the nodes of the B-rep model.
Generally, each plane should be connected to four other planes (i.e. ceiling, floor and two other neighbouring planes) and each closed space (rooms and corridors) should form a closed loop. It can be seen in Figure 6 that there are some clusters which correspond to the closed spaces (rooms and one corridor) in the dataset. Moreover, three clusters are closed loops and three clusters are open. These open parts of the graphs provide valuable information about the missing planes in the 3D model. We use the follwing rules on the adjacency graph to refine the reconstructed model:

Floor
Multi-level ceiling Colors in Figure 5a The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2021 XXIV ISPRS Congress (2021 edition) 3-Each closed loop which the area of its corresponding planes is smaller than a threshold (2m×2m), should be removed. 4-If there is a non-closed part in the graph, the free node (which is just connected to one neighbouring node) should be analyzed for possible connection to the other nodes. For this connection, one remaining segment (a segment which its plane is not available in the reconstructed model), should support the connection. It should be mentioned that we add an attribute to the interpreted surfaces which indicates whether the surface is estimated from data (visible) or interpreted from graph analysis (interpreted). This information might be later used for further verifications. Apllying the above mentioned rules to the adjacency graph of the planes, provides a water-tight and topologically correct reconstruction which is illustrated in Figure 7a Figure 7a, the proposed graph analysis approach makes it possible that indoor space is decomposed to room cells. Each closed loop in the graph resembles a room (or other closed spaces like corridor) and could be visualized in different colors. The topology of reconstructed model is very useful in many applications like indoor location-based services and emergency response.

Evaluation
Comparing our final 3D model (Figure 7) and the ground truth model (Figure 4), the qualitative assessment of the results reveals that most parts of interior spaces of the building are reconstructed and included in the final 3D model. Partial deviation of the dataset from Manhattan-world layout is also correctly detected and the corresponding planes are reconstructed. One small room is missing because of high occlusion on one of the walls which leads to missing the connection with neighboring walls. Also, a wall in lower left part of Figure 7b is not reconstructed with the right angle. From topological point of view, the proposed approach for analyzing the adjacency graph of the planes for consistency check and model refinement, provides a correct topology of the reconstructed rooms which could be used in many applications. In order to better visualize the quality of the generated mode, the overlay of the point cloud on reconstructed model is shown in Figure 8. For quantitative evaluation of the geometric quality of reconstructed water-tight 3D model we use the Boolean and entity-based completeness, correctness metrics as well as accuracy metric. Completeness is measured as the ratio of "reconstructed references" to the number of all reference surfaces. "Reconstructed references" indicates the number of all the planes in the reference model that lie in the buffer around the planes of the reconstructed model . The completeness is defined as:

=
Reconstructed references Cardinality of the reference planes Similarly, the correctness is measured as the ratio of "matched reconstructions" to the number of all surfaces which exist in our final 3D model. "Matched reconstructions" indicates the number of all the planes in the reconstructed model that lie in the buffer around the corresponding plane in the reference model. The correctness is defined as:

= Matched reconstructions Cardinality of the reconstruction planes
(2) Another metric which is used in our experiment is accuracy which is defined in  as the Euclidean distance between the dense points sampled from the reference model and the closest surface in the reconstructed model. Comparing to the point-point distance, this pointsurface approach mitigates the influence of the point density of the sampled model in the accuracy computation. However, existence of the missing planes and incorrectly added planes, affect the point to plane distance. Hence, in ) the accuracy is analyzed as a function of a cut-off distance , where the point-surface distances larger than this cut-off value should be excluded from the computation. We follow the same approach to compute the median absolute distance to robustly estimate the accuracy of the reconstructed 3D model.
In order to compute the completeness and correctness of our model for this dataset we considered a buffer size of 10 cm ( =10). Accordingly, we achieved 77% completeness and 53% correctness. For computing the accuracy, the cut-off distance of 10 cm ( = 10cm) is selected and therefore, the accuracy of the model considering the walls, ceiling and roof is computed as 1.7cm, and just for the wall surfaces accuracy value of 5cm is obtained. The main advantage of our approach is that employing adjacency graph of the planes, the topology of the reconstructed model is considered which also provides the water-tight 3D model with correct topology and separated rooms.
Finally, it is worth mentioning that the accuracy of the ground truth 3D model depends on the accuracy of the data set which is used for manual 3D modelling. Therefore, if the ground truth is not produced from the same dataset, sensor error might also affect the quality of reconstructed 3D model. Moreover, the operators use their knowledge and experience to generate the 3D reference model.

CONCLUSION
We introduced a 3D reconstruction modelling approach for indoor spaces. The Proposed approach is applied on synthetic and real dataset. It works directly on point clouds and no assumption about the sensor is considered. Moreover, the positions or trajectory of the sensor are not used in any step. Furthermore, our approach does not rely on Manhattan-world assumptions. Employing a clustering on normal vectors, the points are segmented and non-planar segments are excluded. Based on segmentation results, for each segment the -shape is computed. The adjacency graph of all -shapes is used to find the intersecting planes. Employing a B-rep approach, an initial 3D model is computed. We believe that the 3D model at this step is not topologically consistent and needs more analysis to have a topologically correct and water-tight 3D model. Hence, adjacency graph of the planes which constitute the initial model is analyzed to refine the 3D model. The proposed approach is applied on synthetic and real dataset.
As a plane-based approach, quality of our proposed method depends on plane detection quality which is mainly influenced by, e.g. clutters and occlusions. However, except for extreme cases like completely occluded wall or vertical planar clutters which span the whole story height, our approach is effective. The main limitation of the proposed approach, similar to the most other approaches , is that it is not applicable on buildings with curved walls which makes a room for the future improvements.. Adding semantic information like doors and windows will also enrich the final model. The authors believe that modern construction technologies like additive manufacturing (Buswell et al., 2020) which offer manufacturing of free form buildings' component and do not hold various assumptions in current 3D reconstruction approaches (such as planarity of the ceiling and walls and verticality of the walls) will bring many new challenges to 3D indoor modelling community.