Point cloud room segmentation based on indoor spaces and 3D mathematical morphology

Room segmentation is a matter of ongoing interesting for indoor navigation and reconstruction in robotics and AEC. While in robotics field, the problem room segmentation has been typically addressed on 2D floorplan, interest in enrichment 3D models providing more detailed representation of indoors has been growing in the AEC. Point clouds make available more realistic and update but room segmentation from point clouds is still a challenging topic. This work presents a method to carried out point cloud segmentation into rooms based on 3D mathematical morphological operations. First, the input point cloud is voxelized and indoor empty voxels are extracted by CropHull algorithm. Then, a morphological erosion is performed on the 3D image of indoor empty voxels in order to break connectivity between voxels belonging to adjacent rooms. Remaining voxels after erosion are clustered by a 3D connected components algorithm so that each room is individualized. Room morphology is retrieved by individual 3D morphological dilation on clustered voxels. Finally, unlabelled occupied voxels are classified according proximity to labelled empty voxels after dilation operation. The method was tested in two real cases and segmentation performance was evaluated with encouraging results.


INTRODUCTION
Space is a term wider referred in many research areas. It acquires different conceptualization on the discipline in which it is applied and on the target application. In the 3D modelling and reconstruction domain, the representation of spaces has been defined and interpreted in very different ways. However, while a formal definition of the space is not provided in the literature for outdoors, the indoor spaces have been more strictly defined. (Zlatanova et al., 2020).
Commonly, indoor space is defined as the space enclosed by permanent structural elements such as floors, ceilings, and walls. The subdivision of indoor space into meaningful subspaces have been addressed in many works applying multiple approaches and with different goals. Since space subdivision provides spatial relations between simpler subspaces, topological representations can be generated from these relations for navigation applications. Also, the space subdivision has been used for facilitating building reconstruction (Nikoohemat et al., 2020).
Beyond the classical indoor space interpretation based on the presence of physical elements (Li, 2008;Yang et al., 2019) new space subdivision approaches have proposed in recent years. Divisions according to space functionality or by human perception of it have entailed a new conceptualization of the space (Zlatanova et al., 2013). A newer space interpretation is proposed in (Diakité and Zlatanova, 2018) based on subdividing the space on the basis of the objects present on the scene and their functionality.
The modelling of complex spaces requires not only geometric models but also a detailed semantic and topological information may be necessary. Generally, these well-defined models do not provide other crucial information for indoor navigation such the distribution of temporary or movable elements in a specific time. While point clouds provide a more realistic and updated representation of environment its processing and analysis involve a greater effort than well-defined models (Zlatanova et al., 2014).
This method proposes the use of 3D mathematical morphology for segmenting indoor point clouds into rooms. Different to previous approaches, this new conceptualization does not require the use of the trajectory followed by the laser scanning system during acquisition, and also it does not need from the extraction and modelling of building structural elements by point cloud processing. Consequently, the method is purely relying on the empty space for room segmentation. For the proposed method, the space division is addressed from indoor empty space perspective. This refers to all space inside building without physical objects. Furthermore, in advance space division will be reference as room segmentation because the resulting divisions of space correspond to the rooms.

RELATED WORK
Most of room segmentation methods are based on previous structural element detection such as walls, ceilings, and floors. Doors also play a key role in many strategies of room segmentation because they act as the connecting element between adjacent rooms. This section reviews previous methods addressing room segmentation from point clouds. In literature, there are several interesting strategies based on floorplans. They are important in the indoor modelling field, but they will not be included in this review.
Line-fitting methods consist in representing walls as lines and then determinate lines delimiting each room. This technique, often used in the reconstruction field, starts with a plane detection in the point cloud. RANSAC and Hough Transform (HT) are the most popular algorithm for that purpose (Grilli et al., 2017). Vertical planes are considered candidate walls and they are often project to XY-plane in order to minimize complexity of 3D processing. This way, walls are represented by lines. (Ochmann et al., 2016) generate a planar graph from candidate lines. Then surfaces classification is formalized as a labelled problem solved by an energy minimization approach. This method requires prior knowledge of the scan positions.
A void-based approach for semantic enrichment of indoor point clouds is proposed in (Armeni et al., 2016). Rooms are separated by identify the hollows between the walls since laser scanner acquire object surfaces. To find these gaps, a set of filters are applied on density histograms on each axis assuming that walls generate peaks on the histogram. The method is strongly limited to Manhattan world and axis-aligned walls.
In recent years, trajectory data provided by Mobile Laser Scanners (MLS) was exploited to carry out point cloud segmentation.  analyse height variations of upper points above trajectory positions to detect door candidates. Then labelled problem is addressed as a minimum energy approach. MLS trajectory is also used by (Nikoohemat et al., 2018(Nikoohemat et al., , 2017 to detect doors from which room segmentation is carried out. Both methods identify structural elements previously by means of plane detection and construct an adjacency graph linking patches that represent different elements. From this classification, openings are searched on detected walls. Morphological segmentation is a well-known approach in the robotic field to divide floorplan into meaningful rooms (Bormann et al., 2016). Some authors (Jung et al., 2017;Li et al., 2018) have addressed point cloud segmentation by morphological operations. However, the operation is performed in 2D space projecting point cloud to XY-plane.
Our method is based on applying morphological operations directly in the 3D voxel-space. Different to other works operating into the pixels representing building elements, in this work morphological operations are applied to the voxels representing the indoor empty space with the aim of breaking indoor space continuity. Besides, the proposed method can overcome the limitations generated by occlusions present in previous 2D methods.

METHOD
This section describes the method used to performed room segmentation based on 3D mathematical morphology. The general workflow is depicted in Figure 1. The input point cloud is transformed to voxels from which the building contour is obtained for extract indoor empty voxels. Then a 3D morphological erosion is applied to indoor voxels allowing spaces identification according room semantic. Finally, identified spaces are morphologically dilated to perform point cloud segmentation.

Voxelization and initial classification
The input data of the method consist in a point cloud composed of a set of coordinates XYZ. In order to reduce the volume and provide a structured representation of the data, the point cloud is voxelized (Figure 1.b). This operation structures the space into voxels which are labelled as 'occupied' if any point is contained in the voxel and as 'empty' voxel if it is not. Voxels are located by the grid coordinates i, j, k related with x, y, z axis, respectively.
Resolution is a critical parameter in the voxelization process. Commonly, the resolution value is determined for trade-off computational costs and accuracy. In our method, voxel resolution is selected in order to ensure that space between walls is not represented by 'empty' voxels.

Contour extraction
Because the voxelization is performed for the extension of the point cloud, the results will include 'occupied' voxels, 'empty' voxels from inside rooms and 'empty' voxels from the outside of the building if the building shape is not a rectangle. With the aim of select the 'empty' voxels belonging to the interior of the building, the building contour is extracted by applying a concave hull operation to 'occupied' voxels (Figure 1.c). As this operation is based on an alpha-shape method, the accuracy of the extracted contour depends on the alpha parameter.
The contour extraction process is computed in 2D and consequently, the point cloud should be orientated. In addition, a z-histogram of 'occupied' voxels is computed to detect the floor and the ceiling of the building in a way that outdoor empty voxels placed above the ceiling and below the floor are discarded.
The result of this step consists on the 'empty' voxels belonging to the building indoor (Figure 1.d, Figure 2). Figure 2. a) Building contour (purple) is computed from occupied voxels (red) to discard outside voxels (grey). b) Indoor empty voxels (green) are those inside building contour.

3D erosion of the empty space
Morphological erosion is applied to empty voxels to break the space continuity between rooms given by doors (Figure 1.e). To apply morphological operations, voxels are binarized to generate a 3D binary image that is divided into indoor empty voxels and non-indoor empty voxels (occupied and outside voxels). By eliminating occupied and outside voxels, indoor empty voxels maintain continuity between rooms through doors. A 3D morphological erosion is applied to break the continuity. Morphological erosion is based on the removal of voxels whose neighbourhood does not match the size and shape of the structuring element. The structuring element used to erode the indoor spaces is a cube whose side l is equal to the door width, which is the typical element that maintains continuity between rooms. After applying morphological erosion, all voxels within 1/2 of an occupied or outside voxel are removed and the continuity between voxels belonging to different rooms is broken. In addition, the existing empty voxels between rooms are eliminated if the space inside walls is wider than l.

Room individualization
Remaining voxels after erosion operation are clustered on basis connectivity between voxels (Figure 1.f). A 3D connected components algorithm is used to perform the clustering. Unlike the popular clustering methods such as K-means, the 3D connected components does not require foreknowledge the number of rooms.

Point cloud classification
The classification of the point cloud is based on a proximity assignment between the voxels belonging to each room and the nearby points. To restore the complete empty space corresponding to each room, a morphological dilation is applied (Figure 1.g). Morphological dilation adds voxels according to the shape and size of the structuring element. Restored dilated voxels have the same room label as voxels from growth. The exact number of empty voxels with respect to the input voxels is not recovered. The combination of erosion and dilation with the same structural element (SE) is a morphological opening and produces the elimination of isolated voxels or narrow shapes. Therefore, empty pixels inside walls are not restored.
Occupied voxels are labelled regarding proximity of labelled empty voxel. For each occupied voxel, a set of neighboring voxels are evaluated. If any of them was labelled in the previous steps, occupied voxel is classified in the same way. In case there are not a unique label, the most frequent one is selected to classify occupied voxel.

Case studies
Two real case studies were selected to evaluate the proposed method. The first case study (Figure 3.a.) corresponds to a laboratory at the University of Vigo composed of three rooms and connected with a corridor partially acquired. The rooms are occupied with office objects such as tables, chairs, computers. The second case study (Figure 3.b.) is provide by the ISPRS Benchmark on Indoor Modelling (Khoshelham et al., 2017). Noise caused by the presence of people in the corridors has been removed. The data of the first study case was collected with ZEB-REVO laser scanner. The raw point cloud containing over 6 million points is shown in Figure 3.a. Otherwise, the second point cloud consist of over 31 million points after cleaning pre-process (Figure 3.b).

Results and discussion
The input parameters used to voxelize the raw points clouds are collected in Table 1 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK The grid resolution was selected in accordance with the width of the empty space between walls. For the first case, the voxel resolution used was 0.1 m and the minimum number of points inside a voxel to consider the voxel an occupied voxel is the one point. In this step, remaining voxels are labelled as empty voxels (Figure 4.a). The selection of the resolution value for the second case study is more compromised due to the different width of the space between walls. This separation becomes more than 1m in some areas. Moreover, a low resolution can lead to an inaccurate space representation. Thus, a value of 0.5 m balances both factors.  In the next step, the side of the cube used as structuring element by 3D morphological operation is defined. The length selected was 7 and 2 voxels in the laboratory and ISPRS case studies, respectively. Results of 3D erosion are depicted in the Figure  6.b and 7d. After, remaining inside voxels (Figure 6.b and 7.b) are clustered by 3D connected components algorithm on basis 26-connectivity ( Figure 6.c and 7.c). Subsequently, a 3D morphology dilation is applied to labelled clusters to retrieve the room form (   Confusion matrix of the ISPRS point cloud classification is collected in the Table 3. The performance is over 95% in five of the rooms and only under 85% in two of them. The worst classification occurs with the room1 which corresponds to the two rightmost segments composing the corridor (Figure 8.a).
Next, we will analyse why the corridor is divided into three segments. A plant-view of the corridor with removed ceiling is visualized in the Figure 8.b.
The Figure 8.c depicts the connection area between two segments belonging to corridor. Since there is a door in the area, the segmentation at this point is carried out correctly. Conversely, the segmentation at the point visualized in the Table 3. Confusion matrix of the classification in the ISPR case study. Figure 8.d is caused by the presence of an object resulting in a narrow in the empty space of the corridor. As this narrowing effect is similar to that caused by door presence in the navigable space, the method segments point cloud at this point. This leads to high level of over-segmentation in Room1. Figure 9.a shows over-segmentation in Room6 (framed in black rectangle), however this miss-classification only represents a 2% of points corresponding to the room. Figures 9.b and 9.c visualise a plant-view and a profile view of the Room6 respectively. Pictures shows the presence of a tripod between classified segments. Since this element do not generate a high level of occlusion the over-segmentation is related with low resolution in the voxelization.
On overall, the method presented performs an accuracy segmentation of most tested rooms. Although people had to be removed manually from the ISPRS point cloud, this limitation is already addressed in the literature. Classification oversegmentation caused by low resolution can be overcome by applying adaptive resolutions or implementing a function to remove inside voxels located between walls. So, it would not be

CONCLUSIONS
This work presents a room segmentation method for point clouds based on 3D morphological operations. The space is represented with a voxel-grid structure and indoor space is modelled from empty space. Unlike previous methods perform room segmentation exploiting trajectory data, this is not required for the proposed method. Also, rooms are individualized independently the modelling of structural elements such as doors and walls.
The method was tested in a couple of real case studies showing promising results. However, some limitations associated with resolution restrictions lead to improve the method in future work. Vertical walls and invariant height of floor and ceil are assumed. Also, the way to perform morphological operations without the prior knowledge of the width door would be addressed.