AUTOMATED DETECTION AND LAYOUT REGULARIZATION OF SIMILAR FEATURES IN INDOOR POINT CLOUD

: Automated identification of high-level structures in unorganized point cloud of indoor spaces Indoor space is an important aspect of scene analysis that provides essential information for many applications, such as building digitization, indoor navigation and evacuation route planning. In addition, detection of repetition and regularities in the organization indoor environments, such as rooms, can be used to provide a contextual relationship in the reconstruction phase. However, retrieving high-level information is a challenging task due to the unorganized nature of the raw data, poor-quality of the input data that are in many cases contaminated with noise and outliers. in point benefit from the apparent regularities and strong contextual relationships in façades. The main observation exploited in this paper is the fact that building indoor is generally constituted by a set of basic shapes repeated several times in regular layouts. Building elements can be considered as similar if they share a set of features and elements in an idealized layout exhibiting some regularities. Starting from this main assumption a recursive adaptive partitioning of the indoor point cloud is carried out to automatically derive a flexible and hierarchical 3D representation of the building space. The presented methodology is tested on a synthetic dataset with Gaussian noise. The reconstructed pattern shows a close correspondence with the synthetic one showing the viability of the proposed approach.


INTRODUCTION
Up-to-date 3D indoor models are increasingly requested for the management of existing buildings.In the last years, Building Information Modelling -BIM (Eastman et al., 2011) become more and more important not only for planning the construction site but also for the management of the building during its lifetime.For this reason, in large renovation projects the availability of "as-build" and "as-found" BIM become a major need not only in the design phase but also for maintenance panning.The availability of a reliable geometrical model is of primary importance for the generation of a BIM (Barazzetti et al., 2015).To this end, laser scanning technologies are gaining a lot of popularity for the documentation of indoor environments.Indeed, ongoing advances in the reduction of size and weight of terrestrial laser scanning sensors, together with improvements in indoor positioning techniques, have led to the development of highly productive Indoor Mobile Mapping Systems (IMMS) that allows rapid acquisition of large area and collection of millions of points in a reduced time.However, the processing of the data and the procedure for the creation of a geometrical model of a building indoor is still mainly a manual task.Even if in the last years several authors focused on this topic (Banfi et al., 2017;Macher et al., 2017) it remains an active research filed.This is mainly due to the fact that captured point clouds often suffers from severe missing data, noise and outliers, making the reconstruction of architectural elements a challenging task.Indeed, both a faithful geometrical reconstruction and topology definition may mostly suffer for incomplete data.This problem is made even more challenging due to the wide variability of architectural styles.Another strictly connected problem is the detection high-level structures in the point cloud.This is due to the fact that in many cases a data driven approach is used for the automating the reconstruction from point clouds.While data driven methods provide highly faithful geometrical reconstructions, they are generally processing the different features independently, missing context-based information.Indeed, man-made environments generally exhibit some regularities (e.g., a floorplan is generally organized considering symmetries, alignments and repetition criterion) that can be used to constrain the reconstruction.Those high-level information are not taken into consideration if the building's rooms are separately extracted.In addition, this context-based information is crucial for obtaining a consistent model less dependent on data quality.This paper presents a methodology for the automated detection and layout regularization of similar features in point clouds of building indoors.Automatically discovering high-level building structures, like rooms, in unorganized 3D point clouds is generally challenging due to the wide variability of possible patterns in room organization, large amount of noise, outliers or clutter.The main observation exploited in this paper is the fact that building indoors are generally constituted by a set of basic shapes repeated several times in regular layouts.Building elements can be considered as similar if they share a set of features and elements in an idealized layout exhibit some regularities (i.e., they are aligned and uniformly distributed along specific directions).Starting from this main assumption a recursive adaptive partitioning of the indoor point cloud is carried out to automatically derive a flexible and hierarchical 3D representation of the building space.
The paper is structured as follows.Section 2 reviews related work on building and indoor reconstruction starting from point cloud data with a specific focus on methodologies introducing strategies for repetition detection and regularization for building modeling.Section 3 presents an overview of the developed methodology, while a detailed description of the room partitioning and layout regularization step is provided in section 4. Section 5 is focused on presenting results for a test carried out a synthetic dataset with Gaussian noise.The reconstructed pattern shows a close correspondence with the synthetic one showing the viability of the proposed approach.The final section is devoted to draw some conclusions and to address future developments.

RELATED WORK
There is a large amount of literature on the reconstruction of indoor environments based on point clouds.This review is focusing only on existing works focusing specifically the metric accuracy of the reconstruction using Terrestrial Laser Scanning (TLS) and Mobile Laser Scanning (MLS) and discovery of repetitive/symmetric patterns as well as other types of regularities (e.g., same alignment, same spacing, etc.).
Early works Yue et al. (2012) and Becker et al. (2015) developed a formal grammars, which describes different building styles, for the reconstruction of 3D indoor models from 3D point clouds.A similar approach was also adopted by Khoshelham and Dìaz-Vilarino (2014).In this case the modeling of building indoor was solved by using an iterative procedure based on placing, connecting and merging of cuboids.All these grammar-based approaches require a pre-design of an according set of grammar rules.Jiang et al. ( 2016) are presenting a methodology for detection and enforcing of 2D regularization that can be applied also for floorplans.Previtali et al. (2014) and Wang et. al. (2017) used a similar idea for generation of building floor plans.Based on some apriori assumption about wall organization, the floorplan estimation was posed in terms of an optimal labeling problem solved by minimizing a score function in the framework of s-t cut.However, no specific analysis are performed concerning the identification of high-level similarities.Armeni et al. (2017), Macher et al. (2017), Previtali et al. (2018) and Jung et al. (2018) presents similar approaches for the generation of semantic parsing of 3D point cloud of building indoors.The presented approaches follows similar steps.Firstly, the point cloud is subdivided into spaces and basic geometries.Secondly, spaces and objects are parsed into their structural and building elements (e.g.walls, columns, etc).Finally, a 3D model of the building is derived.Bassier et al. (2016) combine simultaneously contextual and geometrical characteristics to assign an object category to extracted elements.Ikehata et al. (2015) introduce a new framework that defines a structure graph and grammar for an indoor scene geometry, which can be extended to add more details such as multiple stories.But their method is based on the Manhattan world assumption that restricts the types of geometric expression.Another approach proposed by Mura (2014) exploring the vertical structure (e.g., planar patches from the walls) and visibility check to eliminate occlusions through the positions of the stationary laser scanner.
While most approaches require measurements of high quality, Loch-Dehbi et al. (2017) proposed a methodology for predicting the floorplan and the room arrangement by using a limited number of observations.A set of hypothesis are generated and tested by using the sparse observation available together with strong model assumptions.In the same context, Rosser et al. (2017) presented a semi-automatic data driven estimation of 2D building interior floorplans.Detection of regular patterns was a topic also addressed in building façades reconstruction.Mitra et al. (2006) and Shen et al. (2011) present a pair-matching based approach to detect symmetries in 3D shapes and point clouds.In Pauly et al. (2008) a general regularity detection method for 3D models is presented.This approach can be used for extracting a single façade pattern.However, in this case similarities in the model are detected by considering a local similarity measure of the point cloud curvature which is more prone to output outliers.Lian et al. (2018) present an algorithm for automatically detecting and inferring repetitive elements from façades.Firstly, candidate templates are automatically derived starting on the clustering of similar features.Secondly, an adaptive region descriptor is used to detect partially occluded repetitive elements.Finally, the fully occluded elements are inferred by utilizing the Bayesian probability network, which can be learned from a database of the selected façades.In Previtali et al. (2013) regularities are imposed on building facades assuming that elements are distributed in an irregular lattice schema.Repeated candidates are identified according to a voting scheme and the final regular lattice is estimated minimizing a score function.

OVERVIEW
An overview of the developed method is presented in Figure 1.In this discussion we consider the Manhattan-World-Assumption holds so building rooms are characterized by flat dominant planes.In addition, the point cloud should be aligned such that the Z-axis correspond to the vertical direction.After a pre-processing phase each iteration of the adaptive partitioning algorithm consists of three main steps: point cloud splitting, similar features grouping and layout regularization.Once data are subdivided into different floors, planar pieces are detected in the point cloud using a RANSAC (Fisher and Boles, 1981) approach identifying possible boundary lines.This stage is accomplished by using the segmentation strategy described in (Previtali et al., 2013).In particular, the planar primitive detection is carried out by using a hybrid technique combining RANSAC algorithm and region growing.In this implementation, major attention is paid to the reduction of so called 'bad segmentation' problems.Indeed, spurious results may be due to the fact that points constituting the maximum consensus to RANSAC planes are derived from different objects.
A confidence value indicating the likelihood that a point lies along a true boundary is assigned.The confidence value is evaluating the local structure of the point cloud taking into account eigenvalues computed for the k-nearest point boundary.
A second parameter associated to each point is the length of the line the point belongs to.
As previously anticipated, identification of structure regularities is carried out in three recursive steps:  structure splitting which is aimed at subdividing each building floor into a set of sub-blocks;  similar feature grouping where the similarity of the different detected element is evaluated and similar slices are grouped together;  layout regularization that is devoted to the optimization of the layout.The final output is a top-down hierarchical subdivision of the building space.

HIGH-LEVEL STRUCTURES IDENTIFICATION
Using as input data the outcomes of the pre-processing stage described in the previous section the methodology for identification of high-level structures and similarities is carried out in three recursive steps: structure splitting, similar features grouping, and layout regularization.This section the three stages are discussed and to obtain a top-down hierarchical subdivision of the building indoor.

Structure splitting
Structure splitting stage is aimed at subdividing each building floor into a set of sub-blocks.The initial splitting (number and localization of splitting planes) can be driven considering some weak prior knowledge of building floorplan arrangement.Indeed, it can be easily observer that into an orthogonal building floorplan horizontal splitting planes should be placed where horizontal lines are dense and vertical lines are rare; vice versa vertical splitting planes should be placed where vertical lines are dense and horizontal lines are rare (Muller at al. 2007).To incorporate such prior knowledge, the following formulation is used (for horizontal boundaries): where the product between the direction of the boundary line and the vertical direction measures the contribution of every boundary to the vertical direction.The weight function ( ) takes into consideration the contribution to the op point pi to the splitting line P.The parameter  is related to the size of the walls in the dataset.In particular, for the examples presented in this paper  = 0,2 m was used.Confidence of the boundary point is weighted taking into account the length of the boundary the point belongs to ( ) .Finally, measures the local density around p i .In particular, the local density is evaluated as: (p i ) = k / V(k), where V(k) is the volume of the bounding ball of the k-nearest neighbouring point around p i in the original point cloud.The number of the nearest neighbouring point to be evaluated id dependent to the data density.In the examples reported in this paper a number of k = 10 nearest neighbouring point was used.This factor is of main importance to compensate the influence of non-uniform point distribution in the point cloud.In particular, for a relevant comparison between areas with different sampling density, boundaries point in regions with low density are compensated by larger weights.On the other hand, areas characterized by higher density are compensated with lower weight.Similar considerations can be formulated for vertical boundaries.To take into consideration the previously defined prior we can define the score function: (2) Initial splitting planes are located in correspondence of local maxima F y (Figure 3).Similar considerations can be formulated for vertical boundaries.Once detected the splitting planes they are used to subdivide the current building floor into slices.Splitting planes are added sequentially starting from the ones having higher score (Figure 4).In the case one splitting plane intersect with another one the splitting is interrupted and it is resumed only if a new intersection is found.At the end of the splitting step a set of sub-building blokes is derived.

Similar feature grouping
In the grouping step similar slices are grouped together, so that to identify repetitive building elements (rooms) and generate a regularized layout in the next step.Given two two objects Si and Sj the similarity measure (SM) between them is evaluated.
To tolerate poor quality input data the space of the overlapping region of the aligned slices is quantized and the similarity between Si and Sj is calculated in this quantized space.Specifically, for each resulting voxel two functions vi and vj are defined to indicate the number of points contained in the voxel from Si and Sj, respectively.At the end, the original point clouds are quantized into a tensor representation.The SM between slices Si and Sj is defined as: The adopted SM may range from -1 (full inverse correlation) to +1 (full direct correlation).For this reason, SM values close to +1 indicate high similarity between Si and Sj, while in the case SM is close to zero or negative, they are assumed to be different each other.SM also supports partial matching of two slices, since the similarity is defined on the overlapping region of their aligned versions.
Once the similarity is measured for each pair of slices, the ones having the maximum similarity are automatically clustered by using a bottom-up method as far as no more clusters (C1,C2,…, Cn) can be created.The clustering process is stopped until SM is lower than a user-defined threshold (SM = 0.7 has been used in the experiments).In this way, elements with low similarity concerning the others in the cluster are discarded, improving the robustness of the method.

Layout regularization
Once similar slices are grouped together the optimization of the layout is carried out.In particular each object Si can be described as a cube where the position of the bottom left corner is (x i , y i , z i ), its height is h i , its width is w i and its depth is d i , so: S i ={x i , y i , zi, h i , w i , d i }.In this paper we are looking for three specific sets of constraints: • 'same shape;' • 'same alignment;' and • 'same spacing.' Given a set of objects S whose 'shape similarity' is identified at the previous step the remaining two types of similarity are separately evaluated.First, the 'same alignment' constraint is investigated.Three different kinds of clustering are applied: the first one by considering the 'horizontal alignment' (x and y) between objects, the second one the vertical alignment (z).
(horizontal alignment #1) (horizontal alignment #2) Finally, the 'same spacing' constraints are defined on two element pairs.These can be either in the horizontal or in the vertical direction.In particular, assuming that S i and S j have the same spacing of S n and S m in the horizontal direction #1 and objects are organized in an ascending order with respect to the x coordinate (x i <x j <x n <x m ), his constraint can be formulated as:

=>
A similar definition can also be given for the 'same spacing' constraints in horizontal direction #2: And in the vertical direction: For all objects belonging to the same clusters, pairwise constraints are defined.In particular, the whole group of detected objects is clustered by using the mean-shift clustering algorithm (Comaniciu and Meer, 2002).This exploits the object shape using as bandwidth the user-defined tolerance α.Objects with different shapes will be recognized as different clusters.
The main advantage of the mean shift algorithm is that it is a non-parametric clustering technique which does not require prior knowledge of the number of clusters, and does not constrain the shape of the clusters.For all objects belonging to the same clusters, pairwise constraints defined according to Eq. ( 5), ( 6), ( 7), ( 8), ( 9), (10) are set up.Since in many cases the spacing between objects is obtained as a multiplier of a basic dimension ('unit') the cluster with smaller spacing is used as reference and others are checked it they are the integer multiplier of the 'unit' (to do this test, the mean values of the different clusters are used).A user-defined bandwidth tolerance α is used.In the case the following constraint are added: (11 where k is an integer value. Once the constraints between objects are identified, the second step of the proposed approach is aimed at regularizing the layout under the previously identified constraints.The regularization is performed by using an approach similar to the one presented in Dang et al. (2014).In particular, the regularization problem is addressed by transforming the initial layout, L, into a regularized one, L*, by minimizing the following score function: where the first term represents the element's position while the second one represents the element's shape.The parameter ω balances the influence between the position and the shape constraints.
In addition to the previously identified constraints, some additional constraints are added to limit possible changes of the element's position.In particular, the maximum allowed width change inside a cluster is limited to ( ̅ ̅ ) where ̅ is the average height of the cluster and  is the RMSE (Root Mean Square Error) of the cluster.Similar constraints are added also for object width and depth.By solving the quadratic programming problem of Eq. ( 12) the regularized layout is obtained.Similarly to Jiang et al. ( 2016), constraints are added sequentially to avoid potential conflicts.If any conflict is detected, the constraint will be discarded.

EXPERIMENTAL RESULTS
In this section we show some partition into similar features and layout regularization results.To test the viability of the proposed approach we have used some synthetic datasets.In particular, a two story building was simulated.The floorplan of each story is symmetrical with respect to a central area simulating an hall.At the each side of the hall at the ground floor five rooms having the same shape are connected to a rectangular space representing a corridor.All rooms are aligned along the Y axis and are equally spaced (Figure 5a).Instead, at the first floor the corridor is connected to three rooms (having the same shape of the room at the ground floor) and a larger one.All rooms are aligned along the Y axis (Figure 5b).The point cloud simulating the scanning of the building is composed of 8 million points with a density of about 1.000 points/m 2 .The output of the processing is a set of coordinates representing the hierarchical structure of the data set.In particular, the output represents the centre of the room.In particular, the following tests were performed: For each test the output of the processing were compared with the original synthetic set.In particular, both the coordinates of the room centres and the regularizing constraints ('same shape', 'same alignment' and 'same spacing') were evaluated.In all our examples, we represent features having the 'same shape' as circles of the same colour and features meeting the 'same spacing' constraints lines of the same colour connecting the features.In all tests performed the regularizing constraints detected by the proposed method met the one of the synthetic dataset ground truth.Comparison of the room coordinates is summarized in Table 1.If 'same alignment' constraints are identified between features, a possible misalignment between room centre and ground truth is exemplified as a systematic error over all the features.In other words, since all "Room type 1" are detected as having the 'same alignment' in Y direction the same misalignment is observed on all the rooms (e.g., see for example the test with syntactic dataset without noise in Table 1).
is carried out by a similarity measure function.Finally, similarity constraints in alignment and spacing are exploited and enforced into a regularization step.The presented approach was tested on a synthetic dataset to verify its viability.However, the results presented in this paper are only some preliminary tests.Indeed, a validation with real dataset is of primary importance to have real feedback about the presented approach.In addition, a more in-depth evaluation of precision, recall and F-measure will be necessary taking into consideration a ground truth data set.The presented article is talking only a specific set of translational similarities.Possible extension of the developed methodology may include circular of radial functions.Obviously, buildings completely irregular repetitive patterns are not targeted in this research.
Finally, the presented methodology can be extended to other domain.For example, building facades are characterized by the repetition of a set of elements (e.g., windows, doors, balconies) into a repetitive pattern and the presented methodology can be adapted to face this problem too.

Figure 1 .
Figure 1.Overview of the workflow for the developed approach.
The pre-processing step is aimed at identifying possible splitting planes into the point cloud.Firstly, the whole point cloud is subdivided into floors (Figure2a-b).In particular, this initial subdivision can be performed by considering the distribution of points along the vertical directions.In correspondence of local minima in point cloud density is sought a horizontal splitting between floors.a. b.

Figure 2 .
Figure 2. Synthetic point cloud used as a reference dataset to test the presented methodology (a); and point cloud density distribution in vertical direction (z).

Figure 3 .
Figure 3. Building floorplan for Floor 1 (a) and Floor 2 (d) with score function C hor and C ver for Floor 1 (b and c respectively) and for Floor 2 (e and f respectively).


Processing of the syntactic dataset without noise;  Processing of the synthetic dataset with Gaussian noise having  = 2.0cm  Processing of the synthetic dataset with Gaussian noise having  = 5.0cm  Processing of the synthetic dataset with a large missing area in Rom type 2 a. b. Figure 5. Building floorplan for Floor 1 (a) and Floor 2 (b).