TOWARDS OBJECT DRIVEN FLOOR PLAN EXTRACTION FROM LASER POINT CLOUD

: During the last years, the demand for indoor models has increased for various purposes. As a provisional step to proceed towards higher dimensional indoor models, powerful and flexible floor plans can be utilised. Therefore, several methods have been proposed that provide automatically generated floor plans from laser point clouds. The prevailing methodology seeks to attain semantic enhancement of a model (e.g. the identification and labelling of its components) built upon already reconstructed (a priori) geometry. In contrast, this paper demonstrates preliminary research on the possibility to directly incorporate semantic knowledge, which is itself derived from the raw data during the extraction, into the geometric modelling process. In this regard, we propose a new method to automatically extract floor plans from raw point clouds. It is based on a hierarchical space partitioning of the data, integrated with primitive selection actuated by object detection. First, planar primitives corresponding to vertical architectural structures are extracted using M-estimator SAmple and Consensus (MSAC). The set of the resulting line segments are refined by a selection process through a novel door detection algorithm, considering optimization of prior information and fitness to the data. The selected lines are used as hyperlines to partition the space into enclosed areas. Finally, a floor plan is extracted from these partitions by Minimum Description Length (MDL) hypothesis ranking. The algorithm is applied on a real mobile laser scanner dataset and the results are evaluated both in terms of door detection and consecutive floor plan extraction.


INTRODUCTION
Indoor modelling is a recently trending topic although it has been studied for the last three decades.In addition to significant developments in different techniques such as data structuring and visualization, maturing in outdoor modelling researches naturally paves the way to a transition to indoor spaces.Besides, with the advent of mobile laser scanners, the possibilities as well as the challenges grow hand in hand.
In this context, floor plans can be regarded as a provisional step to proceed towards higher dimensional indoor models.However, they should be given special emphasis as the simplest and most unambiguous representation of information, especially for vital applications such as emergency planning and first respond.Traditionally the employment of floor plans is dependent on the availability of a Computer Aided Design (CAD) model, otherwise is restricted to the digitization from image documents (Macé et al., 2010) or sketch based retrieval (Ahmed et al., 2014) which already presuppose existing reliable plans.Not all buildings satisfy this requirement, both in terms of preservation of the blueprints, and faithful construction or renovation of the architectural design.In this regard, the automatic reconstruction of floor plans from laser point clouds can have valuable contribution to generate efficient inventories.
Compared to geomatics, automatic generation of floor plans is a better-studied area in architecture and computer engineering.

* Corresponding author.
Studies in overcoming floor plan layout problems in urban design (Knecht and Koenig, 2010), or creating real-time floor plans for greedy computer gaming schemes (Lopes et al., 2010) are such examples using procedural methods to generate afresh.On the other hand, automatic extraction of floor plans of the existing structures has roots in robotics for navigation models (Nuchter et al., 2003).In essence, these studies have specific goals, and are not interested in a more general and complete representation which could be used in a range of applications, from energy flow monitoring to law enforcement, and disaster management.
This discourse leads to an aspiration for powerful and flexible representations.Image based reconstructions (Cabral and Furukawa, 2013), or stereo-vision systems providing point clouds (Henry et al., 2012) have apparent limitations with respect to precision demanding projects.Potentially, laser scanners provide immense amount of data appropriate for this endeavour.On the other hand, processing of the data should be carried out with novel algorithms capable of dealing with high data size in an automatized manner.
Computer vision techniques can generate visually appealing automatic models by texturizing mesh representations in a way in which is mostly dependent on some form of raw data, and geometrical correctness is bounded by the acquisition accuracy.In contrast, commercial software packages are capable of creating regularized models with intensive manual labour.Our aim is to substantiate a fast and flexible method, to cover various project demands in a continuous but separable schema.In this regard, we propose to extract floor plans from mobile scanning laser data as direct as possible in 2D.In order to achieve this goal, we focus on reliable primitive extraction, by both benefiting from planar surface extraction robustness in 3D, and semantic information contribution in 2D.Our method is prune to occlusion by using hyperlines as partition primitives in a BSP (Binary Space Partitioning) framework.The drawback of this approach is its vulnerability to the highly cluttered object space which is addressed by a favourable slice selection and a novel door detection algorithm.

RELATED WORK
Architectural floor plans from raw 3D point clouds can be extracted either in 2D or 3D domain, or as a combination of both, differing in the main bulk of the process.According to scene reconstruction method, the existing approaches can be categorized into the following three main groups; mesh based, volumetric fitting, and linear primitive extraction.Mesh based approaches, mainly originated in computer vision community, offer some well visualized but computationally demanding solutions in a triangular form (Turner and Zakhor, 2014).They are inherently not very suitable to extract topological information in a floor plan level, especially considering the typical data size of laser scanners.In contrast, volumetric fitting is generally implemented in rectangular form.Although comparing to 2D approaches, it imposes a stronger regularity, which might be favourable in highly cluttered environments, and it also brings less flexibility; an implicit parallelity and orthogonality requirement to the space.Besides there is still a present complexity issue in volumetric approaches.The problem is addressed in (Xiao and Furukawa, 2012) by decomposing volumetric primitives into rectangular shapes along horizontal slices.The vertical ambiguity is treated by generating an overcomplete set of primitives on the possible combinatory positions.
2D primitive extraction is a widely used technique in indoor modelling for extracting presumed planar surfaces in man-made indoor spaces.Planar surface fitting in laser data is an already extensively studied field in remote sensing community.A comprehensive research by Nurunnabi et al. (2014) compares some of the existing algorithms.A similar comparison for mobile indoor mapping can be found in (Nguyen et al., 2007).For indoor reconstruction implementation, variants of Hough Transform (Okorn et al., 2010), (Oesau et al., 2014) and RANSAC (RAndom SAmple Consensus) (Dumitru et al., 2013), (Ochmann et al., 2014) are generally preferred in 2D processing domain beside plane sweeping by Budroni and Boehm (2010), or EM (Expectation-Maximization) by Thrun et al. (2004) are applied directly in 3D.Sanchez and Zakhor (2012) utilized RANSAC in a region growing method to detect planar primitives.
In order to extract the valuable topological information which establishes the relationship between primitives, space has to be partitioned in a solid framework.Oesau et al. (2014) apply cell decomposition in indoor spaces after line fitting followed by a graph cut optimization.We propose a BSP based method in order to deal with the irregular hierarchical decomposition in indoor spaces.Furthermore we enhance line primitive extraction with a novel door detection algorithm integrating a swinging approach in an energy minimization framework.
The specific application of door detection has come up only in very recent studies.Previtali et al. (2014) classify doors within the more extensive cluster of openings, according to their floor intersections.(Díaz-Vilariño et al., 2015) detect closed doors by making use of both laser point cloud and image data, while (Ochmann et al., 2014) exploit the visibility between two adjacent rooms through wide-open door intervals.Our method is neither dependent on any visibility requirement, nor limited to a predefined certain position of the door, but only vulnerable to inseparability issues, on the occasion that the wall planes and the door plane is insufficiently closer than a given threshold, which occurs only for closed doors allocated at the same wall plane.
Nonetheless, our approach does not primarily focus on the door detection itself, but rather using this knowledge to enhance space partitioning.To our knowledge, the incorporation of door detection to the extraction of the floor plan has not been utilised yet in the literature.

METHOD
The input to our method (Fig. 1) is the raw point cloud, and the output is the floor plan having topological relations between wall units, and some object information to a certain extent; being the detected doors.In the first part, geometric information in terms of primitives is extracted in three phases: the most appropriate horizontal slice is selected volumetrically, plane fitting is applied in 3D to RANSAC determined points, and a regularization is applied to the horizontally projected planes by means of line quantization in 2D.As an intermediate operation, semantic information is obtained by door detection, to be used additionally to refine the primitives which are going to be employed in the next step.And in the final part, decomposition is implemented by over-splitting the space using BSP, driven by the refined primitives obtained in the previous step, and then by optimizing in terms of MDL (Minimum Description Length) formulation.We explain the details of our algorithm in the following sections.

Volumetric Slicing
To start for floor plan extraction, the primary but not the least objective is to extract walls from a given dataset.A 3D laser point cloud partially or fully might consist of all kinds of structures in an indoor environment.This content can be grouped into three geometrically significant clusters with respect to surface normal distributions as: the horizontal structures, vertical structures, and clutter which might be existent as dissolved in any direction, anisotropically-heterogeneously, throughout the whole point cloud.The common approach to leave out walls in this mixed data, is to start with extracting ceiling and floor within horizontal structures, and further eliminate clutter from the remaining cloud (Oesau et al., 2014), (Mura et al., 2013), (Adan and Huber, 2011), (Okorn et al., 2010).For this purpose, the inherent density predominance of ceiling and floor is exploited in a point density histogram along the vertical axis, and clutter is removed by estimating surface normal, and filtering points whose normal are not parallel in the vertical direction.
Keeping in mind the sufficiently simplicity of our aim, we take a slightly different approach by confining the 2D domain in a predefined height below the estimated ceiling.It is assumed that the most appropriate slice relieved from clutter effects is at a certain height below the ceiling.This buffer zone also prevents washboard ceiling structure inference which might arise from having inhomogeneous structures with various surface normal due to objects attached, such as lambs, or the grift structure of the ceiling itself such as cavities and saliences generated from the original design.This assumption is mostly in accordance with architectural design concepts, and space usage practices (Fig. 2).
However it is also necessary to provide robustness to this vulnerable 2D selection process, as this assumption may not accommodate every rare exception.Furthermore, it is considerably more plausible to preserve some other structures in this 2D slice, such as doors, windows and openings for further semantic information extraction to give rise for a more sophisticated floor plan.In order to achieve this robustness and simplicity simultaneously, planar surfaces from a 3D slice are extracted instead of 2D horizontal slices.

Primitive Extraction
The aim of primitive extraction is to detect the line candidates which will take place in the process of space partitioning.In order to assure robust estimation of these lines, we perform an extraction of planes rather than 2D lines, and impose a regularization in 2D by quantizing using a Compass Line Filter (CLF) (Sohn et al., 2008).The final selection of the overall primitive configuration is incorporated within the door detection procedure.
Modern mobile laser scanner data can be identified as a group of very accurately acquired data points, superimposed by nonaccurate points due to various systematic and non-systematic errors.This remark highly implies the existence of points as inliers and outliers, and the employment of RANSAC methods to extract planar surfaces from raw point clouds.The main RANSAC assumption is that from a set of inlier members the planar model parameters can be estimated optimally.As (Schnabel et al., 2007) indicate, the prohibitive hypothesis generation space and the need for effective scoring of the hypothesis are major drawbacks of the method.
In our algorithm we applied MSAC (M-estimator SAmple and Consensus), a variant of RANSAC proposed by (Torr and Zisserman, 2000), which evaluates the quality of the hypothesis with its likelihood.After distinguishing membership points for each plane by MSAC estimation, planes are extracted by minimizing the error in the Least Squares sense.In order to decrease computational cost we further continue our operations in 2D, assuming that only vertical planes are in consideration for detecting walls, and affiliated structures embedded.The 2D plane is regularized by CLF, which quantizes the distance and the angle of the possible line segment positions to certain levels.
The final result of this step is the well aligned primitives representing the wall segments, which are eventually be used in the space partitioning process.Yet, quantization is not cruelly strict to efface the arbitrary orientations of open doors playing disruptive role in space division.

Door Detection
A door can be regarded as a structural building element that establishes connection between enclosed spaces.However in a typical laser scanning application, especially in a mobile platform, it is mostly scanned in its ajar position.In other words, this connection element which provides continuous accessibility between scanning positions turns out to be a disruptive object, effecting the continuity of associated walls, hence the overall floor layout.
We employ a novel door detection algorithm for these ajar doors as an intermediate step between primitive extraction and space decomposition.The main reason for locating detection within reconstruction is to seek for the possibilities in which this semantic information that has been extracted in a relatively early stage of the processing could contribute to an enhanced geometry.In the following section, we demonstrate a preliminary improvement by filtering out the detected doors from the decomposition space, over a simple alternative line filtering procedure which obviously cannot maintain narrower walls having lengths comparable to typical doors.More importantly, we advocate that the benefits are expandable to a regularization of the primitives, and some completion of the raw data by means of primitive grouping.We discuss more on this issue in discussion section.
The process of ajar door detection starts with the selection of potential door candidate lines.Based on these candidates, several hypotheses are generated by the use of the swinging door concept and evaluated by scoring for a global configuration.

Single Door Candidate Selection
Doors as a part of the building form an object class with some general characteristics.For instance, majority of the doors within a building might conform to have a specific width, and revolve around an axis that resides in a wall.Our aim is to exploit these characteristics and formulate as prior knowledge in order to determine single door candidates.For this purpose, we investigate the connectivity of the line segments, and define two different ending types according to the proximity to the neighbouring endings as: a) Anchor (connected) and b) Open (non-connected).The search for nearby line segments is conducted by a greedy searching method.Ending type of a door line is determined as 'Anchor' if the ending point is close to ending point of neighbouring line, otherwise, defined as 'Open'.Once all ending point types are determined, line segments with ending types of 'Anchor-Open (A-O)' or 'Open-Anchor (O-A)' within the threshold values δ lower and δ upper are considered as door candidates (Fig 5a).

Overall Configuration Ranking
Once all the door candidates are determined, a set of hypotheses that constitute the possible angular positions of any candidate line along their fixed ending points, are generated based on a swinging door concept (Fig 3).The idea is to swing the door candidate lines, by imitating an actual door movement.It is presumed that the detected doors would have at least one connected wall and rotate alongside an intersection with their where, n is the number of line primitives and φ, and  are weight values which balance the unary term and the contextual term.

Unary term:
The unary term is designed to measure the likeliness of a line segment length, congruent to be a door.Both, the connection type and the length of the line segment is taken into consideration.A null score is assigned if a line is categorized as O-O or A-A type.Otherwise, the score of a line is calculated by f(x) as shown in Figure 4a where x is the length of the line segment.
Contextual term: This term expresses the context of the CLF configurations with respect to each other in terms of distance.
The smaller the remaining gap, the more likely that the rotating line segment is to be an actual door.For all ending points, distance between the ending point and neighbour ending point is measured and then the distance is converted to a contextual score using following scoring function g(x) (Fig. 4b): Global configuration: Let H = {H 1 , H 2 , … , H m } denote a set of all possible hypotheses.Among generated hypotheses, the optimal model H * is the model with the minimal score.The ranking of all configurations is an iterative process and continue until the total score is minimum.Fig. 5b shows all detected doors based on the evaluation of the presented method.

Floor Plan Generation
A key issue of the floor plan reconstruction is to recover a geometrically and topologically correct configuration from fragmented or missed lines.Therefore, we turn the floor plan reconstruction into a space partitioning problem.Thus, we utilize the BSP to partition the whole indoor space into a set of meaningful subspaces.To extract the final floor plan, adjacent partitions of the BSP tree with similar attributes are merged based on the Minimum Description Length (MDL) principle.
The BSP is a method which recursively subdivides an n dimensional space into convex subspaces by a set of (n-1) dimensional hyperplanes (Schumacker et al., 1969).An advantage of the BSP is that it always provides topologically correct configuration by a hierarchical tree even if the input data is incomplete or fragmented.However, the partitioning result depends entirely on the order in which the hyperlines are applied (Sohn et al., 2008).The determination of the order is achieved by the hypothesize-and-test scheme, in which a line correspondence (LC), and an angle criteria (AC) are taken into account.After all the hyperlines are tested, the hyperline with the highest score h * is selected to partition the space; where, α and β are the weight parameters.LC is the ratio of the actual line's length to the corresponding hyperline, favouring longer lines having point support.In order to maintain only global geometries, AC is set as a binary term being {0, 1} that filters out sharp angles.Fig 6a shows space partitioning result where our proposed score function is applied.
Once the whole indoor space is partitioned into meaningful subspaces, the final floor plan is reconstructed by removing or merging excessive partitions.We use MDL which is introduced by Rissanen (1978), as a simple optimization method.The model that optimizes the balance between the maximum closeness to the data, and the minimum complexity is selected.Considering the partitions derived from the BSP which are over-fitted to observations, the objective function is designed to find an optimal floor plan in a MDL framework as follows: where, H and D represent hypothesis and the data, respectively, and λ being the weight value.Competing hypotheses are generated by merging partitions which share a common edge.
Closeness term: This term measures the description length of the data against the hypothesis, i.e. the goodness to the fit.In this study, the Euclidean distance in 2D is used.Assuming that the error model between the data and a model follows Gaussian distribution, the closeness term can be determined as; where, Ω is the sum of the squared residuals between a model (H) and the observation (D); that is

Complexity term:
The complexity term is the description length of the hypothesis.With the aim of eliminating excessive partitions, perimeters of polygons are encoded as follows; where, L p and L p ′ are the sum of polygon perimeters in initial model and in a hypothesis, respectively.The optimal model is a model which has a minimum DL value.

Data
We evaluate our method on a large scale dataset acquired by a wheeled platform mobile laser scanner instrument.The floor modelled is an office environment, consists of different size and types of rooms, orthogonally and non-orthogonally aligned corridors.As seen in Fig. 7, the rooms are highly cluttered, and walls are partially occluded.The corridors are especially challenging in many aspects.The passages between the corridors are asymmetric, while some different placements such as a kitchen effects the wall regularity as well as the file closets placed along some parts.One final note is on the impermanent office cubical structures accommodated in the middle of the large hall on southern part, which we do not consider as a part of the floor plan, and eliminated in the horizontal slice selection part of the algorithm.
Figure 7.The overall test site, ceiling is removed from the point cloud for the bird-eye view.

Assessment
There are two different subjects for the assessment of our method, of which door detection part could be disjuncted for an individual analysis, whereas the space partitioning and the overall floor plan extraction results are highly dependent on and improved by the success of the door detection algorithm.In order to evaluate the results, a raw data image carved out from the point cloud is employed for visual reference.Furthermore a 3D floor plan is generated manually from this raw image only.
Door detection results are depicted in Table 1, and also shown in Fig. 8.The scoring function parameters φ, and  are set equally as 0.5, the ideal door width is taken as 0.8 m, with the range of δ lower and δ upper values as 0.4 m. and 1.2 m. respectively.In overall, the success rate can be evaluated as high in many parts of the floor, the error is mainly agglomerated in some locations where either the scene is very complex, or scanning density is lower.The final floor plan extraction results are shown in Fig. 9.The BSP parameters α and β are empirically set as 0.8 and 0.2, respectively.Sharp angle parameter SA is considered as 30°.MDL weighting factor and λ is set as 0.5 to equally favour between closeness and complexity.There is substantial improvement in the extracted floor plan after combining the door detection results to drive BSP.Especially the problems caused by the large doors on the western corridor part are mostly eliminated as well as the inaccurate complexity of the southern rooms are reduced.However in both cases, the asymmetric corridor structure with a narrow passage on the eastern part, cannot be recovered successfully.There is a similar problem on the northern corridor as one of the two salient rooms can only be reconstructed partially correctly.But overall extraction could be regarded as satisfactory, considering the challenging environment (Fig. 10).

Limitations
The limitations can be considered both inherent to the acquired data by mobile laser scanner and method-oriented.From an acquisition perspective, the lack of standards, and a substantial layout variation inherently present in indoor environments are the biggest challenges while designing a governing method for all situations.For instance, it is not sufficient to consider a door conceptually either as it is open or closed or in between, it is also necessary to take into account a possible two or more different positions superimposed, generated by multiple scans within the same data set (Fig. 11).
In addition to the environment complexity, another issue concerning reliable primitive extraction is related to the new type of powerful laser scanners; although they provide higher signal/noise ratios, due to their extended scanning range (i.e.130m. in this case), some less precise measurements are superimposed especially through longer indoor corridor sights.This problem should be treated early in the scanner data generation phase, otherwise statistical noise removal methods are needed, though presumably less efficient.
From the methodology point of view, our space decomposition algorithm is not designed to handle finer angular variation of the line primitives; both CLF and BSP act as strict regularizators.In terms of vulnerability to misaligned or degenerative primitives, doors are good but not sufficient enhancers.In addition, the hyperlines utilized for completing missing data such as caused by window units, may not be terminated accurately in all cases.
Finally, for a full automatic framework, parameters employed in the optimization frameworks should be estimated instead of tuned experimentally.

DISCUSSION AND OUTLOOK
With the advance of the state of the art data acquisition methods, it is of crucial importance, to keep up with the developments in modelling side in terms of efficiency.In this regard, we have presented an automatic floor plan extraction method powered by a novel door detection algorithm.
Despite probable precision limitations for uncommon architectures due to hard-constrained angular regularization, it might be an initiation of a valuable automatization alternative for various applications based on manually extracted models.In fact, more effective primitive regularization is in the reach of this algorithm to eliminate or relax the angular constrains.As a natural next step, we intend to make use of the calculated new line segment configurations in terms of primitive grouping.
Reconnected longer line segments can provide an enhancement in BSP, hence more flexible terms can be employed.
From a broader perspective, we use object information to enhance geometrical and topological models; floor plans in this case.Using semantic information as prior knowledge has well proved to be beneficial in several studies in different research fields.However, in indoor modelling, there is a need to develop new ideas on how to incorporate semantics into a reconstruction framework.In general, previous studies treat semantic information as a top layer of information which is to be extracted only after a complete reconstruction.Essentially, we demonstrate a preliminary study of an alternative sequence.
Our framework can be extended to incorporate different object categories.In future work, we plan to design similar strategies to other structural elements, windows in particular, and improve the overall results.Furthermore, we consider to apply the door detection outcome to define the connectivity and accessibility relations between higher semantic entities, such as enclosed spaces like rooms, and corridors, and make benefit of this information recursively in the geometric modelling process.

Figure
Figure 1.The overview of the method

Figure 3 .
Figure 3.The swinging door concept The generation of different hypotheses is formulated as an analogy to an energy minimization problem where the scoring function consists of a unary and a contextual term:

Figure 4 .
Figure 4. Functions used for scoring in door detection.a) Unary term expresses the likely door width priors in a continuous fashion, b) Contextual term penalizes the unmatching door and wall line segments with respect to distance.

Figure 5
Figure 5. a) Extracted door candidates, and b) selected ones after configuration ranking process.

Figure 2 .
Figure 2. Comparison of Clutter and Occlusion involvement in data in different horizontal slices: a) floor level; b) above floor; c) below ceiling; d) ceiling level Fig 6b shows a floor plan constructed by the presented method.

Figure 6
Figure 6.a) Space partitioning, and b) reconstructed floor plan

Figure 8 .
Figure 8. Door detection evaluation; greens indicate the successfully detected doors, while reds are missing, and oranges are false positives

Figure 9
Figure 9. a) Reference floor plan manually generated from raw laser point cloud; b) 3D illustration of the reference floor plan, c) BSP resulted floor plan without door detection d) BSP resulted floor plan with door detection

Figure 10 .
Figure 10.The atypical corridor which has salient rooms and hollow partitions is recovered only partially successfully.

Figure 11 .
Figure 11.Some doors are scanned twice, both in open and closed positions, generating two line segments for the same door.

Table 1 .
Door detection success rates