GENERATION OF INDOOR POINT CLOUDS EXPLOITING GEOMETRIC SYMMETRIES AND REGULARITIES

: 3D building modeling is becoming an important support in civil engineering, architecture and cultural heritage applications. Despite static laser scanning can be considered as the state-of-the-art in such kind of applications, mobile mapping techniques can be considered as a suitable alternative to quickly gather geospatial information. Outdoor mobile mapping can be considered as a mature technique, which takes into advantage of the Global Navigation Satellite System (GNSS)-laser scanning information fusion. Instead, indoor mobile mapping is typically more challenging: the unavailability of GNSS makes the mapping system rely either just on the inertial navigation system, or on some control points. A drift in the navigation solution, and consequently in the 3D reconstruction, is typically visible after a while in the former case, whereas the use of other surveying instruments is required in the latter. Thiswork aims at exploiting geometric characteristics of the buildings, such as symmetries and regularities, to reduce the drift effect in indoor mobile mapping, in particular when dealing with affordable systems. The proposed approach is based on the segmentation of the point clouds acquired with a time of ﬂight camera (ToF), detecting in particular vertical planar surfaces. It is well known that aligning planar surfaces can be a viable way for reducing the drift in this kind of applications. Nevertheless, this paper aims also at investigating the use of geometric symmetries to such aim. The proposed approach is tested on a case study, a building of the University of Padova, whose reconstruction was produced by an ad hoc affordable mobile mapping


INTRODUCTION
Outdoor and indoor building 3D modeling is becoming a fundamental task in the construction, civil engineering and architecture sectors. In particular, the generation of semantic models has recently attracted a lot of attention in these fields.
The automatic generation of large scale semantic models, such as CityGML, has already been considered in the literature by combining both the use of geometric information and machine and/or deep learning techniques (Özdemir and Remondino, 2019, Lin et al., 2019, Masiero et al., 2019a. Similarly, the generation of a building model, for instance as building information model (BIM, (Tucci et al., 2019)), also requires the segmentation and recognition of different parts of the buildings: clearly, both outdoors and indoors parts should be taken into consideration in such semantic segmentation process.
BIM model generation starts from the acquisition of a 3D point cloud describing the building of interest, then such point cloud is segmented in different parts, which finally are classified and semantic/geometric information is extracted and inserted in the BIM description of the detected objects (Bassier et al., 2019, LeCun et al., 2015, Matrone et al., 2019, Barazzetti et al., 2015, Pierdicca et al., 2020.
Building segmentation methods can obviously take advantage from the geometric priors on such kind of environments: regular geometric features, such as planar surfaces, often include * Corresponding author most of the points in the point cloud. Such geometrically regular objects can be quite easily segmented by means of procedures exploiting the local geometric characteristics of the point cloud (Maalek et al., 2018, Belton and Lichti, 2006, Vosselman et al., 2004.
Actually, most of the buildings have some geometric symmetries and/or quite repetitive patterns: for instance, different storeys of the same building often have similar shapes. Furthermore, even several copies of the same objects are often present in the same building, e.g. room doors. (Previtali et al., 2018, Previtali et al., 2013 proposed to take advantage of such geometric regularities in order to improve the performance of the procedure adopted for segmenting and extracting information from the point cloud.
Similarly to such approach, in this work the use of geometric regularities is exploited for improving the indoor mobile mapping results which can be obtained by using affordable mobile mapping solutions (Masiero et al., 2019b, Dabove et al., 2018, Poiesi et al., 2017.

DATA ACQUISITION, INITIAL ALIGNMENT AND VERTICAL SURFACE DETECTION
Time of Flight (ToF) and RGB images of a level of a building of the University of Padova ( Fig. 1) have been collected and aligned by using a in-house simultaneous localization and mapping (SLAM, (Leonard and Durrant-Whyte, 1991)) approach (Masiero et al., 2020), based on the iterative match of visual features in the last two frames (Fig. 2).
On the one hand, such approach is computationally efficient because information of just two frames need to be considered at each step. On the other hand, this procedure limits the overall potential of visual based localization, hence it can have an impact on the final accuracy of the produced 3D reconstruction.  It is well known that the alignment between point clouds acquired in different time instants can be improved by taking into account of the planar surfaces which are in common in such different frames (Forstner andKhoshelham, 2017, Nguyen et al., 2007).
Planar surface detection implemented in this work is based on the use of random sample consensus (RANSAC, (Fischler and Bolles, 1981))-based techniques. Such plane detection algorithms can also be compared with further information based on the local assessment of the point cloud geometric characteristics (Maalek et al., 2018).

SYMMETRY DETECTION
The problem of symmetry detection is faced as a geometric problem iteratively solved during the data acquisition, if executed in real-time, just after the geometric feature detection step. The implemented approach is an extension of (Cicconet et al., 2017).
The assessed symmetry is used in the next section to correct the alignment of the planes which can be matched within a certain threshold error with the corresponding symmetric ones.
This section considers the problem of detecting mirror symmetry on a collected dataset, and, in particular, on a set of detected planar vertical surfaces.
First, a set of points is characterized by a mirror symmetry if there exists a hyperplane such that the mirror reflection of the set of points with respect to such hyperplane produces a new set of points equal to the original one (Cicconet et al., 2017).
Let v be the unit vector corresponding to the normal plane direction, and p0 a fixed point inside of the mirror reflection hyperplane, then the mirror reflection p of a point p can be expressed as follows (Cicconet et al., 2017): The above equation can be equivalently expressed in matrix form: Furthermore, the following can be easily proved: Fact 1 Let (a, b, c, d) be the parameters describing a plane in R 3 , i.e. [a b c d][p 1] = 0 for any point p on the plane. Let also (a , b , c , d ) be the parameters describing the mirror reflection of such plane obtained by applying the reflection described by (v, p0) (or, equivalently, Sv, q, defined as described above). Then, The obtained parameter values shall be normalized, if needed.
The latter equation can be slightly manipulated as follows: In the case study of interest in this work, only vertical planar surfaces are considered, hence the problem can be reduced to that of determining a mirror symmetry in 2D.
Given two vertical planes, deriving the mirror reflection between them is quite easy: considering it as a 2D problem, it is sufficient to compute p0 as the intersection between the two horizontal lines (corresponding to the intersections of the planes The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China with the horizontal surface), and the mirror reflection direction given by the combination of those of the original planes.
Instead, when dealing with more matched planar surfaces, an approximate solution to the problem of computing the mirror reflection is as follows: first, combine the equations (5) Then v is obtained as the first column of U .
Finally, since f = −Svq = 2vv p0, p0 can be computed as follows: where (vv ) † is the pseudo-inverse of (vv ). It is worth to notice that (vv ) is not a full rank matrix, hence p0 can be estimated up to an uncertainty on the direction related to the nullspace of (vv ), which however corresponds to an uncertainty on the position of p0 on the reflection plane.
In practice, if the number of detected planes is small, mirror reflection assessment can be conducted by thoroughly checking all the possible plane matches and checking which one of the estimated mirror reflection achieves the highest number of consensus from the remaining planes (and if the associated fitting error is sufficiently small to be considered as a real mirror reflection). Alternatively, a RANSAC-based procedure can be implemented if the number of possible plane matches is high.

ALIGNMENT CORRECTION
The rationale of the strategy adopted in this work is that the building regularities, i.e. the symmetries, can be used to correct the alignment of the planes which can be matched within a certain threshold error with their corresponding (symmetric) ones.
In particular, a basic assumption here is that the considered building is characterized by a main symmetry, which is then used to for planar surface alignment improvement.
As commonly done with dead reckoning approaches, localization error of the original SLAM algorithm is assumed to progressively increase, becoming eventually quite large to be necessary to be corrected (if loop closures are either not considered or not present in the dataset).
Hence, those plane couples associated to image frames acquired later are assumed to be the ones affected by the higher level of error, and hence to be corrected.
Consequently, the correction is applied by progressively checking the registration of plane couples, i.e. if it is compatible with the currently estimated symmetry transformation.
When matched plane couples do not properly satisfy the above condition, then a rigid transformation is determined in such a way to make the currently considered plane couple/s be compatible with the estimated symmetry. Such rigid transformation is eventually applied to all the successive camera poses as well.

RESULTS
The proposed procedure is applied to the ToF-based data of the University building shown in Fig. 1. According to Fig. 1 and Fig. 3, the mapping platform started acquiring scans close to the top-left corner of the level shown in such figures, then it moved around the level in clockwise direction.
The algorithm for the detection of building symmetries initially detects just the symmetry between the two sides of the first considered corridor (the initial platform track and the symmetry detected correspondingly are shown in red and yellow, respectively, in Fig. 3(a)). Then, once the platform moves on both the left and right sides of the level, as shown in red in Fig. 3(b), an assessment of the symmetry on the overall building level is achieved (solid yellow line in Fig. 3(b)).Such estimate is also improved when more data are available (Fig. 3(c)).
When the localization and mapping error increases the fit between the detected symmetry and the assessed platform positions becomes insufficient to exploit the new data for improve the symmetry assessment. Instead, in the second part of the acquisition (bottom-left part of the trajectory shown in Fig. 3(c)) the estimated symmetry is used to improve the localization ability of the system, as previously described.
It is worth to notice that a key role in this procedure is played by the thresholds used to determine whether either new data shall be used to for the symmetry assessment, or the latter shall be used to improve the platform localization and mapping.
Applying this procedure to the considered case study the final orientation error (the final acquisition is performed approximately with the same position and orientation of the initial one), when completing the acquisitions on the level, was reduced from 6.6 deg to 0.9 deg. Furthermore, the 2D position error was reduced from (0.96 , 1.09) m to (0.06 , 1.08) m (expressed in the local reference system used in Fig. 3).
From the obtained results, it is quite clear that the implemented procedure allowed to notably reduce the orientation error, and the position error for what concerns the x direction, while having negligible effect of the positioning error on the y direction.
For what concerns the effect of the proposed approach on the positioning results latter, it shall be mentioned that most of the planes that contributed to the assessment of the detected overall building symmetry have normals close the x axis direction, hence they provide alignment information mostly along such direction. Instead, those planes with normals approximately aligned with the y axis direction are on the initial part of the data collection (top of Fig. 3(c)), hence providing few information on the alignment along the y direction in the second part of the trajectory.
The caveat is that the procedure may not be efficient on all the directions if the symmetry-corrected matched planes are approximately oriented in the same way, as in this case. Instead, it shall work better if symmetry-based corrections are applied The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China to matched planes oriented in different ways. Hence, improvements shall be made to impose such kind of condition in order to obtain more reliable results on all the directions, if possible.

CONCLUSIONS AND FUTURE WORKS
This paper presented the current state of a project aiming at detecting symmetries in (indoor) 3D reconstructions and exploiting them as a regularization factor, i.e. to improve the geometry of the 3D reconstruction itself, if needed.
The proposed approach has been applied on an in-house developed SLAM algorithm, with the specific goal of reducing the localization and mapping error, which typically increases during the acquisition, as long as loop closures are not considered or not present in the dataset.
The rationale is that since buildings, and more in general human structures, are often characterized by strong geometric regularities, and in particular by symmetries, then such regularities can be exploited to reduce the SLAM error.
Hence, the implemented procedure aimed at properly assessing symmetries in the increasing building map, and then exploiting such information as constraints to reduce the localization (and mapping) error even when loop closures are not present, and hence partially reducing their need.
In the currently implemented procedure, just one overall building symmetry is assumed to be predominant, and its the only one that is then exploited.
The obtained results are quite encouraging, in particular they allowed to properly correct the camera orientation in the last part of the acquisition in the considered case study, remarkably improving both the localization and mapping performance of the system (without exploiting loop closures).
Such result has been obtained by properly assessing an overall symmetry on the building, and then imposing that all the matched vertical planes satisfy it, hence properly adapting the original SLAM solution to make it compatible with such symmetry.
Despite the overall effect of the implemented procedure is quite reasonable, the considered approach shall be improved to: • improve the position correction, • being usable to detect and exploit local symmetries, and other geometric patterns and regularities, in certain parts of the building (hence being applicable right locally in such areas), • improve the harmonization between the camera poses before and after the correction to avoid discontinuities and artifacts.
In particular, the regularities derivable by other geometric features shall be better exploited in our future developments, easing the improvement of localization and mapping tasks all over the surveyed area, even locally, if possible.
Furthermore, the proposed procedure should be validated on other indoor datasets. Some datasets are already available to test data processing of indoor environment models, e.g. "Colored Indoor Laser Scanning Dataset" from the "Multisensorial Indoor Mapping and Positioning Dataset", and a dataset from BIM feature extraction dataset from the ISPRS Scientific Initiatives 2019 -"ISPRS benchmark on multisensorial indoor The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVI-M-1-2021 28th CIPA Symposium "Great Learning & Digital Emotion", 28 August-1 September 2021, Beijing, China mapping and positioning" (MiMAP)   , Wang et al., 2018. The proposed method can be considered as a potential improvement in the process for deriving 3D reconstructions with affordable devices, whose outcome might be affected by a significant error.
A second step, which should be considered in our future work, is represented by the semantic interpretation of the generated data, in order to ease the generation of semantic models, such as a BIM, which may also have positive effects on the implementation of effective strategies for properly exploiting building regularities. Deep learning methods shall be investigated to support the semantic interpretation of the collected data.