UNCERTAINTY MODELING FOR POINT CLOUD-BASED AUTOMATIC INDOOR SCENE RECONSTRUCTION BY STRICT ERROR PROPAGATION ANALYSIS

Accurate digital representation of indoor facilities is a key component for the generation of building twins. 3D indoor scenes are often reconstructed from 3D point clouds obtained by various measurement techniques, which usually show different accuracy characteristics. During the reconstruction process, the uncertainties of data and intermediate products propagate into the accuracy of the vectorized model. Although point clouds-based 3D building modeling has been a hot topic of research for at least two decades, a thorough analysis of error propagation for this problem from a geodetic point of view is still underrepresented. In this contribution, we propose an analytical approach to estimate the uncertainty of 3D modeling results using the analytic approach based on first-order Taylor-series expansion. A general model for the input data is established and the uncertainty expressions of all computed products are symbolically derived. We estimate the uncertainty of 3D data fitting, followed by the derivation of vectorized building parameters and their covariance matrices. The results of the theoretical approaches are tested on real data presenting an indoor scene. The practical example is illustrated, thoroughly analysed, and quantified.


INTRODUCTION
Accurate 3D reconstruction of digital indoor scenes is one of the main components for Building Information Modeling (BIM). Virtual building models are often (semi-)automatically reconstructed from 3D point clouds obtained by various measurement techniques, which usually show different accuracy characteristics. During the reconstruction process errors of the acquired data propagate into the accuracy of a vectorized model. Without the statement of their uncertainties, the final outputs of data processing cannot be reasonably compared with each other or with a reference standard. Especially in the design and development phase, it is needed to estimate the uncertainties associated with a measurement system and modeling method. Such estimation procedures enable the a priori estimation of the achievable accuracy of the data processing chain which is under development (Biljecki at al., 2015). Furthermore, the analysis can also provide knowledge about causes of errors and processing parts that are sensitive to uncertainties.
While automatic 3D building reconstruction from point clouds has been a hot topic of research for at least two decades, a thorough analysis of error propagation for this problem from a geodetic point of view is clearly underrepresented. Research dedicated to quality assessment of the reconstructed 3D models is mostly limited to the comparison against input data or reference models (e.g. Tran 2019; Khoshelham 2020). This fact motivates us to propose an analytical error propagation model based on first-order Taylor-series expansion. We aim at the investigation of how the uncertainties in fitting objects to 3D points propagate towards the positional uncertainty of the reconstructed elements.
Although building structural components can be abstracted in 3D space by various geometric objects, the core idea in most building reconstruction algorithms is to detect planar patches in an input point cloud (Pintore et al., * Corresponding author, 2020). Therefore, the presented approach starts with an analysis of error propagation in algorithms for fitting planes to the sets of segmented 3D points. The estimated plane parameter uncertainties are then used to investigate their influence on the accuracy of the elements which are derived from them. We analyse the following cases: (i) positional uncertainty of the 3D model vertices (computed as the intersection of three planes), and (ii) errors of the 3D line parameters that describe model edges (computed as the intersection of two planes). All the established error propagation models consider the correlations between parameters. The results of the theoretical approaches are tested on a 3D point cloud presenting a single office. The practical example is illustrated, thoroughly analysed, and quantified.

Analytical estimation of error propagation
The error propagation problem can be formulated mathematically as follows (e.g. Heuvelink, 2005): where the function (. ) may represent any operation on the input data. The error propagation analysis aims to determine the error in the output (. ), usually expressed with its variance, given the operation (. ) and the errors in the input attributes (. ).
There are two common approaches to investigate the propagation of errors in computational processes: Taylor series approximation and Monte Carlo simulation. The first approach is analytical and based on the first-order Taylor-series expansion of the process function. It requires establishing mathematical functions that describe the computation process. The alternative approach, Monte-Carlo simulation, is a numerical brute force method based on simulated noise on measurement data. In this contribution, we focus on the analytical modeling of the error propagation to estimate the covariance matrix of a processing result for each step of 3D indoor scene modeling.
Assuming that the input data is represented by the random vector with the covariance matrix and the process is mathematically described by an explicit continuously differentiable function ( ), we aim at the computation of the covariance matrix of the result = ( ) (Clarke, 1998). The Taylor series expansion of ( ) around the expected value ̅ of yields: The first order approximation to the covariance matrix for the estimated vector is thus given by: where ∇ denotes the Jacobian of the function ( ), obtained by computing partial derivatives: . (4)

Uncertainty of 3D plane parameters and derived quantities
A geometric object in 3D space is represented by a set of parameters. During the fitting process, the parameters of the fitted object are estimated by the minimization of a chosen error function. The presented methodological chain starts with fitting planes to the input 3D points by the least-squares method in order to estimate the coefficients of the fitting planes equations and their corresponding covariance matrices. This work is focussed on the reconstruction-based uncertainty, which means that for the time being we assume no measurement errors in the acquired data. A plane in the 3D space is described by a normal vector = [ , , ] and a normal oriented distance , so that for point = [ , , ] on the plane the following relation holds: Considering the notation above, we can rewrite this as: and by assigning = 1 rearrange to: Given a set of > 3 points in the 3D space = ( , , ), = {1 … ), Equation 7 can be presented in a matrix form: and described by: The least square method gives the solution vector : the residual vector for the observations: and the variance factor: where k is the number of minimum required observations.
The covariance matrix of the estimated plane parameters is then computed as: and can be directly used to extract the uncertainties of the estimated plane parameters and their correlations. Additionally, the slope s of the extracted plane is computed using an angle between the plane normal vector and the horizontal plane: The related uncertainty of the plane slope is estimated taking the 2x2 part of the covariance matrix related to , parameters of the fitted plane: and the Jacobian of the function s ( Equation 15): and given by the following relation:

Positional uncertainty of model vertices
In the next part of the presented research, we derive a functional relationship between the covariance matrix of the estimated plane parameters and the parameters of the intersection point. Given three planes , , and their covariance matrices , , , the intersection point p is found by: and , , denotes plane parameters of , , planes, according to equation 6.
Considering the merged covariance matrix of the three planes: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France and the Jacobian of the point coordinate function presented in Equation 18: , ) , the covariance matrix of the intersection point is computed as: providing uncertainties of the derived point coordinates X,Y,Z.

Positional uncertainty of model edges
The uncertainty associated with the parameters of a plane fitted to the input data (c.f. Section 2.2.) is also used to establish a general model of error propagation for the estimation of an intersection line uncertainty. A directional vector of a 3D line generated by the intersection of two planes , is obtained by: Given the merged covariance matrices − , − which are parts (2x2) the covariance matrix for the 3D line directional vector parameters is estimated by: Finally, the inclination of the model edge with respect to the horizontal plane yields: Considering the Jacobian of : the covariance matrix of the model edge slope is computed as:

RESULTS AND DISCUSSION
For the practical application of the derived analytical model of error propagation, we use a raw point cloud generated based on RGB-D data as a part of the Stanford 2D-3D-Semantic Dataset (2D-3D-S, Armeni et al., 2016). The benchmark provides a variety of mutually registered modalities collected in large-scale indoor areas of mainly educational and office use. In our initial experiment, we choose a subset of the data presenting a single office (Fig.1a). The presented methodological chain is applied to a 3D point cloud segmented into planar patches according to the segmentation method presented in (Jarząbek-Rychard and Borkowski, 2016). The input data pre-processed by the segmentation algorithm is visualized in Fig.1b. To avoid singularities in 3D object fitting to the vertical walls, we temporarily align the room layout along the x-axis and rotate the point cloud by 45° around the x-axis and 45° around the y-axis.
Our numerical analyses start with the computation of the uncertainty of plane fitting. The related statistics are collected in Table 1. The numbers present the estimated parameters of each plane (according to Equation 6), together with the values of plane slope that are derived based on them. The average slope error in the test data is 0.0725°. Additionally, we provide the number of points for each segment, and compute a mean absolute distance from 3D points to the plane, which is often used in 3D modeling as an indicator for plane-fitting quality. Figure 2 shows the uncertainties of the estimated plane parameters, presented as a total error of the normalized plane vector and as the error of the parameter . It is clearly visible that the numbers for two planes stand out from the others -plane 7 and plane 8. These are the smallest walls of the test space. Extending the statistics by the visualization from Fig.1, we can infer that the uncertainty of object fitting is inversely proportional to the size of a point segment. This is also proven by the point segment used for the estimation of plane 4, which characteristic is better than the two smallest planes but slightly worse than the rest of them.  In Fig.3 we compare the uncertainties of plane slope estimation against mean residuals calculated between 3D points and the estimated planes. The plot shows that there is no direct dependency correlation between these two indicators. For example, the point segment assigned with plane 6 has large residuals, while its uncertainty of plane fitting remains relatively small.   to the same horizontal plane are nearly the same. We can notice an almost constant offset between the plots regarding floor and ceiling. The lower error values are presented for the ceiling points. This is related to the better quality of the plane fit (cf. Tab.1), which now propagates into the point uncertainty.
The last assessment is related to the propagation of plane fitting errors into the positional uncertainty of model edges generated by the intersection of two planes. Based on the 3D line directional vector and the associated covariance matrix we compute the slope of the model edge (inclination to the horizontal plane) and its uncertainty (Table 3). The mean slope error of all the edges is equal to 0.0741°. The numbers are illustrated in Figures 5-7. Vertical edges present very similar slope errors within the whole group. Although it is noticeable that the uncertainty for edges 5 and 6 are almost twice as large as the others, these differences are not as clear as the differences within both horizontal groups (floor and the ceiling edges). Here the largest slope error (edge 6) is more than 100 times bigger than the smallest one (edge 1 and 3). For all the computed horizontal edges (where one intersecting plane is constant) uncertainty statistics reflect the uncertainty of estimated parameters of the vertical plane used for the generation of the model edge (c.f.

CONCLUSION
This paper gives an insight into error propagation analysis for the estimation of building model parameters, based on the Taylor series approximation. The presented work is an opening step towards establishing a general framework for the investigation of uncertainties in 3D building modeling, which was so far rarely covered. Given the segmented data points, we start with the estimation of an analytical expression for the covariance matrix of the fitted 3D plane parameters, followed by the derivation of error propagation models for each consecutive step of 3D reconstruction. The influence of fitting errors on the uncertainty of the subsequent reconstruction results, computed from the estimated parameters, is thoroughly investigated and quantified. For the presented test data, the analyses reveal the positional error of calculated vertices in a 3D space of 3.7 mm. The mean error [deg]

edge ID
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France of the inclination of model 3D edges to the horizontal plane is equal to 0.0741°. The underlying methodology so far is dedicated to the propagation of uncertainties through the reconstruction process, excluding the influence of data acquisition errors. In future work, we will address this issue and propose a comprehensive error propagation model for the generation of a digital indoor scene. The presented analytical approach will be enhanced by the numerical simulation based on the Monte Carlo method. In the described work, we also assume that the planes are really planar. We plan to verify this hypothesis, using for instance residuals from the input data. Furthermore, we intend to investigate uncertainty distribution to extend the research scope by the applicability of building model regularization rules.