SCAN-TO-BIM OUTPUT VALIDATION: TOWARDS A STANDARDIZED GEOMETRIC QUALITY ASSESSMENT OF BUILDING INFORMATION MODELS BASED ON POINT CLOUDS

The use of Building Information Modeling (BIM) for existing buildings based on point clouds is increasing. Standardized geometric quality assessment of the BIMs is needed to make them more reliable and thus reusable for future users. First, available literature on the subject is studied. Next, an initial proposal for a standardized geometric quality assessment is presented. Finally, this method is tested and evaluated with a case study. The number of specifications on BIM relating to existing buildings is limited. The Levels of Accuracy (LOA) specification of the USIBD provides definitions and suggestions regarding geometric model accuracy, but lacks a standardized assessment method. A deviation analysis is found to be dependent on (1) the used mathematical model, (2) the density of the point clouds and (3) the order of comparison. Results of the analysis can be graphical and numerical. An analysis on macro (building) and micro (BIM object) scale is necessary. On macro scale, the complete model is compared to the original point cloud and vice versa to get an overview of the general model quality. The graphical results show occluded zones and non-modeled objects respectively. Colored point clouds are produced and integrated in the BIM. On micro scale, the relevant surface parts are extracted per BIM object and compared to the complete point cloud. Occluded zones are extracted based on a maximum deviation. What remains is classified according to the LOA specification. The numerical results are integrated in the BIM with the use of object parameters.


INTRODUCTION
A Building Information Model (BIM) is a digital 3D object oriented database of a building.During the past ten years, the use of BIMs has increased rapidly and proved to be one of the biggest game changers in the construction industry.At the moment, the BIM industry is still mainly focusing on new buildings, but there are important movements of both the research community and the laser scanning industry towards BIM for existing buildings, including built heritage (Volk et al., 2014).One of the three typical challenges described in the above literature review article regarding BIM for existing buildings, is "the handling and modeling of uncertain data, objects and relations occurring in existing buildings in BIM" (Volk et al., 2014).These uncertainties include unknown materials, hidden internal element structure, invisible elements, object history, conflicting sources and potential geometric differences between the BIM and the real building.In this paper, we will focus on the last type of uncertainty: what should a client demand regarding survey and modeling accuracy, how should this accuracy be validated and how should this information be integrated in the BIM?When regarding the case of existing buildings and the development of an as-built or as-is BIM, point clouds are often used as a primary source for the digital reconstruction.In the manual scanto-BIM method, the processed point cloud is imported in commercial BIM software and used as an underlay during manual model creation.This process is generally regarded as inefficient, inaccurate and error prone (Thomson, 2016).Several researchers tried to improve this step by introducing process automation and In this article, we argue for more frequently executed geometric quality assessments of as-built and as-is BIMs.A more standardized framework around scan-to-BIM accuracy -including methods for deviation analysis and incorporation of the outputs in the BIM -is needed in order to make the final BIM more reliable and thus reusable by other stakeholders.Such a deviation analysis can also be useful for the evaluation of building element deformations (e.g. a deformed dome versus an ideally modeled dome from the BIM) (Quattrini et al., 2015).Geometric quality assessment is not only useful for manually or semi-automatically constructed BIMs, but also for automatically created models based on point clouds, e.g. for comparing the results of different scan-to-BIM algorithms and to convince stakeholders from other domains of the reliability of the automatically created BIM.
Our research methodology is based on three successive parts: 1. First, the review of the available scientific literature and standards on the subject of geometric quality assessment of BIMs based on point clouds is presented.2. The following part elaborates on the suggested approach for the geometric quality assessment.3.In the last part, the approach from part two is applied on a case study and the feasibility of the initial proposal is evaluated.

REVIEW OF LITERATURE AND STANDARDS
The number of specifications on BIM relating to existing buildings is limited.A starting point in literature was provided by (Anil et al., 2011, Anil et al., 2013).The focus in these successive papers is mainly on the detection and classification of modeling errors or errors that appeared during the data collection or postprocessing phase.In the second paper, the authors demonstrate that by executing a deviation analysis of a point cloud with a BIM -instead of comparing independent physical measurements to dimensions of BIM objects -roughly six times more errors are detected in 40% of the initially needed time.Some researchers executing scan-to-BIM case studies already mention deviation analysis of the assembled BIM versus the used point cloud as a part of their methodology (Lo Turco et al., 2016, Adami et al., 2017).If such an analysis is executed, it is often done with the use of certain commercial BIM plugins to get an idea of the general deviations of certain selected surfaces by using a gradient color map.
We also investigated several standards and guidelines regarding BIM for existing buildings.The Finnish Common BIM requirements (COBIM) series 2 guideline from 2012 (Olof Granlund Oy et al., 2012)

Introduction
The evaluation of the 'measured accuracy' is a relatively straightforward process (Guidi, 2014).On the other hand, the deviation analysis of a BIM based on a point cloud ('represented accuracy') can be executed in several different ways depending on (1) the used mathematical model for deviation calculation, (2) the density of the compared point clouds and (3) the order of comparison (the point cloud to the BIM, or the BIM to the point cloud).If a point cloud is compared to a mesh, no local mathematical model has to be calculated because there is already a global model (the mesh).In some cases, when two point clouds are compared, a local mathematical model can be used to improve the results of the deviation analysis.
The results of the deviation analysis can be represented in multiple ways: graphical with the use of 2D or 3D color maps (binary or gradient, signed or unsigned) (Anil et al., 2011) or numerical (e.g. % of points within a certain LOA range).Many of the available semi-automatic scan-to-BIM plugins can calculate and visualize deviations graphically, but they all use different methodologies and lack the ability to give a precise numerical indication of the accuracy per BIM object.

Macro and micro scale analysis
In the proposed solution, the analysis is divided into two parts: the macro scale (the complete building, or building parts such as building stories) and the micro scale (individual BIM objects, e.g. a certain door).The first scale is used to quickly localize occluded zones (point cloud as reference) and non-modeled elements (model as reference), and zones with serious deviations and modeling errors (see Figure 1 in section 4).The micro scale analysis can be used for a detailed comparison, including a meaningful numerical indication of the geometric accuracy of certain BIM objects.
The macro scale deviation analysis is only useful for producing graphical results of a two-way comparison between the model and the original point cloud.To make the evaluation of model geometry versus original point cloud possible, it is necessary to subsample points on the model surface with at least the same density as the mean density of the original point cloud; the result is called the 'model point cloud' in the remaining of this text.The two entities are compared in both directions without using a so-called local mathematical model.This speeds up the calculation and results in a clearer graphical output.It is not feasible to extract out an exact numerical indication of the general model quality at once because the complete model is compared to the complete original point cloud.Because of the occluded zones and/or non-modeled elements, there is typically an underestimation of the number of points falling within a preconceived LOA deviation range.
After being assured that there are no serious modeling errors found during the previous macro scale analysis, a more detailed study per BIM object can be executed (micro scale): 1. First, the relevant (parts of) surfaces of each BIM object, being the ones that are measured during the data acquisition phase, are selected per room (incl.outside environment).In other words, the internal structure or other not visible parts of modeled objects are excluded because they are modeled based on other information (e.g. a 2D CAD plan, product data sheets, etc.) or assumptions.Onto each collection of relevant BIM surfaces, an equal density of points is subsampled.The point density should be at least the same as the mean density of the original point cloud.This results in socalled 'model point clouds'.If the density of the 'model point cloud' were lower than the mean density of the original point cloud, there would be a noticeable spread on the results.The original point cloud will be used as a reference during the comparison, and is thus not segmented in smaller parts.2. Because there can always be some occluded areas (missing data in the point cloud), it is necessary to segment the 'model point clouds' in occluded and non-occluded parts.
In this work, the segmentation is executed using a certain maximum deviation: all points that have a larger deviation are supposed to be located in occluded zones.Care should be taken to assure that the excluded points do not include points from BIM elements that are not modeled accurately enough.From our first experimental results, a value between 25 and 50 mm seems acceptable for most manually modeled BIM objects.The standard euclidian distance should be used to calculate the deviations that will be used to divide the 'model point cloud' in an occluded and non-occluded part.If a local mathematical model were used to estimate a certain surface through the original point cloud, an underestimation of the occluded part would be measured (see also section 4.3 for more details).3. Every 'model point cloud' is compared to the original point cloud, which is the reference.The remaining points can then be classified per LOA range.This makes it possible to calculate the percentage of acceptable points on the measured object surfaces per room.Note that with a maximum deviation of 50 mm to exclude occluded areas, LOA10 (lower range of 50 mm) will contain zero points.Besides the classification in LOA ranges, it is also useful to calculate the minimum deviation distance for 95% of the points ('95% upper limit').Optionally information such as mean deviation and standard deviation (σ) can also be calculated.4. Subsequently, if the number of points that are located in occluded zones is known, it is also possible to make an estimation of the survey coverage per BIM object and to decide if this is acceptable or not. 5.In a final step, the total values are calculated if an object is located in multiple rooms.These values indicate the 'represented accuracy' BIM object according to the LOA specification.

Results integrated in BIM
The 3D results of both macro scale comparisons can be simply integrated in the BIM as linked point clouds, colored according the calculated deviation value (Adami et al., 2017).
The earlier described numerical results can be stored in the BIM as object parameters (see Table 2 for a logical breakdown of the different results).The raw results from the micro scale analysis are either expressed as a surface area, as a number of points or as a deviation distance ('Maximum deviation for occlusion' and '95% upper limit').Percentage values can be calculated and prove to be very useful when comparing multiple BIM objects of different sizes.
To keep the number of geometric quality assessment parameters per BIM object reasonable, we propose to only store the parameters from Table 2 that are marked in bold, and only as percentages (except 'Total' as area, and 'Maximum deviation for occlusion' and '95% upper limit' as deviation distances).This results in a number of quality assessment parameters p per BIM object if it has parts in n rooms (including outside environment), given by equation 1.
It is of course still possible to visualize the results of the detailed analysis of individual BIM objects in a gradient and/or binary color map and to integrate them as colored point clouds in the BIM.

Multiple surveys and point clouds
Some constructions will be surveyed multiple times over their lifetime, for example to measure changes in deformation.Other Raw parameter data types: a surface area ; b number of points ; c deviation distance Table 2. Calculated quality assessment parameters of a checked BIM object with parts located in n rooms.For every parameter (except 'Total', 'Maximum deviation for occlusion' and '95% upper limit') a percentage value can be calculated.The parameters indicated in bold are added as percentage parameters to the BIM object, except 'Total' (area), 'Maximum deviation for occlusion'(distance) and '95% upper limit' (distance).
buildings will have parts that are modeled using point clouds from different sources.In case of a saddle roof for example a drone with a camera will be selected over a TLS.The drone images will be processed by photogrammetric software to calculate a point cloud, which can be used as a basis for the BIM modeling.In most cases, it is expected that by far the largest part of the building will be surveyed with a TLS, except complex roofs and other hard to reach places (very narrow corridors, complex staircases, etc.).
In order to differentiate between these multiple surveys (and resulting point clouds), it is necessary to add some type of unique survey identifier to the quality assessment parameters of the BIM objects.Extra metadata (survey method, equipment, survey company, date of survey, scan resolution, etc.) about the survey and used processing methods can also be stored in the BIM, ideally on project level.

Introduction
The case study consists of a 3600 m 2 university building with a single floor (mostly classrooms and corridors) and a flat roof.
A BIM was modeled manually -starting from a processed, uncolored point cloud -using Autodesk Revit 2017 BIM software.This set of points was measured using a Leica P30 Terrestrial Laser Scanner (TLS), operating at a scan resolution of 6.3 mm at 10 m to assure a dense point cloud.The outputs (a point cloud and a derived BIM) had to satisfy a relative LOA30 (Level of Accuracy), meaning that respectively the relative measured and represented accuracy should be maximum 15 mm (specified at the 95% confidence level) (U.S. Institute of Building Documentation, 2016).
The check of the measured accuracy was performed with the use of an accurate, dense network of control points, surveyed with a total station.The represented accuracy was checked by the authors following the suggested approach as defined in section 3. We used the open-source CloudCompare software (version 2.8.1), to compute and analyze the deviations between the point cloud and the BIM.

Execution of macro scale analysis
For the first part of the analysis, the whole model was exported as one '.obj'file, containing all surfaces of all BIM elements (floors, roofs, ceilings, walls, beams, columns, stairs, doors and windows).This mesh was then subsampled to a 'model point cloud' and compared to the original point cloud without using a local mathematical model.The results of this general comparison are visible in Figure 1.

Execution of micro scale analysis
For the second part of the quality assessment, a plugin with the Autodesk Dynamo visual programming tool was developed.This Dynamo script allows to extract the relevant surfaces (the ones that can be visible) of the Revit elements (Figure 2).The material of the surface is also taken into account, to immediately filter out modeled glass surfaces.Objects that contain glass will typically contain occluded areas in the point cloud because a standard TLS cannot measure surfaces of this material.Each collection of surfaces was combined in one, separate '.obj' file per Revit BIM object.
The In Figure 3 (bottom), one can visually assess that a maximum deviation of 25 mm for non-occluded parts is acceptable.A small part of the points classified as occluded are actually part of a minor modeling error (the front corner of the floor is modeled with a sharp edge instead of a radius fillet, see red dashed rectangle), while other points are classified as non-occluded while they actually are (see white dashed polygons).A local mathematical model should not be used to extract non-occluded areas from te 'model point cloud', as this results in an underestimation of the occluded area (see Figure 3 (mid)).
In a following step, the part of the 'model point cloud' that was classified as non-occluded, was again compared to the original point cloud.At this moment, the local mathematical model called 'least square plane' with six nearest neighbors is used to compare the 'model point cloud' to the original point cloud.By using the 'least square plane' local model, the calculated deviations are closer to zero, compared to a situation without local model.With the use of the earlier described method, we can now divide the results of our comparison into five groups: the first contains the points that have a deviation larger than the maximum deviation for occlusion (here set at 25 mm), while the remaining points were classified as respectively LOA50, LOA40, LOA30 or LOA20 (see Table 1).Note that LOA20 contains only the points between 15 and 25 mm, if the described approach is used.LOA10 contains no points because the complete LOA10 range is classified as occluded.Figure 4 shows the locations of all the classified points of the studied floor object when no local model is used for the LOA classification (top) and when a local mathematical model is applied (bottom).From the numerical results collected in Table 3 the two calculation methods for LOA classification described earlier are compared.From the numerical results, it is clear that the floor with ID 1191172 does not meet the requirements of the LOA30.The results of a calculation with a local mathematical model are clearly better: 92.06 % of the relevant floor surface complies to at least LOA30 when no local model is used, compared to 94.03 % when a local model is used.Around 17.25 % of the relevant object surface is occluded.A large part of this occlusion however is located on the floor riser, which is almost entirely classified as LOA20.
As a conclusion, we can assume that the top surface of the floor complies to LOA30, but the riser does not.

Integration of results in BIM
In a final step, the results of the macro and micro scale deviation analysis are integrated into the BIM as mentioned in section 3.3.For the macro scale analysis, the original point cloud and 'model point cloud' are RGB colored in CloudCompare according to their deviation value and color display settings.In a next step, both point clouds are exported as '.e57' point clouds, which are, prior to import in Revit, converted to an indexed format that can be read directly by the BIM software ('.rcs' or '.rcp'), as shown in Figure 5.
The point clouds in Revit can be cropped easily by using selection/section boxes around objects of interest.One of the downsides regarding this workaround, is that the original RGB values This results in data duplication as only the color (RGB) will change between the point cloud files.Regarding the micro scale analysis, only numerical information is included in the BIM as the colored point cloud from the macro scale provides sufficient qualitative information.

CONCLUSIONS
A standardized method for geometric quality assessment of BIMs based on point clouds -as part of the scan-to-BIM process -will improve the reliability and usability of the model.The Level of Accuracy (LOA) specification is used here as a starting point.Multiple methods, with potentially different results, exist for the calculation of the deviations, the type of output and the integration of the results with the BIM.In the proposed approach, it is suggested to perform a quality assessment on a macro scale, followed by an analysis of the individual BIM elements (micro scale).
On macro scale, it is necessary to perform a comparison in two directions (point cloud versus BIM and BIM versus point cloud).3D gradient color maps are useful as an output here, to detect occlusions, non-modeled elements and/or serious modeling errors.These results can be integrated into the BIM by using linked point clouds, RGB colored according to the calculated deviation.BIM modeling and viewing software however should not only be able to view point clouds, but should also allow a broader range of view settings in order to efficiently represent the results of previous deviation analyses.
Regarding the micro scale analysis, we first developed an initial methodology for extracting the relevant (parts of) surfaces with the use of a script implemented in Dynamo for Revit.The current algorithm however cannot find the side jambs of BIM objects embedded in Walls, Floors, Ceilings and Roofs.In the following step, the surfaces per BIM object are subsampled to a point cloud with a density equal to at least the mean density of the original point cloud.Occluded zones on the object surface are detected with the use of a maximum deviation range, calculated without a local mathematical model.In real world situations, it can be difficult to precisely set the maximum deviation range to classify the occluded parts of 'model point clouds', as they often interfere with modeling errors.The limit for occlusion can be set to a higher value to include these modeling errors, which will typically result in a slight underestimation of the object's ac- curacy.Finally, the numerical accuracy rate of every individual BIM element is determined via a deviation analysis using a local mathematical model.The numerical results of the micro scale analysis can be integrated in the model using BIM object parameters.These parameters allow for a quick visualization and BIM database querying according to the geometric accuracy and occlusion rate of its objects.With the integrated results of both the macro and the micro scale analysis in the BIM, it is possible for the next user to quickly check if the geometric accuracy of the model complies with his or her specific needs.

FURTHER RESEARCH
In our research, we only applied the suggested macro and micro scale analysis on a BIM in the Revit environment.In a broader BIM environment, the neutral IFC (Industry Foundation Classes) BIM format of buildingSMART should be taken into account.A similar algorithm as described in this paper can be developed to extract relevant (parts of) surfaces of BIM objects stored in the IFC format.The current Dynamo script for extracting the relevant (parts of) surfaces per BIM object can be improved to also include the side jambs of Windows and Doors embedded in other room bounding elements.
Future research also includes the development of more accurate algorithms to detect occlusions, which can result in a faster and more precise accuracy calculation of BIM objects based on point clouds.The hierarchical breakdown of the quality assessment parameters on micro scale can also be extended to include the orientation of the relevant (parts of) BIM object surfaces per room.
In this way, specific local modeling errors, can be detected faster and more accurately.In CloudCompare's proprietary file format and several other point cloud formats such as '.pts' and '.pcd' this information is efficiently stored as a scalar field parameter per point.It would be more efficient to integrate point clouds in a similar way into the BIM, instead of linking a RGB colored point cloud.
Further implementations should also focus on the automation of the quality assessment process as a whole.If a BIM contains for example 100 objects, 100 micro scale analyses will have to be executed including deviation calculation, segmentation, extraction of qualitative and quantitative results and integration of these results in the BIM.

Figure 1 .Figure 2 .
Figure 1.Results from CloudCompare (macro scale) -focus on classroom 31 and corridor 3 Top: 'model point cloud' as reference (BIM compared to point cloud), non-modeled objects are marked Bottom: original point cloud as reference (point cloud compared to BIM), occluded areas are marked (deviations in [m])

Figure 4 .
Figure 4. Results from CloudCompare (micro scale) -floor with ID 1191172 -original point cloud as reference (BIM compared to point cloud) -no local model for classification of occlusionmaximum deviation for occlusion = 25 mm Top: no local model for LOA classification Bottom: local model for LOA classification Colors according to LOA class, grey is classified as occluded (deviations in [m])

Figure 5 .
Figure 5. Results of macro scale analysis imported in Revit as colored point clouds Top: colored 'model point cloud' Bottom: colored 'original point cloud'

Table 3 .
Total a = 24.62 m 2 Calculated quality assessment parameters of floor with ID 1191172.Parameters in bold are added to the BIM floor object -if measured by the TLS -are overwritten by the results of the deviation analysis.In addition, a deviation color legend (such as available in CloudCompare) cannot be created as the BIM software has only access to the RGB values.If the user wants different visualizations of the deviations (e.g.gradient and binary), one point cloud for each visualization should be created in Cloud-Compare and converted to a point cloud format readable by Revit.