AUTOMATIC GEOMETRIC INSPECTION IN DIGITAL FABRICATION

: Additive Manufacturing in construction allows to create complex objects of different materials. Accordingly, an appropriate co-registration and comparison between the printed object and its digital model is needed for several purposes: quality control (QC) to ensure that tolerances are maintained and the realisation of a digital twin which holds the actual geometry. In this paper, we introduce an automated robotic data capturing and direct co-registration method. That is, a terrestrial laser scanner (TLS) is mounted on a robot to be moved freely in the printing room. In addition, various strategies are employed for quality control. The scanner is also mounted on the tripods, to validate the accuracy of our data capturing and co-registration solution. Furthermore, the experiment on real data is conducted for two different objects: a shotcrete printed object and a wax material object. Our experiments revealed that there is almost no influence on the scanner position accuracy compared to classical setup (mounting the TLS on tripod). The scanner position accuracy when mounted on tripods is 0.76 mm and the accuracy achieved by mounting the scanner on the robot is 1.03 mm. From the simulation it is noticed that C2M cannot detect missing extruded parts. To cope this problem, we used C2C and M3C2 algorithms. For real data scenario, without edge trimming and surface finishing it is challenging to interpret the data directly. However, M3C2 provided better results but it requires parameters tuning. In case of the surface finished object C2M and M3C2 algorithms have almost similar results.


INTRODUCTION
The traditional construction process is performed in sequence, that is, planning and the actual construction are decoupled and feedback loops are cumbersome and lead to additional cost and delay. In addition, concrete object production, depends on a previously formed mold which adds cost and decreases flexibility in shape and it is not sustainable while it is tailor-made for individual parts. With Additive Manufacturing in Construction (AMC) the design and manufacturing of the production process along with the higher possible complexity of the fabricated components is inherently coupled . Building components will not be produced by several different trades as erecting a wall, cutting slots to lay electrical cables, plastering, etc. Rather, AMC allows to print previously modelled objects in unprecedented and novel ways, pushing the boundaries of what was previously possible in construction (Kloft et al., 2021). By seamlessly embedding the process of the design directly into the printing process, ever more complex and free form objects can now be realized (Kumar et al., 2017). To ensure that a robust process is followed and that the printed object adheres faithfully to the designed model, continuous data capturing and inspection of the process is required . Quality control would ensure the original vision of the designer comes to light and that the combination of components into objects actually would work. Since traditional quality control in construction relies heavily on highly skilled operators, and demands considerable time and cost, the importance of the automation in quality inspection is s attracting the attentions both in research and industry (Wang et al., 2018). In this paper we will introduce an automated workflow for quality inspection within a digital fabrication lab.
In QC workflow, the problem of robust and accurate coregistration of the printed object, reflected in this case in a TLS-based point-cloud, with as-designed model needs to be solved first. Most common approaches for co-registration of a point cloud to a model or another point cloud are namely: ICP-like distance minimization algorithms, or direct coregistration. The ICP algorithm suffers from the local minima problem and it needs a coarse registration between the two objects. However, coarse registration is usually performed manually or by prior knowledge concerning the relative location of both data instances. Direct co-registration, in contrary to ICP, is of advantage since it is independent of the actual object geometry and problems imposed by an iterative approach are not expected (Mawas et al., 2022).
In this research, we utilize a fully automatic co-registration of a TLS-based point-cloud to the model space. In order to acquire data from all parts of the printed object as well as having an automated data capturing, the scanner is mounted on a robot which is also used during object production as it is depicted in Figure 1. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France The pose of the TLS is known for each station, since the robot coordinate system and its transformation to the model space is known. Hence, the as-built point cloud and asdesigned model can be compared instantly. The main focus of this paper is on co-registration and automatic quality inspection of digitally fabricated components. The very first essential step for inspecting the conformity of the fabricated component with its digital twin is co-registration of the asbuilt data and as-designed model. As shown in Figure 1, the TLS is mounted on the robot to facilitate the scanning of the object from proper viewpoints, efficiently. This is especially useful for scanning the parts of the objects which may not be visible from classic TLS stationing on the tripod. On the other hand, since the position and attitude of the robot is known in the as-designed model coordinate system, we could directly bring the TLS measurements, i.e. as-built data to the component's design model. This automatic co-registration of two datasets, enables more reliable quality inspection of the printed objects with respect to its digital counterpart. Details on the computation of transformation parameters between the coordinate systems are given in (Mawas et al., 2022).
In this paper we will introduce the concept of our direct coregistration by mounting the scanner on the robot. In addition, algorithms for direct QC will be investigated as well. Section 2 will introduce the related work on coregistration methods as well as various algorithm for direct comparison between two datasets. The co-registration method as well as the analyzation of C2C, C2M, and M3C2 on simulated data will be given in Section 3. Experimental setup and results of a real shotcrete wall and a wax object using our co-registration approach will be discussed in Section 4. Lastly, conclusion and further research directions will be addressed in Section 5.

RELATED WORK
Quality control generally is categorized from the methodology perspective as follows: i) direct comparison with the as-designed model, and ii) comparison of the corresponding features between the as-printed object with their correspondences in the as-designed model such as the edges or relative dimensions (Guo et al., 2020).
QC approaches can also be classified based on their employment throughout the printing life cycle, i.e. online, layer-wise, preassembly (surface finishing), and assembly verification. Online QC inspects one of the robot parameters, such as the height of the nozzle (Wolfs et al., 2018). Layerwise QC captures the printed object after every printed layer and compares the data with the designed model . While preassembly QC is proceeded after the surface finishing (Wang et al., 2018;Xu et al., 2020). Lastly, assembly verification QC is to verify whether the whole assembly process is within the defined tolerances . This classification is crucial since it covers the different phases of the AMC life cycle. It reveals the importance of the robot's parameters, online inspection, and their influence on the deformation of the printed object. Moreover, after printing inspection aims to ensure that the object's deviations meet the tolerances. Xu et al. (2020), provided an overview of the different geometric aspects that could be inspected through the quality assessment process. These include e.g. flatness, symmetry, position and perpendicularity, etc. The dimensions of the object and its tolerances are also two critical parameters whose deviations from the as-designed model must be monitored. Wang et al. (2018), extracted the geometric features as edges from TLS data in order to compare them with their corresponding features in the as-designed model.
Utilizing a direct comparison between two datasets during the printing process is useful to obtain the deviation information between the current state of the printed object and its digital model. This is important due to the fact that the object has a rough shape and no clean edges are formed yet. This is seen clearly for additive manufacturing using the shotcrete printing method before the surface smoothing and edge trimming step .
Co-registration methods can be categorized in two main groups based on the used features: i) known correspondences, and ii) unknown correspondences. For the first category the most common approach in surveying is using artificial targets i.e. spheres or checkerboard targets. Minimum of three non-colinear targets must be in common between the two datasets for registration. In the second category which the correspondences should be estimated, a variety of algorithms can be employed. If corresponding features are not known explicitly, Iterative Closest Point (ICP) proposed by (Besl and McKay, 1992) is by far the most commonly used algorithm for registering two-point clouds. Generally, the pipelines of the ICP-like algorithms consist of: i) Selection, ii) Matching, iii) Filtering, and iv) Alignment. The interested reader can refer to (Rusinkiewicz and Levoy, 2001;Cheng et al., 2018) for a comprehensive review of ICP-based algorithms.
ICP method is not the best choice for QC in AMC, as it was explained in our last paper (Mawas et al., 2022). It is due to the facts that i) ICP tends to average out and distribute the error between the two data sets, and ii) it may get stuck in local minima. We introduced a target-based co-registration approach in (Mawas et al., 2022). However, decreasing the role of targets and mounting the TLS on the robot, improves the efficiency and automation of the process, which is the aim of this paper.
A common QC technique is to measure the deviation of an as-build point cloud from a reference dataset. The deviations could be computed using Cloud-to-Cloud (C2C), Cloud-to-Mesh (C2M), and the Multiscale Model to Model Cloud Comparison (M3C2) algorithm proposed by (Lague et al., 2013). C2C distance is the distance between the points in the target dataset and the nearest points in the reference dataset (Girardeau-Montaut et al., 2005). This method has been utilized by numerous researchers (Chen and Cho, 2018;Buswell et al., 2020 ;Lu et al., 2020). While, the cloud-tomesh distance is the orthogonal distance of each point in the target point cloud to the nearest triangle in the reference model. Otherwise, the distance is considered to the nearest edge (Cignoni et al., 1998). Mesh-based distance is more robust to outliers than the C2C distance (Girardeau-Montaut et al., 2005;Maboudi et al., 2018). C2M algorithm is being employed in different research papers especially when the designed model is available as: (Lim et al., 2011;Nguyen and Choi, 2018;Buswell et al., 2020;Frangez et al., 2020). M3C2 algorithm measures the normal vector of every point by considering its neighbors. In addition, the algorithm projects search cylinders along the local normal vectors to compute the averaged distance between the two point clouds.
M3C2 operates directly on point clouds without the need for meshing. This is an advantage, where an accurate mesh for the reference data is not available, in contrary to AMC, where the designed model is always available. However, M3C2 accuracy depends on how well the surface mesh is able to model the surface of the object without over interpolate the original geometry of the point cloud.
Moreover, M3C2 has number of parameters which their tuning could be challenging.

METHOD
In this section we will discuss the proposed co-registration method as well as the analysis of the direct quality control. This analysis will be conducted first by data simulation to investigate and verify the characteristic of the used algorithms namely: C2C, C2M, and M3C2. In addition, the analysis would help to reveal the superiority of the direct coregistration in comparison with ICP.
In the case of mounting the TLS on the robot (see Figure 1), the position of the TLS can be obtained from the robot position. Nevertheless, in order to bring the scanner to the robot coordinate systems the orientation of the scanner is still missing. Since we used Z+F IMAGER 5010X which comes with fixed zero-direction of the azimuth angle, it would be sufficient to compute the azimuth angle just once. We verified this property before starting our experiments: If we move the robot without rotating its spindle we guarantee that we have the same orientation for every scanning station. As a result, we need to solve the translation between the scanner and the model coordinate system only. Figure 2, presents our strategy for analyzing various deviation measurement approaches. Firstly, we start by a defined model then we modify it (see Figure 3) to analyze the performance of the different algorithms as well as to have a better understanding of the registration effect. As it is shown in Figure 3, depicted in red, the left and right side of the original object are shortened by 10 mm and 2 mm respectively. Also, two of the prominent short cross sectioned cones were deleted in blue (their height is 20 mm). Lastly, the top surface of the modified object is 5 mm higher than the original (shown in green).
we changed the pose of the modified object and co-registered it with the original model by utilizing ICP. Thirdly, all three models namely: original model, modified object-we will call it here as Ground Truth pose (GT-pose), and Modified object with ICP registration are sampled in CloudCompare by assigning the total number of points in order to retrieve point clouds. Lastly, the modified data (GT-pose and ICP) are compared with the original data with one of the direct comparison algorithms as it will be clarified later in this section. Before describing our comparison strategy, it is beneficial to summarize the pros and cons of each comparison algorithm. C2C does not provide signed distances. Furthermore, in case of a local modeling, the accuracy of the computed distance depends on the density of the point cloud. Although C2M delivers signed distances, the distances are computed to the nearest neighboring part of the mesh. The problem is that if the deviation of the point from the corresponding ground truth surface is bigger than the distance of the point to its closest surface, the distance value would be wrongly assigned based on the closest surface. M3C2 also provides signed distances. Furthermore, the computed distance is along the local normal vector. M3C2, in contrary to C2C and C2M, requires a variety of parameters tuning.
The results of the deviation analysis of the simulated data by employing C2M between the original model and GT-pose and ICP point clouds is shown in the Figure 4. As it is visible in Figure 4, the ICP algorithm tends to average out and distribute the error between the two objects. This can be seen from the distance histogram of the GT-pose that has distinctive and separated bars, compared to the ICP results. The overall registration error from the ICP algorithm resulted in 1.8 mm (RMS).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B1-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France C2M revealed a transition colored area around the boundaries. This clearly can be seen from the blue colored distance results. This indicates that the computed distance accuracy by C2M algorithm is being affected around the boundaries. Moreover, in regards to GT-pose, the two prominent short cross cones are not detectable (see Figure 4 (a)). This is due to the fact that there are no points from the target (modified object as point cloud) to be compared with in the reference (original model). While it is slightly visible in the ICP case due to the inaccurate ICP co-registration. Therefore, to be able to detect the missing extruded parts, the original model is converted to point cloud (as-target) then compared to the point cloud of the modified model (asreference) by utilizing C2C (Figure 5). In other words, employ C2C algorithm by changing the assigning order compared to C2M. As it is shown in Figure 5, the modified object's point cloud (as reference) compared to the original object's point cloud (as target) by utilizing C2C for ICP scenario. The missing parts are now detectable. It worth mentioning that the true height value of the cross cones is 20 mm. Lastly, for utilizing M3C2 algorithm for both modified object (GT-pose & ICP) as point cloud is compared to the original object as point cloud as it is shown in Figure 6. In contrary to C2M, the results reveal the two cross section cones. Also, the figure shows a non-transition colored area around the boundaries. However, M3C2 has a variety of parameters that would affect the results. Thus, this needs to be tuned with caution. The main parameters are normal scale, projection scale and max depth. The normal scale is the diameter of the spherical neighborhood extracted around each point of interest to calculate the local normal. This normal is used to align a cylinder to search for equivalent points in the other cloud. The projection dimension is the diameter of the search cylinder. The max depth parameter is the height of the cylinder in both directions. The larger the radius, the smaller the effect of local surface roughness (and noise). However, more points will be averaged and the computation cost will be higher as well. From the results we see that the algorithm has over estimated distance for the top surface (see Figure 6). This mostly happens around the boundaries of the object. Nevertheless, majority of the points do give the right values. These biased results are due to the max depth parameter. Since the as-designed model is available we find the best practice for setting those parameters are as follows: the normal vectors can be imported from the model. Secondly, the max depth of the search cylinder parameter could be set from the max expected difference of the as-designed model. In this case, the max depth was set to 20 mm. Due to the fact that the two cross section cones are the max difference between these two datasets.
In summary, M3C2 provided better results in comparison with the other algorithms. In contrary to C2M, finding the proper initial values for parameters of M3C2 might be challenging. Nevertheless, if the reference model is available, it is easier to use the normal vectors from the source directly. In addition, one can use the knowledge of the reference model to set the max depth of the search cylinder. In case of missing parts and by employing C2M, we recommend employing C2C by changing the assigning order compared to C2M.

EXPERIMENT
In the experiment section, real data which is captured by Z+F IMAGER 5010X TLS, is used for two objects as shown in Figure 7 by utilizing our co-registration solution. Then the captured data is compared with its as-designed model using C2M and M3C2. The first object is a shotcrete wall and is captured by TLS before the surface smoothing and edge trimming step, while the other object is made of wax using a subtractive method. For scanning the shotcrete wall, TLS is utilized by mounting on the robot as well as on tripods in order to check the mounting approach with respect to tripod. The co-registration of the different stations performed by utilizing a plane-based solution implemented in Scantra (Wujanz et al., 2018) in order to check the positional accuracy of the scanner mounted on the robot. The scanner stations are registered to the robot coordinate system by measuring the center coordinates of six spheres which are known in the robot coordinate system. The average standard deviation of the adjusted translation from the tripod is 0.76 mm, while the average standard deviation for the mounting robot is 1.03 mm. This parameter indicates the deviation of the scanner position (after transformation) from robot coordinate system. Concerning the local registration of the different stations together, the tripod-based network resulted in 0.2 mm max error. On the other hand, the robot-based network showed 0.7 mm max error. This difference could be due to mounting the TLS on the robot, where the compensator is switched off, since it is upside down. Lastly, both networks have almost similar misclosures between the stations and the spheres in the robot coordinate system (approx. 4.5 mm max). As a conclusion, mounting the TLS on the robot has a neglectable effect on the registration accuracy. Furthermore, the transformation matrix between the fabrication lab and the digital model is known. The co-registration of the point cloud and the digital model can be achieved instantly as it is just a translation and the rotation parameters are identical for any station. The transformation matrix, for mounting the scanner on the robot, could be estimated before printing process or for the first TLS station only.
These results indeed have a room for improvements. Coregistration of the point cloud to the robot coordinate system is subjected to the spheres' center measurement which has its limitation in terms of accuracy. Nevertheless, these results are still promising, and the remaining uncertainties are smaller than the requirements from AMC, at least for objects we are concerned with currently.
After employing the proposed direct co-registration method, the point cloud of the shotcrete wall and the wax object could be compared with their as-designed model using C2M and M3C2. Figure 10 shows the shotcrete printed wall which is captured before the surface smoothing and edge trimming.  Figure 9.  The results of C2M (see Figure 10) shows a slightly underestimated results for the sides of the objects. In addition, the top surface reveals a transition area around the boundaries. In other words, the max value in purple refers to 70 mm surrounded by the blue which is inside the green area. This can be due to the inconsistency of the normal vector computation. Furthermore, this is because of what we have stated in section 3 that C2M distance is to the nearest surface plane.    Figure 11 shows the results of the distance analysis of the wax object. Both algorithms revealed almost similar distance deviations. Nonetheless, M3C2 shows some outliers around the boundaries. This is due to the inconsistency of the point around the boundaries. Furthermore, the points seem to be a bit off from the object. This is due to the computed M3C2 distance is the average distance between the two datasets. Moreover, the highlighted areas in Figure 11 around some points as well as around the cross-section cones are because of high reflected signals from the object's surface to the laser scanner. This distortion in the point cloud could be due to the wax surface that acted like a mirror. Also, the incidence angle should be defined carefully to eliminate this effect.
In summary, when the objects are in theirs rough state before surface finishing and edge trimming, it is challenging to estimate a proper normal vector. Thus, it will be challenging for the algorithm to give a reasonable and clear results. In this case with high (intentional) deviations, M3C2 performs better than C2M but it requires parameters tuning and some more insight investigation in comparison with C2M. The available as-designed model can help to have a better estimation for the parameters.

CONCLUSION
In our experiments, objects from 3D shotcrete printing wall as well as wax objects were analyzed. The paper provided results for an automatic registration between the different robot-mounted TLS stations and the as-designed model in addition to direct QC using C2M and M3C2 algorithms.
Using our registration solution, quality inspection could be performed without having the error-averaging effect (which is common in ICP-based algorithms) automatically and efficiently. The absolute accuracy of the registration with respect to the model coordinate system could be enhanced by using different artificial targets. A comparison between the as-built object to the as-designed model using different approaches is performed for simulated datasets as well as for the real data. As concluded from the simulated data that C2M is parameters free, in contrary to M3C2. However, C2M fails to detect the missing parts of the as-built data. This is due to the lack of data to be compared with. To cope this C2C could be employed in the reversed assigning order of C2M.
The combination of the direct co-registration of as-designed model and as-built object for subsequent quality control leads to an efficient production pipeline in AMC projects. Furthermore, the suggested automatic registration with the digital model allows to have a direct segmentation of the point cloud of the printed object since the position of the object is known in advance. In addition, the proposed strategy for installing the sensor on the highly flexible robot with known coordinate system could be also employed for photogrammetric image-based inspections where the camera could be installed on the robot to capture the images with known positions from previously well-designed positions.
The dimensions of the object and its tolerances are also two crucial parameters whose deviations from the as-designed model must be monitored. Thus, extracting the geometric entities like lines and surfaces and inspecting their geometric properties could be an interesting direction for future research.

ACKNOWLEDGMENTS
This research is funded by the German Research Foundation (DFG) -Project number 414265976 -TRR 277. The authors would like to thank the DFG for the support within the SFB/Transregio 277-Additive manufacturing in construction. (Subproject C06).