VAULT MODELING WITH NEURAL NETWORKS

: Nowadays, the digital reconstruction of vaults is carried out using photogrammetric and laser scanning techniques able to capture the visible surface with dense point clouds. Then, different modeling strategies allow the generation of 3D models in various formats, such as meshes that interpolates the acquired point cloud, NURBS-based reconstructions based on manual, semi-automated, or automated procedures, and parametric objects for Building Information Modeling. This paper proposes a novel method that reconstructs the visible surface of a vault using neural networks. It is based on the assumption that vaults are not irregular free-form objects, but they can be reconstructed by mathematical functions calculated from the acquired point clouds. The proposed approach uses the point cloud to train a neural network that approximates vault surface. The achieved solution is not only able to consider the basic geometry of the vault, but also its irregularities that cannot be neglected in the case of accurate and detailed modeling projects of historical vaults. Considerations on the approximation capabilities of neural networks are illustrated and discussed along with the advantages of creating a mathematical representation encapsulated into a function.


INTRODUCTION
Laser scanning and photogrammetric point clouds can capture the visible surface of vaults (intrados) with a high level of detail. This method reveals the geometric irregularities of the surface, making geometric survey a powerful non-destructive inspection and analysis tool in conservation and restoration projects. 3D models generated from point clouds can be delivered in different formats. Photogrammetric software usually provides a mesh interpolated from the original point clouds, which is then textured to produce 3D models with color information and digital orthophotos. Similar results can be obtained with laser scanning point clouds, notwithstanding the quality of color information is not comparable with the one achievable from images. Meshes generated with the automatic interpolation of point clouds have some disadvantages, especially when they have to be used for conservation purposes. The first issue is related to the size of the model, which can be made up of several million triangles, resulting in heavy models. An additional (probably more important) issue concerns the interpolation algorithms that do not consider the different elements of the vault. This gives a single surface that does not follow the logic of construction of the vault. An alternative solution for vault modeling consists of the use of software for modeling with NURBS surfaces. Examples are Rhinoceros and Autodesk Maya. Commercial software allows the user to import point clouds and generate 3D surfaces with manual, semi-automatic, and automatic algorithms. The user plays a fundamental role in the modeling process. He/she can identify the different structural elements and separate the different surfaces, obtaining a model that is not only a "pure" geometric representation, but it also includes the interpretation carried out to understand better how the vault is built. After defining the generative geometry and modeling the discontinuities between constructive elements, modeling can become a more automatic procedure using both edges (generally NURBS curves) and point clouds. A surface is fitted to the point cloud using curves as boundaries to obtain a perfect element-to-element correspondence in the final model. Meshes and NURBS surfaces are powerful tools to reconstruct the visible surface of the vault. However, what happens for those elements that cannot be surveyed with point clouds? This could be the case of the internal structure of the vault as well as the extrados that (in some cases) it is not visible. Such problems are relevant in the case of BIM projects, which cannot be limited to the visible surfaces. BIM requires 3D objects with the opportunity to add information about the internal structure (layers) of the considered architectural elements. In other words, a vault for BIM must become a sort of "solid" object with a thickness. The advantage of BIM consists in parametrization procedures that allows one to modify the shape of the modeled elements without redrawing. In the case of a vault with unknown thickness, the extrados can be generated using an offset-based parametrization of the intrados. This is not a static solution since changing the value of the thickness in the BIM database modifies the 3D representation. The solution proposed in this paper relies on the reconstruction of the vault intrados with a neural network, which is trained using point clouds. The network is able to learn and approximate the geometry of the vault. It is obvious that geometry plays a fundamental role when dealing with vaults. Vaults can be intended as a "combination" of basic geometric primitives (cylinders, planes, spheres, etc.) with an additional aspect that cannot be neglected in the case of historic vaults: the irregular shape, i.e. the deviation from the ideal geometry made up of geometric primitives. Laser scanning and photogrammetric point clouds are able to capture the visible surfaces with millimeter-level accuracy, so that they can reveal the actual shape. Let us suppose that the surface of a "perfect" vault (i.e., the vault is made up of just a combination of basic primitives) is described by a function = ( ), where is a vector. We can assume that the geometric irregularities can be modeled by a second function = ( ).
The function describing the entire vault will be the sum = + . It is not difficult to find a closed-form expression form for . On the other hand, the deviation term is not a constant for different vaults, since every vault has its own deformations. We propose a solution able to reconstruct the entire vault surface (without separating the contribution of and ) using the point cloud as representative of discrete values computed for . The advantage of the proposed method consists in the representation of the vault with a mathematical function that can be evaluated on a domain to reconstruct the vault. The approach includes modeling without drawing, since the vault is expressed using a closed-form expression. Sections can also be extracted by evaluating the estimated function along particular directions, for instance using a 2D line 2 = m 1 +q in the horizontal plane. The paper is structured as follows. Section 2 presents an overview of the developed method addressing in detail the concept of surface approximation with neural networks. Section 3 is focused on testing the viability of the proposed method by presenting some preliminary tests on synthetic datasets. Those tests were mainly devoted to determining the influence of the number of hidden layers of the neural network and the number of neurons to be used in each layer. Section 4 presents results for some real datasets. In particular, the reconstruction of some sets of real vaults with different shape is presented and discussed. The reconstructed vaults show a close correspondence with the input point cloud demonstrating the effectiveness of the proposed approach. The final section is devoted to draw some conclusions and to address future developments.

OVERVIEW OF THE PROPOSED METHOD
This work is an extension of what presented in Barazzetti (2018), here applied to the specific case of historical vaults. As mentioned, we assume that the surface of a vault can be approximated by a function : → ⊆ ℝ. The traditional way to define is to study the shape of the vault and implement a (static) computer program that generates the required mapping. Such a task is not a trivial problem, especially in the case of irregular vaults like those of historic buildings, which have geometric irregularities and anomalies. The mathematical problem behind the definition of the function is not a simple task. Let us suppose that the idea is to implement an algorithm able to recover a suitable mapping for a specific vault. This is a challenging task not only for the intrinsic difficulties in finding a closed-form expressions with sufficient metric accuracy. The intrinsic deformations of historic vaults make the approach not replicable for other case vaults, which would require specific mapping functions. In other words, working with different vaults requires a continuous change of the mapping function. The use of photogrammetric or laser scanning point clouds allows one to collect metric information about the unknown mapping. A set of measured values ( 1 , 2 , 3 ) measured (index m) can be used to recover = + without separating the contribution of and . The aim is to find a mapping function that provides a set of correspondences ( 1 , 2 ) → ( 3 + ), i.e. an approximate mapping with a "small" error , which is acceptable and comparable with the expected metric accuracy of the final output or the chosen scale for the model. The mapping problem may be cast in the form : ⊆ ℝ 2 → ℝ. The function may also be expressed in the more common form 3 = ( 1 , 2 ), which assumes that there is a direct mapping between the spatial coordinates 1 , 2 and the vertical one 3 . The proposed (automatic) solution that realizes this process without manual intervention is based on a neural network : ⊆ ℝ 2 → ℝ that has the capability to learn and define the mapping from the measured punctual values ( 1 , 2 , 3 ). In other words, the algorithm has to learn on its own without the need to explicitly write the code, and the neural network has to approximate the function with an "acceptable" error ‖ − ‖ < , so that we can say that ( 1 , 2 ) is an -approximated interpolant of the function . For an in-depth study of neural network, the reader is addressed to Haykin (2008). Here, only the main concepts will be briefly reviewed. To describe neural networks, we can start considering the most straightforward possible neural network, the one that comprises a single "neuron" (Figure 1). This "neuron" is a computational unit that takes as input x1,x2 (and a +1 intercept term), and outputs hW,b(x3)=A(W T x)=f(∑ 2 i=1Wixi+b), where A:R↦R is called the activation function. In these paper we have used the sigmoid as activation function: A neural network is obtained by hooking together many simple "neurons," so that the output of a neuron can be the input of another. In the neural network presented in Figure 2 the leftmost layer of the network is called the input layer (x1, x2, +1), and the rightmost layer the output layer (which, in our examples, has only one node, x3). The middle layer of nodes is termed hidden layer. (1) + 12 2 2 (2) + 13 2 3 (2) + 1 (2) ) (2) A vector consisting of W(i,k) is called weight vector. These weights are what makes each neuron unique. They are fixed during testing. During training such values are the numbers that the algorithms will change to 'tune' the network. Neural networks may present different architectures (i.e., patterns of connectivity between neurons). In particular, we may have neural networks with multiple hidden layers. The most common choice is a nl-layered network where each layer li is densely connected to layer li+1. In this setting, to compute the output of the network, we can successively compute all the activations in layer l2, then layer l3, and so on, up to layer ln. Figure 3. A diagram denoting a neural network with 2 hidden layers.
An example of the presented method based on neural network is shown in Figure 4. The vault was scanned with a Faro Focus 3D, obtaining point coordinates with an expected metric accuracy of about ±2 mm. A feed-forward neural network with a single hidden layer and 40 neurons ( Figure 5) was trained using a subset of the original point clouds decimated using curvature information. Regression information on training, test and validation (70%, 15%, 15% of original dataset, respectively, see Figure 6) and a mean squared error (MSE) of 4.02·10 -3 , 4.12·10 -3 , and 4.06·10 -3 (the relationship between MSE and the more common root mean squares is RMS = (2·MSE) 1/2 ) reveal that the network can approximate vault geometry with millimeter-level precision.

TEST WITH SIMULATED DATA
A presented in the previous section the main element constituting a neural network are the "neurons" and the architecture of the neural network itself (i.e., the number of layers in which the "neurons" are organized). This section aimed at investigating the viability of the proposed method for vault fitting and, in particular, to investigate the influence of the number of hidden layers net and the number on neurons in each of them.
To perform this analysis, a set of syndetic data was generated. The aim of using simulated data sets was to discard the influence of noise in the data and allow an easier comparison between the output of the neural network and the original point cloud.
In particular, two synthetic datasets were generated:  "Barrel vault" data set; and  "Cloister vault" data set Some details about the three datasets are provides into Table 1.  sharp boundaries seem more problematic than regular surfaces (Figure 7 and Figure 9);  points characterized by vertical tangent planes are critical areas for the neural network (Figure 7).

Number of points
Results related to the number of hidden layers in the network along with the neurons into each layer show that:   Table 2. "Barrel Vault" and "Cloister vault" data set results.

Barrel vault
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W9, 2019 8th Intl. Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 6-8 February 2019, Bergamo, Italy A numerical comparison of the results achieved with the different neural networks was also performed. In particular, the percentage of points at absolute distances less than 0.005 m and 0.001 m is estimated. Results of this comparison are reported in Table 2. These data confirm the previous considerations and the viability of the proposed method.

TEST WITH REAL DATA
Different tests were carried out with laser scanning datasets acquired with a Faro Focus 3D. Shown in Figure 11a are the results for an irregular barrel vault characterized by the uniqueness of its shape, for which basic primitives (e.g. a cylinder) cannot be used to approximate the surface when accurate reconstructions are required. The point cloud was decimated to 3,595 points to allow rapid estimation of network parameters. 20 neural networks with a variable number of neurons in the hidden layer (from 57 to 62) were estimated and the best 5 networks were selected. Results on a spatial domain that contains the planar projection of the vault were evaluated through a regular grid of 0.01 m The second case study is a cloister vault. Different scans were acquired to capture the vault but some occlusions were found after inspecting scans in the office, as shown in Figure 12a. 20 neural networks were trained like in the previous case study, using 4,561 points and a variable number of neurons (from 65 to 69). Data processing took less than 3 minutes and the results of the best 5 neural networks (average value) were used to create a new point cloud on a regular grid of 0.01 m x 0.01 m (Figure 12b). The final discrepancy estimated with CloudCompare between the averaged results and the original point cloud is 0.006±0.003 m. The last case study is a pavilion vault in the "Centro Culturale Chiesa Vecchia" a former church in Calusco d'Adda (Italy). The point cloud of the vault is composed by more than 4 million points. In this case, different configurations of the neural network architecture were tested ( Figure 13 and Table 3). Results seem to confirm the conclusions that were discussed about the simulated data sets.

CONCLUSION
The paper presented a novel procedure for the reconstruction of vaults. The process could be intended as a strategy for "modeling without modeling" since the vault is mathematically reconstructed with a closed-form expression encapsulated in the numerical coefficients of a neural network. The method has advantages regarding memory occupation. The model of the vault is available after evaluating the computed network on a spatial domain. The parameters that must be stored are the code behind the network and the computed numerical coefficients. This is a simple textfile with very limited memory occupation.
Network architecture has an essential role not only in the final metric quality of the method but also in the time required to estimate network coefficients. Other issues are related to the convergence of the solution, the number of hidden layers, the number of neurons on each layer, and the decimation procedure for the original point cloud. Additional work will focus on these issues, addressing the case of other input parameters. Indeed, more information captured by the laser scanner or digital cameras (e.g., intensity, RGB, …) can be used to train the network, extending the output to other parameters rather than just using z coordinates.