POINT CLOUD EXPLOITATION FOR STRUCTURAL MODELING AND ANALYSIS: A RELIABLE WORKFLOW

The digitization and geometric knowledge of the historical built heritage is currently based on point cloud, that rarely or only partially is used as digital twin for structural analysis. The present work deals with historical artefacts survey, with particular reference to masonry structures, aimed to their structural analysis and assessment. In detail, the study proposes a methodology capable of employing semi-directly the original data obtained from the 3D digital survey for the generation of a Finite Element Model (FEM), used for structural analysis of masonry buildings. The methodology described presents a reliable workflow with twofold purpose: the improvement of the transformation process of the point cloud in solid and subsequently obtain a high-quality and detailed model for structural analyses. Through the application of the methodology to a case study, the method consistency was assessed, regarding the smoothness of the whole procedure and the dynamic characterization of the Finite Element Model. The main improvement in respect with similar or our previous workflows is obtained by the introduction of the retopology in data processing, allowing the transformation of the raw data into a solid model with optimal balancing between Level of Detail (LOD) and computational weight. Another significant aspect of the optimized process is undoubtedly the possibility of faithfully respecting the semantics of the structure, leading to the discretization of the model into different parts depending on the materials. This work may represent an excellent reference for the study of masonry artefacts belonging to the existing historical heritage, starting from surveys and with the purpose to structural and seismic evaluations, in the general framework of knowledge-based preservation of heritage. * Corresponding author


INTRODUCTION
In the field of the 3D digitization of the historical built heritage, point cloud represents the reference object to obtain a digital twin of the existing constructions. The point cloud that used to be the core of innovative workflows, represents nowadays the starting point. In fact, it constitutes the unsegmented and non-informed material that opens up to different applications, at the service of several purposes. A highly investigated topic in current literature is the ScanToBIM, facing all problems of Heritage Building Information Modelling (HBIM) and the use of the point cloud for the management and documentation of the historical buildings, in which interesting research perspectives and solutions are observed. Another studied area is the Information and Communication Technology (ICT) applied to the dissemination of knowledge of historical buildings, including all the potentials of Augmented Reality/Virtual Reality/Mixed Reality Virtual Reality (VR) opens up to data collected and georeferenced at various scales of definition and detail, while Augmented Reality (AR) allows the visit of architecture while enjoying additional information placed next to the real physical element and Immersive Reality (IR) allows an immersive journey into architecture, collecting scattered elements or remote worlds. In the structural field, two are the reliable approaches to study the historical masonry buildings: Finite Element Models (FEM) (Acito et al., 2014;Giordano et al., 2019) and Discrete Element Models (DEM) Ferrante et al., 2021). The first is most used since it requires less modeling and computational times. Nonetheless, the Finite Element Method (FEM) does not yet have solid workflows based on point cloud. Currently, the most diffused technique consists in using the point cloud for the processing of traditional 2D drawings (plans, elevations and sections) on which 3D modeling is then based within the tools integrated into the FEM environment. This workflow presents two main weak points. On the one hand, it causes loss of accuracy and details: the accepted approximation, in fact, does not allow to consider the relative inclination, the imperfection and the lack of material in the masonry walls, constituting some recurrent peculiarities of a historical building, pertaining to Heritage. On the other hand, this approach takes a long time considering all the steps: definition of the level of detail (LOD), choice of sections, modeling, meshing, load application, analysis, etc. The presented work tries to face the challenge of interoperability between two disciplinary separated fields with specific software environments, to automate the transformation from high-quality reality-based digital facsimile to Finite Element Analysis. Evaluation and assessment of new workflows aimed at creating FEM models solidly based on the point cloud are needed, moreover when we come to implementing intelligent and convenient modelling strategies. The work starts from the analysis of some similar approaches already available in literature, it highlights their partial results, such as the fact that the majority is not structured in a broad logic, and then proposes a methodology that allows to semiautomatically transforming the point cloud into a 3D FEM, dealing with a whole building.
In order to demonstrate usability and reliability of similar models in the structural field, the case study was also used to perform a numerical updating based on Operational Modal Analysis (OMA). The paper is structured as described below. The paragraph on the state of the art offers a picture of the current situation with particular regard to the two areas under study, presenting at the moment only a few contact points: the point clouds data acquisition and uses and, on the other hand, the different approaches to the structural analysis of existing masonry buildings. The second paragraph describes the methodology applied on the case study of the bell tower of the cathedral of San Ciriaco in Ancona. The various phases are described and hierarchized in detail, from data acquisition to the results of the structural analysis. The third paragraph offers a critical assessment of the results obtained, highlighting the strengths of the proposed methodology.

Research aim
The research theme of this work can be identified in historical artefacts survey (with particular reference to masonry structures) aimed to structural analysis and assessment. In detail, the study proposes a methodology capable of employing directly, or semi-directly, the original data obtained from the Tele Laser Scanner (TLS) survey for the generation of a FEM that may be used for structural analysis of masonry buildings. To achieve this aim, a specific case study which allowed a gradual process of workflow optimization was selected. In particular, considering the interest to work on whole heritage buildings, which have proved to be particularly vulnerable under earthquake both for their complex geometries and for their inadequate state of conservation the workflows were instantiated on bell towers. The first attempts, mainly expensive and limiting, were improved in order to achieve a streamlined and fairly performing workflow.

STATE OF ART
Over the past twenty years, the architectural survey sector has undergone huge evolution thanks to exponential technological innovation. Point cloud today represents the main result of the data capturing: in fact it stems from the data processing of TLS or photogrammetry and all problems of data merging and integration are easily managed, not only in the research field but also for professionals. In parallel with the development of acquisition tools and techniques, desktop or mobile cloud-based systems have been developed in order to manage the enormous amount of data coming from the survey. Some projects are realizing the full potential of on-line repositories for point cloud sharing and use for research across institutional and national as well as disciplinary boundaries (Niccolucci and Richards, 2019) also focusing on AI exploitation (Yang et al., 2020). The idea of Building Information Modelling intended as an information system was thus declined in Heritage Building Information Modelling (HBIM) and a large part of the debate on the exploitation of reality-based models coming from point cloud is currently dealing with HBIM and management systems able to respond to the goals of restoration and predictive conservation. An extensive literature is already available on the theme of HBIM (Yang et al, 2020) and the different concept compared to BIM: there are many challenges and needs, such as the differences between standardization and irregularity of semantic objects or non-segmented geometries, as well as the presence of parametric intelligence derived from accurate geometric data. (Apollonio et al., 2016) Great attention has been paid to the fruition and management of the object (Godinho et al., 2020) even to the assessment of context-aware risk (Lee et al., 2019)

Structural model generation approaches
Over the past few years, point cloud has aroused the interest of civil engineering. Indeed, even if masonry structures are normally studied with the equivalent frame model, where the structural geometry is simplified in piers and spandrels beams elements, connected with rigid nodes (Lagomarsino et al., 2013), sometimes this modelling technique can be not suitable for buildings with complex geometry, like historical heritage structures (Valente and Milani, 2019). These types of structures, especially in the academic environment, are normally analyzed with FEM, requiring an adequate definition of geometry and material parameters. The purpose of this research is to obtain a 3D model representative of the object, from the point cloud directly. In recent years, the first studies where aligned with this approach, but they did not provide reliable workflows, thus their diffusion is nowadays weak. The point cloud is not used directly to create the computational model necessary for finite element analysis. As mentioned, although this is a rather recent research area, there are several attempts pursuing this objective in literature. The study of (Korumaz et al., 2017)offers a classification of the different types of approach to the problem, in (Quattrini et al., 2019) we reviewed it and updated the framework. In this regard the present study offers a "direct use" methodology performed on the basis of the previous one and completely assessed. An example of the "indirect use" approach is offered by (Fortunato et al., 2017). According to (Quattrini et al., 2019), the most performing approach in terms of accuracy is the "direct use" type, since it is less affected by the operator judgment. This same approach presents two modifications with growing refinement. The first, tested by (Bassier et al., 2016), is rather expensive from the computational point of view that grows according to the complexity of the object. This method is very suitable in the field of mechanical engineering, but it is time consuming for complex and large buildings. Due to the complexity and irregularity typical of historic masonry buildings, the "Create mesh and solid model" approach is considered the most performing (Bagnéris et al., 2017). In this line, the Level of Detail (LOD) of mesh is an element of great importance to obtaining a reliable structural analysis (Spangher and Visintini, 2015) The objective of an ever-increasing LOD highlights two sides of the same coin: on one hand it improves the reliability and quality of the analysis and carried out evaluations (Giaccone et al., 2020), on the other it causes interoperability issues. Finite element software has a different mesh concept than triangular meshes that are generated directly from the point cloud. From a merely mathematical point of view, the mesh represents discrete 3d surfaces (consisting of single flat faces); while nurbs are numerical surfaces, capable of continuously representing complex shapes. As mentioned above, the construction of nurbs curves that generate a nurbs surface is unsuitable for the complexity of the geometry of an historical architecture. There is, however, an convincing approach (Barsanti and Guidi, 2018), based on quadrangular meshes instead of triangular ones. It mainly consists in rearranging the initial topology of the mesh, thus it is The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) called Retopology. Similar techniques are used very often for visualization purposes of reality-based models. (Cipriani and Fantini, 2017)In this way, the resulting meshes easily draw near to a nurbs surface in compliance with FEM software. This step therefore allows to avoid multiple interoperability problems between software, as well as it enables variable LOD within the same mesh, leading to a great number of patches only where the features of the building require them. In this way, with the same total number of faces, it is possible to obtain a variable refinement of the 3D model with a consequent accuracy.

Structural Health Monitoring of historical structures
Historic masonry structures constitute both an economic resource and a cultural legacy for many Countries. However, these buildings are extremely fragile, especially when subjected to horizontal forces, due to several factors: complex geometries, structural evolutions differentiated over time, and decay (Valente and Milani, 2016). For the preservations of these structures, it is necessary to identify methods that keep their state of health under control to intervene as soon as possible.
With this in mind, it must be remembered that any operation on these buildings shall be properly studied since invasive interventions can compromise the historical and artistic value of the asset. From this point of view, the research is turning towards the Ambient Vibration Tests (AVT) (or also known as OMA) consisting in acquiring the dynamic properties of the structures through accelerometer sensors placed in significant places, without interrupting the common use of the building. The dynamic parameters are functions both the mass and the stiffness of the buildings; hence their variation could indicate the triggering of a degradation phenomenon (Salzano et al., 2020). There are two types of OMA: short-term (Doebling et al., 1996) and long-term monitoring (Gentile et al., 2019;Venanzi et al., 2020). In this article, we will focus on the shortterm and the realization of a FEM able to faithfully represent the real behavior of a structure. To achieve this goal, it is essential to have a geometric model that is as responsive as possible to reality which, as mentioned above, can be obtained with a point cloud. Once the survey is obtained, the linear parameters of the used materials are varied to minimize the difference between experimental and numerical results. Most of the studies in literature on this topic use manual calibrations, even thought the researchers have been heading towards its automatization over the past few years (Bianconi et al., 2020)

MATERIAL AND METHODS
The methodology consists in a complete workflow for structural analysis and seismic behaviour characterization applied in a whole building. Starting from the on-field acquisition and survey campaign, including historical sources recognition, the performed pipeline deals with a geometrical model high descriptive and able to be semantically segmented via semiautomatic procedures and interoperable with FEA, in order to study and characterize the building with modal analysis. The workflow is assessed in an exemplary case: the belfry of Saint Cyriacus in Ancona.

Case study
In order to test the previously stated objectives, the whole workflow was applied to a case study. The considered tower is the belfry of the Metropolitan Cathedral-Basilica of Saint Cyriacus in Ancona, Central Italy. Its isolated position, detached from the Church, is probably due to its constructive origin. It was a former military tower belonging to the defensive walls system on the top of the hill. The tower has a rectangular transversal section. The roof is compose by a large cornice and an elliptical dome with an octagonal basis. Over the years, several modifications have been made to the original layout of the tower: the opening of the previously missing monoforas, reconstruction of the roof, replacement of the attic of the bell cell. Now the tower looks as it is described below. The walls vary in thickness and consist of an external curtain almost entirely made of squared limestone blocks of Mount Conero and an internal predominantly brick curtain. The internal space is divided into 3 rooms vertically connected by a steel staircase fixed to the perimeter. The ground floor is vaulted, the upper room is covered by a reinforced concrete floor. This concrete slab also hosts the bell castle (with a structure in steel profiles) and then stands the pointed dome.

Data collection
The first step was the acquisition of on-site data through different campaigns, performed on separate days: a terrestrial campaign with laser scanner and an aerial campaign carried out with UAV (Unmanned Aerial Vehicle).
Regarding the first survey campaign, it was carried out using the Leica Scan Station P40. Twenty-nine scans were acquired of which: 1 frontal at the entrance of the tower which allowed the alignment between external and internal scans, 10 scans inside and 18 scans outside. The result of this campaign allowed to detect the entire square and part of the external elevations of the church, as well as the internal and external ones of the bell tower. The roof was excluded from this point cloud. A second campaign was then planned to scan the roof through an imagebased technique. For this purpose, the UAV DJI Spark device was used, as it is capable of reaching the upper surfaces of the roof, which could not be detected through TLS. 78 photos were then taken to model the roof by using the Structure from Motion technique.

Complete Point Cloud (Leica Cyclone-Agisoft
Metashape): first of all, the scans acquired with P40 were managed inside the proprietary Leica Cyclone software. In succession, steps performed were: import with automatic filtering, mapping and manual cloud-to-cloud alignment. Final point cloud registered (of the whole complex) has an alignment error of 11 mm (RMS= 2 mm). Subsequently, the tower was isolated eliminating all the points deriving from the acquisition of the surrounding space. In order to complete the survey with these parts, we proceeded to create a point cloud from the photos taken via UAV. The UAV used is the DJI Spark which has the following characteristics: 1/2.3" CMOS sensor, Field Of View of 81.9° with a Focal Length of 25 mm (35 mm equivalent). 78 photos were taken in jpeg format with a resolution of 3968×2976 pixels: the flight followed a circular trajectory around the dome (54 shots) to which 24 zenith photos were added. The average distance from the object varies from 12.5 m for zenith photos to 10.5 m for diagonal photos, obtaining a Ground Sample Distance of 0,746 mm/pixel in the horizontal direction and 0,734 mm/pixel in the vertical direction. The camera alignment and cloud generation operations were performed using Agisoft Metashape software. The initial cloud was first cleaned, isolating the points of the dome, and subsequently it was oriented, through the use of 3 markers on points common to the two clouds, with the previously created point cloud reference system The final result was a point cloud of 23 756 115 points ( Figure  1). The average point density is 20 640 points per square meter, equivalent to 1.44 points per cm. The farthest areas from the scanner (the top portions of the external elevations) have a density of 0.52 points per cm.

Mesh generation (Geomagic Wrap)
: the next step consists in mesh generation from the point cloud: it was then imported into Geomagic software and was then generated through the "Wrap" command mesh. All the holes in the mesh were finally closed through the command "fill holes", obtaining a watertight mesh of 1 352 349 triangles (Figure 2).

Retopology:
it represents one of the most important upgrades in comparison with our previous works (Quattrini et al., 2019). Thanks to it, in fact, it was possible to drastically reduce the timing of model optimization. The resulting mesh, made up of millions of triangles, was adjusted taking into consideration two fundamental factors: the non-uniformity of the LOD and the quadrangularity of the mesh.

Figure 2. Watertight mesh generated into Geomagic
The first result is enabled by different sizes given to the patches of the mesh network, taking into account the trend lines of the geometry (  Figure 3.a). The second factor concerns the shape of the mesh patches with fundamental implication: quadrangular meshes are used in the meshing process within the FEM. Quadrangular patches are obtained from a series of NURBS curves in two directions (called "U" and "V") interpolated to create a surface as a tensor product of two NURBS curves. Therefore the workflows was able to avoid the majority of incompatibility problems. In our case, Zbrush software was used and the mesh (  Figure 3.b) was obtained from the original one, by applying "ZRemesher".

. Model subdivision in Rhinoceros software
The model was then segmented by taking into account semantics, structural behaviour and materials, using the same Rhino software. In order to properly model the structure, vertical walls were separated into two parts, according to two curtains of different material (see par. 3.1). The various parts in which the model was divided are shown in Figure 4.

Dynamic characterization
When the investigation via NMs is the only option available, these must be reproduced as faithfully as possible. In the previous section, a metrology to obtain an accurate reproduction of the geometry was shown, while the parameters of the materials remain an issue. The parameters of the materials used to characterize the models are usually taken from local codes, even though this simplification may not be representative of reality. Exposure to environmental factors, the presence of cracks or retrofitting interventions may change them in some parts of the structure. Since it is absurd to carry out tests on the materials of composing the entire building, both due to theirs costs and to preserve the artistic value of the structures, noninvasive techniques such as OMA are currently used. As seen in section Errore. L'origine riferimento non è stata trovata. with the OMA real frequencies and modal shapes are obtained to calibrate the NMs by acting on the linear materials' parameters. This first calibration already gives an idea about the current state of conservation of the building. In this section, the FEM of the San Ciriaco Tower, obtained from the point cloud, was used to perform a numerical calibration based on the experimental data obtained by OMA.

Operation Modal Analysis:
to record the real frequencies and modal shapes, the tower was monitored under the operation conditions using five tri-axial Piezo-MEMS accelerometers (dynamic range 120dB, sensibility 1 V/g, range of frequency 0.8-100 Hz) connected with a Sinchub for the synchronization (Ribilotta et al., 2018).
The experimental campaign consisted of four measurements where two accelerometers on the last accessible floor were kept fixed, meanwhile the remaining were moved in other locations to collect a greater amount of data and have a better dynamic characterization. Each measurement lasted 40 minutes, and data were collected with a sampling frequency of 1000 Hz.
The recorded time histories were first filtered and decimated, to delete the noise and reduce the data in a range of common frequencies for the towers 0.5-16.67Hz, and then processed with Cov SSI-PC (Covariance Stochastic Subspace Identification Principal Component) that allowed to extract the frequencies and modal shapes of the structure. (Table 1, Figure  6)  While the accuracy of this procedure may be verified in different ways, it was checked in this article by using the percentage of frequency error and the Modal Assurance Criteria (MAC) between the modal vectors (Pastor et al., 2012). The numerical calibration was then performed in three steps. At the beginning, the materials identify with the survey (Section 3.1) were assigned to the NM. The values were assumed following the Italian Code (Ministero delle infrastrutture e dei trasporti, 2019; Ministero delle Infrastrutture e dei Trasporti, 2018) ( Figure 8). The tower is fixed at the base. The bell castle load was assigned as an additional mass on the concrete floor.
The model underwent to linear dynamic analysis to obtain the numerical frequencies and modal shapes. A huge difference between the frequencies was found by comparing the results with experimental campaign, while modal shapes were quite in accordance with the reality showing a Modal Assurance Criteria (MAC) values included in a range between 56% and 88% ( Figure 8, Table 2).

Figure 7 Assigned materials at the starting model
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2021 XXIV ISPRS Congress (2021 edition) The real structure appeared stiffer than the NM, probably due to the armed injections intervention executed on the masonry panels in 1963. For this reason, the second step consisted of an increase of the elastic modulus of the bricks and the limestone masonry. This approach allowed to fit well the first two frequencies, even though the others were still too distant from the experiments. (Table 3 Table 3 Comparison between experimental and numerical data considering the numerical model updated to the second step Observing the real modal shapes in Figure 6, it may be noticed that the 3 rd , 4 th and 5 th modes involve the upper part of the tower more than the rest of the building, since it is more exposed to ambient factors that may have reduced the masonry quality over the years. For this reason, Young's Modulus of the masonry under the bell cell was increased meanwhile those of the bell cell and the dome were reduced until obtaining percentage differences with the experiment's frequencies less of 5%. , Figure 9.a, Figure 9.b) Comparing the MAC values of the last step with the first model, an improvement of the 2 nd 3 rd and 4 th modal shapes is also evident (4, Figure 9.b).  Table 4 Comparison between experimental and numerical data considering the numerical model updated to the last step

RESULTS AND DISCUSSION
The evolution and refinement of the methodology already tested (Quattrini et al., 2019) has a twofold objective: the improvement of the transformation process of the point cloud in solid and subsequently a better FEM for structural analyses.
Concerning the improvement of the workflow for transforming the raw data into a solid model, the main innovation was the introduction of retopology (see 3.3.3), contributing to a quality leap. In another case study (Torre del Borgo in Recanati), in fact, a slightly different workflow was applied to obtain a segmentation of the model as close as possible to the semantics of the object. It is here reported for the shape of completeness of the reader. It was carried out cutting off the solid through the Autodesk software Inventor, without retopology. This intermediate attempt, however, did not lead to calibration results comparable to those obtained now ( Figure 10). The retopology step was the turning point for obtaining a reliable and accurate workflow, as it enables in fact to obtain a truly high Level of Detail (LOD) without the defects and possible interoperability problems. The histogram of the maps of the deviation between the point cloud and FEM used for the analyses shows a quantitative assessment of the accuracy (Figure 11.a). Deviation maps graphically show the distance between the points of the point cloud and the solid model generated by it. Only a marginal part of points, at the sharpest edges, deviates by an order of more than ten millimetres from the final model (Figure 11.b). a) b) Figure 11.

a) Deviation histogram point cloud -solid model expressed in meters; b) Deviation maps point cloud -solid model expressed in meters
Another very significant aspect of this optimized process is the possibility of faithfully respecting the semantics of the structure, leading to the discretization of the model into different parts depending on the materials. Indeed, to be calibrated with respect to the real characteristics, the NM did not need substantial modifications. The only operation that became necessary was to divide the material bell tower in height. This procedure was easy to implement, as it did not require a subdivision of the geometry, but it was enough to assign different materials directly to the elements of the meshes. Finally, observing the OMA results, the tower analysed presents the classic modal shapes of isolated towers, as the first two are translational while the third is rotational. Comparing the real characteristics with the first model, built according to engineering hypotheses, it is evident that the interventions carried out over the years have increased the stiffness of the structure. However, the calibration showed that, as in many other case studies, the most exposed parts tend to degrade very easily.

CONCLUSION
The initial goal of this study was to obtain a performing workflow for the direct use of the point cloud aimed at FE structural analysis. After various attempts, the present case study obtained an optimal degree of reliability on both fronts, regarding the smoothness of the whole procedure and the dynamic characterization of the FEM. Improvements in the workflow were achieved thanks to the severe reduction, if compared to the workflow of previous studies, of the steps with operator's discretion, thanks to the introduction of retopology. Thanks to the latter, a more streamlined workflow was obtained, in which a) computational times have been significantly reduced; b) high Level of Detail, significant for the historical architecture features, was managed. The use of the point cloud allowed to remove the manual construction step of the geometry, necessary to create a correct numerical model. Indeed, it is a step that usually requires time and operator's skills, especially for monumental structures featured by complex geometries. The usability of the obtained model was verified by carrying out a numerical calibration of the real frequencies and modal shapes recorded during an OMA. The results showed that through quick and easy steps it was possible to obtain frequencies and modal shapes of the numerical model close enough to the real ones, with a maximum deviation of less than 2.5% and minimum MAC value of 65%, respectively. This work may represent an excellent reference for the study of masonry artefacts belonging to heritage, starting from surveys and leading to structural evaluations with the aim of preserving the heritage, under seismic solicitations. As future works, it is foreseen to implement this process on other case studies to test it and further optimize it. In future, it will plan to use the points cloud to also realize DEM models. Indeed, this technique will be allowed to represent accurately the real blocks' sizes that compose the structures. A challenge is undoubtedly represented by complex buildings in which the workflow can be stressed and assessed.