DIGITAL PHOTOGRAMMETRY, TLS SURVEY AND 3D MODELLING FOR VR AND AR APPLICATIONS IN CH

The world of valorization of Cultural Heritage is even more focused on the virtual representation and reconstructions of digital 3D models of monuments and archaeological sites. In this scenario the quality and the performances offered by the virtual reality (VR) and augmented reality (AR) navigation take primary importance, improving the accessibility of cultural sites where the real access is not allowed for natural conditions or human possibilities. The creation of a virtual environment useful for these purposes requires a specific workflow to follow, combining different strategies in the fields of survey, 3D modelling and virtual navigation. In this work a specific case of study has been analyzed as a practical example, the church of ‘San Giorgio dei Genovesi’, settled in the Historic Centre of Palermo (Italy). The acquisition of geometric information has been obtained with the integration of Terrestrial Laser Scanner (TLS) technologies and the photogrammetric reconstruction from mini Unmanned Aerial Vehicle (UAV) equipment. The obtained point cloud has been georeferred considering a network of Ground Control Points (GCP) acquired by a Global Navigation Satellite System (GNSS) receiver. The final point cloud has been processed and properly simplified through 3D modelling procedures, to obtain a realistic and light 3D model reconstruction. The model has hence employed into a VR WEB navigation system and will be used for AR outdoor application in the future, allowing to obtain different solutions for empowering the accessibility of the cultural good. The strategy of 3D CH model reconstruction, followed in this work, could be considered a reference methodology for the development of VR gaming applications finalized to CH valorization and AR applications, applied to museums or touristic paths in historical centres.


INTRODUCTION
The area of Digital Humanities related to art, architecture and archaeology and the tools linked to it have gone through as strong growth in a few years. The recent development of 3D virtual applications deeply changed the approach to accessibility in many fields of life. The diffusion of portable devices like smartphones and tablets has opened new possibilities of uses, allowing virtual-navigation as in the case of Virtual Reality (VR), or even offering additional information to the fruition of the real environment as in the case of Augmented Reality (AR). At the same time, the topic of Cultural Heritage (CH) valorization achieved even more importance in the last decades, highlighting the key role of these new technologies, in the fields of conservation and fruition of cultural goods (Ioannides andDavies, 2018, Ferracani et al., 2017;Mah et al., 2019;Campisi et al., 2019). The development of these new strategies, related to VR and AR fruition, increased the necessity of new requirements in the fields of the survey and 3D modelling of CH. In fact, both VR and AR solutions require to show complex 3D models with detailed textures associated with them, but strongly limited in the number of polygons and in the texture resolution (Scianna et al., 2016;Scianna et al., 2019). These limitations are related with the network capabilities of WEB browsing (as in the case of WEB navigation of VR 3D models) and the storage size of apps to download from the WEB (as in the case of AR and VR applications). In light of these limitations, the 3D modelling phase represents a crucial step because, in this practice, the original 3D information obtained from survey operations should be simplified and manipulated, reducing clusters and overabundant number of polygons, to make a navigable model. For this reason, it's necessary to choose the best procedures of simplifications in order to obtain a lighter 3D model and, at the same time, to maintain the necessary Level of Detail (LoD). Considering the acquisition of the base information, survey operations, recent advances in technologies allow obtaining 3D point cloud restitutions of cultural sites with satisfactory levels of accuracy, useful for conservation and valorization aims (Campisi et al., 2019;Mineo et al., 2018;Guarnieri et al., 2017). Terrestrial Laser Scanner (TLS) technologies grant the acquisition and the merging of different point clouds of thousands of points. The association, to models, of a network of Global Navigation Satellite System (GNSS) targets grants its correct georeferentiation. At the same time, the strong evolution of digital photogrammetry based on Structure from Motion (SfM) algorithms allows today the rapid virtual reconstruction of models from chunks of photos.
In this field, the development of Unmanned Aerial Vehicle (UAV) technologies allowed the acquisition of aerial images from different angles, strengthening the possibilities of digital photogrammetry and the diffusion of its applications (Murtiyoso et al., 2017). Today, digital photogrammetry and TLS technologies are different strategies that can be combined for CH survey operations trying to take advantage of the strengths of both solutions (Pepe et al., 2016). The choice of the best solution depends on the natural and urban features of the cultural good to represent. The proper acquisition of 3D documentation relative to cultural goods is fundamental for the construction of the virtual environments. It represents the base for achieving conservation and fruition purposes, aimed to the dissemination of CH information for touristic and scientific uses. Indeed, the necessity of sharing complex 3D environments on WEB led scholars and developers to study new solutions for networking 3D models. In particular, some strategies related to VR are based on the experimentation of touristic WEB applications, sometimes connected with a real 2D token (the QR code) printed on a paper guide or placed in museums or archaeological sites installation. Other alternatives are based on the creation of a virtual environment loaded inside a .html webpage, through the use of javascript code linked to external libraries and 3D models. This is the WebGL solution, a system of javascript libraries based on HTML5 standard, that allows the networking of 3D environments simply opening .html pages through the most common WEB browsers (Scianna et al., 2016). At the same time, the diffusion and the development of rapid prototyping technologies opened new possibilities for connecting real scaled 3D models, generated by 3D printers, to virtual environments, creating specific AR applications that could improve the accessibility of cultural goods (D'Agnano et al., 2020;Scianna et al., 2019). In fact, the recent development of WEB interactive applications granted to share virtual CH information around the world with a simple click. These new opportunities allow users to virtually discover and visit monuments and archaeological sites through the WEB, considering also those sites that, for many reasons, are not accessible. Indeed, many cultural sites cannot accept a large number of visitors, or are not accessible for a particular placement, or, even, are settled on submerged locations (Bruno et al., 2017). Also missing parts of ancient castles can be seen and explored from different angles in real-time just using a smartphone (Canciani et al., 2016). Hence, the virtual WEB navigation partially solves these problems connected to the accessibility. About this, it's necessary to underline that the virtual fruition of CH couldn't substitute the real one, but could be considered as a useful integration for the sharing of its knowledge. The present work shows a case study, the church of San Giorgio dei Genovesi in Palermo, where different surveying strategies, 3D modelling procedures, and VR fruition are involved. The virtual model developed has been included in a WEB application making it navigable and is also ready to be used for AR application. The experimentation regarding this work outlines an operative workflow that could represent a new strategy finalized to the empowering of CH accessibility.

SURVEYING STRATEGIES AND 3D MODELLING FOR VIRTUALIZATION OF CH.
As affirmed before, TLS acquisition and digital photogrammetry applied to survey and to the digital reconstruction of CH achieved a high level of development in terms of speed and accuracy. The possibility of obtaining point cloud restitutions of complex CH environments, with a good geometric level of accuracy, represents an important strength of these technologies. In facts, the dense clouds obtained are useful for exporting orthogonal projections, obtaining traditional sections, finding deformations or breaks, etc. These data could be also processed for the creation of meshes and are the base for the creation of virtual cultural heritage representations. However, there are some limitations in laser scanning or in photogrammetry. These different acquisition strategies have different strengths and weakness, depending on the final use of the acquired data (Fassi et al., 2013) and on the target of the survey (Vacca et al., 2012). In particular, TLS technology allows a quick point cloud acquisition, but the visualization, the management and processing are very time-consuming. These elaborations also require a huge amount of RAM and memory, depending on the data dimension of the considered point cloud. In addition, a good level of accuracy is not always guaranteed. It depends on many factors, such as the features of instruments, the distance from the target, the position, the material of the target surface and so on. Also, processing steps such as registration of the point cloud, filtering and meshing could produce additional errors that affect the level of accuracy of the final model. In the end, the original point cloud acquired by TLS instrumentation achieves a satisfactory level of accuracy. Still, its management generally requires specific software, which sometimes lacks several useful features for subsequent processing. Digital photogrammetry, instead, has the great advantage of the rapid survey acquisition, using not too expensive instrumentation, like a single-lens reflex (SLR) camera. The automatism based on SfM, achieved in this field, is advanced and allows to obtain aligned dense point clouds of indoor and outdoor environments. If before, using easily recognizable targets as tie point in the elaboration process (Remondino et al., 2012) was possible, today the good quality of the photos, coupled with some algorithms, can be enough for the point cloud reconstruction. Also, the generation of textures produces excellent results in terms of quality, even in elaboration coming from historical images (Maiwald et al., 2017). However, the problems of this kind of acquisition are related to the kind of objects to survey. The photogrammetric reconstruction of large cultural goods, like buildings and monuments, needs a great number of photos taken from different points of view. Many photos mean much time and computing power required by the software for the next alignment and virtual reconstruction. The placement of the monument and the light conditions are also important aspects to consider for survey operations in order to obtain photos good enough for the correct alignment by the software. These conditions are also useful for the correct texture generation in the final model. When necessary, point clouds obtained from a TLS or photogrammetric reconstruction, can be transformed into 3D models like meshes, NURBS, etc. This kind of procedures for 3D virtual reconstruction represents a very important element in the field of Digital Humanities for architecture and archaeology. In facts, these virtual 3D representations could be used in several fields for conservation and valorization purposes, thanks to the high level of detail achieved by modelling software processing. 3D models represent a great instrument for studying possible reconstructions, light effects, awareness of spatial relationship and comparative researches (Von Schwerin et al., 2013). As affirmed before, the management of virtual 3D reconstructions represents a complex process, due to the huge dimension of the point clouds obtained after TLS or photogrammetric restitution. In particular, WEB navigation of 3D models requires a strong simplification process. Besides, point cloud management software allows simplified automatic meshes generation, very useful tools that return 3D models with a reduced number of polygons. In this case, some further manual operations could be required, but for the most part, the work is done. Emerging issues start when the point cloud data are incomplete: the dense cloud displays some holes, that in certain cases can have relevant dimensions, and, during the creation of the mesh, the software will try to fill them. This operation often presents some errors in the shape of meshes and obviously also the generation of the texture in these areas is compromised. Timeconsuming manual operations are so required to fix these lacks. Another strategy to adopt could be manual modelling using the dense cloud as a base reference, skipping the automatic reconstruction. This is a good method if the aim is the realization of a medium/low-quality model useful for virtual WEB navigation that takes advantage of a high-quality texture to maintain a good level of realism in networking or visualization by APPs. As said before, WebGL and in-browser virtual fruition of CH are allowing to rapid share 3D models of both little objects that museum cannot exhibit (Santos et al., 2014) and big ancient buildings not easily accessible (Scianna and La Guardia,2018). The advantages of this technique stay in the speed of real-time virtual navigation, possible without the necessity of download large amount of data or apps for the user. However, as said before, to reach this goal, the development of low-sized 3D virtual models without losing in quality is essential. In the following part of the paper a mixed approach is applied to a particular case of study, the Church of San Giorgio dei Genovesi in Palermo, considering different survey acquisition strategies, 3D modelling elaborations and virtual fruition solutions. The survey operations are finalized to the realization of a 3D virtual model navigable on WEB, analyzing strengths and weaknesses of different methods of processing in the different phases, in order to define a successful workflow for virtual fruition of monuments. This work defines a strategy for CH valorization finalized to the enhancement of monuments accessibility, considering different kinds of technologies, with a particular focus to open-source solutions.

THE EXPERIMENTATION OF SAN GIORGIO DEI GENOVESI.
The Church of 'San Giorgio dei Genovesi' is an interesting example of Renaissance architecture in Palermo. Like other basilicas, this one has a cruciform drawing, but it can't be seen from the outside, and an octagonal dome on the top (Fig. 1). The facade is horizontally divided into two orders by a cornice while four half pilasters divide it vertically into three parts. There is a round window on the top center highly decorated and an adorned fronton on the top. The building is located in the 'La Loggia' quarter and it has been realized between the 1576 and 1596 by the architect Giorgio Di Faccio. The experimentation, described in this paper, concerns the steps of a workflow characterized by the survey operations and the 3D modelling, necessary for the virtual fruition of the building (Fig.  2). As will be described below, the survey operations involved both laser scanning and photogrammetric reconstruction. This mix of procedures has been an opportunity to compare the techniques and the results and to evaluate how they influence the 3D modelling phase. The next step has been the experimentation of different 3D modelling approaches, necessary for the creation of the final 3D model to visualize on WEB. Then, the last phase has been the creation of the WEB navigation model, where users can freely navigate around the 3D model of the monument using WEB browser navigation. The adopted strategy followed for the geometric information acquisition was the integration of different survey technologies: Terrestrial Laser Scanning (TLS), Global Navigation Satellite System (GNSS) and Unmanned micro Aerial Vehicle (UAV) equipment for photogrammetric reconstruction. The network of ground control points (GCP) has been the base of the survey. With the help of a GNSS receiver, a network of eleven ground control points has been carried out, covering with a uniform distribution on the ground the entire area all around the monument and the neighborhoods. Then, a TLS acquisition (Fig.  3), connected to GNSS survey of 12 stations around the monument, allowed to fully cover all the lower sides of the building and the environment around it. The choice of the laser scanner station placements has been accurate, taking into account differences in the height, viewing angles, distances and, not least, accessibility and visibility problems in the surrounding areas. As instrumentation, a Faro Focus3D X 130 terrestrial laser scanner has been used, obtaining a level of accuracy of about 5-10 millimeters (Tab. 1). However, the point cloud restitution (Fig.  4) couldn't cover the roofs and the hidden surfaces that were not visible from points of view on the roads surrounding the church.  Here comes into play the micro UAV equipment, the DJI Tello, that allowed to take pictures all around the monument, flying at 17 meters about over the ground level. A chunk of 96 images has been obtained, flying all around the roofs of the church, in order to cover all of the missing surfaces of the TLS survey. The collected photos have been next processed for the photogrammetric restitution, based on SfM algorithm, obtaining a dense point cloud of all of the roofs, the dome and the vertical walls of the church.

3D modelling.
Survey operations allowed the restitution of two different dense point clouds, the first obtained from the TLS survey, and the second obtained from the photogrammetric reconstruction of the images acquired by micro UAV equipment. Obviously, the level of definition of the two dense clouds was different because the density of the TLS point cloud restitution was higher than the UAV point cloud. The matching of the two dense clouds has been an interesting challenge to carry on because this procedure allowed to exploit different survey acquisition technologies. The process followed, due to the different origin of the point clouds, didn't allow their automatic alignment. The manual intervention has been, in fact, necessary and obviously the risk of losing the necessary level of accuracy was enhanced. Two solutions have been explored for the mesh modelling of the two clouds (Fig. 5): -using the automatic mesh reconstruction procedures of the acquisition software, both for the TLS and photogrammetric solution, and, then, integrating the two meshes in a unique geometric result, generating the final model; -exporting the separated point clouds and merging them using external software, and then meshing and texturing the final model.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 5. the 3D modelling tested solutions.
In the first case, the dense cloud reconstruction generated from the TLS acquisition, based on GNSS survey, has been chosen as reference. The acquisition software, Faro Scene, gives few options for mesh generation, like the gap filling, the smoothing, the color generation and the number of triangles. In this process, the need to have a low-sized geometry has been taken into account in order to produce a final model easily shareable on WEB. So, in this phase, it has been decided to create a mesh with a medium number of triangles. Also, the automatic gap filling has been maintained at a low-medium level in order to avoid the creation of incorrect excess mass that could complicate the cleaning operations in the next 3D modelling phase. The SCENE software allows the texture reconstruction from the point cloud, but this operation doesn't involve any external photo integration (for instance from the UAV), considering only the color information of the points taken by the TLS camera. Considering the photogrammetric reconstruction of the second dense cloud, based on the chunk of mini UAV acquisition, the Photoscan software has been used. The same software has been used for the next mesh reconstruction. With reference to the first case, the quality of the geometric reconstruction obtained in both of the clouds resulted similar. Considering the Level of Detail (LoD) of textures, both solutions achieve a good quality for the creation of a WEB navigation model. The generated meshes have been then exported with relative textures to the open-source 3D modelling software Blender for further necessary editing implementations.  These operations highlighted the main weaknesses of this first solution. First of all, the orientation of the normals of the faces for each triangle, of the mesh, was casual. This represented a relevant problem, in particular for the WEB navigation of the 3D model, where the faces with negative normals are invisible. Then, the presence of many overlapped geometric elements, like vertices, edges and faces, affected the size of the file and denied some automatic operations on the 3D model. Hence, the first action has been cleaning up the model, operation that, fortunately, can be done automatically. Instead, the orientation of normals required manual intervention. As said before, this final mesh was composed by a low number of triangles (Fig. 6), so it didn't need further geometry decimation. The manual filling of holes and adjustment of the texture in some areas was instead a necessary operation (Fig. 7).
In the second followed solution, both point clouds (from the TLS acquisition and from the photogrammetric reconstruction) have been exported into the 3D point cloud editing and processing open-source software Cloud Compare. In this case, it has been possible to correctly calculate the orientation of the vectors of the entire environment, necessary for the next mesh modelling. The clouds have been aligned ever considering the TLS dense cloud as the main reference. The 3D mesh has been obtained with the Poisson Surface Reconstruction, managing the main regulation parameters appropriately. In particular, these parameters have been chosen: Octree depth 11, samples per node 5, full depth 10, point weight 20. At the end of this operation a complete 3D model of the monument has been obtained, instead wasn't possible the automatic texture generation. Once again, the final model has been modified in Blender. This software allowed some automatic procedures necessary for the geometry decimation and the merging of vertices, very useful to decrease the weight of the model and the size of the file. Anyway, the manual modelling operations was necessary for eliminating redundant surfaces automatically created during the Poisson reconstruction, and some holes have been filled to generate a uniform mesh model ready to be textured. The texture has been initially realized through the projection in Blender of the orthographic views relative to every side of the monument. These projection views of facades have been obtained from the color information of the point clouds, increasing the size of the point visualization in Cloud Compare.
However, the quality of the result was not much different from the texture visualization obtained in the first solution. For this reason, another photogrammetric survey has been carried out in situ to take a large number of photos of the four facades of the church from different angles. The next photogrammetric reconstruction of this last chunk of photos has been necessary for improving the texture quality of the final model. The pictures have been straightened and merged in an orthomosaic for every facade in Agisoft Metashape. The obtained orthophotos have been then projected on the 3D model through Blender, creating a realistic virtual representation of the monument (Fig. 8-9).  In the end, this last reconstruction proved to be the best solution, because the final geometry and the final texture has been more homogeneous, maintaining, at the same time, a limited size. Hence, this model has been chosen for the next development of the WEB navigation model. The features achieved in size, geometric quality and texture definition allowed a possible further implementation of the 3D model into an AR application. In fact, both AR and VR technologies often need the same 3D modeling requirements.

The WEB navigation model.
As said before, the 3D model obtained through the second solution has been used for the creation of a WEB navigation system developed for a virtual exploration of the exterior parts of the church. The followed solution allowed the free online fruition of the model, with an HTML5 compliant template, without the necessity of any application to install on the client-side. Using a Blender's plug-in based on WebGL structure (Fig. 10), Verge3D, the model has been exported in an HTML page. Figure 10. Positioning of the points to view relatively to the WEB navigation model.
The interface of this Blender plugin allowed to define the scene in order to respond to specific actions of users and to create an interactive navigation model (. The created application works both on desktop and mobile devices (pc, tablet and smartphone), and a path around the monument, with a navigation system on click-based, has been developed: when users click on a specific point, the camera changes its position going to that point and offering a new perspective of the church. The environment visualized into the WEB application involved not only the church but also the neighborhoods around it, to give the user a realistic navigation experience. In detail, the outdoor environment that surrounds the building has been recreated using the spherical photos taken by the Laser Scanner. In order to achieve a good integration between the 3D model visualization and its surroundings, the spherical photos have been projected into 3D spheres that involved the monument (Fig. 11-12). Figure 11. The creation of the environmental spheres in Blender that involved the 3D model of the monument.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 12. Positioning and texturing of the environmental spheres.
Each sphere allowed to obtain a 360 degrees visualization relative to the point of view of its center position. In correspondence of the center of every sphere, it has been positioned a 3D object. This, once clicked, moves the camera to its location. The visualization of each sphere is inactive until the navigating user clicks on the object positioned on the corresponding center. In that instant the sphere becomes active, offering the user a realistic 360 view of that position (Fig. 13). Figure 13. Visualization of the result obtained from the combination of a sphere and the 3D model.

RESULTS AND CONCLUSIONS
In this paper, a comparison between different approaches regarding survey and 3D modelling finalized to virtual fruition has been discussed, considering, at the same time, the possibility of their integration. Moreover, particular attention was given to outline a workflow finalized to the creation of a 3D virtual navigation model of CH accessible through the WEB. In the followed case of study regarding the virtual reconstruction of the church of 'San Giorgio dei Genovesi', the mixed approach to the survey has been revealed to be the best solution considering the large extension of the target and the use designed for the final model. In fact, an advanced level of detail of the geometric model was not enough to give a realistic virtual representation, but, at the same time, a good definition of texture was fundamental. Considering these aspects, TLS acquisition has been necessary for guaranteeing the geometric accuracy of the 3D digital model, while the digital photogrammetric reconstruction solved the task of a detailed texture tessellation of monument surfaces. This last requirement, in fact, was not obtainable from the resolution of TLS camera acquisition because of low resolution of laser scanner camera. Moreover, the integration of photogrammetric and UAV technologies has been fundamental for the acquisition of the geometric information of the surfaces not visible from TLS survey, like the roofs. Hence, only the integration of TLS and photogrammetric reconstruction could satisfy these requirements, necessary for the virtual fruition of the model. Regarding the modelling phase, another aspect to consider was the size limit of the geometric model, necessary for guaranteeing correct and fast WEB navigation. For this reason, it has been necessary to find the best balance between the level of accuracy in geometry and the texture resolution in order to obtain a realistic virtual model ready for the WEB browsing. Indeed, a higher texture level of detail made up for the shortcoming linked to the necessary geometric simplification of the final 3D model. The two approaches studied in the 3D modelling phases shown that the integration of the point clouds (generated from TLS and photogrammetric acquisition) and the next common mesh reconstruction represents the best solution for the creation of a 3D model finalized to WEB navigation. In the end, the approach followed in survey and 3D modelling operations, that mixed different technologies and methodologies, turned out to be the best solution. The risk of this approach is the time to spend in processing phases considering the huge amount of data to manage, that could be definitely more than the time spent following only one methodology. For this reason, the definition of the workflow delineated in this work offers an exhaustive solution to optimize these processes and achieve the best result on WEB navigation. The final integration of the model into the WebGL environment developed with Verge 3D Blender plugin allows smart WEB navigation in real-time, and the construction of the virtual visitors' path around the monument built with the 3D model of the church. Besides the 360° active spheres guarantee a realistic perception of immersive tour. To conclude, the presented work offers an integrative solution that allows to create a virtual navigation model of a real monument, with possible integration into an AR application, considering multiple technologies and testing different strategies. The followed workflow is finalized to the improvement of the accessibility for monuments and archaeological sites, through the development of virtual navigation models freely available on WEB. This research could be considered a further step in geomatics for the valorization of CH conceived as dissemination of cultural information.

FUNDING
This research was funded by Interreg I-ACCESS project -"Implementing the Accessibility to Urban Historic Center's Use and Knowledge", project code C1-1.1-41 -Axis 1.