THE VIRTUALIZATION OF CH FOR HISTORICAL RECONSTRUCTION: THE AR FRUITION OF THE FOUNTAIN OF ST. GEORGE SQUARE IN VALLETTA (MALTA)

Improving accessibility to Cultural Heritage (CH) is an increasingly urgent challenge today. It is not only a matter of physical inaccessibility but also temporal, considering that part of CH now lost. Fortunately, the most modern technological tools are helping to break down both space and time barriers. In facts, recent advances in representation, 3D modelling and survey methodologies opened new scenarios for valorization and conservation of CH. In particular, the improvement of quality in resolution and sensor sensitivity of cameras allowed to achieve the right level of 3D reconstruction through digital photogrammetry procedures. In the same field, terrestrial laser scanners (TLS) allowed acquiring dense point clouds of complex environments with a millimetric level of accuracy. At the same time, the application of Augmented Reality (AR) and Virtual Reality (VR) technologies is an excellent solution for improving the accessibility to monuments, museums and archaeological sites. It is possible to share new levels of information about CH, in space and time, for touristic, managerial and scientific aims. This work is focused on the virtualization of CH, considering the study case of the fountain of Wignacourt, today present in St. Philip Garden in Floriana and initially located in Valletta (Malta). The application presented allows the virtual fruition of the monument placed in its original location, St. George Square. A simplified plant of the square will enable tourists to make a temporal journey in the past with their mobile device. The work is part of the Interreg Italia-Malta European project named I-Access, dedicated to the improvement of CH accessibility. It focuses the attention to the experimentation of new specific procedures in Geomatics necessary to solve big data issues of complex environment visualization.


INTRODUCTION
In recent years digital technologies have offered significant advantages to the world of CH (Ioannides et al., 2017;Ramos et Remondino, 2015, Fissore et al., 2018. Knowledge of sculptures or monuments can be shared more efficiently thanks to the use of modern devices (Chianese and Piccialli, 2014). The use of smartphones and tablets for the virtual fruition of CH is a perfect solution for the study and exploration of architectural or archaeological goods (Scianna et al., 2016). The achievement of this level of CH digitalization is not simple and needs specific procedures in the survey, restitution and 3D modelling fields (Gonizzi Barsanti et al., 2013). In the area of survey operations, recent advances in technology have become fundamental for this aim. Through the application of photogrammetric reconstruction, allowed by the Structure from Motion (SfM) algorithms, it is possible to create 3D dense point clouds of complex environments starting from chunks of photos taken by Single Lens Reflex (SLR) cameras. The association of this technology with a network of ground control points acquired by a global navigation satellite system (GNSS) survey allows obtaining a 3D representation of large environments with a centimetric level of accuracy. Moreover, the recent diffusion of several Unmanned Aerial Vehicle (UAVs) vehicles opened new possibilities to the digital photogrammetric acquisition of CH (Chiabrando et al., 2018;Vacca et al., 2017). Furthermore, Terrestrial Laser Scanner (TLS) technology offers today significant improvements in survey operations and digital reconstructions of monuments and historic buildings (Dominici et al. 2013), allowing the reconstruction of dense point clouds with millimetric accuracy. Regarding the restitution and the 3D modelling phase, 3D data management software and tools are now improved and made fast, also becoming accessible to a broader audience. VR and, even more, AR offer new immersive ways to explore CH. The creation of three-dimensional meshes starting from point clouds allows a realistic reconstruction of the detected objects, with the help of high-quality texture application. The possibility of overlapping virtual additional information to the real monuments, thanks to the increasingly widespread use of smartphones and tablets, guarantees particularly effective dissemination of knowledge as it involves the user through the charm of the game and entertainment. The virtual objects overlapped on reality can show past CH configurations no longer existing or architectural elements that still exist but which are no longer in their place of origin. The latter case is the theme of the application covered by this article. In the next sections of the paper, after a look at state of the art on CH virtualization, the case study concerning the virtual repositioning of the Wignacourt Fountain in its original location (S. George square in Valletta) through the use of augmented reality will be described. All the phases, from the survey to the creation of an application on accessibility to CH, will, therefore, be analyzed.

VALORIZATION & VIRTUALIZATION: THE NEW LIFE OF CH.
The classic approach of CH valorization is based on the concept of conservation of cultural goods. Critical conditions of many monuments and archaeological sites caused by anthropic and natural factors have ever led the interest of scientists to safeguard the structural conditions of ancient buildings. Recently, with the evolution and the growing of the touristic sector, the CH valorization is conceived from a new and more complex point to view. Besides the conservation, the quality of fruition of the cultural good acquired even more importance. The dissemination of CH knowledge become a fundamental aspect of its valorization. A right level of fruition could be hence useful for better conservation of the cultural good because it stimulates any funding for necessary restoration interventions. In this field, the level of accessibility of CH sites takes primary importance, and the development of AR and VR technologies play a crucial role in experimenting new ways of fruition based on virtualization (Bruno et al., 2009, Skarlatos et al., 2016. At the same time, the evolution of rapid prototyping processes recently allowed to generate scaled 3D printed models with an advanced level of accuracy (Balletti et al., 2017, Inzerillo and Di Paola, 2017, Scianna and Di Filippo, 2019. The use of multiple technologies in the field of CH contributes to enhancing both the visitors' experience and the work of those involved in the management of CH. Virtual reality, developed for the first time for videogames and recently also for educational games (Mortara et al., 2014), allows the virtual navigation of the monument, simulating the real navigation in situ. Augmented reality instead enriches the real visit with additional information also in the form of entertainment. Finally, rapid prototyping combines the previous experiences with the tactile one, which is interesting for both blind and visually impaired people (Jung et al., 2017). These technologies opened so new scenarios of accessibility, allowing the fruition of CH also for people with visual or motor disabilities. The virtual fruition can't be seen as a substitute of the real one, instead represents a powerful tool for the diffusion of its knowledge and the generation of a complementary level of information. AR, in particular, is a technology that can improve the virtual fruition of CH (Clini et al., 2017). The use of mobile devices (like smartphones and tablets) with specific targets makes it possible to explore an element belonging to CH in a more immersive way. For this reason, AR is a particularly powerful tool if applied in the context of the enhancement and use of CH. The possibility of superimposing virtual elements on reality, which is how it is augmented, mainly stimulates the user who, in this way, feels himself much more involved in exploration and discovery. The overlap can concern the reconstruction of missing parts, the insertion of animations, writings and everything that contributes to help a more in-depth knowledge of what is observed . The rapid spread of this technology is mainly due to its economic accessibility, ease of use, and the vast diffusion of devices that support it. The work shown in this paper represents just an example of how AR and VR technology could be applied to CH valorization, allowing users to discover the ancient historical setting of St. George Square, different from today's layout.

THE AR FRUITION OF THE WIGNACOURT FOUNTAIN IN ST. GEORGE SQUARE IN VALLETTA
The study case considers the fountain of Wignacourt (Fig. 1) immersed in the ancient historical context of St. George Square in Valletta (Malta). This monument was built in 1625, representing the first fountain connected to the Wignacourt Aqueduct and was initially located in the centre of St. George Square (Fig. 2). The original shape, the decorations and the placement itself were modified along the centuries. The Grand Master Pinto modified the structure of the fountain in 1746, and the monument was afterwards relocated close by the Palace of Justice and, finally, settled into St. Philip Garden in Floriana, where it is possible to admire the monument today.  The changes that involved the monument represent an essential chance to test the possibilities offered by survey and 3D modelling restitution finalized to creation of a virtual journey along the centuries supported by AR technology. Both the fountain and the ancient configuration of the square have been The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK objects of research. The aim was to offer users a virtual journey in the past through AR. The workflow followed in this application considered several necessary steps: the historical documentation, survey operations, 3D reconstruction and 3D modelling, creation of AR and VR applications (Fig. 3).

The historical documentation.
The workflow started from historical documentation, necessary to define the original configuration of the fountain inside the square and the facades involved in the represented environment. Some ancient representations of the square have given useful information about the original configuration of the spaces. Still, the lack of philological documentation about plants and facades of buildings led not a few difficulties for virtualization. Anyway, typological research has been carried out, and the 3D representation has been left basic, where the acquired historical information was not enough. This disposition has been followed to avoid the generation of historical fake. The found historical documentation shows the presence of the fountain inside the St. George square in the early 18th Century (Fig. 4-5). This one wasn't the only difference in the square configuration from the today asset. In fact, in the original configuration, it's possible to observe the total absence of the North-West facade portico and the presence of the Verlade Column in the North-East side of the square. In the end, the lack of the portico and the presence of the obelisk and the Wignacourt fountain change the perception of the original square.

Survey operations.
The geometric information necessary for the virtual environment reconstruction has been acquired during survey operations. During this step, different technologies have been experimented, considering the dimensions and generally the geometric features of the cultural goods involved, the St. George square and the Wignacourt fountain. The square has been surveyed using TLS instrumentation, in particular Faro Focus3D X 150, obtaining a level of accuracy of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK about 1 cm. Six scans have been necessary for acquiring the essential information for the reconstruction of an exhaustive point cloud of the square (Fig.6). Instead, the Wignacourt fountain, today placed into St. Philip Garden in Floriana, has been surveyed through digital photogrammetric reconstruction, starting from a chunk of 209 photos taken from different angles using an extensible pole, acquiring the whole surface of the monument at different levels (Fig. 7).  An SLR camera Sony Alpha 6000, with a resolution of 24 megapixels, has been used for these operations, to obtain the necessary geometric accuracy but also a high level of texture details. The main dimensions of the fountain have been obtained with TLS instrumentation, Faro Focus3D X150. In the case of the geometric reconstruction of the fountain, the digital photogrammetry solution has been preferred to TLS, considering the more homogeneous definition of the dense cloud granted by this choice, avoiding the generation of shaded areas not visible from the laser scanner. The fountain has been reconstructed obtaining a centimetric level of accuracy, with an error of about 1,5 cm (Tab. 1). In the last phase, the main environment is prepared for VR and AR fruition. It represents the base of the final result. In particular, the mesh imported in Blender needs to be further simplified, eliminating clusters and correcting the normals of all the polygons. The maximum size of the 3D model is a substantial limitation that represents the main challenge of CH fruition in AR and VR applications. Considering this limitation, the quality of mesh texturization played a crucial role in obtaining a high level of realism. For this reason, different chunks of photos have been taken to be projected in all the facades of the mesh model through UV mapping procedure. Hence, the main environment was ready (Fig. 8). The other elements to insert into the virtual visualization were the Verlade Column and, obviously, the fountain itself. The Verlade Column has been modelled following a historical painting, that represents the best reference of this architecture left to the present The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-4/W1-2020, 2020 3rd BIM/GIS Integration Workshop and 15th 3D GeoInfo Conference, 7-11 September 2020, London, UK day. For this reason, it has been chosen to model only the main shape of the monument, leaving the details (Fig. 9). The perspective of the painting allowed following the correct proportions to apply the exact measures in Verlade Column reconstruction (Fig. 10).  The Wignacourt fountain has been reconstructed with the digital photogrammetry process using Agisoft Photoscan software. It allowed the construction of the dense cloud and the processing of the textured mesh. The final version of the model is well detailed and not oversized, all requirements necessary for VR and AR visualizations (Fig. 11). Once reconstructed all the models involved into the 3D environment, the final 3D configuration has been developed in Blender software, correctly positioning the Verlade Column and the Wignacourt fountain inside St. George square, according to historical references.

The VR and AR app development.
The virtual environment created in Blender software has been imported in Unity software in conjunction with the Vuforia plugin for the creation of the AR application. The 3D model should be associated with a 2D target reference. The aim of the experimentation is the creation of an AR/VR application connected with a totem settled in loco inside the real St. George Square. Hence, the target should be a 2D element printable and replicable into the totem installation. The choice has been to use a schematic representation of St. George Square plant with the indication of the original position of the fountain (Fig. 12). Figure 12. The AR visualization of St.George Square connected with the 2D target reference. The test has been carried out using the 2D target printed in a sheet of paper.
Tourists could download the application through a QR code and directly frame the 2D target printed into the totem installation. In this way, the virtual representation of the square in its past configuration, with the Wignacourt fountain in its centre, will appear. By clicking inside the fountain model, it's possible to navigate closer to the fountain opening a browser window, allowing tourists to admire the details of the monument (Fig. 13). This VR implementation is realized with the Blender plug-in Verge 3D, using WebGL libraries. These tools based on javascript and HTML5 compliant will enable the web visualization of 3D models with browser navigation. About the usability, the AR application developed is Android operating system compliant and available for tablets and smartphones. The WebGL visualization is compliant with the most common browsers (Chrome, Safari, Firefox, Edge). Figure 13. The browser visualization on internet of the fountain in WebGL.

CONCLUSIONS AND OPEN SCENARIOS
This experimentation is a real example of enhancement of the level of CH accessibility, offering an added value to the real fruition of cultural goods. Much attention has been paid to the survey techniques adopted for the three-dimensional reconstruction of the CH to highlight and better define a specific workflow. The experimentation has brought to light some interesting aspects of the combined application of TLS acquisition and digital photogrammetry. Both of them provide geometrically accurate point clouds, integrating the reference measurements of TLS acquisition with the processing of photogrammetric reconstruction. This process allows to achieve a high level of geometric accuracy and high quality in texture resolution. Particular attention should also be paid to the reconstruction of missing parts, considering the correct historical configuration of the place to be represented. About avoiding historical fake and errors, historical representations and paintings have proved to be fundamental and particularly useful as a reference for geometric virtual reconstructions of missing monuments. Starting from these data and combining the techniques of graphic representation, it is possible to make a basic reconstruction of cultural goods, as in the case of the Verlade Column. In the end, the development of a reliable historical reconstruction of a cultural asset is a fundamental aspect to consider for the creation of a virtual fruition experience in the past. The creation of the AR application, which allows virtual access to the artistic heritage and arouses the interest of users, is only the final step of the process, and its success depends on all the work that precedes it. The opportunity offered by the AR application with the discovery of the ancient configuration of St. George Square does not replace the real exploration. Still, it integrates it, valorizing the experience in situ. It proves to be a particularly powerful tool because it allows to reach a superior knowledge of what is observed. This study case is an example of how geomatics and computer science applied to monuments or cultural sites could give an essential contribution to the valorization and enhancement of accessibility to CH and its history. This kind of application could be applied in several examples of monuments, opening new scenarios to a new smart approach finalized to CH valorization.

FUNDING
This research was funded by Interreg I-ACCESS project -"Implementing the Accessibility to Urban Historic Center's Use and Knowledge", project code C1-1.1-41 -Axis 1.