3D Virtual CH Interactive Information Systems for a smart web browsing experience for desktop PCs and mobile devices

: Recently, the diffusion of knowledge on Cultural Heritage (CH) has become an element of primary importance for its valorization. At the same time, the diffusion of surveys based on UAV Unmanned Aerial Vehicles (UAV) technologies and new methods of photogrammetric reconstruction have opened new possibilities for 3D CH representation. Furthermore the recent development of faster and more stable internet connections leads people to increase the use of mobile devices. In the light of all this, the importance of the development of Virtual Reality (VR) environments applied to CH is strategic for the diffusion of knowledge in a smart solution. In particular, the present work shows how, starting from a basic survey and the further photogrammetric reconstruction of a cultural good, is possible to built a 3D CH interactive information system useful for desktop and mobile devices. For this experimentation the Arab-Norman church of the Trinity of Delia (in Castelvetrano-Sicily-Italy) has been adopted as case study. The survey operations have been carried out considering different rapid methods of acquisition (UAV camera, SLR camera and smartphone camera). The web platform to publish the 3D information has been built using HTML5 markup language and WebGL JavaScript libraries (Three.js libraries). This work presents the construction of a 3D navigation system for a web-browsing of a virtual CH environment, with the integration of first person controls and 3D popup links. This contribution adds a further step to enrich the possibilities of open-source technologies applied to the world of CH valorization on web


INTRODUCTION
The valorization of CH, especially in recent years, is strongly connected with the diffusion of knowledge on cultural goods and archaeological sites, for scientific and touristic aims.The possibilities of virtual 3D reconstructions of CH have grown considerably for the recent development of new indirect techniques on survey operations (Gonizzi Barsanti et al, 2014).These techniques make use of Unmanned Aerial Vehicles (UAV ) technologies and Global Navigation Satellite System (GNSS) in the short-and close-range domain, giving, in fact, new low-cost solutions on aerial image acquisition in the fields of archaeology and architecture, due also to the growing development of compact digital camera resolution (Rinaudo et al, 2012).In this scenery, new photogrammetric reconstruction methods offer the possibility of obtaining high-resolution datasets with a user-friendly and cheaper procedure.These methods are based on Structure from Motion (SfM) algorithms (Westoby et al, 2012), that allow to extract 3D point clouds of an environment from the acquisition of many overlapping photographs.Anyway, these new acquisition technologies based on photogrammetry haven't just achieved the same precision obtainable with traditional topographic equipments like laser scanners, but their use is daily increasing due to their improvements and the development of new algorithms for data processing (Piras et al., 2017).For example, potential improvements have been achieved including information of the navigation system into the 3D reconstruction algorithm (Fissore et al, 2017).Furthermore, the diffusion of heritage digitalization has been increased from a rising demand, that lead researchers to experiment new cross cultural approaches for educating people on Monuments based on the EU Europeana Space project (Ioannides et al, 2016).At the same time, the spread of fast internet connection for desktop and mobile devices has strongly increased the needs of web browsing for sharing information all over the Internet.Nowadays people prefer to access Web information and services using smartphones and tablets rather than desktop Pcs.In this scenery, virtual reality (VR) technologies applied to desktop Pcs and specially to mobile devices have been even more diffused.Furthermore, the development in recent times of Information and Communication Technologies (ICT) has given a fundamental support for the documentation and the digitalization of CH.Besides the development of Digital Heritage interactive representations become even more common, opening new possibilities on the implementation of virtual and augmented reality technologies of CH exhibitions (White et al, 2007).

THE DEVELOPMENT OF VR ENVIRONMENTS FOR THE NETWORKING
Considering these scenarios, the development of HTML5 elements and technologies, based on the JavaScript Web Graphics Library (WebGL) has given a smart solution for the creation of virtual environment on web (Nazarov et Galletly, 2013).HTML5 elements and JavaScript features generally have been experimented for the development of games, giving developers the possibility of creating 3D environments and interactive real-time interactions (Curran et George, 2012).In the compound of VR, many studies have been proposed in last times, focusing the attention on tracker orientation for first person controls in order to obtain a more realistic experience Commission II, WG II/8 (Smith and Graham, 2006), a fundamental aspect to consider for the construction of 3D navigation systems.The choice of the WebGL open-source technologies has allowed to load a 3D scene inside an .htmlpage, giving users the possibility of visualize 3D models, navigating into a virtual 3D environment without the need of any app installation on the client-side.These libraries inserted into an HTML5 environment have been used also for creation of home automation applications, enabling users to control home appliances navigating into a virtual model (Cheong et al, 2012).Besides, these JavaScript and WebGL technologies presents some issues related to latency sufferings (due to the difficulties in downloading of complex 3D models) and adaptation to client devices (Lavoue et al, 2013).In the light of these considerations, the creation of a 3D interactive information system for a free web navigation of cultural goods represents a smart solution for developing a web browsing experience on CH, that could work both on desktop and mobile devices.The possibilities for visitors, during a first person navigation, of receiving also historical information make a more exhaustive experience, useful for the diffusion of knowledge about cultural goods.The developed system, applied to the cultural good of the Arab-Norman church of the Trinity of Delia, is the result of the multidisciplinary use of different technologies, carried out in successive steps.

THE WORKFLOW
The work has been divided in different steps, where every phase needed different skills in different sectors: -acquisition of historic information on the church of the Trinity of Delia; -survey operations in external and internal environments; -photogrammetric restitution of surveyed environments; -3D mesh simplification for web-browsing; -networking of the 3D interactive environment with WebGL libraries, developing a first person navigation control for desktop and mobile devices.

Historical information about the church of the Trinity of Delia
The first phase of the work has been the collection of historical information of the cultural good, and, at the same time, the execution of a site survey for discovering the environments.This first approach to the monument is fundamental for focusing the main points of interests and the environment conditions in order to correctly plan the next survey operations.The acquisition of historical references is also fundamental for collecting semantic information to be implemented on queries for the 3D navigation of the final model.The Trinity of Delia was founded in the first half of the twelfth century and is considered one of the major examples of Arab-Norman architecture in Sicily.
The plan with a square shape, in fact, leads some scholars to consider the construction of the building even from the previous era, associating the type of "cuba byzantine".The artifact is presented in the perfect geometry of the original configuration thanks to the fundamental historical contribution made by the architect Giuseppe Patricolo in 1880 (Volpe, 2007).He, in fact, instructed by the owners to design a hypogeum plan below the building, immediately realized that the overlapping surfaces of the building, layered over the centuries, indeed, concealed an important monument of the XIIth Century.Subsequent restoration work immediately gave the architect the right to restore the entire building, bringing it back the original architectural configuration.In particular the perfectly square shape, the 4 columns that support the central dome, the perfect symmetry of the parts, the architectural language and the materials are just some of the analogies that make the Trinity of Delia one of the most important Arab-Norman monuments in Sicily.

The survey operations
One of the purposes of the work has been to test different ways of rapid acquisition for the subsequent photogrammetric reconstruction.In order to correctly scale the 3D models, a local network of distributed markers based on a local reference system has been created.
Both for the survey of the external and the internal environment, the main geometries have been measured using a laser distance meter, considering an exhaustive number of recognizable Ground Control Points (GCP).During the image capture operation, different solution have been followed for the survey of the external and internal environments.
In particular, the external environment has been reconstructed using two different groups of images, obtained from two different close-range solutions, one on terrain and another on flight.The network of images has been obtained taking pictures from different points and angles, trying to avoid issues caused by bad light situations.

The photogrammetric restitution
After survey operations, photogrammetric restitution allowed the reconstruction of external and internal environments.For this operation the software Agisoft Photoscan, based on Structure for Motion (SfM) algorithms, has been used, importing and aligning images, and correcting the results using the network of GCPs.Before reconstruction, every camera has been carefully calibrated, in order to obtain the best possible results.The best reconstruction results have been obtained, as we could imagine, from the images taken from the survey of the external environment (using UAV and SLR camera), even if the internal reconstruction obtained by the smartphone camera has achieved surprising exhaustive results.
In particular, external light conditions and the position of the building in relationship with the near presence of vegetation lead two sides of the external surface in the shade, causing some difficulties for the reconstruction based on UAV camera images.
In fact, the integration of SLR camera images with UAV images solved these problems, increasing the quality of the reconstruction of external facades.The internal environment reconstruction has been satisfactory, except for the glossy surfaces of the tombs and the columns, that have been modeled in subsequent steps.
Figure 8. Dense cloud reconstruction of the internal environment using smartphone camera.

The 3D mesh simplification
Once obtained the 3D meshes from the photogrammetric reconstruction, they were processed in next steps, in order to be implemented into a WebGL system for the web-browsing experience.The 3D models have been necessary strongly simplified, in order to maintain, as much as possible, the original appearance, but, at the same time, to become lighter.For this operation it's been used Agisoft Photoscan, the same software used for the restitution, in order to strongly simplify the number of triangles, but at the same time to maintain a good texture Level of detail (LoD).
Figure 9.The 3D mesh of the external model in Blender.
After this first simplification, the 3D models have been imported into another software, Blender (a professional opensource modeling software), where the internal and external environment have been revised and improved for creating a realistic reconstruction for the networking.
In particular, regarding the external environment, the reconstructions from the UAV (close-range on flight images) and SLR camera (close-range on terrain images) acquisition have been matched, creating a unique 3D model.During this operation the 3D elements of the surrounding environment that wouldn't be visible on the web-browsing experience have been deleted, in order to make the model lighter.Considering the internal environment, the columns have been recreated and the meshes of the tombs have been pushed and appropriately modeled.Also the texture associated to the internal model has been edited, in order to uniform lights and details of 3D surfaces.
In the lights of what has been done, the 3D model restitution could be only considered a starting reference point for the next creation of a web-browsing experience, because many procedures should be done until a usable 3D model could be ready for the networking.The models, should be strongly simplified, because the final 3D environments must not exceed 10 Mb, in order to avoid trespassing of browser download limits.In fact, it's necessary to find the right balance between number of polygons of 3D meshes and the Level of Details (LoDs) of the textures.

The networking of the 3D virtual interactive environment
Every 3D simplified mesh of the environment has been exported using .objformat, associated with an .mtlfile and textures.The networking has been developed creating a specific WebGL template based on HTML5 standards, inside the main .htmlpage.Some navigation functions, like movement controls have been managed connecting the main .htmlpage with external WebGL libraries (three.js),other ones have been directly developed inside the JavaScript code of the page, like light controls, loading properties, gravity system, popup operations and elements.These JavaScript strings have been properly modified inside the .htmlpage, and also some parameters have been personalized inside the external libraries.
Considering the WebGL environment, different 3D elements have been inserted, in particular external 3D models (that have been loaded as .objfiles) and internal 3D geometries (created inside the JavaScript lines).The 3D interactive virtual scene has been implemented with these 3D elements: -the loaded 3D model, that has been positioned on the centre of the scene; -a sphere geometry, that has been inserted in order to envelop the visible elements, where a 360°degrees photo of the surrounding environment has been projected on its internal surface (in order to add a realistic visualization); -a system of invisible boxes that define the navigation boundaries, 4 boxes on the 4 directions and another one that represents the ground; -an invisible sphere geometry that represents the navigator, which reacts to movement commands; -a system of spherical clickable popups, that once hovered, change colors.In order to offer a more realistic experience to users, a gravity simulator (phisi.js)has been implemented into the system, giving the possibility to choose the G force value, its direction, and the reaction of every elements.

The navigation system for mobile devices.
Compared to previous works, in this 3D virtual navigation system has been improved the customization of the interface for mobile devices.In order to make the navigation more userfriendly for tablets and smartphones, a system of first person navigation controls has been developed, based on the movement of a 3D invisible sphere.The idea was that the camera view could move using touch controls, following the direction of movement.Furthermore, navigation controls of the camera should react to the gravity simulator.Considering the satisfaction of these requirements an element of primary importance, an invisible sphere has been created, that could react to gravity, laid on the ground box like a marble on a table.In our case the "table" is horizontal and the force is vertical, but changing the force direction or the inclination of the ground surface, the sphere would roll on that direction.The developed system of controls allows to move the sphere all over the surface, activating different horizontal forces on every direction with "on click" functions.Thanks to these controls, the invisible sphere could navigate all over the model limiting the navigation on correspondence with the invisible borders.The camera has been merged to the sphere, avoiding the possibilities of rolling movements on its surface, but allowing the movement linked to the translation of the centre of the sphere.These operations have been followed both for internal and external environments.
controls in the WebGL environment.
In order to enrich the visualized information, the 3D navigation system should be completed adding historic and architectural descriptions of some representative elements of the environment.At the same time, the possibility of exploring spaces, passing from one environment to another was also considered an important ability to add.In order to satisfy these needs, a network of clickable popups has been created, where some popups have been linked to .htmlinformation windows (giving historical and architectural description of 3D elements), and other ones send visitors to other environments (for example in correspondence with doors).

RESULTS
The developed 3D interactive navigation system has been tested and the networking has achieved the satisfaction of user needs, guaranteeing a simple navigation using common browsers (Chrome, Firefox, Safari).
Considering previous solution developed in recent years (Scianna et al. 2016), this last work has obtained new goals, regarding the interface efficiency.With the aim of developing a smart solution, that should be more user-friendly as possible, and useful for visitors everywhere localized, the attention has been focused on the use of the system on mobile devices, with particular attention to the navigation controls and to the implementation of information windows.
In particular, the improvement of mesh simplification operations has lead to achieve a good LoD of 3D models during navigation, avoiding browser crashes.At the same time, the development of a first person "on touch" controls navigation, that reacts to the G force, with the possibility of eye-rotation, added a realistic impact to the virtual 3D visualization and a more user-friendly movement.

CONCLUSIONS AND OPEN SCENARIOS
In the light of developing new technological applications for CH valorization, this work consists in a smart solution for the fruition of cultural goods using VR open-source technologies, based on the free 3D online navigation through the use of personal devices (PCs, tablets, smartphones).The diffusion of these examples applied to archaeological sites and museums could allow, in the future, a further increasing of visitors, due to the spreading of knowledge of CH VR online representations.In fact, the possibilities of virtual interactive navigation can never replace the impact given by the real cultural good, but, instead, can increase the curiosity and the need of visiting the monument.In this scenery the development of new possibilities for adding semantic information to the 3D VR representation (considering connection to external databases) will be fundamental in order to enrich the virtual experience.In the future the multidisciplinary interaction between different sciences like architecture, engineering, archeology and computer science, will be even more relevant in order to evolve new solutions for CH valorization.

Figure 1 .
An external view of the Trinity of Delia.

Figure 2 .
Figure 2. The different phases of the workflow.

Figure 3 .
Figure 3.An internal view of the Trinity of Delia.

Figure 4 .
Figure 4.The different techniques followed during survey operations.

Figure 5 .
Figure 5. UAV survey for the reconstruction of the external environment.

Figure 6 .
Figure 6.Photogrammetric reconstruction of the external environment using UAV camera images.

Figure 7 .
Figure 7. Photogrammetric reconstruction of the external environment using SLR camera images.

Figure 10 .
Figure 10.The 3D mesh of the internal model in Blender.

Figure 12 .
Figure 12.An external view of the interactive navigation model accessible on web.

Figure 15 .
Figure 15.An internal view of the interactive navigation model accessible on web with the linked popup information window.

Figures 13 -
Figures 13-14.Schemes of the developed system of first person

Table 1 .
Calibration parameters of the Zenmuse X3 camera in Agisoft Lens.

Table 3 .
Calibration parameters of the smartphone Elephone P9000 rear camera in Agisoft Lens.