THE WINCKELMANN300 PROJECT: DISSEMINATION OF CULTURE WITH VIRTUAL REALITY AT THE CAPITOLINE MUSEUM IN ROME

: The best way to disseminate culture is, nowadays, the creation of scenarios with virtual and augmented reality that supply the visitors of museums with a powerful, interactive tool that allows to learn sometimes difficult concepts in an easy, entertaining way. 3D models derived from reality-based techniques are nowadays used to preserve, document and restore historical artefacts. These digital contents are also powerful instrument to interactively communicate their significance to non-specialist, making easier to understand concepts sometimes complicated or not clear. Virtual and Augmented Reality are surely a valid tool to interact with 3D models and a fundamental help in making culture more accessible to the wide public. These technologies can help the museum curators to adapt the cultural proposal and the information about the artefacts based on the different type of visitor’s categories. These technologies allow visitors to travel through space and time and have a great educative function permitting to explain in an easy and attractive way information and concepts that could prove to be complicated. The aim of this paper is to create a virtual scenario and an augmented reality app to recreate specific spaces in the Capitoline Museum in Rome as they were during Winckelmann’s time, placing specific statues in their original position in the 18th century.


INTRODUCTION
Conservation of Cultural Heritage, considered as a cultural capital, is a fundamental task not only for economic purposes but also for social and cultural development.Following this, the possibility of fruition of the Heritage is a main concern for the institution in charge of the management of our past because it allows everyone to be involved in the process of preservation.The use of 3D reality based technologies for documenting and preserving Cultural Heritage has been widely used since decades because of their accuracy and precision in the documentation (Remondino, 2011;Gonizzi et al., 2013).Despite the large diffusion and use of these technologies, 3D products do not fully accomplish their potential until they are not used to interactively communicate their significance to nonspecialists (Canevese, Gottardo, 2017).Virtual Reality is an important and easy-to-use tool to interact with 3D models, can help in adapting the museum's contents to a wider public, based on the different typologies of visitors (Reffat et al., 2013).The VR technologies are useful to create an environment inside which visitors can move and navigate, along with information as texts, images and 3D models that not only lead to a better comprehension of the site of Museum but also can recreated hidden or lost sites (Fernández-Palacios et al, 2017, Kersten et al., 2017, Deggim et al, 2017).The possibility to navigate a 3D virtual environment reconstructing the ancient legacy is fundamental for non-experts to better understand what they are looking at.In this way, the visitors will be also aware of the importance of our past and will be more involved in its conservation.This statement goes with the one related to the awareness that ancient buildings, sites or artefacts are our legacy and need to be preserved in order to permit their fruition and understanding to future generation (Malegiannaki, Daradoumis, 2017).Another important issue that goes with the use of VR in Cultural Heritage and Museums expositions is the educational approach that these installations can provide (Fonseca et al, 2017).Using VR installation, the knowledge of a site, a building or an artefact is increased because along with the model of the item in 3D metadata and documentation are added in a way that better explain the importance of what is shown.The use of 3D virtual environment as a powerful tool for allowing a deepen knowledge of Cultural Heritage to nonexperts has been widely investigated (Gonizzi et al, 2015) also with evaluation of the VR systems in Museums through uses experience (Barbieri et al., 2017).VR technologies have been also used for virtually restore ancient artefacts that confirm the importance of the use of this technologies in the field of CH (Liarokapis et al., 2004;Girbacia et al., 2013).Regarding web-based services and their deployment in a museum, the growth of open-source, scalable and crossplatform runtime environments such as Node.js 1 offer developers a powerful and portable ecosystem to deploy serverside services (Wok et al., 2017) within public exhibits.Performance-wise, past researches (Charland et al, 2011) already investigated advancements of web technologies compared to native applications.Nowadays the gap is further reduced, open specifications such as WebVR 2 are already employed to deliver 3D content still maintaining 90fps, and javascript overall performances consistently improved on mobile devices during the last 5 years.Major players such as SketchFab3 took advantage of such technologies in order to create a "universal viewer" with scalable approaches.Opensource WebGL libraries such as the well-known three.js4are used in several web-based 3D visualization (Rego and Koes, 2014) and also games (Sukin, 2013).Solutions like 3D-Hop (Potenziani et al. 2015) allows the creation of interactive visualization for complex multi-resolution models, although support for mobile is still not complete and advanced presentation features (WebVR, PBR5 , etc..) are not provided out-of-the-box.X3Dom (Limp et al., 2011) is also used to present 3D model collections interactively, resulting from common acquisition, processing and optimization workflow.The aim of this project is to create a virtual and augmented reality app to show the spaces in the Capitoline Museum in Rome as they were during Winckelmann's time, placing specific statues in their original position in the 18th century.The process involved the use of photogrammetry to acquire the models, with a double goal: provide the curators of the Museums with high-resolution 3D models to create a digital database of the statues and create simplified models to be put in the VR application.A virtual environment was created of specific rooms of the Museums where the statues were placed in their original position and, finally, a 3D infrastructure was built to create the interactive application.The interactions in the VR environment used in CH are different and always include the use of devices, able to support the user's intents to the digital contents.In this project, it was requested to create an app to be installed inside the selected rooms in the Capitoline Museums and used by visitors to navigate the reconstructed space.This work is related to the exhibition "Il Tesoro di Antichità.Winckelmann e il Museo Capitolino nella Roma del Settecento", that aims at celebrating the anniversaries of birth and death of Johann Joachim Winckelmann (1717-1768), the founder of modern archaeology.

METHODOLOGY
The focus of the project was to create a VR platform to be used by visitors for understanding the shape of the rooms in the Capitoline Museums during Winckelmann's period.The request from the customer was to have 3D models of specific statues that had to be placed in their original position in the 18th century in a virtual environment representing the rooms of the museums.This meant the 3D survey of the selected statues (76 in total), the simplification of the final 3D models and the creation of the environment and the infrastructure.The first part of the project regarded the survey of the objects.The technology selected for the survey was photogrammetry, considering the shape, texture and material of the statues and also their position in the rooms of the Museum.Six statues have been modelled in a CAD environment due to the impossibility of a photogrammetric survey or because they were loaned to other expositions.The second step regarded the simplification of the 3D model in order to obtain models suitable to be inserted in the virtual environment, allowing a lighter and faster rendering.The last part of the project involved the creation of an infrastructure for the 3D visualization of the data.

Photogrammetry
The survey of the selected statues was done through photogrammetry.The technique was imposed by the position of the statues close to the walls, the short time available for the survey and the post processing and the material of the statues (Figure 1a-c All the selected statues (more than 70) were captured with APS-C camera: a Canon 60D coupled with a 20mm lens, a Canon 600D with 18-55mm lens, a Canon 100D coupled with a 18-55mm lens and a Canon 1100D with a 10-20mm lens.
Except from the fixed focal length, the Canon 600D and the Canon 100D lenses were blocked at 18mm and the Canon 1100D lens at 20mm.The reason why several cameras have been used, it's because of the very short time given to acquire all the statues, so is was necessary to do the survey in parallel with 4 operators.A proper focal length was maintained by generally using small apertures (f8 -f16) with the Canon 60D and the Canon 600D, especially in the most critical cases such as at short cameraobject distances.For the Canon 1100D the focal length varied from f5.6 to f8, due to the specification of the camera, while for the other two cameras, the focal length was set at f8 ( The distance to which the images were taken was variable (0.5-3m) due to the disposition of the objects in the Museum: some of them were halted to the walls and was not possible to move them (Figure 2Figure 1a-b).The images were taken maintaining around 70% of overlapping between adjacent images.Manual focusing on zoomed areas of the statues was used in the most critical conditions, avoiding possible undesired blurring due to unmanageable autofocus errors (Figure 3a-b).The ISO has been kept at a level adequate with small apertures and the normal illumination conditions of museum, avoiding to install -unless strictly needed -special illumination devices.The GSD was considered taking into account the final destination of these models, choosing a shooting distance reasonable to have a sub-millimetre GSD needed for the 3D database requested by the curators of the Museums and compatible with the space available around the statues.The illumination was the most difficult problem to face because the Capitoline Museums have large windows that was not possible to cover.Furthermore, the Museums are always open, so the space provided for the survey was limited.In case of insufficient light, it has been preferred the use of a tripod for increasing the shooting time with no movement blurring but in some cases, the use of professional lights was necessary to maintain a homogeneous light and texture of the object, avoiding shadows and reflections (Figure 4).All the statues have been exported in *.obj extension with their own texture, scaled using at least 4 targets, two placed horizontally and where possible two vertically in front and beside the statue.
A special attention was reserved to the texture, fundamental for the visualization of the simplified models in the AR app.All the 3D models were obtained using Agisoft Photoscan, considering a maximum number for the final mesh of 4M polygons.The post processing to obtain a closed 3D model was in some case difficult and long due to big gaps in the back part of the statues.The 3D models obtained were simplified in order to fit inside the reconstructed environment.This process was done imposing a fixed number of polygons for the simplification, in order to have the models more or less of the same size.This passage was fundamental to permit an easier import of the 3D models inside the reconstructed environment and to reduce the time needed for the rendering (Figure 5a-c

3D virtual environment
The first step for creating the virtual environment was examining the photos of the rooms selected for the app, interpreting the space to deduce the architectonic style of the rooms.Then, the cornices, mouldings, frames and shelves have been identified in order to be put in the reconstruction of the rooms.Subsequently, the images where the walls were recognizable, have been edited in photoshop to create facądes.
In this way, it was possible to identify the symmetry and repetitiveness of several architectonic elements and the facądes have been drawn, scaled and proportionally deformed in Autocad, following the measurement given by the direction of the Museums (Figure 6a-c).

Infrastructure and 3D visualization
The virtual exploration of reconstructed environments was realized through a responsive application, allowing the visitors to navigate the spaces and inspect 3D models obtaining basic information.
In order to centralize development efforts and create scalable components, the whole infrastructure is composed by two macro sets: 1. Server-side services leveraging on node.js6 for content delivery 2. Visualization Front-Ends based on modern HTML5 standards and open-source 3D libraries, for end-user fruition From a server-side perspective, node.js and its large ecosystem of open-source libraries offer a robust playground to build flexible and reusable applications.The cross-platform runtime environment guarantees high portability and ease of configuration even on cheap devices (e.g.Raspberry Pi or dedicated devices).Furthermore, such approach allows both centralized and decentralized configurations, depending on the museum requirements and needs.Figure 8 shows a centralized setup (left) where a device (green) is serving application content (3D models, text, multimedia and other) to visitor tablets, while a decentralized configuration (right) does not require local network or content streaming although a per-device configuration is needed.
For each room (e.g."Atrio", "Sala degli Imperatori", etc..) a json database is served (e.g."room1-db.json")containing general room information and a collection of items.The latter is a list of all interrogable objects with local metadata including multi-language description, multimedia resources and 3D model inspection url, where present.The latter can be used to overlay a 3D viewer (typically within an iframe) to easily inspect the model.This allows to employ different presentation layers or viewers where needed (e.g.employing different WebGL libraries), granting flexibility and reuse.From user presentation perspective (Figure 9), a responsive approach has been employed in order to maintain consistency among different devices (kiosk browsers, large touch screens, smartphones, and tablets).This allows customized behaviours and adaptive layouts by exploiting modern CSS capabilities and HTML5 API.During initial stages of the project, a survey of different open-source presentation tools was performed, in order to measure performance and scalability, especially in terms of mobile fruition requirements, VR presentation and cross-device responsiveness.
Customized presentation layouts are used on handheld devices depending on screen size and orientation (landscape or portrait), automatically detecting device capabilities for optimal performance from mobile to desktop.For mobile dissemination (smartphones and tablets) device-orientation capabilities are employed for augmented reality.The overall web application is built on top of several open-source libraries and frameworks.The 3D fruition is based on the interplay of two interactive layers mapping two tasks: room exploration and object inspection, respectively (Figure 10).Regarding the first layer (explore), the open-source DPF library7 (Fanini, D'Annibale 2016) is employed as egocentric interface to explore (using multi-touch or device orientation) and query the virtual room.The library automatically adapts shaders and served content to device hardware capabilities.Creation of semantic masks was carried out through common 2D graphics software by cultural heritage professionals (Figure 11).The efficient encoding of semantic masks offered by the library is specifically conceived for compact web transmission (semantic map size per single node ranged from ~12 to ~29 kb).Such masks are linked to specific item entries in the current room json database, in order to present metadata information such as multi-language formatted text, audio-visual content and 3D model for selected item.Special attention was dedicated to CSS layout to present such information through scrollable containers and HTML5 elements arrangement for landscape and portrait modes.Within the infrastructure, a single room (e.g."Salone") holds one or more DPFs (DPF network) automatically connected by the library for user locomotion (where enabled) once the room database is loaded at runtime.Regarding the second layer (3D inspection) a viewer application was developed on top of a minimal component of open-source library ATON 2.08 , used to inspect fully textured 3D models on mobile devices.The component is based on the open-source OSG.js library9 (the same SketchFab10 is based on) and offers out-of-the-box: advanced camera manipulation / POV management, device orientation, adaptive shaders and responsive UI, WebVR presentation, support for compressed geometry and texture, including glTF format11 .From a fruition perspective, common and well established multi-touch interactions such as pinch, pan and double-tap to re-target are provided.A simplified and responsive user interface was specifically designed for the viewer to grant a liquid experience across mobile and desktop devices.Some 3D models required camera constraints, in order to optimize or limit multi-touch fruition on specific spots or areas.
Figure 12. 3D viewer and inspection of a few items on mobile devices with reference layers.
Due to a large UI simplification for mobile devices, a single home button is present in order to reset camera position to its original configuration.Measurement tools are supported but were disabled, using instead dynamic reference layers (Figure 12).To maintain optimal performances on mobile devices the two layers communicate in order to pause rendering and save computational resources (e.g. when user tap to 3D inspect an object, the egocentric interface background rendering process is paused).Such interplay allows to obtain maximum supported framerate and minimize memory footprint on tablet devices, thus granting a smooth experience for both exploration and inspection tasks.

INGESTION AND AUTOMATION
Due to the amount of 3D models, ingestion processes were automated through web-based services based on node.js.The main identified processing task requires conversion of a single 3D model (in this case obj format) and corresponding atlas texture into a minified json descriptor and one or more binary data (vertex arrays).Multiple binary streams can be employed for multi-request approach on server, although within this context a merged and compressed binary data was used.Automation of such procedures was necessary during the project in order to ease web conversion phases.For this reason a server-side processing service on a LAMP virtual machine was deployed to consume entire input folders (including OBJ and atlas texture) to boost the overall workflow.The service automatically takes care of: • Pre-processing of geometry of incoming OBJ files for OpenGL ES 2.0 specification and up • Production and compression of binary data (vertex arrays) and corresponding JSON descriptor • Production and compression of texture atlases at different resolutions (2048, 4096 and 8192) Similarly to SketchFab approach, different resolutions allow the viewer to automatically adapt to different devices while maintaining high frame-rates.

CONCLUSION AND FURTHER WORK
The use of VR technologies to increase and reinforce the information provided by museum's expositions is well defined nowadays and built in different ways, related to the possibilities of space, control and funding in the museums.These technologies are a powerful tool in the educative function of CH, allowing to describe and explain in an easy and engaging way concepts sometimes complicated.The combination of information, images and 3D models, interactively manipulated by the user permits the visitor to explore an ancient word without really moving and to actively participate in the virtual environment.This project was carried out in accordance with the requests of the Capitoline Museums, that wanted to deploy a VR app inside its spaces.
The realized web architecture is highly modular and scalable, including its interactive web components (viewers): they can be reused for instance as single components to embed single object presentation on a webpage or CMS, customized through local parameters and advanced visualization modes.Future work will also focus on automation improvement, including encoding and processing of input data in order to make it more accessible, flexible and adaptable to other contexts.One issue raised during the deployment of the web-app on desktop kiosks (totems) related to lack of minimum hardware computational power (only integrated intel graphics available).This forced totem kiosks to display less detailed models compared to museum tablets were 3D inspection is smoother.The main intent is to install proper GPU hardware in order to display and inspect more detailed models and offer smoother experience to totem kiosks users.

Figure 1 .
Figure 1.Three examples of statues held in the Capitoline Museums that have been surveyed: (a) the Young Centaur; (b) the Capitoline Venus; (c) the dying Galata.

Figure 2 .
Figure 2. The difficulties in acquiring the images for the photogrammetric survey inside the rooms of the Museums.
Figure 3.The problems with the shooting distance and the use of manual focusing when the distance and the lights in the Museum were favourable: (a) the tripod that was placed in a corner of the room with a good illumination; (b) the Capitoline Venus.

Figure 4 .
Figure 4.The use of professional illuminators was necessary when the rooms were dark or had bad illumination.In this figure is shown the preparation for the survey of the Sarcophagus of Achille.

Figure 5 .
Figure 5. Three examples of 3D models obtained: (a) the old centaur; (b) a female statue; (c) the Capitoline Venus.

Figure 6 .
Figure 6.The process followed for the virtual reconstruction of the rooms of the Museums, as they were in the 18 th century: (a) examination of the photos; (b) extraction of the profiles; (c) drawing and extrusion in CAD environment.

Figure 7 .
Figure 7. Examples of the 360° renders: (a) the Emperors' room; (b-c) the Atrium taken from two different views.

Figure 9 .
Figure 9. Hand-held devices responsiveness (A and B) and desktop-based static installation (e.g.: totem, desktop kiosks, etc.) for large multi-touch screen option inside the museum.

Table 1 .
The settings of the different cameras used for the photogrammetric survey