BEYOND POINT CLOUDS AND VIRTUAL REALITY INNOVATIVE METHODS AND TECHNOLOGIES FOR THE PROTECTION AND PROMOTION OF CULTURAL HERITAGE

The morphometric and photogrammetric knowledge, combined with the historical research, are the indispensable prerequisites for the protection and enhancement of historical, architectural and cultural heritage. Nowadays the use of BIM (Building Information Modeling) as a supporting tool for restoration and conservation purposes is becoming more and more popular. However this tool is not fully adequate in this context because of its simplified representation of three-dimensional models, resulting from solid modelling techniques (mostly used in virtual reality) causing the loss of important morphometric information. One solution to this problem is imagining new advanced tools and methods that enable the building of effective and efficient three-dimensional representations backing the correct geometric analysis of the built model. Twenty-year of interdisciplinary research activities implemented by Virtualgeo focused on developing new methods and tools for 3D modeling that go beyond the simplified digital-virtual reconstruction used in standard solid modeling. Methods and tools allowing the creation of informative and true to life three-dimensional representations, that can be further used by various academics or industry professionals to carry out diverse analysis, research and design activities. Virtualgeo applied research activities, in line with the European Commission 2013’s directives of Reflective 7 Horizon 2020 Project, gave birth to GeomaticsCube Ecosystem, an ecosystem resulting from different technologies based on experiences garnered from various fields, metrology in particular, a discipline used in the automotive and aviation industry, and in general mechanical engineering. The implementation of the metrological functionality is only possible if the 3D model is created with special modeling techniques, based on surface modeling that allow, as opposed to solid modeling, a 3D representation of the manufact that is true to life. The advantages offered by metrological analysis are varied and important because they permit a precise and detailed overview of the 3D model’s characteristics, and especially the over time monitoring of the model itself, these informations are impossible to obtain from a three-dimensional representation produced with solid modelling techniques. The applied research activities are also focused on the possibility of obtaining a photogrammetric and informative 3D model., Two distinct applications have been developed for this purpose, the first allows the classification of each individual element and the association of its material characteristics during the 3D modelling phase, whilst the second allows segmentations of the photogrammetric 3D model in its diverse aspects (materic, related to decay, chronological) with the possibility to make use and to populate the database, associated with the 3D model, with all types of multimedia contents.


INTRODUCTION 1.1 Surveying with laser scanner systems
Before we examine the processes, 3D modeling techniques and the various types of informative three-dimensional representations (including the possibilities of sharing them amongst operators of the Cultural Heritage field) it is important, in our opinion, to remind the basics of surveying, starting from the data acquisition activities, since they represent the foundation on which the 3D restitution and designing operations will be developed.From experience, operationally competent topographic methodologies, are often underestimated, especially after the introduction of point clouds automatic alignment systems, with a higher or lower degree of automatization or in certain cases even robotization.Actually competent topographical practices allow, in addition to the performance of an effective work, to know and control the quality level of the final result, the 3D model.The quality level of the processed 3D model can be determined by two aspects, the first being represented by the accuracy (degree of correspondence between the measured and the real) of the modelled surfaces compared with the acquired point clouds on which they were processed in order to determine the maximum deviation.
The second is represented by the reliability of the entire final 3D model, which allows us to identify the errors (deviations) committed at any point of any surface of the 3D model.This latter aspect is determined by summing up the accuracy of each mesh with the deviations obtained in topographical work.For measurement tasks we mean all surveying activities that represent the "skeleton" of the work on which the entire laser scanning operations lay, specifically: precise geometric levelling, traversing, and the topographical survey of referral points (sights) and the alignment operations of the acquired 3D laser scans.Hence the choice of the 3D laser scanner type assumes a significant importance, because the accuracy of the modelled surfaces will also depend on the accuracy of the employed instrumentation.We want to briefly remind the three types of terrestrial 3D laser scanners, and their degree of accuracy that can vary according to the distance between the instrument and the object, and the material and chromatic characteristics of the artefact: • "Time of flight" laser scanner: medium accuracy up to 20 m ± 3 mm -has a range of more than 1 km.(Fig. 1  • Phase comparison laser scanner are recommended in architecture, archeology and hypogeal environments (Canevese et al., 2008;Canevese et al., 2009;Canevese et al., 2011); • Triangulation laser scanners for the survey of statues, capitals, goldsmith, carvings, design objects, doors and windows, etc.
In addition to the above mentioned 3D laser scanning technologies, the market offers today other types of survey instruments.Since several years now, thanks to the development of various photogrammetric surveying software technologies, it is also possible to generate 3D models with non metrical cameras through the use of epipolar geometry (stereoscopic vision geometry) (Hoiem, 2011).An example is represented by the advent of UAV-Drone systems that allow aerial surveys at a low cost, thus making this technology that a decade ago was restricted to few, potentially aviable to everyone.The same applies to structured light systems implemented on mobile devices (smart phones and tablets) such as Google Project Tango, Structure Sensor 3D Scanner and the like.Certainly these devices will enjoy a great popularity in the future, because if we consider the growing demand for 3D printers, more and more non-expert users will be able to create 3D models of their houses, artworks and so on.
The following question springs to mind: why should we invest tens of thousands of Euros on professional 3D laser scanning technologies, when is it possible to create 3D models, including BIMs at a much lower cost?The answer lies in the quality degree of the acquired data, since the accuracy of the latter "entry-level" systems is not comparable to the accuracy of the measures provided by more expensive laser technologies.
In fact, in architectural and archaeological surveys processed with acquisition systems such as Google Tango and the like, including photogrammetric techniques, the accuracy level reached on surfaces (without considering the reliability of the entire 3D model) is in the range of a few centimetres.An acceptable margin to build 3D models as well as BIMs (that they can also be produced by means of the old simple measuring tape and/or distal) yet unacceptable to produce true to life Advanced 3D models (Virtualgeo, 2017).

Classification of 3D laser scanner data
After having chosen the most appropriate technology and carried out the survey operations, the data acquired with the various stations (so-called point clouds) will be subjected to a first treatment involving the alignment of the point clouds in a single Cartesian axes reference system.At this stage it would be appropriate to follow competent topographic practices in order to produce a correct validation of the 3D model degree of reliability (checking) rather than relying on automated or even completely robotic processes.Once the point cloud of the whole survey is processed into a single Cartesian axis system, in order to optimize 3D modelling (BIM) and reverse modelling (Advanced 3D), it is important to divide the data in subsets, given the huge amount of resulting data.The classification of laser scanner data into sets and subsets, called segmentation, presents several advantages.In fact, apart from optimizing the creation processes of the three dimensional representation, it allows to quickly perform both punctual queries (metrology) and the validation (checking) of each modelled surface.An example of what just mentioned is represented by the segmentation performed on the 3D laser scans (phase comparison system) of the Red Castle in Tripoli -Libya (Canevese et al., 2012) processed on 1123 stationings, with over 45 billion points acquired on a surface that covered approximately 17 000 m² and about 1000 rooms.In this specific case, after obtaining a single point cloud of more than 45 billion points from the alignment of the different stationings, the first segmentation generated the clouds of each section and of the exterior facades of the historic complex.Subsequently, a second segmentation was made for each component of the architectural, structural and decorative elements, resulting in a total of 5802 clouds on which 15 operators have created the Advanced 3D informative model with millimetric accuracy.(Fig. 6)

Methods and 3D geometric model typologies
The methods and types of 3D models that we will illustrate below are the most commonly used in various areas of geomatics, especially architecture, archeology and environment.
It is important to take into account that software technologies are designed and developed mostly to meet and satisfy the specific needs of various sectors; and as we will see, for each sector, the simultaneous use of more than type of three dimensional representation is often required.As for the 3D model types the first distinction can be made between surface 3D models (polygonal mesh) and solid 3D models.The first are mostly used to represent the territory, artefacts, and in the engineering industry for metrology activities, while the latter are mostly used in the fields of virtual reality production (movies, videogames) and design (BIM, design and industry).Surface three-dimensional representations (polygonal meshes) can in turn be 2,5D or 3D.What differentiates them is the method with which the points are triangulated (acquired by any type of instrumentation).In the case of 2,5D model, commonly used in the field of territory (topography and GIS), the triangulation is performed without considering the third dimension (Z -height) that will be only used to elevate the vertices of every single polygonal element in order to obtain the final 3D model.This entails the impossibility to represent convex, concave and hypogeal surfaces.In the case of 3D models, on the other hand, the triangulation is done considering all three XYZ coordinates.

Polygonal meshes 3D models and their use:
The various fields, for the realisation of complex models In fact, if we take into consideration the automotive industry, we can clearly see that the design of every single part of the car is carried out with solid 3D parametric technologies and subsequently, after the prototyping of moulds for the series production of each individual designed element, surface 3D models (developed on the basis of 3D scans) are used for their validation and testing.Now thinking about the architectural and archaeology built artefacts, we must determine whether the object of our survey is more similar to the design phase or the phase of validation and testing of the project.At Virtualgeo we believe that we should approach the three-dimensional restitution activities with accuracy and precision, since we are not facing with planning and designing phases but rather in the surveying phase of an existing artefact.We do this by using 3D modelling techniques that allow us to faithfully reproduce reality.This is why, with a solid know-how acquired in more than twenty years of experience in various areas of geomatics and metrology, we have conceived and developed tools and methodologies for the production of Advanced 3D models, called LIM -Lidar Information Model, anticipating the research activities promoted by the European Commission Reflective 7 -Horizon 2020 Project (European Commission, 2013).
Due to their high accuracy, Advanced 3D models allow the implementation of metrological capabilities, a discipline taken from the mechanics field, to be able to make accurate and effective analysis of the geometry of the building (off plumb elements, bulging, depressions, walls and floors thickness on their whole surface).
Another unique advantage of accurate models as those provided by Advanced 3D, consists in the possibility of monitoring the geometry over time, to identify failures caused by natural and/or man-made disasters, unlike parametric solid models.
A further consideration, what we have previously stated about the automotive, aeronautics and other mechanical industries we assume should be applied as well to the construction site set-up of new buildings.In support of this consideration, Virtualgeo has already performed this type of intervention many times, one in particular, involved the design of the new asphalt of the Mugello racetrack (Geodromo, 2011).On that occasion, laser scanning of the entire racetrack was performed twice, once for the creation of the Advanced 3D model used for designing purposes, whilst the second took place after the completion of the project for testing and checking the completed work.
When we consider historical and artistic artefacts the correct association of chromatic and material properties to three-dimensional representation assumes a significant importance.
We could say that the photogrammetric 3D model is a "speaking" model because of the information offered by the photographic images, and thus it becomes an unique and valuable tool for all operators of the cultural heritage field in support of their study, analysis and diagnosis activities.
The "dressing" of the Advanced 3D model (Fig. 8), unlike the texturing techniques typically employed in the fields of virtual reality and 3D solid models (Fig. 9), takes place by digital photogrammetry processes based on threedimensional homologous points.Such a process and dressing methodology enables to obtain also a chromatically accurate 3D representation, allowing to carry out metric investigations and/or additional segmentations for associating information of different nature such as: materic aspect, deterioration, cracking, stratigraphic and temporal data at each element of the 3D model.The photogrammetric 3D model, besides offering important advantages to all the experts involved in cultural heritage, is also a scientific and rigorous tool available to the general public, the consumer or end-user of cultural heritage, allowing him to go beyond the well-known virtual reality.Therefore wishing to identify which is the best 3D model to use for the creation of the photogrammetric 3D model between surface models and solid models, in order to obtain the best metric accuracy of the images on the threedimensional representation, for what has been said till now, the choice should fall on the surface model.

Photogrammetric 3D Model
In support of this claim it is sufficient to recall the irregularities of morphologies of architectural and archaeological elements and buildings, especially in the case of hypogeal environments (off plumb, bulging, depressions, etc.), and consequently the ineffectiveness of solid 3D representations that we should consider as a schematic and approximated representation of what is built.
For these reasons, Virtualgeo focused on the development of a technology for the production of 3D photogrammetric models based on Advanced 3D geometric models (LIM -Lidar Information Model) and not on solid 3D models, such as BIM.This choice allowed us to develop methods and photogrammetric techniques that enable the production of 3D models that are true to life, both in geometric and chromatic terms.We have called these Advanced 3D models PDM -Photo 3D Model (Figure 8).

Informative 3D model
Both polygonal surface and solid 3D models can be enriched with additional technical, scientific or chronological information, in order to make them informative and multimedia-friendly to support both design activities, future maintenance of the artefact and also for the enjoyment of public end-users (tourists, art and history lovers, students, etc.).In this way we obtain a so-called "informative" 3D model, enriched with multimedia content that can be also georeferenced on the 3D model itself using clickable objects (POI -Point of Interest) so they are easy and fast to access and look up.
The insertion of these information may occur either during the 3D modelling and the reverse modelling phase, but also after the completion of the 3D model, be it BIM (Fig. 10) or Advanced 3D (Fig. 11).
Figure 9. HBIM (Revit -Autodesk) Figure 10.HBIM -Database Specifically, in the 3D modelling and reverse modelling phases, information will mostly concern the classification of the various elements that form the model such as the structures, architectural elements, decorative and material elements, etc.
Later you can expand the information by taking advantage of a database linked to the 3D model which can also be georeferenced, as with the GIS platform, it is possible to link and view content (divided by type of format such as videos, drawings, texts, etc.) using POIs inserted on the various 3D model elements or the successive segmented surfaces (done on the bases of the Photogrammetric 3D model, such as: analysis of the degradation of walls and frescoes, deteriorations of the structures, stratigraphy studies, etc.) (Fig. 11).Informative 3D models can be queried on various levels, the first based on the segmentations performed during 3D modelling (BIM) or reverse modelling (Advanced 3D) activities, and subsequently on the photogrammetric 3D model for the decay or stratigraphic analysis and so on.
Additionally it can be interrogated using the geospatial and temporal databases for the different multimedia contents.
The ability to achieve an informative 3D model is also remarkable for the social and professional aspects as it opens new horizons and prospects for new generations of experts in the field of cultural heritage and promotion.
Before the introduction of new technologies such as 3D laser scanners, BIM and Advanced 3D, topographic and architectural surveys were the prerogative of topographers, mostly surveyors, but today things have changed.
In fact, the ones that most took advantage of this technological innovations are experienced professionals in the fields of protection and promotion of cultural heritage, such as architects, archaeologists, engineers, etc., because they are able to independently produce 3D models and enrich them with knowledge and contents that traditional surveyors do not possess.

Operators in the cultural heritage field
In its applied research activities Virtualgeo has paid particular attention to all the experts and scholars of public cultural institutions and individuals professionals or businesses, who wish to interact with Advanced 3D models so that they can take advantage of technological innovation.
Especially those that for various reasons (limited or no experience with the 3D world, or held back by the idea of being unable to use these types of technologies) have remained far from the profound change brought by the "digital era".
We believe this represents a gap because technology should and can be improved and enriched with the humanistic contributions of this audience of experts and scholars.That is why we considered the development of a particularly user-friendly technology (Virtualgeo, 2017), which allows everyone to query and enrich with contents the Advanced 3D models, a very important task.(Fig. 12) Figure 12.Decay segmentation

Cultural institutions and people
By following (and anticipating) the directives of the Reflective 7 -Horizon 2020 Project, besides realizing Advanced 3D technology and models (GeomaticsCube Ecosystem), Virtualgeo has designed and developed a technology called ECH -Edutainment for Cultural Heritage, for the scientific dissemination and the enjoyment among cultural audiences (education, tourism, enthusiast) of rigorous, semantic and multimedia-friendly 3D representations.
The ECH platform has been already tested in various exhibitions and events (Fig. 13), and has received a technological innovation award from the Polo Tecnologico di Pordenone (FVG -Italy) (Archeomatica, 2014).It is the first time that scientific technologies derived from the geomatics field are applied to the cultural heritage and its promotion, offering Advanced 3D models to the public that allow new possibilities of interaction with cultural heritage, encouraging a direct and "experiential" approach (Fig. 14).

CONCLUSIONS
Possessing a varied experience of more than twenty years as leaders in the diverse fields of geomatics, surveying services, designing, applied research and communications, with this intervention we considered it important to focus our attention on the basics of topographic and architectural surveying and on an accurate and informative 3D representation.
The correct elaboration of the restoration project can not be separated from a solid knowledge of the object of study, since it is only trough knowledge that the action can be "guided" at both the theoretical orientations level both at the procedural level, by reducing, at the same time, the risk of "arbitrariness" of the design sphere (Franceschi et al., 2007).
Starting from this assumption (on which we agree), given for granted the importance of topographical practice, we should ask whether the solid modelling is the right approach for creating 3D models of historic-artistic building.
Obviously the result to be achieved depends on the goals, not the cost, set by the customer and the restorer.If you need a schematic solid 3D model the choice will likely be directed on BIM technologies, called HBIM when used in the cultural heritage field.
On the contrary, if you want to perform metrology operations and geometrical monitoring, Advanced 3D becomes necessary in order for you to take advantage of true to life geometries and colours.Both types of 3D models mentioned above are semantic and multimedia-friendly, because they allow to segment the various elements that form it, and afterwards to connect each element to information from a database.Assuming that with Advanced 3D models it is possible to manage architectural and historical complexes of all sizes (as it was done with the Red Castle in Tripoli), the ability to connect it to an information database can be used to integrate and enrich existing HBIM representations with Advanced 3D models of important components that can be an object of study (i.e.fresco areas to be restored, structural failures, etc.) thus making these two technologies complementary and not necessarily conflicting.
Virtualgeo has created a variety of technologies for the production, management and publishing of Advanced 3D models, that have been grouped into the GeomaticsCube Ecosystem.These technologies allow a full-spectrum management of the survey, starting from its restitution up to the final 3D model issued to clients, end-users, or divulged in museums or websites.Therefore we deal with mature technologies that have been extensively tested in the field, in fact offering the possibility to immediately implement the cultural changes and the different approach to the protection and promotion of cultural heritage that we hope will be applied in the future.
) • "Phase Comparison" laser scanner: medium accuracy up to 20 m ± 0.5 mm -has a range of approximately 200 m.(Fig. 2) • Triangulation laser scanner (laser or structured light) : accuracy from 0.05 to 0.1 mm -range up to 1 m.

Fig. 5 .
Fig. 5. UAV-Drone -Google Tango -Structure Sensor 3D Into GIS -Geographical Information System -platforms • GRID: territory representation with quadrilateral meshes.• DEM -Digital Elevation Model: representation of the land elevation quotes without the classification of cartographic detail.• DSM -Digital Surface Model: representation of the land elevation quotes with the classification of cartographic elements.The DSM is particularly useful in the 3D modelling of the territory.• T I N -Tr i a n g u l a t e d I r r e g u l a r N e t w o r k : representation done with an irregular network of triangles without any classification.Points and surfaces 2,5D models and their use : In designing infrastructural projects: • DTM -Digital Terrain Model: topographic threedimensional representation of the terrain, without the classification of topographical details. • DITIM -Digital Information Terrain Model: topographic three-dimensional representation of the terrain, with the classification of topographical details.More effective and efficient in designing and planning as well as visual simulation works.