HIGH RESOLUTION 3D ACQUISITION AND MODELLING IN CULTURAL HERITAGE KNOWLEDGE AND RESTORATION PROJECTS: THE SURVEY OF THE FOUNTAIN OF NEPTUNE IN BOLOGNA

: In 2016, the Municipality of Bologna (Italy) has undertaken the restoration of one of the symbols of the entire city, the Fountain of Neptune, in evident state of degradation. The works have touched upon all the aspects of this complex object and the project has seen involved the Municipality and the University of Bologna, the Istituto Superiore per la Conservazione e il Restauro (ISCR) (Rome) and the Visual Computing Lab of the CNR – ISTI (Pisa), in a modern and highly multi-disciplinary approach. One of the key elements of the project was made up by the creation of an information system ad hoc developed to permit, in an innovative, efficient and user-friendly way, the collection, sharing, management and analysis of all the information and data related to diagnostics and restoration actions . The base of the information system is a very detailed 3D model of the monument, realized by means of the most modern techniques for objects 3D modelling (laser scanning, digital photogrammetry and 3D scanning) integrated together with the aim to obtain a photo-textured 3D model characterized by a sub-millimetre precision level in the geometric description and a high perceptive fidelity of colour reproduction. The surveying activities and data processing, performed by the DICAM Geomatics group of the University of Bologna (with the collaboration of the MCM Company of Rome), are described in the paper, with considerations on the problems encountered and the procedures and solutions adopted. The information system has been developed by CNR-ISTI.


INTRODUCTION
Each project related to the analysis, monitoring and preservation of an historical building starts necessarily with an accurate description of the object of interest in its different points of view: geometry, materials, history, structural behaviour, damaged areas, etc. Nowadays, Geomatics offers many technological solutions able to generate very realistic 3D products characterized by high geometric and radiometric accuracy, which can be exploited for different purposes: digital recording and documentation, restoration, preservation, monitoring, structural and seismic analysis, HBIM establishment, database for different applications (e.g.touristic exploitation) (Bitelli et al, 2017;Chiabrando et al., 2017;Bitelli et al, 2018).In this context, the 3D survey represents the first step of the entire workflow, as an accurate 3D model constitutes the base on which all the subsequent analyses can be developed, often in a multidisciplinary approach.Laser scanning and digital photogrammetry have become today the standard surveying techniques used in the field of 3D Cultural Heritage documentation and modelling.Each of these two techniques presents peculiar potentialities and limitations.Laser scanning is fast, reliable, able to capture very dense and accurate measurements on the object surface; photogrammetry is inexpensive, flexible, and easy in the acquisition phase, and its efficiency has grown considerably thanks to the integration with the automatic algorithms based on the digital images processing derived from the Computer Vision community.(Bitelli et al., 2007).The choice between the two techniques is influenced by many factors: the objects material characteristics, the light conditions, the shape of the surface to be surveyed or the logistic constraints, as for example the distance range for data acquisition.Seen all these considerations, it is today well demonstrated that an integrated approach is often the best solution to survey all the geometric characteristics concerning the object of interest (Bitelli et al, 2011).This is especially true in the field of Cultural Heritage, where each case study presents peculiar characteristics needing often ad hoc solutions, and where the requirements of 3D modelling are various and complex: not only the high geometric accuracy but also representation of all details, colour fidelity and photo realism, together with the efficiency of the model size.The restoration project of the Fountain of Neptune in Bologna (Italy), occurred in 2016-2017, can be considered a significant and innovative experience in the international scene under different points of view.It was a complex project seeing involved the Municipality of Bologna, the University of Bologna, the Istituto Superiore per la Conservazione e il Restauro -ISCR (Rome), the Visual Computing Lab of the CNR -ISTI (Pisa), the newspaper "il Resto del Carlino" and the citizen itself, which partly financed the restoration participating in fundraising.The project availed itself of the support of a complete and very high fidelity 3D digital model, base for an information system developed ad hoc for data management and sharing during the restoration phase (Apollonio et al., 2017b); these data have different origins and nature, collected and generated by the different specialists involved.This approach is innovative, because the idea is that the restorers could directly work on the textured 3D model to identify and sketch the damaged areas and their interventions, or to derive information about the materials (Callieri et al., 2004, Apollonio et al., 2017a).
In the multidisciplinary program developed for the diagnostic and cognitive phase, the DICAM Geomatics group has dealt with the design, planning and execution of the activities aimed at generating the 3D model of the Fountain, characterized by an uncommon accuracy (average mesh edge of 0.5 mm), especially for an object of this size and complexity.For the upper part of the monument, the one including the bronze statues, the MCM company (Rome) also operated.Considering the extremely stringent requirements, both on the geometrical accuracy and on the radiometric fidelity of the 3D model, as well as the morphological and material characteristics of the different parts, different integrated techniques were adopted.Traditional 3D terrestrial surveying techniques were used for the object georeferencing, time-of-flight (TOF) laser scanner for a first scan of the entire monument, above and under-ground, and the surrounding buildings.Lastly, structured-light projection scanner and automatic digital photogrammetry were utilized for generating the very highresolution textured 3D model of the monument.The need to work with very short deadlines and in a complex logistic context has influenced the choices relating to the instruments and operating methods, and the final multi-resolution model interfaced with the information system is the assembling of different parts in the single reference system provided by the first general scan of the Fountain.After a brief historical background and description of the monument, symbol of the city, the main operations of surveying and data processing are described in the paper, with considerations on the problems encountered and the procedures and solutions adopted.

THE FOUNTAIN OF NEPTUNE
The fountain of Neptune is probably the most symbolic monument of the city of Bologna and, together with the Two Towers, is its icon.Defined as one of the most spectacular examples of renaissance fountain, it was designed and realized thanks to the collaboration and work of two artists: the Sicilian architect and painter Tomaso Laureti and the Flemish sculptor Jean de Boulogne of Douai, called Giambologna, between 1563 and 1567 (Tuttle, 2015).The monument presents a squared symmetric structure, in which several bronze sculptural groups adorn a tank and a "castellum" realized in Istrian stone, on which it stands the imposing statue of the god Neptune.The three steps leading to the fountain are made of red Verona marble, and the same stone decorates the basement, which houses the tank (Figure 1).
The square of the steps has a side of about 11 m and the entire fountain is about 9 m tall.

3D SURVEYING AND GEOREFERENCING
The process that has conducted at a very high detailed 3D model of the fountain has actually requested a complex surveying, where the complexity of the object and the need for products at different scales of detail, required the integration of four different techniques.
The activities were structured in three measurement campaigns: -In 2012: traditional 3D terrestrial topographical survey by Total Station for the object georeferencing and TOF laser scanning for acquiring a general point cloud of the object and neighbouring context.
-In 2013: TOF laser scanning to integrate the previous point cloud with that of the underground tunnels and complete the surveying of the entire monument.
-In 2016: 3D scanning and photogrammetry for generating the detailed photo-textured 3D model of the fountain.
The TOF laser scanner Riegl VZ400 (declared precision 3 mm) was used to produce a 3D point cloud of the entire fountain, complete with the underground (Figures 2 and 3).In total, 27 scans were realized, 12 for the above part and 15 for the underground one.The alignment of the acquired point clouds was solved using many cylindrical and double-side retro reflecting target, placed on the fountain basement, on the façades of the neighbouring buildings and around the small hatch in front of the fountain (Figure 3c), visible also in Figure 1, which represents the only direct connection between the above and the underground part.In fact, in this first phase, the survey of the subterranean tunnels, which preserve the technological heart of the fountain from the 16th Century, has represented one of the most critical aspects.The reduced spaces required a considerable number of scan positions and was particularly difficult the alignment of the underground part with that of the above one in a single local reference system (Figure 3).The position of 6 targets were measured by traditional topographic survey using a high precision Total Station, for the establishment of a univocal and rigorous three-dimensional reference system and for the estimation of the transformation parameters that serve to move the acquired point cloud in the large-scale technical cartography of the city (Figure 4).In addition, the calculation of the offset in altimetry with respect to the municipal levelling network was calculated.
The topographic survey was also used to make vertical carefully the overall 3D point cloud.After an accurate phase of manual cleaning of the clouds from numerous useless points, corresponding for example to people, pigeons, electrical and hydraulic tubes, etc., the final complete point cloud of the fountain consists of about 140 million points, characterized by an average spacing of 5 mm (Figure 5).
Figure 5.The 3D point cloud of the Fountain of the Neptune (above and underground in the same reference system), obtained by the TOF laser scanning.
The final product of this first campaign, perfected and reworked during the multidisciplinary program started for the documentation and diagnostics of the fountain, was essential in the subsequent and more deepened second survey campaign.This general model was, in fact, the reference for the necessary and fundamental georeferencing of the following more detailed digital 3D models, as well as the support for different analyses conducted by the University of Bologna, for example in structural or hydraulic fields.

HIGH DETAILED 3D MODELLING
The second surveying campaign, aimed at generating the 3D model of the fountain with very high geometric detail and radiometric fidelity, was performed between the months of January and April 2016, adopting a decomposition of the object in defined portions based on the characteristics morphology and materials, in agreement with ISCR.The steps, the basement, the tank, the "castellum" and the Neptune were taken on separately, by adopting different surveying approaches and working for parts.This choice was also linked to technical reasons, connected to the management of the heavy amount of digital data acquired, to the problems posed by the different surface and by the characteristics of the different materials, as well as to the peculiarities inherent in each of the acquisition techniques adopted.

Digital photogrammetry for the 3D modelling of the steps
For the 3D modelling of the steps at the base of the fountain was chosen to adopt the photogrammetric technique, using for the acquisition a DSLR full frame Canon 6D camera and for the data processing the modern Structure from Motion approach.The models were scaled using direct precision measuring between many little singularities in the marble surface.The steps were divided in 8 portions (4 for the central part and 4 for the corners) and the radiometry of all the acquired images was also corrected using a 24 elements ColorChecker, photographed in each surveying day and in different working hours, in consideration of the changing of the light conditions also during the same day.The characteristics of the marble together with the predominant planarity of the shape, has created problems of reflections on the polished planes, and other problems derived from water and irrelevant objects on the surfaces.For this reason, an important manual pre-processing phase was necessary to correct the radiometry and also to mask the images and delete from the elaboration those areas that they could have generated outliers and noise in the extracted point clouds.It should be noted that the shape of the steps, characterized by strongly angulated flat surfaces (the steps have risers of 12 cm with treads with a run of about 60) and by very protruding prominences in relation to the size of the portions photographed, has forced to a particular attention in the acquisition phase of the images.It was necessary, to reach the required very high degree of detail, to have large photographic datasets (about 250 pictures for each side and for each corner of the steps, with a total amount of 2000 images), characterized by reduced fields of view and often with limited transversal overlap, generating critical situations for the processing algorithms.
Essentially the acquisition scheme consisted in (Figure 7): -two rows of nadiral close-up images, one for each of the first two risers; -two rows of convergent images permitting to acquire contemporary the treads and to connect the treads and the risers; -a row of images for the top tread; -some overview images from afar of the object as a whole, to reinforce the acquisition scheme and help the alignment phase.
Figure 7. Example of the image acquisition scheme: perspective views of the South side mesh of the steps with the photos location (blue rectangles).
These difficulties were resulted in the necessity, during the elaboration phase, to carefully control the semi-automatic processes and to manually insert numerous tie points (Figure 8), checking at each elaboration the problematic areas.The produced meshes finally required an accurate manual editing intervention, before the texturing phase, to correct some residual anomalies (e.g.filling little holes and smoothing the surfaces by the noise).

Structured-light projection scanning for the 3D modelling of the basement
For the basement, for the "castellum" with the sculptures and for the bronze statue of Neptune, the surveying was carried out using the structured-light projection scanning technique (Figure 10).The two adopted instruments (both Artec scanners) have an intrinsic precision of 0.1 mm and allow the direct generation of a mesh already enriched with radiometric information.
The surveying of the basement has presented some of the problems already encountered for the steps.In particular, the need to work without wet surfaces and directly lit by the sun, has conditioned the activities, limiting the useful working days.
The huge amount of acquired data, also considering the required accuracy and the complexity of the object, has forced to work for small surface portions.A considerable effort in terms of time was required, both in the acquisition phase and in the subsequent data processing phase.By means of specific software tools, the single meshes, after a first processing phase, were edited in order to eliminate the outliers, to smooth the noise and to fill any small holes, before proceeding to the alignment, fusion in a unique model and finally to texturing.
Figure 11 shows the acquired scans for the East side of the basement (8 GB of data) and the textured mesh obtained at the end of the entire workflow, which consists of about 20 million of triangles.
Figure 11.East side of the basement; the numerous scans acquired (top); the complete photo-textured mesh (bottom).
Similarly to the models referred to the basement, the individual parts have been one by one georeferenced on the reference system previously created for the whole monument by the topographic and TOF laser scanner surveys.At the end, the complete high resolution model was adopted for the implementation of the information system developed by CNR-ISTI.The last had to provide a multi-resolution approach for data visualization, having to take in consideration the huge amount of data (Potenziani et al., 2015).The final product allows the conservators to assess in real time and with a very high fidelity the situation of the single areas of the object, in terms of shape and colour, managing the information cycle before, during and after analyses and treatments.

CONCLUSIONS
The modern 3D digitalization techniques are one of the key factors, together with the technologies of efficient management of high-resolution 3D models and the solutions for 3D drawing on the web, that nowadays permit a new and innovative approach to diagnostics and restoration of objects and monuments.In the case of the restoration of the fountain of Neptune in Bologna, was realized ad hoc an information system having as base of the entire data organization a high-detailed 3D model of the object.This case study is peculiar in the international panorama: for of a so complex, large and remarkable object, the entire amount of data concerning the study, documentation, diagnostics and restoration, is managed in real time on a 3D web-based system.The 3D model that is the heart of the information system was generated using different surveying techniques integrated together: TOF laser scanner, digital photogrammetry and 3D scanning by structured-light projection system.Several problems related to the surface material, to the logistic constraints, to the acquisition scheme in relation to the geometry of the object and to the elaboration of a large set of data, emerged during the work, forcing the adoption of surveying and processing procedures specific to the case study.The overall 3D model, which consists in more than 600 million of textured triangles after data fusion and optimization, combines high geometric quality and high radiometric fidelity (Figure 12).Restorers and other involved personalities have used it throughout the duration of the entire restoration project, to organize and georeference the different data collected during the phases of analysis and documentation of the fountain, mapping directly in a 3D georeferenced environment the damaged and intervention areas (Figure 13).

Figure 1 .
Figure 1.The fountain of Neptune, placed in the square of the same name in the centre of Bologna.

Figure 2 .
Figure 2. Two moments of the laser scanning survey (top); one of the acquired point cloud (bottom).

Figure 3 .
Figure 3.A phase of the underground surveying (a) and the placement of the targets used to align the clouds of the tunnels (b); some targets around the hatch used for the alignment of the above and the underground parts (c).

Figure 4 .
Figure 4.The scan positions of the laser scanner survey (yellow squares) and the targets used for the definition of the reference system (green circles).

Figure 6 .
Figure 6.Image of the North-East corner of the fountain.Examples of problematic areas are highlighted: light reflections (green), changing of the background for people presence (orange) and out of focus objects (blue).

Figure 8 .
Figure 8. Manual intervention in the images alignment phase for a portion of the North side steps: a) sparse cloud obtained by automatic processing, the red lines highlight erroneous trends of points; b) the final mesh with the markers manually added to correct the images alignment.

Figure 9 .
Figure 9.The 3D final model: a) photo-textured mesh of the North-West corner composed by about 12 million triangles; b) and c) detail of the area in the red square of the model, textured respectively with an artificial colour and with photos RGB (about 300.000 triangles).

Figure 10 .
Figure 10.Surveying activities for the basement 3D modelling.

Figure 12 .
Figure 12.Overall image of the 3D model of the fountain as available in the information system.

Figure 13 .
Figure 13.An example of the mapping of damaged areas, by means of direct identification and sketch on the 3D model of the basement.