3D DIGITAL AND PHYSICAL RECONSTRUCTION OF A COLLAPSED DOME USING SFM TECHNIQUES FROM HISTORICAL IMAGES

The historical images preserved in archives and in private collections represent not only a valuable documentation of objects belonging to Cultural Heritage; sometimes they are the only remained evidence of destroyed assets of our past. In the last few years, the improvement of the technologies in the framework of photogrammetric vision and the implementations of new Structure-fromMotion (SfM) algorithms allow to extract metric information’s from this kind of images in order to carry out a digital reconstruction of these lost masterpieces. The study presented in this paper aims to evaluate a SfM approach to perform the 3D reconstruction of a dome collapsed in 1971 by using historical images. The final goal is to provide not only a digital replica but also a physical reconstruction of a portion of the collapsed dome as a support for the recovered fragments of the fresco originally present on the surface of the dome. * Corresponding author


INTRODUCTION
Geomatics techniques provide valuable instruments and technologies in the framework of three-dimensional metric survey for a continuous Cultural Heritage documentation. By considering the risks to which built heritage is subjected (from continuous risks, like natural deterioration of a building, to emergency such as an earthquake) a particularly sensitive subject concerns the documentation of no longer existing assets. 3D reconstruction of lost objects belonging to the built heritage (for documentation purposes or providing a support for recovery works) is possible by using historical images retrieved in public archives or private collections. In the last few years the improvements in photogrammetric computer vision technologies and implementation of image-matching algorithms have made possible to accurately reproduce the geometry and the connected shape of a lost asset and provide to the user usable 3D model (Condorelli and Rinaudo 2018;Rodríguez Miranda and J. M. Valle Melón 2017). The case study presented in this paper refers to the collapsed dome of the Basilica of San Pietro in Tuscania. In 1971 an earthquake caused the loss of a portion of the dome and the destruction of a medieval fresco present on its surface (depicting the Pantocrator Christ encircled by four angels). Some parts of the fresco have not gone destroyed after the collapse and have been collected and preserved. In 2006 the first step of a conservation project was designed and carried out by the Istituto Superiore per la Conservazione ed il Restauro (ISCR) in order to perform a restoration of the destroyed masterpiece and to try to rebuilt the original shape of the dome. The project was started with Dr. Werner Schmid in 2005, who has studied and restored the fragments and promoted collaborations until he completed the reconstruction of the right angel in 2014. To define the real model of the dome and support the reconstruction of the found fragments of the fresco in the left portion of the destroyed dome (the angel in the left part of the Pantocrator Christ) a traditional photogrammetric approach has been carried out in 2010 with two stereoscopic highresolution images. During the survey campaign a LiDAR acquisition of the apse has been performed as well (Chiabrando and Rinaudo 2014). Thanks to the integrated use of different Geomatics techniques, a carbon fiber composite structure with the real shape of the analysed dome part was constructed in order to reproduce the left side angel. In 2017, a new restoration campaign has been developed by ISCR in the Basilica of San Pietro and the reconstruction of the angel in the right part of the fresco has begun. During 2017, a different approach was followed according to the improvement of modern photogrammetric methodologies connected to the implementation in recent years of Structure-from-Motion (SfM) algorithms. During the research activities the geometry reconstruction of the entire collapsed dome has been achieved not only from two stereoscopic images but using all the historic images (acquired before 1971) that an accurate and fortunate research allowed to retrieve from various archives and private institutions (from Italy, Germany and U.S.A.). Historic images have been processed with a SfM based photogrammetric software and a dense point cloud of the entire dome has been generated. From the point cloud a three-dimensional mesh has been carried out and then the texture was produced using highresolution images.
Another aim of the work was to provide to the restorers a twodimensional basis on which they can work for reassemble the not destroyed pieces of the fresco connected to the right angel. In order to follow this objective and to limit the deformation of the final results the surface was segmented into eight parts and for each part a cylindrical orthoprojection at scale 1:1 has been achieved (Sammartano and Spanò 2017;Gottardi et al 2017). Finally, each part (each orthoprojection) has been plotted by using the TATTOOWALL®system and subsequently positioned on the support structure (made by the restorers in carbon fiber composite like the ones built in 2012 on the basis of the obtained 3D model). The experience described in the next sections aims to demonstrate the consolidated role of the Geomatics, as a discipline able to respond to different needs, in the framework of restoration works of objects belonging to Cultural Heritage, not only as far as the survey during the preliminary stages is concerned, but also as an active support during the restorers' works. Another goal is to evaluate how the development in the photogrammetric algorithms made in recent years by computer vision-based techniques enables the digital 3D documentation of artefacts belonging to build heritage and, when and where is needed, the recovery of assets that today have been lost.

The collapsed dome of S. Pietro Church in Tuscania
San Pietro is a church located in Tuscania (a small town near Viterbo, 100 km North of Rome). The building is sitting on a hilltop (Civita Hill) where presumably an Etruscan acropolis was located and where today it is possible to observe the ruins of the ancient Bishop citadel (Figure 1). Some historians think that the most ancient nucleus of the church was built during the VIII century A.D.; some more recent researches argued that the building was built in XI century A.D. instead. Good authority attest that the actual façade and the two naves were built between the XII and the XIII century A.D. Up to 1573 San Pietro church was the venue of the Cathedral of Tuscania's Diocese and the main religious centre of the area. Over the centuries, the church has been subject to numerous restoration works and it is characterised by different architectural styles. Actually San Pietro is composed by the Romanic Basilica (with three naves), two medieval towers (originally there were a higher number because of the strategic importance of the site) and a palace used in the past as Bishop venue (Cerasa 1993). In 1971, as stated above, an earthquake struck the central part of Italy involving also Tuscania and San Pietro church; although the structural parts of the building did not suffer for significant damages, during the earthquake event the dome of the apse collapsed (Figure 2 -right) with its medieval fresco. After the collapse a concrete dome has been rebuilt in the place of the original one; as regards the original fresco, some of its fragments have been recovered and classified in order to proceed with the restoration operations. Tuscania before the earthquake of 1971. On the right, the collapsed dome after the earthquake that struck the church.

The "traditional" photogrammetric approach (2010)
In 2010, a data acquisition campaign has been carried out to digitally reconstruct the geometry of the dome and to support the restorers during the reassembling of the first part of the fresco (portraying the angel placed on the left of the Pantocrator Christ). Even then, the main goal was the digital reconstruction of the dome; coherently with the technologies available in 2010, a traditional photogrammetric approach, starting from two stereoscopic historical images was applied (Chiabrando and Rinaudo 2014). The two selected images have been digitalized from the original film (a b/w film 4.5x6 cm, stored in ISCR's archive) by using a photogrammetric scanner. For each of the two images a self-calibration has been performed, even if the frames were presumably acquired with the same camera; however, the calibration has been necessary since the typical information used for photogrammetric purpose like focal length, principal point coordinate and the possible different deformation occurred for the two original films were unknown. After the survey of some Ground Control Points (GCPs) which have been measured by a total station in portions of the apse still existing today (the ciborium and the areas of the apse not involved in the collapse), the exterior orientation parameter estimations of the two images has been performed through the Leica Photogrammetric Suite software.
Starting from the oriented model about 4000 points of the old dome have been plotted using simplified matching techniques (Eisenbeiss et al. 2005;Wang et al. 2004) with a huge integration of manual plotting (1 point each 5 mm). This procedure has been extremely challenging as regards the manual involvement of an operator. Furthermore, at that time the 3D reconstruction of the complete area of the dome has not been possible since the dome acquired in the employed images was partially occluded by the ciborium located in the apse. The point cloud obtained at the end of this procedure has allowed to interpolate a sphere (a least square interpolation has been applied) with a radius of 3.35 m and to estimate the discrepancies between the sphere and the plotted point cloud (used as reference).
The obtained sphere has been digitally inserted in the 3D model of the apse (reconstructed by a point cloud acquired by a time of flight LiDAR system RIEGL LMS-Z420). Before proceeding with this operation, the 3D mesh has been segmented in order to remove the actual modern concrete dome (which was built after the collapse of the original one). In its place, a quarter of the best fitting sphere has then been digitally superimposed in order to reproduce the original geometry of the apse as it was before the collapse of 1971.

DOME'S SHAPE RECOVERING: THE STRUCTURE-FROM-MOTION APPROACH (2017)
Contrary to what has been done in 2010 (when, as stated above, a traditional stereoscopic approach has been proposed), in this case a SfM workflow was used. This was possible thanks to the progress made in the last few years by photogrammetric methodology and to the developments of SfM algorithms. In 2017 (when restoration works have begun, specifically to reassemble the area of the fresco with the angel on the right of the Pantocrator Christ) the main aim was to obtain a 3D model to support the operations of the restorers. For this reason, a SfM approach has been applied in order to digitally reconstruct the geometry of the dome. Despite in some cases (as the one presented in the current paper) historical images represent the only evidence of a lost architectural asset, the use of this kind of images for photogrammetric purposes has some significant restrictions (Zawieska et al., 2017;Adami, 2015).
The first matter to be dealt with is the quality of the images collected from the archives. In many cases the retrieved frames are characterised by a very low-resolution; bad lighting conditions during acquisitions must be considered; often the images have not been acquired at the same time or with the same camera; in same cases the original film has been lost and the frame is a scan of a printed image. Another issue is the absence of metadata with the information and parameters about the employed camera mainly the focal length, and the interior orientation parameters. However, as already stated, the further development in the framework of this kind of photogrammetric technology allows nowadays achieving acceptable results (Maiwald et al. 2017). In the case presented in this study, according to the lack of images acquired with metric purposes several images from historical archives (from Italy, Germany and U.S.A.) have been employed.

3D dome geometric reconstruction
To perform the digital reconstruction of the geometry of the dome, the photogrammetric block (composed by 23 historical images) have been processed through a commercial SfM based photogrammetric software (e.g. Agisoft Photoscan platform). Images have been selected from those found to better fulfil the requirements of the presented study (as regards image quality and resolution). Even though the images obviously have not been acquired for metric purposes in accordance with the wellknown criteria that need to be observed during a photogrammetric acquisition, the overlapping between the selected frames was acceptable for image relative orientation and sparse cloud matching ( Figure 3). Since it was not possible to trace back any information about the cameras used to acquire the images retrieved from the various archives, the calibration of the cameras has been evaluated by the self-calibration approach implemented in Agisoft Photoscan.
As stated above, during the 2010 survey campaign some points of the apse and the ciborium surfaces placed near the destroyed dome (identified in the elements still existing today, placed on the areas not involved in the collapse during earthquake of 1971) have been measured with traditional topographic method. From these points, 11 (selected based on their detectability in the historical pictures) have been used as Ground Control Points (GCPs) and Check Points (CPs) during the photogrammetric process in order to evaluate the metric accuracy. Seven of these natural points have been used as GPCs, four as CPs.
In Table 1 Table 2. Main details of the photogrammetric process.
As it is possible to observe in Figure 4, the obtained point cloud is characterised by a high noise and the density is not homogeneous. The density of the obtained point cloud has been analysed with open-source software CloudCompare: from Figure 5 and Table 3 is possible to observe the high value of the standard deviation, especially in those regions in which the overlap of the images is lower (Figure 6 -right). This is due to the fact that the images have not been acquired for 3D metric documentation purposes and therefore the overlapping in the external areas of the fresco is lower than the central ones.

3D Modeling
From the obtained results, a first irregular mesh has been achieved. Since the generated point cloud is extremely noisy, the achieved raw 3D model (automatically generated from the used software) is characterised by several topological errors. Consequently, before further processing operations, an editing phase is necessary with manual and semi-automatic procedures.
In the present research, the editing phase has been performed using the 3DReshaper platform by Hexagon, in order to obtain a smooth surface (Figure 7) on which it is possible to correctly projecting a high-resolution texture.
In order to reassemble the remaining fragments of the fresco a textured model is necessary since the restorers need a plane representation of some portions of the test site area. In this case the 3D model is used as a digital surface on which a highresolution texture can be applied; subsequently the single portions of the mesh need to be orthoprojected to obtain the required two-dimensional images. The images have to be printed in 1:1 scale, so re-texturing the 3D model is necessary since texture generated from historical images does not have an adequate resolution and a sufficient level of detail for the requirements of the restorers. In order to obtain a better quality of the final product a very highresolution image has been used. The selected image is a highresolution digitisation (45628 x 34540 pixel) using a planar scanner of the original film (a b/w film 4.5x6 cm, stored in ISCR's archive). In any case, although low-resolution texture was not suitable for the final aims of this study, it was still very useful in order to provide the references during the collimation process of the homologues points, in order to orient the frame from ISCR's archive and to project it on the surface of the 3D mesh ( Figure 8). Once the test area of the second angel (placed in the right of the Pantocrator Christ) has been identified, in accordance with the requirements of the restorers, the mesh with the high-resolution texture has been segmented in 8 parts (Figure 9) in order to start with the next processing steps: orthoprojection and printing requirements of the restorers. Between each area and the adjacent ones, an overlapping of approximately 10 cm has been provided, in order to facilitate the subsequent reassembly operations. Figure 9. Segmentation of the texturized mesh in eight parts (angel on the right side of the Pantocrator Christ).

Cylindrical orthoprojection of the segments
In order to apply the TATTOOWALL®system technique, an orthoprojection of each segment is necessary. The aim is to obtain two-dimensional images in scale of 1:1 so that they can be printed and, successively, attached to the support structure in carbon fiber (as explained in the following sections).
To perform these projections, the unrolling tools of 3DReshaper commercial platform has been applied to achieve a cylindrical orthoprojection. For each segment, the best-fitting cylinder has been interpolated (least square interpolation) (Figure 10 -left) and the central axis has been extracted in order to unroll the tubular shape along it. Cylindrical orthoprojections are commonly adopted for cartographic uses in order to provide a planar representation of the Earth, even if the areas more distant from the point of tangency with the unwrapping cylinder (generally the Equator) result extremely distorted. For this reason the 3D model has been segmented in eight portions in order to minimise the distortions in the areas where the tangency between the mesh and the interpolated tubular shape is reduced. If the segments (belonging to a surface approximated to a sphere) were bigger, the distortions on the external parts would be excessive and the orthoprojections would not be fit for the purposes of the restorers. Each segmented mesh has been unwrapped around the axis of revolution (which in this case is the barycentre axis of the best fitting cylinder) in order to obtain a high-resolution twodimensional projection in 1:1 scale of the desired part of the model (Figure 10 -right). Since the mean value of the discrepancies between each portion of the 3D mesh and the interpolated cylinder is millimeter-level, the deformations are almost inexistent. Figure 10. On the left: best fitting cylinder of one of the segmented 3D mesh and its central axis. On the right: orthoprojection of a portion of the 3D mesh.

Construction of the support panels and structures
The structures designed to hold the wall painting fragments needed to copy the shape of the collapsed masonry, so as to offer a similar shape for their position and orientation in the space. A similar problem has been faced in 2005 for the reconstruction of the vaults of the Basilica Superiore in Assisi after the 1997 earthquake, where the need was to adapt the shape of the panels to that of the distorted masonries.
In that case (Iaccarino Idelson and Serino, 2011), the counterform needed to be shaped according to a 3D drawing copying the existing walls, and the technology used was that of a skilled carpenter, cutting the rib elements with a band saw according to a 1:1 print on paper of the vast project. The panels measured approx. 70 square meters. A similar procedure was used to manufacture the support panel for a wall painting that had been detached with the "strappo technique" from a demolished building in Rome, and still on canvas. The shape was recovered from the measures of the wall painting itself, with a reverse procedure (Borzomati and Iaccarino Idelson, 2007). For the apse of S. Pietro in Tuscania the situation was simpler, as the masonry had been reconstructed according to the shape extracted as previously described, and then approximated to a sphere. Moreover, the rear face of the fragments had been regularized, so that their thickness could be regarded as an even layer of approx. 8 mm. For this reason, the whole surface of the panels was defined by one single information: radius 3.350 m. The panels needed to have an orientation coherent with that of the floor of the display area in order to appreciate the images from a point of view aligned to that of the original fruition of the church, though from a much smaller distance. The orientation was also needed to design the supporting structures in steel. Another problem was that of the dimensions of the entrance doors and stairway, which obliged to build the panels in separate elements in prevision of a reliable reassembling of the large panels in the Museum's premises. The cutting lines needed to be accurately defined in order to minimize the perception of the discontinuity, which should not be found in areas that should host larger fragments or clusters of fragments. The panels were built with a layered sandwich structure, Carbon fibre/epoxyend grain balsa wood -Carbon fibre/epoxy. Each layer is soft and flexible during the construction and, while the epoxy resin completes its curing, needs to be held in place within a vacuum envelope on the surface of the counterform. In this case, a portion of sphere shaped with hot wire from large blocks of polystyrene.
The sandwich structure provides the required rigidity of the wide panels, independently from the display stands in steel, thanks to the choice of the carbon (416 g/m 2 ) and of the thickness of the balsa core material (20 mm). On the rear of the cutting line, a broad molded rabbet plate was built with a lay-up of 8 layers of carbon, to be used for securing all the way the joining area with bolts. The structures were built by Equilibrarte srl under the direction of Antonio Iaccarino Idelson and Carlo Serino. In the following Figure 12 the final result is shown. Figure 12. The support panel and structure.

Reassembly of the fresco
After the earthquake of 1971, the fragments of the fresco collected from the area of the apse of the Church of S. Pietro in Tuscania have been subjected to a long and meticulous preliminary examination. During this preliminary study every fragment has been cleaned, consolidated and carefully observed in order to be classified and stored in accordance with chromatic, consistency and conservation status criteria ( Figure  13). This study has allowed the discovery of fragments belonging to the same area and the consequent and gradual reconstruction of small portions of the fresco.
However, in such a complex context, the principal findingwhich represents the main aim of the projectis represented by the opportunity to recontextualize these little portions in order to document the lost masterpiece through a correct and comprehensible presentation of its fragments. The most relevant figures of the fresco (the four angels on both sides of the Christ Pantocrator) have been full-scale printed from the historical images of the dome of the apse before the earthquake. Both manual and digital reconstruction attempts have been performed on these images. The followed workflow has been developed by Laboratorio Mantegna (founded by Università di Padova) in order to assist during localisation of the fragments of the paintings by Mantegna in Cappella Ovetari. Although in this case the manual searching has shown to be more effective, digital attempts have been very useful, perhaps even essential, during the preliminary phase of the study. In fact, the digital approach has allowed identifying very quickly and with high precision a very high number of fragments. Many of these fragments belong to areas with monochromatic background and this approach has provided some references to manual searching operations. Furthermore, in some other cases, it was possible to identify adjacent fragments characterised by a very different chromaticity (for conservation reasons).
After the reconstruction attempts, the obtained results have enabled to proceed with the project and to obtain two physical prototypes in order to effectively present the final work. For this purpose, the images of the two angels placed in the upper part of the dome have been selected. As already stated, thanks to 3D model of the dome of the apse, it was possible to build the support structures and to orthoproject the original images in order to obtain orthophotos in scale 1:1. The concave side of the structures has been prepared in order to support the printed image as is reported in the previous section. The surface has been covered with a thin layer of mortar with acrylic composition; the mortar has been applied in an irregular way in order to simulate the original appearance of preliminary layers of the fresco. On this surface Graphic Report snc (from Padova) has applied the photographic background (the orthophotos), carried out by using TATTOOWALL®system.
The thin membrane has been attached to the mortar with a specific glue (synthetic emulsionated resin glue screeds) and furthermore the surface of the membrane has been protected with a two-pack paint with opaque effect. At this point all fragments assessed as fit during the preliminary searching have been thickened to 10 mm and then they have been glued to the structure through acrylic mortar with very fine grain size. The adopted method is fully reversible and it allows the possible rectification of the fragments already placed and the progressive insertion of other portions, leaving the door open to a future of new possibilities for research and image enhancement for improving the collective memory.

CONCLUSIONS
During the study reported in the previous sections and from the comparison between the different used approaches, various aspects have been explored. On one side it can be observed how the technological progress of the last few years and the continue research have provided and continue to provide increasingly effective tools to users operating in the fieldwork of Cultural Heritage especially the ones involved in conservation projects As is well known the implementation of automatic or semiautomatic procedures as regards the geometry reconstruction process allows to reduce significantly the data processing time; for example the stereo-plotting of the dome by traditional photogrammetric approach (a procedure which had required a huge manual involvement of the operator) could be replaced nowadays with procedures characterised by a fully automatic approach.
As regards the metric accuracy of the two followed methods (in 2010 and 2017), it has been observed that the accuracy of the obtained metric product is totally coherent. By comparing the point cloud generated thanks to SfM algorithms and the sphere (with a radius of 3.35 m) interpolated by around 4000 points plotted in 2010, the following metric discrepancy (Figure 16-17 and Table 4) could be observed.   Table 4. Mean and standard deviation of the cloud-to-mesh comparison.
Naturally one of the main advantages of the use of SfM approach is the save in terms of manual work from the operator. Nevertheless the automatism, the monitoring and the management of the dataset by the operator remains highly important as regards accuracy check of the obtained metric results and the interpretation of the final products. However the use of these kinds of techniques by a competent user might mean a significant save of time. Another pro is the higher completeness of the obtained data. The photogrammetric block (composed of images acquired from different angles) has allowed to rebuild even the part of the dome which were partially occluded by the ciborium in the stereoscopic images used in 2010. In connection with this, one of the most interesting future perspectives can be considered the possibility to perform a co-registration between the historical images (acquired before the earthquake of 1971) and more recent images (in this case acquired during the survey campaign of 2010, following a photogrammetric approach). In this way it is possible to reconstruct the geometry of the original complete apse (not only the shape of the dome) as presumably it should be before the collapse. As is reported in Figure 18 coregistration was possible by applying masks on the images of 2010 (in order to cover the concrete spherical dome built after 1971). In this way the software perform the tie points extraction only in the portions of the images with common elements between Time 1 (before 1971) and Time 2 (survey campaign of 2010). The photogrammetric block has been assembled by the 23 historical images acquired before 1971 and 43 images acquired in 2010 by a DSLR full frame camera (Canon EOS 5D Mark II), the images acquired in 2010 were transformed in b/w in order to preserve a radiometric coherence between the different dataset.
As a first test in order to evaluate metric accuracy during the photogrammetric process, 10 points have been used as GCPs and 5 as CPs. In Table 5 it is possible to see total error (3.5 cm as regard the residual values on GCPs and 5.  Finally, is possible to underline that another interesting aspect of the research presented in this paper is how very often, despite the intrinsic fragility that characterises Cultural Heritage, a proper documentation can provide a significant increase of the resiliency of the assets belonging to our built heritage. During this work (as in the one of 2010), even though the lack of metric data of the original dome, the final aim has been reached thanks to the precious contribution given by images stored in historical archives.
In conclusion, as observed in the previous sections, in these cases a continuous and adequate documentation is not only desirable; in such an eventuality it represents the only way to recover any information about buildings and artworks, which run the risk of being lost.