D MODELLING OF A HISTORICAL BUILDING USING CLOSE-RANGE PHOTOGRAMMETRY AND REMOTELY PILOTED AIRCRAFT SYSTEM ( RPAS )

The photogrammetric survey of architectural Cultural Heritage is a very useful and standard process in order to obtain accurate 3D data for the documentation and visualization of historical buildings. In particular, the integration of terrestrial close-range photogrammetry and Remotely Piloted Aircraft Systems (RPASs) photogrammetry allows to create accurate and reliable 3D models of buildings and to monitor their state of conservation. The use of RPASs has indeed become more popular in Cultural Heritage survey to measure and detect areas that cannot normally be covered using terrestrial photogrammetry or terrestrial laser scanner. The paper presents the results of a photogrammetric survey executed to document the monumental complex of Villa Lampedusa ai Colli in Palermo (Italy), one of the most important historical buildings of the town. An integrated survey by close-range photogrammetry and RPAS photogrammetry was planned and carried out to reconstruct the 3D digital model of the monumental complex. Different images configurations (terrestrial, aerial nadiral, aerial parallel and oblique to the façades) have been acquired; data have been processed to verify the accuracy of the photogrammetric survey as regards the camera calibration parameters and the number of Ground Control Points (GCPs) measured on building façades. A very detailed 3D digital model and high-resolution ortho-images of the façades were obtained in order to carry out further analysis for historical studies, conservation and restoration project. The final 3D model of Villa Lampedusa ai Colli has been compared with a laser scanner 3D model to evaluate the quality of the photogrammetric approach. Beyond a purely metric assessment, 3D textured model has employed to generate 2D representations, useful for documentation purpose and to highlight the most significant damaged areas. 3D digital models and 2D representations can effectively contribute to monitor the state of conservation of historical buildings and become a very useful support for preliminary restoration works.


INTRODUCTION
Conservation and restoration of cultural heritage involve protection of historical buildings using all new technologies that are effective in keeping their original condition as long as possible; they are often associated with the geometric survey, the monitoring and the documentation of the heritage.In particular, when historical buildings are in a state of neglect and deterioration their survey and monitoring are the basic steps for the correct planning of conservation and restoration.The demanding task to survey historic buildings, in order to document their geometric and morphological features with very high accuracy, becomes an important incentive for the use of innovative geomatics approach.The geomatics survey is generally focused to obtain 3D point clouds and 3D models, but some traditional products, like ortho-images or 2D drawings, are still very usefully for Cultural Heritage study.The complexity and multiplicity of built heritage sites define a wide range of possibilities and solutions to acquire good quality 3D data.Therefore, it is difficult to define standard procedures and the integration of different technologies seems often the best solution.Integration of several 3D data acquisition techniques is becoming an important resource in Cultural Heritage surveying of valuable historical monuments and it provides complete and multi-scale information according to the monument complexity, the accessibility of the area and the different level of detail required (Gagliolo et al., 2017).New strategies and new techniques coming from photogrammetry are becoming standard approaches in Cultural Heritage survey.The progress for fully automated processing (by the so-called Structure-from-Motion and Multi-View Stereo approaches) allows to process large image datasets and deliver 3D (both sparse and dense) results with a level of detail and precision variable according to the applications (Frahm et al., 2010;Crandall et al., 2013, Remondino et al 2017).Moreover, the widespread of low-cost and user-friendly Remotely Piloted Aircraft Systems (RPASs) and the availability of many types of sensors allow greater use of these systems for photogrammetric applications to obtain accurate 3D models.The potentiality of RPASs for a photogrammetric survey of Cultural Heritage was shown in many works (Lo Brutto et al., 2014;Bolognesi et al., 2015;Caroti et al., 2015;Lo Brutto et al., 2017;Murtiyoso & Grussenmeyer, 2017).Actually, in case of wide and open sites or monuments with vertical development, the integrated use of both terrestrial and aerial images from RPASs becomes useful to gather complete 3D data of vertical and horizontal features (Russo & Manfredini, 2014).Moreover, the integration of oblique images, acquired with RPASs, strengthens the camera network geometry, allows creating more complete datasets and to overcome the nadir view of the typical aerial photogrammetric survey (Nocerino et al., 2013).The considerable advantages of the integration of the terrestrial and RPAS images, with the joint use of nadir and oblique cameras, allow achieving textured 3D models of a monumental complex with high accuracy and reliable features (Chiabrando et al., 2017;Bolognesi et al. 2015).Textured reality-based 3D models are not the only product of a 3D survey of historic buildings; they can be used to extract further metric data, such as conventional 2D drawings (plans or sections) or 2D raster data (façade ortho-images).All these data could be used to plan restoration and monitoring activities (for damaged historical buildings), to promote built heritage assets, (by means of tools for visualization and virtual reconstructions), and to share heritage information (Achille et al., 2015;Barazzetti et al., 2017;Mezzino et al., 2017).In the town of Palermo (Sicily, southern Italy), are still present several examples of historic buildings, more or less important, which are partly or totally abandoned without being insured adequate maintenance actions.These buildings are part of that particular urban phenomenon that interested the town of Palermo since 15 th to the 18 th century, with the construction of aristocratic villas outside the town walls.These villas were complex architectural systems that generally included a residential building, a "casena" (buildings used as servants quarters) and a garden.In this work we have focused our study on the 3D survey of Villa Lampedusa ai Colli in Palermo; this Villa, built in 1721 and belonged to Prince Giulio Tomasi di Lampedusa (whose life inspired the great-grandson Giuseppe Tomasi di Lampedusa in his novel "The Leopard" -in Italian "Il Gattopardo") is a typical example of these neglected and deteriorated historical buildings.The aim of the work was the creation of an accurate and reliable 3D model of the Villa, through the integration of different acquisition methodologies, in order to use it for future restoration and conservation activities.The 3D survey of Villa Lampedusa ai Colli was realized by a photogrammetric survey combining terrestrial images acquisitions and RPAS images acquisition.The terrestrial acquisition was realized with the aim of generating a whole coverage of the façades, while the RPAS was employed to acquire both vertical and oblique images of the façades and of the roof.In this way, using images captured from different points of view, it was possible to obtain a very dense point cloud and a very detailed 3D model of the building.To evaluate the influence of the camera calibration phase on the final accuracy different calibration approaches were performed during data processing.Furthermore, some tests were realized to evaluate the images orientation accuracy as regards the number of GCPs used and the geometric accuracy of the 3D model through the comparison with 3D data achieved by a terrestrial laser scanning survey.The 3D data were also used to obtain ortho-images and 2D drawing of the façades that allowed performing a preliminary study of the state of conservation of the building façades.

CASE STUDY: VILLA LAMPEDUSA AI COLLI
Villa Lampedusa ai Colli (Palermo -Italy) is an ancient summer residence of the local nobility, built in the 18 th century and representative of an architectural trend of that period.Between the 17 th and 18 th centuries, an important architectural development involved whole Italian regions.In this context, the outside expansion of the inhabited area of Palermo, which had already begun since the 16 th century, increased outside the city walls.In a little over a century, rural areas around Palermo became important settlement centres, especially for the intellectual elite.The nobility left the medieval idea of fortification and appropriated these rural areas to build their summer residences, known as "sollazzi", and their hunting houses.This summer residence trend became a European tendency too.The Villa emulated the great European residences, by detaching itself from the original rural activities.In many cases, rural buildings and "bagli" (from latin "Valium") were converted into summer resorts.In the 18 th century, this urban expansion achieved its highest manifestation: the nobility, with its entourage, moved from the city to the country and the summer residence became a place of entertainment in Sicilian temperate days.Nowadays, Palermo could boast beautiful and inestimable noble houses, sumptuous buildings and summer residences but they are in an absolute state of decay and abandon, often victims of vandalism.The architectural structure of the Sicilian villas was depending especially on Spanish and French artistic influences.According to the classical typology of these residences, Villa Lampedusa ai Colli is characterized of the main body (the patrician house), low bodies that housed staff service and accommodation rooms ("casene"), a staircase and a wide garden surrounding three sides of the building (Figure 1).The patrician house presented a U-plan and a courtyard in a longitudinal position with it.The house had a ground floor and the main floor (or "piano nobile").The "casene" were perpendicular to the main façade, defining a rectangular courtyard closed on three sides.Outwardly, the façades were richly decorated with refined architectural elements.Inwards, rooms were decorated with frescos, stuccos, gold decorations, ceramic floors and painting doors (typical of the Spanish influence and the local tradition).The family's coat of arm was located at the top, in a symmetric location regarding the main façade.
The main façade of Villa Lampedusa ai Colli was also dominated by a pincers staircase (Figure 2).It was the fulcrum of these villas and often placed in a symmetric position with the main façade.The Villa Lampedusa staircase was characterized by two symmetrical flights which connected the ground floor with the "piano nobile".The garden was arranged behind the Villa, laterally to the building.It was separated from the residence with a perimeter fence and it housed greenhouses, pavilions and café-house.Despite its indisputable artistic and cultural relevance, the neglected current state of Villa Lampedusa ai Colli is the result of many changes over the years, since the end of 17 th century, and, above all, of a long period of abandon.At the present time, the Villa is affected by a high level of deterioration, due to infiltrations, cracks in the walls, detachments of plaster, roofs and floors unstable or even collapsed (Figure 3).

MATERIALS AND METHODS
The 3D data acquisition was realized by a photogrammetric survey, with terrestrial and RPAS images acquisition.The terrestrial acquisition was realized with the aim of generating a whole coverage of the façades, while the RPAS acquisition was carried out to acquire images of the façades and of the roof.In this way, using images captured from different points of view, it was possible to obtain complete images coverage of the building.
The survey was based on a topographic network all around the Villa, measured by means of a Leica TPS 1105 total station.The topographic survey was also used to measure the coordinates of 44 natural points, uniformly distributed on the four façades; these points were used in the orientation phase as Ground Control Points (GCPs) and as Check Points (CPs).

Terrestrial images acquisition
The terrestrial images acquisition was planned to obtain complete coverage of the façade and was carried out using a digital camera Nikon D5000, with a 12.3 megapixels CMOS sensor of 23.6 mm x 15.8 mm and a focal lens of 35 mm.Convergent images were acquired from north-east, north-west and south-west sides at a distance of about 28 m to achieve a Ground Sample Distance (GSD) of 5 mm (Figure 4).The photogrammetric acquisition of the main façade was more problematic because of the presence of two smaller bodies.
Seven different images blocks were planned, with convergent and parallel photos of the main façade, from the ground plane and from the bodies' roofs, at variable distances between 5 m and 15 m, with a GSD variable from 2.3 mm to 5.5 mm (Figure 4).Overall 172 terrestrial images were acquired.

RPAS images acquisition
A complete RPAS images acquisition, with parallel, nadir and oblique camera's optical axes, was carried out at different distances from the building and varying flight height.The acquisition of aerial images was realized using an RPAS NT4 Contras produced by Airvision.This system is a multirotor, in carbon fibre, composed of eight motors electrically powered; the electronic equipment required for both the remote control and the automatic flight.It has a payload of about 600 gr, with a maximum flight time of about 15 minutes.
The system is equipped with a digital camera Nikon Coolpix S3300 with a 16 megapixels CCD sensor of 4.4 mm x 6.16 mm.The camera has a focal length variable from 4.6 mm to 27.6 mm but, for the survey, the minimum focal length was used.The camera is mounted on a servo-assisted gimbal that allows controlled rotations of 90° along the horizontal axis, to acquire vertical or oblique images.
In order to acquire a complete documentation of the building three flights, with a GSD variable from 2 mm to 5.5 mm, have executed; the three flights were planned to take images with a nadiral view to the roof, an oblique and a parallel view to façades (Figure 5).The camera has been set up with a tilt angle of 45° to obtain images of the roof and of part of the façade (Figure 6).These images can be used to link the nadiral and the parallel blocks.
Parallel images of façades were acquired at 7 m altitude from the ground and 15 m of distance from the building.Both for oblique and parallel flights, a stereoscopic strip was carried out, with a forward overlap of 70% for long and short sides of the building; for building corners, some convergent images were taken.To evaluate the influence of the camera calibration phase on the final accuracy, different calibration processes were performed during data processing.Moreover, some tests were performed to evaluate the image orientation accuracy as regards the number of GCPs used.

Camera calibration of terrestrial images
The camera calibration of the terrestrial images was carried out using Agisoft PhotoScan package and PhotoModeler Scanner by EOS Systems Inc. package.The first set of camera calibration parameters was obtained using the images of the Villa, Agisoft PhotoScan and the typical SfM approach.
A second dataset was instead obtained by a typical photogrammetric approach, using a test field with PhotoModeler Scanner coded targets and a multi-station convergent imaging network.This dataset was used in PhotoModeler Scanner to estimate the second set of camera calibration parameters.These parameters were converted to be used in an Agisoft PhotoScan project for terrestrial images.
To compare the calibration parameters of the Nikon D5000 terrestrial images, two projects were created in Agisoft Photoscan.The first one using the parameters estimated with Agisoft Photoscan (Project PS) and the second with the parameters estimated with PhotoModeler Scanner (Project PM).
In both projects were taken into account 21 GCPs and 23 CPs, homogeneously placed on façades.The quality of these two solutions was evaluated by considering the Root Mean Square Error (RMSE) of GCPs and CPs (Table 7).
The results are very similar; the RMSE differ by a few millimetres.This result shows that the terrestrial image network is enough strong to compute reliable camera calibration parameters even using the SfM approach.So, we decided to use, for next steps, the Agisoft PhotoScan calibration parameters for the terrestrial block.
Table 7. Results of terrestrial image orientation using PhotoScan (PS) and PhotoModeler (PM) camera calibration parameters.

Camera calibration of RPAS images
The RPAS acquisition was performed using the digital compact camera Nikon Coolpix S3300.In this camera autofocus cannot be deactivated; so even if all images were taken with the same focal length by changing the camera-to-object distance, it is possible to have slight variations in the calibrated focal length.The RPAS images were processed only using Agisoft PhotoScan and the SfM approach.
To check the influence of camera calibration parameters in the accuracy of image orientation three different approaches were evaluated: i. estimation of camera calibration parameters by three different self-calibration projects, one for each flight altitude; ii.
estimation of camera calibration parameters by one self-calibration project, in which the images have been divided into calibration groups according to flight altitude; iii.
estimation of camera calibration parameters by one self-calibration project, without dividing the images into groups and calculating a single set of calibration parameters for all the images.After the camera calibration process, three overall projects with all acquired RPAS images were created.The first one with camera calibration parameters estimated from each flight altitude single project (Project 1).The second with camera parameters estimated using all the images splitting into calibration groups, according to flight altitude (Project 2).The third using a single set of camera calibration parameters for all images (Project 3).To assess the images orientation accuracy of each project, 21 GCPs and 23 CPs were used.The quality of each project was evaluated by considering the RMSE of GCPs and CPs.Also, in this case, the residues show very similar results (Table 8).This result would seem to show that the output of an SfM approach is influenced by the camera calibration process than traditional photogrammetry.This hypothesis requires a more in-depth analysis, however, which has not been carried out in this work.The parameters obtained from the project in which the images have been divided into calibration groups according to flight altitude were chosen for the subsequent image orientation step.

Image orientation
The image orientation was carried with Agisoft PhotoScan using all the images (terrestrial and aerial) and the camera calibration parameters previously computed.Also in this phase, some tests have been carried out to verify if it is possible to decrease the number of GCPs maintaining, however, the same levels of accuracy.
The GPSs number was changed from a minimum per façade (4 on the long sides and 2 on the shorter ones) to a maximum per façade (8 GCPs on the long sides and 6 on the shorter).In total three projects were carried out with 28 GCPs (Project 28 GCPs), 20 GCPs (Project 20 GCPs) and 12 GCPs (Project 12 GCPs) respectively.The accuracy check was evaluated by calculating the RMSE of 16 CPs.
Although the number of GCPs decreases to less than half compared to the initial project, all projects have an RMSE within 0.025 m (Table 9).This value confirms the good geometry of the photogrammetric block configurations which, even with a fairly limited number of GCPs, does not show any deformation.This aspect is very important because it allows affirming that with very rigid photogrammetric block geometries, even using SfM approaches, it is possible to obtain reliable results even if a limited number of GCPs is used.
Table 9. Results of RPAS image orientation using a different number of GCPs.

3D model production
The "Project 28 GCPs" was selected to calculate the 3D model of the Villa and the ortho-images of the façades.The 3D models were achieved with Agisoft PhotoScan according to the following steps: calculating of a dense point cloud by dense image matching, building of a 3D mesh and texturing of the 3D mesh.
A dense point cloud of about 17 million points was obtained using the "ultra-high" quality setting of Agisoft PhotoScan; this quality setting means processing of images using the original resolution and allows getting the best results as regards the level of detail of the 3D model.A very defined mesh, in terms of numbers of polygons and geometry, was calculated from the point cloud; about 14 million faces and over 7 million vertices were indeed generated.In order to increase the quality of the 3D model, a texture from the images of the photogrammetric survey was produced (Figure 10).
To evaluate the metric reliability of the photogrammetric 3D model, a comparison with a Terrestrial Laser Scanner (TLS) model was carried out.A total of 12 scans around the building were realized with a Faro Focus 3D, with an average resolution of 6 mm, for a total of about 37 million points.To facilitate the comparison between the photogrammetric mesh and the laser scanner 3D point cloud, single meshes for the south-east, southwest and north-east façades were selected form the overall photogrammetric 3D model.Because of the presence of a scaffolding and a creeping plant, the comparison between the north-west photogrammetric mesh and TLS point cloud has not been realized.
The photogrammetric meshes and the laser point cloud were compared in CloudCompare using the C2M.This algorithm allows computing Cloud-to-Mesh distance.For each point of the point cloud, the algorithm searches the nearest triangle of the compared mesh and computes the distances (Figures 11-13).
As visible in the histogram next to pictures, the distribution of residuals is included between 0 and 4.5 cm for the south-east façade and between 0 and 1.5 cm for the north-east and the south-west façades.Table 14 shows the statistic distances computed in CloudCompare.
Longer distances between the 3D models are focused along the splayed windows, the adjoining walls and along the TLS unattainable areas (the upper part of cornices and the window brackets).Overall, the results of the comparison show a good accuracy of the photogrammetric approach, with an RMSE below 0.0015 m for the 80% of data.
Polygonal reconstructed models from photogrammetric point clouds are noisier but more complete than laser scanner ones.Instead, these last present more shadow areas caused by façade overhangs.The integration of terrestrial survey with different aerial configuration block allowed overcoming laser scanner data collection limits.

2D documentation
2D data is an essential part of a complete documentation in Cultural Heritage field.Actually, for an architectural analysis and restoration work, as in this case, 2D representations become a resource for other experts, like restorers, which can be involved in such a multidisciplinary project.For this reason, ortho-images for all façades and for the building roof were realized with a resolution of 5 mm .In order to document the architectural complex, for a complete and reliable knowledge of the monument and for a future restoration project, ortho-images were employed to get 2D information.Figure 19 shows together the south-east orthoimage and to 2D drawing realized from it.By means of 2D drawing, a descriptive map of degradation was carried out in order to document the state of conservation of the façades and to highlight to most significant damaged areas.The ortho-image is used to document some of the most significant damaged areas, clearly visible on the 3D reconstructed model too, waiting for future conservation activities.Figure 20 shows the map of significant damages on the south-east ortho-image.The whole building is in a dreadful state of conservation and it needs urgent consolidation and restoration works.As visible in the last picture, the main façade, as well as the whole building, show extended and extensive cracks, local loss and fragmentation of material, peeling of surface layers, holes, black deposits, biological colonization.Moreover, the risk of floors and slabs collapse is high.

CONCLUSIONS
The paper presents a complete survey workflow to document Villa Lampedusa ai Colli, in Palermo.Although this historical monument has an incontrovertible cultural value, it is currently in a state of neglect and decay.An integrated photogrammetric survey, by close-range terrestrial photogrammetry and closerange aerial photogrammetry by RPAS, has been planned and carried out in order to document and monitor this historic building.The employment of RPAS acquisitions, with both nadiral and oblique images, gave the possibility to completely survey the building, from the roof to the ground level.
The work was also used to perform some tests about the camera calibration process and the number of suitable GCPs.
The camera calibration parameters of terrestrial dataset were calculated with two different software packages and two diverse approaches (an SfM approach and a typical photogrammetric approach).The result shows that the terrestrial image network is enough strong to compute reliable camera calibration parameters even using the SfM approach.
The camera calibration parameters of aerial dataset were calculated using only an SfM approach, but with three different strategies.Also, in this case, the result shows that the SfM approach is suitable for camera calibration process and the different strategies adopted do not substantially change the final results.
To achieve the overall documentation of the monument, a complete project was created, integrating terrestrial and aerial dataset and importing the camera calibration parameters from previous projects.The accuracy of final project was evaluated by varying the number of GCPs.Three different GCPs configurations were tested, from a minimum number of GCPs per façade (4 on the long sides and 2 on the shorter ones) to a maximum per façade (8 GCPs on the long sides and 6 on the shorter).The results show that a minimum number of GCPs per façade is sufficient to guarantee good survey accuracy even using SfM approaches.
A very detailed 3D model of the historical building was generated.The metric reliability of this 3D model was evaluated also by means of laser scanner 3D model.The comparison confirmed the accuracy of the photogrammetric survey, with differences below 0.002 m for the 80% of data.Finally, the ortho-images production for the façades and for the roof allowed to obtain 2D drawings and maps of damages of the building.The realisation of 2D products and the possibility to identify significant damages and degradations directly on 2D and/or 3D data could be a very useful support for restoration works.
In conclusion, nowadays the integration of different photogrammetric approaches proved their indisputable potentiality as the best way to the Cultural Heritage documentation and for monitoring purposes.

Figure 3 .
Figure 3.The state of conservation of the north-west façade.

Figure 5 .
Figure 5. Schema of RPAS acquisition.The flights were carried out both in an automatic way, previously planned and managed by the software that connected the platform to the ground station, and in a manual way.A first flight, for the vertical survey of the roof, was carried out at 25 m altitude from the ground and a distance of 15 m from the roof.The forward overlap among the images was of 80%; the lateral overlap among the strips was of 70%.The flights direction was oriented parallel to the longer side of the Villa.The flight to acquire oblique images was carried out at 20 m altitude from the ground and 15 m of distance from the building.The camera has been set up with a tilt angle of 45° to obtain images of the roof and of part of the façade (Figure6).These images can be used to link the nadiral and the parallel blocks.Parallel images of façades were acquired at 7 m altitude from the ground and 15 m of distance from the building.Both for oblique and parallel flights, a stereoscopic strip was carried out, with a forward overlap of 70% for long and short sides of the building; for building corners, some convergent images were taken.

Figure 6 .
Figure 6.Oblique image from RPAS.4.DATA PROCESSINGThe data processing was executed to obtain a 3D model of the building and ortho-image and 2D drawing of the façades.The image orientation and the 3D reconstruction were carried out with Agisoft Photoscan Professional Edition package.Image orientation and 3D reconstruction were performed with typical Agisoft Photoscan workflow: automatic image orientation by Structure-from-Motion (SfM), bundle block adjustment, dense image matching to generate a 3D point cloud, 3D reconstructing by polygonal mesh, texture mapping and ortho-image production.To evaluate the influence of the camera calibration phase on the final accuracy, different calibration processes were performed during data processing.Moreover, some tests were performed to evaluate the image orientation accuracy as regards the number of GCPs used.

Figure 11 .
Figure 11.Comparison between the south-east photogrammetric mesh and TLS point cloud.

Figure 12 .
Figure 12.Comparison between the north-east photogrammetric mesh and TLS point cloud.

Figure 13 .
Figure 13.Comparison between the south-west photogrammetric mesh and TLS point cloud.

Figure 20 .
Figure 20.Map of significant damages on the south-east façade.

Table 8 .
Results of RPAS image orientation using the three different camera calibration parameters.

Table 14 .
CloudCompare statistical parameters between photogrammetric meshes and TLS point cloud.