3D MODELING OF GIRIFALCO FORTRESS

: Despite photogrammetry has been longly used for 3D model production, the recent development of structure from motion based reconstructions and the spread of unmanned aerial vehicles (UAV) are increasing its usage even further. Actually, the combined use of nadir and oblique UAV photogrammetry allows to cross the bridge between close range and aerial photogrammetry, enabling the quick acquisition of images for detailed 3D model production. This paper assesses the accuracy of UAV photogrammetry in the reconstruction of a cultural heritage building, namely the Girifalco fortress (Cortona, Italy), and it evaluates advantages and issues of this kind of 3D model generation. Furthermore, this work aims also at investigating the possible use of the generated model in order to extract geometric information about the building. First, elementary geometric shapes are extracted from the photogrammetric point cloud: the results of this step can be exploited for enabling the automatic generation of compact models of the building, which can be useful for instance in 3D city modeling applications. Then, a proper analysis of the geometric information included in the extracted shapes can be used in


INTRODUCTION
The worldwide UAV market is continuously growing as also the number of applications that involve the usage of aerial drones.During the last years the use of UAVs for aerial surveys is becoming a consolidated application, commonly used for obtaining 3D models of the outer side of buildings.In accordance with this trend in outdoor building modeling, this paper considers the generation of a 3D model of the Girifalco fortress (Cortona, Italy, Fig. 1).Furthermore, in this work point cloud segmentation and classification algorithms have been applied in order to enable the automatic extraction of certain valuable information from the 3D point cloud produced by means of photogrammetric processing of images acquired by a UAV (Vosselman et al., 2004, Rabbani et al., 2006, Makuti et al., 2018).
A UAV photogrammetric survey of the Girifalco Fortress has been done by acquiring 960 images of the area of interest.Such survey, conducted on September 8, 2017, has been done by using a DJI Phantom 4, flying at an average altitude of 40 m over the ground (ground sample distance was 1.4 cm, approximately).Flight plan was designed to ensure (at least) 60% of overlapping area between both successive images on the same strip and closest images from different strips.Furthermore, both nadir and oblique images have been acquired in order to improve the reconstruction accuracy of the walls of the fortress, and, concurrently, ensuring a better camera network geometry (Fraser andStamatopoulos, 2014, Chiabrando et al., 2017).
Several widely used commercial software are available in order to obtain 3D reconstructions of such buildings from images acquired by UAVs (Agisoft PhotoScan, n.d., Pix4D, n.d., Gonzalez-Aguilera et al., 2018).These tools, usually based on the Structure from Motion approach, enables 3D reconstruction with camera self calibration.Despite this can be a clear advantage in terms of saving the time required for the camera calibration, it can actually increase the reconstruction error of the obtained 3D model.The introduction of GNSS measurements in the photogrammetric reconstruction procedure, corresponding to the camera locations during image acquisitions, reduces the error level.However, ground control points should be used to ensure best performance of the photogrammetric reconstruction (Fraser, 2018).Motivated by the above considerations, 27 targets were distributed on the area and their positions were measured by means of GNSS RTK survey (average accuracy of the topographic survey was 1.6 cm).Ten of such targets were used as ground control points (GCPs), whereas the remaining seventeen were used as check points (CPs) for validation of the photogrammetric reconstruction, which has been performed with Agisoft PhotoScan 1.3.2.
Then, the obtained point cloud has been analyzed in order to extract information that can be useful to characterize the building geometric shape and structure, and to detect potential damages (Korumaz et al., 2017, Masiero et al., 2015).Despite commercial software is available in order to make 3D model geometric analysis, this paper considers the development of in-house procedures in order to make the obtained solution independent of the usage of any specific software.More precisely, the considered approach has been implemented in MathWorks Matlab, however such implementation can easily be exported in any other kind of programming language.
The obtained point cloud has been analyzed in order to detect approximately planar surfaces, which mostly corresponds to the fortress ceilings and walls.The identified surfaces can be processed further in order to obtain a synthetic simplified and com- The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W9, 2019 8th Intl.Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 6-8 February 2019, Bergamo, Italy pact model of the building (which might considered for automatically setting building geometry for producing for instance Building Information Models (BIMs) or CityGML models, (Angelini et al., 2017, Kolbe, 2009, Haala and Kada, 2010, Fissore and Pirotti, 2018, Guarnieri et al., 2017)).
Finally, a depth analysis can be used to determine windows, doors and damaged surfaces on the building walls.Actually, a classification between such cases is also investigated by using a machine learning classifier.

PHOTOGRAMMETRIC RECONSTRUCTION
DJI Phantom 4 is provided with 1 12.4 Mpix sensor, 1/2.3"CMOS.The camera focal lens is 8.8 mm, equivalent to 20 mm in a 35 mm sensor format, which ensures a field of view of 94 • .Phantom 4 flied at an average altitude of 40 m over the ground, hence ground sample distance was 1.4 cm, approximately.
Overlapping area between images along and across strips was (at least) 60%.Both nadir and oblique images have been acquired in order to improve the reconstruction accuracy of the walls of the fortress, and, concurrently, ensuring a better camera network geometry.

Photogrammetric reconstruction and validation has been performed
with Agisoft PhotoScan 1.3.2: a dense cloud of 40 Million points was generated with an average closest-point distance (e.g.resolution) of 1.5 cm.
Target positions (in WGS84 UTM-ETRF2000) were obtained by a GNSS RTK survey.According to GNSS measurements, average accuracy of the topographic survey was 1.6 cm (Root Mean Square Error (RMSE)).
As reported in Table 1, 3D RMS error was 1.2 cm on GCPs, whereas it was 3.1 cm on CPs (planimetric RMS error was 2.0 cm, altimetric RMS error was 2.4 cm).

DETECTION OF THE FORTRESS WALLS
Point cloud obtained by Agisoft PhotoScan has been analyzed in order to automatically detect planar surfaces, which can be useful in order to produce a compact but geometrically comprehensive model of the fortress (e.g. to be used in CityGML).
To this aim, a Random Sample Consensus (RANSAC) (Fischler and Bolles, 1981) planar surface detection approach has been implemented.
First, it is worth to notice that a significant part of the surveyed area is covered by vegetation.This can lead to a negative impact on both the quality of the generated photogrammetric point cloud (in terms of both completeness and accuracy) and on the detection of planar surfaces (walls are partially, or even completely, covered by vegetation).
A local surface fitting has been used in order to check whether points approximately lies on a planar/smooth surface.The smallest eigenvalue of the sample covariance matrix of points in a local spherical neighborhood (with radius 0.25 m) of the considered point has been considered in order to check the smoothed surface hypothesis (Habib and Lin, 2016), e.g. a surface locally well approximated by a plane leads to a two dimensional geometric description (i.e.small value of the third eigenvalue of the covariance matrix).This check allowed to filter out a significant part of the vegetation.
Normals to the 3D model have been computed on the local neighborhoods described above.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W9, 2019 8th Intl.Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", 6-8 February 2019, Bergamo, Italy Then, a RANSAC algorithm has been used to detect subsets of the original point clouds that can be well approximated with a planar surface, and whose points have associate normal directions quite similar to that of the detected plane (less than 35 • ).
An example of segmented planar surfaces can be seen in Fig. 3. Given the prior knowledge about the orientation of the obtained 3D model, the RANSAC algorithm can be tailored in order to detect vertical planar surfaces, which are those corresponding to the fortress walls, Fig. 4. Given the negative effect of the vegetation on segmentation results, walls have been delimited also by taking into account of point density and point quantity variations along each wall line.
Actually, the characteristics of the detected walls, shown in Fig. 4, can be improved further by taking into account of continuity constraints between consecutive parts of the walls.The implementation of this consideration will be considered in our future developments of this work.The detected wall planar surfaces can be exploited to automatically produce side/orthographic views/projections of the walls, as shown in Fig. 5.

DATA ANALYSIS AND INFORMATION EXTRACTION
Planar surface detection can be considered as a first step in order to obtain a geometric shape-based point cloud segmentation and Each wall has been analyzed searching for areas outside of the fitting plane, e.g.Fig. 7 ((a) has been obtained by using a proper threshold on the point-to-plane distance, which should be set depending on the photogrammetric reconstruction resolution and error).
Each connected region (with area larger than 200 cm 2 ) in Fig. 7(a) is considered as a different area to be classified.For each area the following geometric information has been extracted: height, width, area.
Then, three classes were considered for classification purposes: doors, windows, and damaged parts.A decision tree has been properly trained for the classification of such classes based on the geometric information extracted as described above.Results on this case study gave a 94.1% of correct classifications.Fig. 8 shows a misclassification example (damaged area classified as window).It is quite clear that the use of just geometric information provides relatively good results, which however should be improved by the introduction of different information (e.g. on the appearance of the area).
Furthermore, a more detailed analysis on the geometric characteristics of damaged areas can be done by checking the local pointto-planar surface distance, as shown in for instance in Fig. 9.

DISCUSSION AND CONCLUSIONS
This paper presented results obtained in 3D modeling of the Girifalco fortress obtained by means of UAV photogrammetry.
UAV data collection lasted for approximately one hour, which confirms one of the typical advantages in UAV photogrammetry, i.e. the possibility of surveying relatively large areas quite fast.
The 3D model (40 M points), obtained by processing the 960 images collected by a DJI Phantom 4, has been georeferenced using 10 ground control points, and validated by means of 17 check points, with a RMS error of 3.1 cm (GSD was 1.2 cm).Furthermore, standard processing of the imagery allows to obtain orthophoto and DTM of the surveyed area (He et al., 2018, Hadadin et al., 2002).
Given the obtained model accuracy, which according to the check point validation was at centimeter-level, the point cloud has been used also to automatically extract certain geometric information.First, geometric shape recognition (e.g.extraction of planar surfaces and recognition of architectural elements) has been considered: the obtained results shows the possibility of using this kind of data in order to support the production of BIMs/cityGML models (Scianna et al., 2018, Quattrini et al., 2017).
Furthermore, geometric analysis of the reconstructed walls has shown the potential use of the obtained model in order to monitor damages on the building.
Despite the aim of this paper was that of showing several potential uses of 3D models obtained by means of UAV photogrammetry, actually the obtained reconstruction allows to show also certain limits of this kind of methodology.For instance, the limitations on the flying area imposed by safety reasons (avoiding damages to the building and to the UAV) caused the impossibility of ensuring a complete coverage of the entire building by a sufficient number of images.In particular, obstructions (mostly due to the presence of trees and vegetation) caused the presence of certain gaps in the model (Fig. 5), typically close to the ground.This suggests the usefulness of properly integrating UAV photogrammetry with terrestrial surveys (e.g.close-range image acquisitions (Lo Brutto et al., 2018)).
Furthermore, despite the effect of vegetation has been partially reduced by properly filtering it out thanks to considerations on the local regularity of building surfaces, actually more accurate results can be obtained by using multi-spectral data acquisitions.Actually, the latter case, integrated also with LiDAR (Aicardi et al., 2016, Nex and Rinaudo, 2011, Remondino et al., 2005, Ravi et al., 2018), if possible, will be investigated in our future research work also in order to enhance the automatic recognition of objects and building architectural parts, which should be useful for instance in the production of BIMs.It is worth to notice that the considered 3D reconstruction has been obtained by exploiting highly accurate GNSS measurements: the extension of the proposed methodology to GNSS denied environments will also be considered in our future investigations (Masiero et al., 2017, Mostafa et al., 2018).

Fig. 2
Fig.2shows an orthophoto and a digital elevation model (DEM) of the Girifalco fortress obtained from the photogrammetric reconstruction.

Figure 3 .
Figure 3. Segmented planar surfaces in the central part of the fortress.

Fig. 5
Fig. 5 clearly shows the effect of obstructions in the 3D photogrammetric reconstruction: indeed, certain areas of the walls are missing in Fig. 5(a) (bottom-left) and (b) (bottom-central part) due to the obstructions of stairs and vegetation, respectively, on images taken by the UAV.

Figure 5 .
Figure 5. Side views of the central part of the fortress (a) inner wall view, (b) outer wall view.
Figure 7. (a) Areas (in white) detected as not part of the inner wall of the fortress.(b) A photo of the wall analyzed in (a).

Figure 9 .
Figure 9. Close front view of a Fortress wall (a) and corresponding depth analysis with respect to the planar surface locally fitting the wall (b).