UAV PHOTOGRAMMETRY WITH OBLIQUE IMAGES : FIRST ANALYSIS ON DATA ACQUISITION AND PROCESSING

In recent years, many studies revealed the advantages of using airborne oblique images for obtaining improved 3D city models (e.g. including façades and building footprints). Expensive airborne cameras, installed on traditional aerial platforms, usually acquired the data. The purpose of this paper is to evaluate the possibility of acquire and use oblique images for the 3D reconstruction of a historical building, obtained by UAV (Unmanned Aerial Vehicle) and traditional COTS (Commercial Off-the-Shelf) digital cameras (more compact and lighter than generally used devices), for the realization of high-level-of-detail architectural survey. The critical issues of the acquisitions from a common UAV (flight planning strategies, ground control points, check points distribution and measurement, etc.) are described. Another important considered aspect was the evaluation of the possibility to use such systems as low cost methods for obtaining complete information from an aerial point of view in case of emergency problems or, as in the present paper, in the cultural heritage application field. The data processing was realized using SfM-based approach for point cloud generation: different dense image-matching algorithms implemented in some commercial and open source software were tested. The achieved results are analysed and the discrepancies from some reference LiDAR data are computed for a final evaluation. The system was tested on the S. Maria Chapel, a part of the Novalesa Abbey (Italy). * Corresponding author


INTRODUCTION
The arisen interest of the scientific community and software developers in using airborne oblique images to build 3D models made the advantages of the technique evident (Höhle, 2008;Gerke, 2009). The method was experimented starting from the years 2008-2009, however, the more recent integration of computer vision algorithms (Szelinski, 2010) in common software tools, critically increase the method potentialities. In 3D city modelling, dense image matching of oblique images permits to include the façade description and the building footprints in the models. This enriches the representation from simple DSMs (Digital Surface Models) from one single point of view (generally nadir) to more complex 3D models, which can be useful for improved analysis with different aims (energetic policies, urban planning, city management, etc.). The technique permits to collect a very complex and complete information in short time, which is a great advantage for mapping and documentation purposes. Furthermore, the method is absolutely non-invasive, since it can collect data completely from remote areas and without touching the site at all. It also permits to reach little accessible parts of the buildings (such as the higher parts, which are difficult to be recorded by means of other techniques, e.g. Terrestrial Laser Scanner, TLS). However, performed benchmarks (Nex et al., 2015) and many researches (Frueh, 2004) used images acquired from high flight altitudes that permit to reach accuracies suitable for usual cartographic mapping and urban-scale analysis. Others use the data acquired from oblique images for building 3D models using a parametric approach (Feifei et al., 2012;Püschel et al., 2008). These are though not sufficient for specific architectural heritage applications, which require higher levels of detail. This research aims at testing a methodology for acquiring images useful for 3D reconstruction of a single historical building from UAV (Unmanned Aerial Vehicle). Surveys from UAVs can follow many methodologies, proposed in last years. They are well-affirmed for traditional nadir images acquisitions in a number of application fields and sites. These permit to obtain highly detailed DSMs of the surveyed objects. However, the façades, and vertical elements and surfaces were usually not modelled. Some previous researches experimented the 3D model reconstruction of a unique historical building for documentation aims from UAV (Fiorillo et al., 2013;Irshara et al., 2010). In the present contribution, however, a higher number of images are used and a closer acquisition is performed, in order to reach the maximum possible detail and to evaluate the behaviour of different software tools in processing close range oblique images acquired from UAV. The flight was realised at close distance (mean flight height of about 20 m) for acquiring images about a chapel, which is part of the Novalesa abbey complex. The safety conditions (isolated building rounded from fields) permitted to perform such a flight and experiment the complete methodology ( Figure 1). A method was tested for acquiring a high number of data from a UAV in a unique very close circular flight, plus some nadir images to further tie and strengthen the block. The images GSD (Ground Sample Distance) is about 0.5 cm; being the object quite compact, there are not meaningful differences between the nearest part of the object and the most distant part, from the camera. Figure 1 The complex of Novalesa Abbey (in the foreground, the S. Maria chapel) Superabundant data in a short time were acquired since the aim was to optimize a workflow also usable, for instance, for emergency situation, or simply for economic considerations (often critical in cultural heritage management). Also, for considering limited accessibility which can characterise some cultural heritage sites, and the frequent limited budgets of cultural heritage operators, the employed instruments were as compact as possible and low cost devices were preferred. The used camera is the same usually employed for nadir acquisitions, but it was positioned with oblique axis; expensive specific multiple cameras used in many researches (Feifei et al., 2012) were therefore not necessary. In the processing also low cost or open source tools were tested. Only the data from UAV images were considered, without any integration to the model with other sort of data (e.g. TLS points or terrestrial photogrammetry results). Dense image matching techniques were then used to build a dense 3D point cloud, to be used as a support for historical investigations, stratigraphic units identification (Donadio & Spanò, 2015), deterioration mapping, static analysis (Bertolini et al., 2015) and any other activity which could be supported by a dense and high-level-of-detail 3D model (Chiabrando et al., 2016;Castellazzi et al., 2015). As it is mentioned before, the acquired images were processed using some of the different software tools implementing photogrammetry and computer vision algorithms, in order to evaluate their behaviour in the processing and results. The resulting dense point clouds have been finally compared using as reference the TLS point cloud.

The case study
The object site of this study is the S. Maria's Chapel (Figure 2), which is part of the Novalesa Abbey, a Benedectine monastery in Val Susa (Piedmont -Italy) along the major pilgrimage road, the 'Via Francigena'. The abbey, which was founded in the year 726 AD by Cistercian monks, is still operative as an active Benedectine monastery and it is one of the most important historical and artistic witnesses of the Western Alps. Nearby the monastery there are four chapels including S. Maria. The S. Maria chapel is placed on a hill about 300 m from the monastery. Built in the VIII century, with restoration of the XI century, it was symbolizing the entrance to the monastery and was the place where visitors would stop for a first moment of prayer. The church is composed by a unique rectangular nave with a quadrangular apse on the east. The nave of the chapel is characterized by a double pitch roof, while the apse is constituted by a barrel vault surmounted by a roof. it is visible how a part of the object is hidden by trees, which can be a problem for the use of the oblique images to build a part of the complete 3D model

THE ACQUISITION METHODOLOGY
The photogrammetric acquisition from UAV are nowadays well established, since during the years different tests have been performed trying to understand the best practices and accuracy (Eisenbeiß, 2009). In the case of oblique images, the choose of the best flight planning approach in order to cover the whole object with the minimum time of flight still remain the main problem. In relation to the specific case study, different approaches can be evaluated since the use of oblique images can allow the observation of vertical structures, but the occlusion related to the contest must be taken into account. For this reason, different configurations of the used camera were evaluated by the scientific community during the years (Petrie, 2009): -the Maltese Cross configuration: that uses one nadir camera and 4 oblique (45°) sensors pointing the cardinal directions; -Fan configuration: it increases the swath width along the track to cover more area.
Right now, the most common acquisition strategy for the oblique data is related to the Pictometry System that follows the Maltese Cross configuration using a cluster of five cameras pointing the cardinal direction and a vertical one.
Since the system is used on aircrafts, image data are always combined with GPS/INS data to have position and attitude of each image.
Obviously, this type of system cannot be used on mini UAVs, so another strategy is required. For the survey of the Santa Maria chapel, a common Hexacopter (Mikrokopter) mini UAV was used ( Figure 3). It has a payload of 2 Kg and a flight time of about 10-12 minutes. The system is composed of 6 motors, 6 adaptor cards (in order to control the speed and the rotation of each motor), 1 flight control adaptor card, 1 remote control, 1 navi control, 1 GPS receiver, 1 MK3 MAG sensor equipped with a three-axial magnetometer (in order to control the vehicle's attitude), 1 wireless connection kit and 1 computer serving as ground control station.  Other two linear flights were also performed on the chapel with the camera oriented at 45°: one flight cover a side of the chapel and the other flight acquired the other part of the abbey. This is another possibility to perform oblique acquisitions. In the case of this specific study, the two acquisition strategies were completely equivalent since the object is very small and there was no occlusions (except for the tree to one side). For the oblique acquisition an FPV (First Person View) system was employed in order to control the orientation of the camera during the flight. All the used software give the possibility to consider approximate or precise orientation parameters (internal and external).
The used UAV has navigation sensors whose accuracy is not comparable to the system commonly adopted in aircrafts for oblique images. For this reason, the parameters were considered unknown and external markers were used as Ground Control Points (GCPs) to reference the model. Since the aim was to evaluate also the accuracy in terms of geometry and georeferencing model, two type of markers were used ( Figure 6): -LiDAR markers that were placed on the chapel's façade and used to georeference the reference LiDAR data; -photogrammetric markers that were put on the ground around the chapel. Both the markers were measured with topographic instruments. As usually, first, a reference network was realized with GNSS techniques to have known high precision vertices. Afterwards a RTK approach was used for the markers on the terrain and the total station for the markers on the chapel façade. The positions were determined with a centimetre accuracy. Using these points as reference to orient the point clouds, they were realized in a common reference system, which also was useful to easily compare the results.

Ground truth
The reference data for our comparison is obtained from a terrestrial laser scanner (TLS) survey, to assess the geometric quality of the entire 3D point clouds of the chapel, processed from oblique images. The used instrument was the FARO Focus 3D. It is a well-known phase-shift laser scanner with a distance accuracy up to ± 0.002 m and a range from 0.6m up to 130 m. It has the advantage to be low-cost, compared to similar instruments, and to have small dimensions, which is useful for portability reasons. 11 scans have been performed for the whole chapel, including 8 scans for the exterior (Figure 7) and 3 scans for the interior part.

Photogrammetric data-processing
In the present work, some first tests are presented, performed with the purpose to analyse the possibility of acquire and use oblique images for 3D modelling and to compare different software packages that follow different workflows, in order to evaluate their effectiveness and weaknesses. The alignment of 190 UAV images sequences, both nadir and oblique, was realized. The models, as mentioned before, was then georeferenced in the same reference system using the measured GCPs. This also allows to easily compare the different results.
The acquired data were processed using the Structure from Motion ( (Furukawa, 2010). All these software tools differ in the algorithms used for the image processing and matching, and in the different possibilities on the various parameters setting. However, they all lead to the images alignment, generation of dense point clouds and, subsequently, to the production of a triangulated mesh and to DEMs and orthophoto extraction. Generally, the input data required by these tools to perform the 3D model reconstruction process are only the acquired images and some GCPs, since it is not even necessary to know a-priori the exterior orientation parameters of the cameras. The only exception is SURE, which needs a set of already oriented input images to extract the 3D point cloud, representing the scene surface (Rothermel et al., 2012). In this case the alignment performed by Photoscan was used as input.
As it is mentioned before the tests were performed using two of the most popular commercial tools: Agisoft Photoscan Professional, which is an advanced image-based solution produced by the Russian-based company Agisoft LLC, and Pix4D, from the well known Swiss software house, especially dedicated to process UAV images (www.pix4d.com). Analyses were also conducted with 3Df Zephyr, produced by the Italian company 3Dflow (www.3dflow.net) and on the last released program Context Capture by Bentley (www.acute3d.com/ contextcapture/), which is the evolution of the previous Acute3D, now acquired by the Bentley software house. The first part of image processing is often performed by these tools through the use of modified versions of the Scale-Invariant Feature Transform (SIFT) algorithm (Lowe, 2004). Subsequently, using the GCPs a bundle block adjustment is performed (Chiabrando et al., 2015). The 3D dense representation of the object surface geometry is usually generated using multi-view stereo (MVS) algorithms, which allow to extract a detailed three-dimensional point cloud and to transform it in a triangular mesh. The algorithms are different and sometimes the commercial software could be defined as "black box" since the real made of operation of the program algorithms are not described in deeper (Photoscan, Context Capture and 3d Zephyr could be considered in this category).
On the other hand, in other tools the information are known, this is the case of the commercial version of SURE (now distributed by nFrames) and Pix4D (option not employed in this case study), which uses a semi global matching (SGM)-based stereo method, for an accurate pixel-wise matching with higher stability (Wenzel et al., 2013 According to the acquired data, the obtained point clouds offer a complete 3D model of the exterior of the chapel, including both the façades and the roof (which was reached with only few points by the TLS). In the following Figure 9 two views of the achieved point clouds are reported. The north-east part of the chapel is obviously disturbed by the presence of the trees. A survey inclusive of that part could be obtained performing the data acquisition in a season in which the trees have no leafs. Otherwise, an integration with the TLS acquisitions should be realized to complete the model. Different density of points are characteristics of each software, but they are, however, always sufficient to describe the small details in the geometry. In the following Table 1 a summary (in terms of millions of points) of the generated clouds is reported. Table1. Millions of points of the generated clouds.

PS
As it is possible to notice from the table, the denser cloud was obtained by the Context Capture software, more than 19 million of points. A very interesting and promising result for the next operation that could be performed using a point cloud.
On the other hand, the other softwares allow to extract less dense clouds (around 4 million of points) except for 3DZephyr and VisualSfM-CMVS, where the obtained point clouds were both achieved with less than 1 million of points in the employed configuration. Since probably some bad configuration have been set up in 3DZephyr a deeper analysis of this software is on-going in order to improve the results. A different output, close to the double in terms of density, has been achieved by SURE, where the obtained point cloud was very noisy. In this case an automatic procedure for removing the outliers using a statistical approach, implemented in CloudCompare, was employed. However, the cloud remained noisy, as it is visible in Figure 12 (b) and from the values in Table 2.
(a) (b) Figure 9. Generated point clouds: software MicMac (a) and ContextCapture (b), which shows the lacking part of façade (covered by the trees).

RESULTS AND COMPARISONS
Some first analysis has been performed using the software CloudCompare, on a part of the generated point clouds using as ground truth the TLS point cloud.
The southern façade was considered since it does not present any element, which could disturb the geometry reconstruction (as it happens with the trees on the north-east part of the chapel). This guarantees that the statistical results of the comparisons are truthful with respect to the reliability of the generated geometry from the oblique images. A quantitative result is given by the computation of the minimal distance between every point of the models using the Nearest Neighbour algorithm. The software allows also to calculate statistical values, such as minimal distance, maximal distance, average distance, standard deviation.
These values are reported in Table 2. Figures 10-16 report the maps of the discrepancies on the façades. The scale bars on the right part of the images also show a graph of the error distribution.  Table 2. Statistical values of the comparisons between the TLS data and the generated point clouds from oblique images (in m).  (a) (b) Figure 12. Discrepancies map between the TLS cloud and SURE results(a); in (b) it is possible to notice how the obtained cloud is noisy, although it was already filtered in order to eliminate the more noisy points.

DISCUSSION
Comparing images, graphs and tables referring to each point cloud deriving from different software, it is possible to observe that each 3D obtained cloud is very rich and could be realized in short time according to the actual potentiality of the employed PC and software. Observing in deeper the results, using each software, some problems have been noticed and sometimes solved during the data processing. Some of that could be ascribed to the skilled of the operator, others to the difficult of the algorithms and finally some to the particular geometry of the acquired images.
In order to underline some pros and cons according to the achieved outputs it is possible to state that: the point clouds generated by SURE has the main error as final Standard deviation (close to 4 cm), probably due to the important noise that is possible to deduce from the previous Figure 12.
In the Context Capture cloud, probably a systematic error is present, as it can be seen in the discrepancies distribution in Figure 10. Using this software some problems were founded in the georeferenciation process, since we notice that if the cartographic coordinates are employed all the steps are working very well except the point cloud generation that delivers a cloud like voxels ( Figure 17). Once these problems were fixed, the generated output was quite incredible: 19 millions of point without noise. According to the achieved results Photoscan and Pix4D confirm again if need be necessary the optimal performance of the algorithms that the software employed, as is possible to notice for both the tools the evaluated standard deviation are the lowest (it does not reach 2 cm for both). Using Photoscan the markers on the facade of Santa Maria were not employed since the software didn't deliver good results in the Bundle Block Adjustment (BBA), moreover the dense point cloud computation were very time spending. As general remark, it is possible to affirm that these software tools (PS and Pix4D) combine the user-friendly approach with good results. The open source tools were extremely effective, on the one hand, it is known that VisualSfM is not the perfect software for generating very dense point clouds but the achieved results in terms of accuracy are totally comparable with the ones obtained by Photoscan and Pix4D. MicMac, on the other hand, with the new automatic tools (C3DC using the option Bigmac) improves a lot the production with an excellent accuracy in both the analyzed phases: the BBA results and the final generated point cloud. The improvement connected to the generation of a 3D Mask are excellent and, as it is possible to see in the previous table, the point clouds accuracy is comparable with the other software. As was mentioned before, the problem connected to 3Df Zephyr was the generation of the point cloud, that need to be drastically improved. The last performed tests, under analysis, confirm that the new updates of the software, which is now divided in two main products: Pro and Aerial, are able to deliver denser clouds.

CONCLUSIONS AND FUTURE WORKS
The use of oblique images acquired from UAV is an effective tool to survey Cultural Heritage objects, which are characterized by limited accessibility, need for detail and rapidity of the acquisition phase, and often reduced budgets. For similar reasons, the method is also suitable for emergency cases. The flight plan should be cared, since multiple values than the images overlay and GSD are to be taken in account. The angle of the camera axis must be determined considering the height of the object, the height of the flight, the kind of eventual obstructions, the kind and position of details to be acquired on the objects. Moreover, for high objects some attention must be paid to survey the nearest and most remote parts with similar GSD (solutions could be to change the focal length of the camera in similar flights, to perform different flights, or integrating the model or parts of it by means of points measured through other techniques). The used software tools are able to realize suitable product in a quite similar way although some slight differences in the results, including the free and open source software. The only one that gives back results that are not immediately usable, but need some further processing to eliminate the noise, is SURE. Maybe in future investigations of some critical parameters will be adjusted in order to avoid the problem.
The tests will be continue in order to, first of all, fix the problem noticed in this first part of the works, moreover future works will be addressed to the realization of a new study in the correct strategy for oblique data acquisition using UAV platforms. Regarding the production, it will be moved to the next step of the workflow usually implemented in the most common software used for point cloud generation: the 3D modelling phase (Figure 18). This aspect will be analyzed starting again from the TLS survey. In this case the complete 3D model generated by the TLS will be employed as the ground truth for the next comparisons. Finally, it is important to underline that thanks to these last developments it is possible to state that actually we are arriving to realize a photogrammetric workflow for everyone. We do not know if this was the objective of the pioneer researcher in the photogrammetric field but we are sure that our past professors had this aspiration when in the early '70 introduced the photogrammetric course in the area of the Architecture and the Engineering science.