EVALUATING UNMANNED AERIAL PLATFORMS FOR CULTURAL HERITAGE LARGE SCALE MAPPING

: When it comes to large scale mapping of limited areas especially for cultural heritage sites, things become critical. Optical and non-optical sensors are developed to such sizes and weights that can be lifted by such platforms, like e.g. LiDAR units. At the same time there is an increase in emphasis on solutions that enable users to get access to 3D information faster and cheaper. Considering the multitude of platforms, cameras and the advancement of algorithms in conjunction with the increase of available computing power this challenge should and indeed is further investigated. In this paper a short review of the UAS technologies today is attempted. A discussion follows as to their applicability and advantages, depending on their specifications, which vary immensely. The on-board cameras available are also compared and evaluated for large scale mapping. Furthermore a thorough analysis, review and experimentation with different software implementations of Structure from Motion and Multiple View Stereo algorithms, able to process such dense and mostly unordered sequence of digital images is also conducted and presented. As test data set, we use a rich optical and thermal data set from both fixed wing and multi-rotor platforms over an archaeological excavation with adverse height variations and using different cameras. Dense 3D point clouds, digital terrain models and orthophotos have been produced and evaluated for their radiometric as well as metric qualities


INTRODUCTION
During the recent years the use of unmanned aerial systems (UAS) of any kind for large scale mapping and in the geospatial domain in general has experienced an incredible boom (Eltner & Schneider, 2015).Over a short period of time a plethora of platform types and, most importantly a multitude of cameras and other sensors have been made available and are being implemented with different and often uncontrollable results.Sometimes their combinations also challenge the implementation of conventional aerial photography for certain applications (Colomina & Molina 2014).Legislation is another critical issue when unmanned aerial platforms are involved, as it varies a lot from country to country, if rules apply at all.Gradually, however, most of the countries pass pertinent laws governing the flight and use of these UAS.A short review of the current situation is initially attempted, in order to enlighten potential users of the applicability of such systems.On the other hand, automated or semi-automated photogrammetric software implementing Structure from Motion (SfM) and Multiview Stereo (MVS) algorithms have become extremely popular making photogrammetry, a rather obscure science up to now a commonly used tool among the non-expert community.Several pieces of automated software implementing SfM/MVS algorithms coming with user-friendly graphical interfaces are already available, both commercial and freeware.In the literature a variety of several comparison reports for such algorithms can be found mainly using benchmark images (Seitz et al. 2006, Remondino et al. 2014).This "automation without deep knowledge" situation has many pitfalls, as uncontrollable processing may easily lead to nice looking, but completely and most severely slightly wrong products.Consequently, their implementation should be done with caution and always under the supervision of an expert in evaluating and assessing photogrammetric products.These two facts are the main motivation behind this project, whose first results were reported in Oikonomou et al. (2015) and which aims to investigate the applicability of UAS and the images that these aircraft are capable of acquiring in combination with automated SfM/MVS software.Relative recent literature includes studies about the usability of UASs in cultural heritage mapping or surface measurement (Sauerbeier et al. 2010;Remondino et al. 2011;Neitzel et al., 2011), using different UAS acquisition platforms (Lo Brutto et al. 2014), combination of terrestrial and aerial UAS data for 3D reconstruction evaluation (Koutsoudis et al. 2014) or assessments on UAS data dense matching (Haala & Rothermel, 2012).To realize an objective evaluation of the generated point clouds, DSMs or orthophotos, ground truth data is needed, such as LiDAR data or evenly distributed ground control points (GCPs) measured with GNSS.The current study focuses on a short review on UAS platforms and cameras, using some data scenarios of a rather limited cultural heritage site to assess the performance of state of the art algorithm SfM/MVS implementations.Several experiments are conducted in order to evaluate systems, sensors and software based on the final results i.e., DSMs and large scale orthophotos in this case.Although the range is not yet complete, it is believed that it presents an overview of the available systems, software and their usability.In section 2 of this paper a short review of the Unmanned Aerial Systems (UAS) is presented along with reference to payloads and legislation issues.Τhe data sets used for the practical experimentation are described in section 3.In section 4 the algorithm implementations are briefly described.Description of the processing stage and the presentation of results and their evaluation follows in section 5 and finally in section 6 some concluding remarks are attempted.

Body frames, characteristics and applicability
Modern UAS are commonly divided into two categories.The first includes fixed wing UAS with relatively better body aerodynamics, which result in long durations of flight up to more than 1hr, capable of carrying heavy sensors (Eisenbeiss, 2009).These systems can be used in CH mapping, also for larger areas, as for archaeological excavation sites -because they pose arduousness to manoeuvre, usually require bigger flying heights for safety reasons and elongated areas to lift off/land.They are generally more inexpensive to maintain.The second category refers on multi-rotor UAS, more commonly used in CH large scale mapping than fixed-wing due to their enhanced manoeuvrability (Lo Brutto et al. 2014).They are stable, having the ability to move vertically and hover.Multi-rotor UAS can thus be used to reach inaccessible or not easily approachable places, and buildings which lack of static adequacy (usually after a seismic event) as in Fregonese et al. (2015).In addition to that, they can be used to acquire images from a short baseline when it is required, resulting to big overlaps, for example for objects with very intense depth differences.They can also reach the object of interest from close distance with the purpose of exporting imaging derivatives that could not be produced by traditional aerial or terrestrial photogrammetry e.g.top view-orthophotos.On the other hand, rotary wing UAS involve greater mechanical and electronic complexity which translates into generally more complex maintenance, repair work and time consumption.This type of vehicles is able to carry smaller payload, affecting the number, size and weight of sensors which can be mounted on the system, especially for smaller or lower cost categories.
Finally, because of their lower speed and the shortest flight duration (10-25 min), the operator will require many extra flights to cover larger areas, leading to another increase in time and operational costs.For CH applications, finding the appropriate balance between overall cost, body frame type and weight, sensing payload precision and accuracy for given standards is challenging (Colomina, Molina 2014).In the following sections, a short review and comparison between state-of-the-art sensors for UAS commonly used for CH large scale mapping is presented.

Optical cameras
Visible band sensors are widely used for CH photogrammetric applications using UAS, mainly aiming for orthophotos, orthomosaics, 3D models, and surface and elevation models generation.Some of the important parameters to take into consideration while in the project planning phase are: the sensor type and resolution, the pixel size, the frame rate, the shutter speed, the focal length to be used as well as the weight of the camera/lens system.The most commonly used types of optical sensors used are CCD and CMOS DSLR cameras, while mirrorless cameras are becoming increasingly popular, mainly because of their small weight.Table 1 summarizes the latest and most representative optical cameras suitable for CH mapping with UAS.The use of high resolution and small pixel size optical sensors allows for the generation of high precision photogrammetric products, though built-in optical cameras in fixed winged UAS, are usually of smaller resolution and usually of unspecified precision, so that these systems represent a cheaper solution for low-precision applications.

Manufacturer and model
Sensor

Thermal cameras
In the recent years the miniaturization of infrared sensors has enabled their usage in unmanned platforms (Colomina, Molina 2014).This integration is ideal for real time surveillance, monitoring, risk management, calculation and improvement of the energy efficiency of buildings etc.Moreover, acquisition and further processing of thermal images may allow precise temperature measurements per pixel, or even producing "thermal" 3D models with them.Regarding CH applications, these imaging derivatives can be useful in revealing previously undocumented architectural features and non-destructive building diagnostics including moisture or deterioration inspections.Latest near infrared imaging technologies include uncooled micro bolometers as thermal sensors.These are lowcost, lightweight sensors, but their analysis is significantly lower than that of the visible spectrum sensors, mostly 320 x 256 pixels, 384 x 288, or 640 x 512 (up to the most expensive models) and also are less sensitive than cooled thermal and photon detector imagers with increased noise factor.

Flight Rules
The EU has not issued a directive regarding the UAS or any restrictions on their use, but the European Data Protection Supervisor has expressed in a text without legal force his opinion on the matter.In this text, the social and economic benefits from the use of UAS are highlighted, stressing the need to harmonize the aviation related UAS policies of the Member States' safety, referring to liability and public security and noted that the proliferation of UAS can be achieved by ensuring compliance in matters concerning personal data, privacy and security.This text does not set out any technical rules, yet citations are made to other regulations of the EU or of the Member States.In general, those of the other EU states that have issued instructions regarding the UAS are consistent with each other.Namely, in most EU countries the flight of a UAS is prohibited unless it is inside the operator's field of view or at a height of below 250-400 feet, at long distance from people or residential areas and outside military installations or airports.Concerning CH applications, in most cases specific permission is required to use UAS inside or indeed above archaeological sites.In the USA, the Federal Aviation Administration on a memorandum makes clear that the use of proprietary UAS for news gathering for the media or for other business purposes requires permission from the Federation, while the use of UAS for recreational purposes does not require permission.Hence, even in the USA federal authorities do not adopt technical rules restricting the use of such systems.These restrictions are enacted at state level and in some cases are strict or almost nonexistent.To operate UAS commercially in South Africa, the operators need to hold 5 certificates, including registration of all UAS used for commercial purposes.Operators must meet the legal requirements set out by the SACAA (Civil Aviation Authority) and are audited regularly for compliance.In Oceania and Asia similar licenses and permissions are required, while all of the above airworthiness certificates are issued for UAS by the respective aviation authority.Canada and India are about to issue new directives due to the large number of incidents including UAS.

DESCRIPTION OF THE DATA SET
The evaluation and comparison of the photogrammetric products, produced by different algorithm implementations were based on data sets available from previous projects.In particular, the data sets used for this study were acquired during the Erasmus Intensive Programme HERICT2013, an international Summer School organized by the Laboratory of Photogrammetry of NTUA for the documentation support of the archaeological excavation in Vassilika settlement in the archaeological site of Kymisala in Rhodes (Acevedo et al. 2013).The Vassilika settlement lies within the wider archaeological site and it actually is the ruins of an organized urban network covering an area of approximately 200x250m with some 10m of height differences.Careful flight planning was crucial for the correct positioning of the pre-marked GCP's.For both UAS employed flight planning was carried out using proprietary software which enabled introduction of the parameters based on Google Earth.After each flight and image acquisition, the data was downloaded and checked for their integrity.The data acquisition process also included the collection, archiving and storing of all necessary metadata.The UAV's employed (Figure 1) were the Swinglet fixed wing airplane by Sensefly and an octocopter developed by the Hafen City University.For the needs of the Erasmus IP also a Dunford Flying Machine kite and smaller UAV with a built-in camera for some experimental flights were also employed.However for this project only the data from the fixed wing and the octocopter will be used.The octocopter is a lightweight multi-rotor system with 8 rotors and has a maximum payload of approximately 1kg.The maximum duration of the flight is around 15 minutes and a compact digital camera Samsung NX1000 was integrated for the image capturing.This camera has a 23.5x15.7mm 2 sensor with 20MP resolution and a pixel pitch of 4,3μm The Swinglet Cam manufactured by Sensefly (https://www.sensefly.com/drones/swinglet-cam.html), is an electrically-powered fixed wing platform and is approximately 0.5kg heavy.It is so lightweight that the user can launch it by hand and let it land on almost any surface without using any supporting system, such as a parachute.Its small impact energy also reduces significantly the risks of third-party collision damage.Two cameras were available for this UAS.A Canon Ixus 220HS compact camera recording 12.1 MP (sensor size 6.14x4.55mm 2 and pixel pitch of 1.5μm) optical images and a Canon PowerShot ELPH300HS compact camera (sensor size 6.14x4.55mm 2 and pixel pitch of 1.5μm) capable of recording infrared images of 12.1MP.The mission planning with both sensors has been automated for the application using e-motion software.Ground control points were measured using GNSS and the Real Time Kinematic method (RTK) with an accuracy of 2-3 cm.
They were signalized with a 20cm 2 black and white checkerboard pattern and were distributed in order to cover the entire area in the most effective possible way.Particularly, 15 points were used as control points and another 15 as check points.For this project several processing strategies were considered.After careful examination it was decided to process the data from each UAS separately and in addition to process the data from the two flights, i.e. flying height of 60m and 90m of the Swinglet together.In Table 3 an overview of all the examined scenarios is presented regarding the acquisition equipment, the flying height and ground sampling distance.The images captured from the octocopter form a more stable block configuration with great overlaps among them, while the acquisition points using the fixed wing platform are more random.This resulted in higher overlap percentages providing, thus, a more stable solution.Moreover the camera used in the octocopter was of superior resolution and significantly greater pixel pitch and the results are expected to be better.In Figure 3 the camera stations, i.e. the image distribution is presented for each scenario.(Haala and Rothermel, 2012;Rothermel et al. 2012).It allows high parametrization and thus flexibility by the user and can deliver dense point clouds, triangulated meshes and true orthophotos given the camera orientations.In this study, as opposed to Oikonomou et al. (2015), VisualSfM, a freeware software, was excluded, as GCPs are not considered in the same way while performing bundle adjustment.The datasets were thoroughly processed using the above mentioned software and the resulting point clouds, orthophotos were compared for their qualities.(Oikonomou et al., 2015).Given the orientation parameters from those two, SURE is then used for dense image matching and orthophoto production as well.Hence four different processing methods were implemented as far as the point cloud production is concerned.Firstly the two pieces of software on their own and secondly introducing the orientation, i.e. alignment, results of PhotoScan and Pix4D into SURE to perform the dense image matching.The differences of this processing for Scenario 2 are presented in Figure 4.

DATA PROCESSING AND EVALUATION
The three scenarios described in section 3 were used to evaluate the performance of the aforementioned image-based reconstruction algorithms and thus the quality of the acquired data with the different platforms, based on the metric and visual quality of the results.No LiDAR data were available as reference dataset and for this reason, yet Ground Control Points were used as ground truth data.
(i) (ii) (iii) (iv) Figure 4: Detail of the dense cloud of Scenario 2 processed with Agisoft PhotoScan (i), Pix4Dmapper (ii), SURE (Agisoft) (iii) and SURE (Pix4Dmapper) (iv) respectively A first evaluation of the performance of the image-based reconstruction algorithms is conducted by assessing the alignment results.In scenarios 2 and 3, Pix4Dmapper produces 25% more points in the sparse cloud in average than PhotoScan (Table 4).Only in the first scenario the results are contradictory as PhotoScan triangulates almost 2.2M points, whereas Pix4Dmapper 1.2M points.In both cases, this stage is successfully performed, yet few images could not be aligned in scenarios 2 and 3.It is stressed that for all cases the same PC configuration was used, in order for the results to be comparable.

Scenario # Points Time (mins)
Table 4: Sparse point clouds generated using the same PC configuration In Table 5 the RMS errors of the ground control and check points for all scenarios are shown since they constitute reliable indicators for assessing the aerial triangulation and alignment accuracy.As shown, the results are of greater accuracy for Pix4Dmapper.In the third scenario, where NIR images were processed the results are quite similar.However, both solutions present acceptable accuracy for the target scale of the final product.
Regarding the dense 3D point cloud results, it is observed that PhotoScan generates a larger number of point correspondences between the images.Indeed, PhotoScan has produced almost 140M points in the first scenario, almost four times more points than the corresponding solution in Pix4Dmapper.Using the image orientations coming from PhotoScan and Pix4Dmapper as input to SURE, a very dense point cloud is generated for scenario 1, while for scenarios 2 and 3 the resulting point clouds are smaller than those coming from PhotoScan densification, but still larger than those of Pix4Dmapper (Table 6).In all cases, it is noted that the algorithms perform better when (a) the imagery is strongly overlapping; (b) the camera moves in a relatively stable trajectory and (c) the images are of good quality.4).Digital Surface Models arising from more dense point clouds offer great detail of the objects and precise edges, significantly improving the metric and visual quality of the products.The DSMs generated from Pix4Dmapper are quite smooth, especially in the areas where the dense cloud is sparser, without losing in metric accuracy (2-4cm), taking into consideration the measured absolute vertical differences between the DSM and the check points.SURE DSM models are of similar quality, yet the results differ depending on the orientation input and thus the dense cloud produced.As expected, the greatest errors appear for all algorithms in scenario 2, which has the wildest image spatial distribution.The combination of Pix4Dmapper and SURE gives inferior results, which are not considered as representative as they occur just in this scenario.Here the lack of information produces gaps and mislead interpolation.In Table 7 the absolute vertical differences between the DMS elevations and the check points' measured height are presented for all software solutions.These results are refined in the sense that, in areas where the software was unable to match satisfactory number of points, i.e. due to vegetation coverage, the check points in those areas were not taken into account.In all cases, however, more than 6 check points were used.

Scenario
In order to assess the metric quality of the derived orthophotos, a number of distances between ground control points and the corresponding ones in the orthophotos were measured.These measurements were performed manually and hence include the estimation errors of the operator.On the other hand, the measurements in the DSM are automatically calculated.These results are presented in Table 8.  5 and 6).The radiometric quality is better without blurry areas and the colours are sharper, especially in the areas where drastic height differences occur and break lines would be otherwise needed.However, the geometric quality is similar in all four software combinations.On the other hand, for scenarios 2 and 3 the results are inferior (Figure 7).They are noisier irrespective of the software combination, obviously because of the irregular camera positions, which inevitably result to lack of information in the DSM and to lower overlap percentages.It should also be remembered that the ground sampling distance for Scenarios 2 and 3 was drastically less favourable.

CONCLUDING REMARKS
The results of UAS imagery for mapping strongly depends on the chosen platform.More stable flight trajectories mean dense overlaps and thus more complete point clouds, with no lack of information, as it was obvious with the images from the octocopter in this project.Hence, multi-rotor platforms are more suited for large scale mapping than fixed wing platforms which are more effective in mapping larger areas in shorter times.Concerning the software used, it is very important for the user to deeply understand the algorithms implemented and the required parameters, as they have proved to be very critical for the quality of the results.Hence this parametrization is a valuable tools for expert users, but extremely complex for nonexperts who prefer fully automated fit-it-all solutions, leading thus to uncontrolled results.However, automation has enabled everyday users to produce qualitative and accurate results, depriving them at the same time of the necessary control on the procedures.For the project described, however, quite satisfactory results were derived with all algorithms implemented.The optimistic outlook is that with the constant evolvement of the computer vision techniques the future is really promising.All in all, UAV mapping is already widespread and constitutes a solution in a wide range of applications, especially in cultural heritage documentation.
The authors sincerely thank Mr. Konrad Wenzel for his support and advice regarding SURE.

Figure 1 :
Figure 1: The UAV's used for acquiring the digital image data

Figure 2 :
Figure 2: Area of interest and distribution of the Ground Control Points

Figure
Figure 3: Camera stations for Scenario 1 (above), 2 and 3 (below) 4. METHODOLOGY It was decided to use contemporary automated photogrammetric and computer vision algorithms for processing the data sets.This implies the implementation of Structure-from-Motion (SfM) and Dense Image Matching for orienting the images and producing 3D information about the object.Structure from Motion approaches (SfM) (Szeliski, 2010; R. Hartley & Zisserman, 2003) in computer vision are image based techniques using epipolar geometry concepts to calculate the

Figure 5 :Figure 6 :
Figure 5: Orthophotos produced by Agisoft (left) and Pix4D (right) for Scenario 1 Orthophotos produced by Agisoft (a), Pix4D (b), Agisoft+SURE (c), Pix4D+SURE (d) for Scenario 3 Computational time, as expected, is directly proportional to the size of the dataset and the image resolution.Thus, the first scenario consisting of more images and of higher resolution required more time than the other two for the complete process in both PhotoScan and Pix4Dmapper.It is ascertained that, apart from the second scenario, data processing is generally quicker in Pix4Dmapper.It was also remarked that in the stage of point cloud densification, PhotoScan requires almost double computation time than Pix4Dmapper, whereas in the stage of DSM and orthophoto generation the opposite relation was observed.Summary of the total processing time

Table 1 :
Common and/or representative optical cameras suitable for UAS

Table 2 :
Table 2 summarizes the latest commercially available thermal cameras suitable for CH UAS mapping.Upon choosing a thermal camera, the most important factors are pixel size, spectral range and thermal sensitivity.Common and/or representative thermal cameras suitable for UAS

Table 3 :
Overview of the datasets orientations were calculated using the two commercial software applications available, i.e.Agisoft PhotoScan and Pix4Dmapper as in Data processing includes camera pose estimation, point cloud densification and orthophoto generation.The camera

Table 5 :
Summary of RMS errors of the control points and the check points in PhotoScan and Pix4Dmapper (in m) Dense point clouds resulting from Pix4Dmapper present large gaps in vegetation areas, however the point density increases in areas of rock and soil.On the other hand, the point clouds resulting from PhotoScan have fewer gaps and describe better the vegetation areas, giving a more realistic representation of the area of interest.SURE dense point cloud is detailed enough and noiseless for scenario 1, yet some gaps and noise in the vegetation areas are observed in scenarios 2 and 3, however they are less noticeable when the Agisoft PhotoScan image alignment is used as input (Figure

Table 7 :
Mean absolute vertical differences between check points and the Digital Surface Models and their standard deviation * In this case the results are not representative as already explained

Table 8 :
Mean distance differences between check points and on the orthophotos with their standard deviationThe results have shown that scenario 1 has produced better quality 3D models and orthophotos, mainly due to the more complete and evenly distributed image acquisition positions (Figure3).Indeed, for this dataset SURE outperforms the other two software solutions in terms of true orthophoto refinement.Less occlusions and similar defects like double images are