UAV PHOTOGRAMMETRY FOR MAPPING AND 3D MODELING – CURRENT STATUS AND FUTURE PERSPECTIVES –

: UAV platforms are nowadays a valuable source of data for inspection, surveillance, mapping and 3D modeling issues. New applications in the short-and close-range domain are introduced, being the UAVs a low-cost alternatives to the classical manned aerial photogrammetry. Rotary or fixed wing UAVs, capable of performing the photogrammetric data acquisition with amateur or SLR digital cameras, can fly in manual, semi-automated and autonomous modes. With a typical photogrammetric pipeline, 3D results like DSM/DTM, contour lines, textured 3D models, vector data, etc. can be produced, in a reasonable automated way. The paper reports the latest developments of UAV image processing methods for photogrammetric applications, mapping and 3D modeling issues. Automation is nowadays necessary and feasible at the image orientation, DSM generation and orthophoto production stages, while accurate feature extraction is still an interactive procedure. New perspectives are also addressed.


INTRODUCTION
According to the UVS International definition, a UAV is a generic aircraft design to operate with no human pilot onboard.The term UAV is used commonly in the geomatics community, but also terms like Remotely Piloted Vehicle (RPV), Remotely Operated Aircraft (ROA), Remote Controlled (RC) Helicopter, Unmanned Vehicle Systems (UVS) and Model Helicopter are often used.Based on size, weight, endurance, range and flying altitude, UVS International defines three main categories of UAVs:  tactical UAVs which include micro, mini, close-, short-, medium-range, medium-range endurance, low altitude deep penetration, low altitude long endurance, medium altitude long endurance systems.The mass ranges from few kilograms up to 1,000 kg, the range from few kilometers up to 500 km, the flight altitude from few hundred meters to 5 km, and the endurance from some minutes to 2-3 days;  strategical UAVs, including high altitude long endurance, stratospheric and exo-stratospheric systems which fly higher than 20,000 m altitude and have an endurance of 2-4 days; and  special tasks UAVs like unmanned combat autonomous vehicles, lethal and decoys systems.The primary airframe types are fixed and rotary wings while the most common launch/take-off methods are, beside the autonomous mode, air-, hand-, car/track-, canister-, bungee cord launched.
In the past the development of UAV systems and platforms was primarily motivated by military goals and applications.Unmanned inspection, surveillance, reconnaissance and mapping of inimical areas were the primary military aims.For geomatics applications, the first experiences were carried out by Przybilla and Wester-Ebbinghaus (1979).In the last years, more and more applications of UAVs in the geomatics field became common.UAV photogrammetry indeed opens various, new applications in the close-range aerial domain and introduces also a low-cost alternatives to the classical manned aerial photogrammetry (Colomina et al., 2008;Eisenbeiss, 2009).This development can be explained by the spreading of low-cost platforms combined with amateur or SRL digital cameras and GNSS/INS systems, necessary to navigate the UAV with high precision to the predefined acquisition points.The small size and the reduced pay-load of some UAV platforms limit the transportation of high quality IMU devices like those coupled to airborne cameras or LiDAR sensors used for mapping.GNSS is mainly used in code-based positioning mode and thus it is not sufficient for accurate direct sensor orientation.The use of RTK techniques would improve the quality of positioning to a decimeter level, but the system would become too complex, expensive and heavy.Simple and hand-launched UAVs which operate autonomously using its GPS-driven autopilot and, generally, an IMU sensor, are the most inexpensive systems, although platform's stability in case of windy areas might be a problem.More stable systems, generally with a gasoline engine, thanks to a higher payload, allow more professional camera onboard or even surveying with LiDAR instruments (Nagai et al., 2004).Everaerts (2008) reports on UAV projects, regulations, classifications and application in the mapping domain.Niranjan et al. (2007) presented an overview of UAVs civilian applications.The typical domains were UAVs images and photogrammetrically derived 3D data or orthoimagess are generally employed include:  forestry and agriculture: producers can take reliable decisions to save money and time (e.g.precision farming), get quick and accurate record of damages or identify potential problems in the field (Newcombe, 2007).Assessments of woodlots, fires surveillance, species identification, volume computation as well as silviculture can also be accurately performed (Restas, 2006;Grenzdörffer et al., 2008);  archaeology and cultural heritage: 3D documentation and mapping of sites and structures are easily achieved with a low-altitude image-based survey (Cabuk et al., 2007;Lambers et al., 2007;Sauerbier et al., 2010);  environmental surveying: quick and cheap regular flights allow land and water monitoring at multiple epochs (Thamm and Judex, 2006).Post-disaster response and mapping, excavation volume computation and natural resources documentations are also feasible;  traffic monitoring: surveillance, travel time estimation, trajectories, lane occupancies and incidence response are the most required information (Haarbrink and Koers, 2006;Puri et al., 2007). 3D reconstruction: unmanned aerial vehicles are a valuable source of image data for general 3D reconstruction purposes of man-made structures (Wang and Li, 2007;Irschara et al., 2010).A typical image-based field surveying with UAV systems require a flight or mission planning, GCPs measurement (if not available and required for geo-referencing), image acquisition, camera calibration and image orientation, image processing for 3D information extraction (Fig. 1).

FLIGHT PLANNING
The mission (flight and data acquisition) is planned in the lab with dedicated software, starting from the area of interest (AOI), the required ground sample distance (GSD) or footprint, and knowing the intrinsic parameters of the mounted digital camera.Thus fixing the image scale and camera focal length, the flying height is derived.The camera perspective centers ('waypoints') are computed fixing the longitudinal and transversal overlap of strips, while the presence of GNSS/INS onboard is usually exploited to guide the image acquisition.The take-off and landing operations are strictly related to the employed vehicle and characteristics, but normally controlled from ground by a pilot (e.g. with a remote controller).During flight, the platform is normally observed with a control station which shows realtime flight data such as position, speed, attitude and distances, GNSS observations, battery or fuel status, rotor speed, etc.Most of the systems allow then image data acquisition following the computed 'waypoints', while low-cost systems acquire images according to a scheduled interval.

CAMERA CALIBRATION AND IMAGE TRIANGULATION
Camera calibration and image orientation are two fundamental prerequisites for any metric reconstruction from images.In photogrammetric applications, the separation of both tasks in two different steps is preferred (Remondino and Fraser, 2006), also for UAV blocks without cross-strips.Indeed, both steps require different block geometries, which can be better optimized if they are treated in separated stages.On the other hand, in many applications where lower accuracy is required, calibration and orientation can be computed at the same time by solving a self-calibrating bundle adjustment.The camera calibration is generally performed in the lab although in-flight calibration are also performed (Colomina et al., 2007).Camera calibration and image orientation tasks require the extraction of common features visible in as many images as possible.In aerial photogrammetry this task is accomplished today by exploiting automatic aerial triangulation (AAT) techniques.In close-range photogrammetry, the automation of this task is a more complex issue due to large (and often varying) image scale, convergent image geometry, irregular overlap, strong geometric and radiometric changes.In many cases, image blocks acquired by using UAV systems are more similar to close-range than aerial blocks.Consequently, standard AAT procedures do not work out properly.Procedure based on the manual identification of tie points by an expert operator or based on signalized coded markers are well assessed and used today in close-range applications.In recent years some procedures for the automated extraction of a consistent and redundant sets of tie points from markerless close-range (or UAV) images have been developed for photogrammetric applications (Foerstner and Steffen, 2007;Barazzetti et al., 2010a;Irschara et al., 2010;Pierrot-Deseilligny and Clery, 2011).Some commercial solutions have also appeared on the market (e.g.PhotoModeler Scanner, Eos Inc; PhotoScan, Agisoft).The collected GNSS/INS data help for the automated tie point extraction and could theoretically allow the direct georeferencing of the captured images.But a bundle adjustment is generally computed, starting from the approximated exterior orientation (EO) parameters, to further refine the sought accurate camera poses and attitudes (Eisenbeiss, 2008;Grenzdörffer et al., 2008).In other applications with low metric quality requirements, e.g. for fast data acquisition and mapping during emergency response, the accuracy of direct GNSS/INS observation can be enough (Zhou, 2009).But there are many surveying applications where the scene and location are not suitable (even partially) for direct geo-referencing techniques, like an object with prevalent vertical extension (e.g. a big building façade - Pueschel et al., 2008;Scaioni et al., 2009), or an environment where satellite visibility is limited or impossible at all (e.g. in downtowns, rainforest areas, etc.).In both cases the GNSS positioning can be hardly used even for autonomous flight modes as the signal is strongly degraded and thus the orientation phase can rely only on pure image-based approach (Eugster and Nebiker, 2008;Wang et al., 2008;Barazzetti et al., 2010b).

SURFACE RECONSTRUCTION AND FEATURE EXTRACTION
Once a set of images has been oriented, the following steps in the 3D reconstruction and modeling process are surface measurement and feature extraction.Starting from the known exterior orientation and camera calibration parameters, a scene can be digitally reconstructed by means of automated dense image matching techniques or interactive methods for manmade features and vector information extraction.Interactive approaches deliver sparse point clouds which need structuring and editing in order to create accurate 3D data (e.g.building models).Automated methods produce a dense point cloud describing the surface of the surveyed scene (DSM), which has to be interpolated, maybe simplified and finally textured for photo-realistic visualization.A powerful image matching algorithm should be able to extract dense 3D point clouds with a sufficient resolution to describe the object's surface and its discontinuities.Therefore the point density must be adaptively tuned to preserve edges and, possibly, avoid too many points in flat areas.At the same time, a correct matching result must be guaranteed also in regions with poor textures.The actual stateof-the-art encompasses the multi-image matching technique (Seitz et al., 2006;Remondino et al., 2008;Vu et al., 2009;Zhu et al., 2010)  Finally orthoimages can be produced and delivered for further processes, analyses and decisions.

Veio archaeological area, Italy
Almost 100 images were acquired over the archeological area of Veio, Italy (Fig. 2).A 4-rotors Microdrone system coupled with GNSS/INS and equipped with a Pentax Optio A40 (8 mm lens, 12 Mpx, pixel size 1.9 m) was used.The images of the block have an average scale of 1:4,400 with a mean flying height of 35 m (ca 1 cm GSD).The image orientation is achieved using the ATiPE method (Barazzetti et al., 2010a) which delivered more than 330,000 image correspondences, afterwards reduced to roughly 18,000 for a faster bundle adjustment computation.The analysis of the covariance matrix of the bundle solution gave a sigma-nought of 1.6 μm, (i.e. in the order of the pixel size) and a theoretical precision of the computed object coordinates of σ x σ y = ±0.6 cm; σ z = cm.The computed RMSE on 5 targets were 4 cm, 3 cm and 7 cm in the X, Y, and Z directions, respectively.These discrepancies are surely due to the manual measurement of the target coordinates in the images (the center of each target was not clearly visible) but enough accurate for archaeological needs.The successive surface measurement was performed with the Mic-Mac algorithm which delivered a point cloud of ca 40M points.Figure 3 shows the reconstructed surface model.

Copan archaeological area, Honduras
The two main courts (Great Plaza and East Court) of the Maya site in Copan, Honduras (Remondino et al., 2009) were surveyed using an UAV model helicopter (Eisenbeiss, 2009) equipped with a Nikon SRL digital camera (12 Mpx).The flying height was approximately 100 m above ground and the acquisitions produced ca 250 images (average GSD of 1 cm).The image triangulation was automatically achieved using ATiPE (Fig. 4) while the GCP measurement phase required the user interaction.Due to the dense vegetation coverage, the successive automated image matching procedure produced some blunders but the non-occluded man-made structures were correctly reconstructed (Fig. 5).

Pava excavation area, Italy
The heritage area in Pava (ca 60 x 50 m, Fig. 6a) is surveyed every year at the beginning and end of the excavation period to monitor the advances of works, compute the exaction volumes and produce multi-temporal orthoimages of the area.The first 2010 flight (35 m height) was performed with a Microdrone MD4-200 which delivered ca 40 images with an average GSD of 1 cm.The image were then processed in order to produce a DSM at (5 cm geometric resolution) and orthoimages for archaeological documentation and analyses (Fig. 6).

Agriculture analyses
The area of interest (ca 2 ha) consists of a Prosecco wine yard with two different cultivations characterized by a different distance between the rows of vines.The survey was done at an average height of 130 m using a Pentax Optio A40 for the images in the visible spectrum and a Sigma DP1 for the images in the NIR spectrum.The combination of the image channels and spectrum allowed the derivation of an NDVI index classification (Fig. 7) which helped in the study of the wine yard health and cultivation.

ADVANTAGES, LIMITATIONS AND FUTURE PERSPECTIVES
The article presented an overview of existing UAV systems with particular attention to those platforms used for geomatics applications.The great advantage of actual UAV systems is the ability to quickly deliver high temporal and spatial resolution image information and to allow a rapid response in a number of critical situations where immediate access to 3D geoinformation is crucial.Indeed UAV feature real-time capability of fast data acquisition, transmission and, possibly, processing.Rotary wing UAV platforms can even take-off and land vertically, thus no runway area is required.For small-case applications, UAVs can be a complement or replacement of terrestrial acquisition (images or range data).The derived highresolution images can be used, beside geometric modeling purposes, for texture mapping on existing 3D data or for mosaic, map and drawing generation.If compared to traditional airborne platforms, they decrease the operational costs and reduce the risk of access in harsh environments, still keeping high accuracy potential.But the small or medium format cameras which are generally employed, in particular on lowcost and small payload systems, enforce the acquisition of a higher number of images in order to achieve the same image coverage at a comparable resolution.The stability and endurance of low-cost and light platforms is also an open issue, in particular in windy areas, although camera and platform stabilizers can reduce the weather dependency.High altitude surveying can affect gasoline and turbine engines while the payload limitation enforce the use of low weight IMU thus denying direct geo-referencing solutions.A drawback might also be the necessary presence of at least two persons for system maneuvers and transportation.The acquisition of image blocks with a suitable geometry for the photogrammetric process is still a critical task, especially in case of large scale projects and nonflat objects (e.g.buildings, towers, rock faces, etc.).While the flight planning is quite simple when using nadiral images, the same task becomes much more complex in case of 3D objects requiring convergent images and, maybe, vertical strips.Future work has to be addressed to develop tools for simplifying this task.
Despite the fact that automated image processing is already feasible with quite reliable and precise results, in the near future there are still possible improvements.High-end navigation sensors, like DGPS and expensive INS would allow direct georeferencing of the acquired images directly on site while advanced DSM generation algorithms could deliver surface models in short time thanks to GPU programming.In case of low-end navigation systems, real-time image orientation could be achieved with advanced SLAM methods (Nuechter et al., 2007;Konolige and Agrawal, 2008;Strasdat et al., 2010) although working incrementally on a frame by frame basis can lead to error accumulation and drift errors (Steffen and Foerstner, 2008).Sequential estimation algorithms based on Givens transformations (Gruen and Kersten, 1992) could be also exploited for (real-time) image analysis and object space feature extraction.In any case, lab post-processing and some user interaction will be most probably always mandatory for applications requiring high accuracy.
UAV regulations are under development in several countries all around the world, in order to propose the technical specifications and the areas where these devices can be used, e.g. over urban areas, increasing the range of their applications.

Figure 2 :
Figure 2: Automated image triangulation results for the UAV block over the area of Veio, Italy (ca 35 x 20 m).

Figure 3 :
Figure 3: Micmac matching results with the UAV images over Veio, visualized in shaded mode.

Figure 5 :
Figure 5: Produced DSM over the entire East Court in Copan (left) and a closer view of the reconstructed Ball Court (shaded and color mode) inside the Main Plaza (right).