AN EXPERIMENTAL EVALUATION OF 3D TERRAIN MAPPING WITH AN AUTONOMOUS HELICOPTER

We demonstrate a method for unmanned aerial vehicle based structure from motion mapping and show it to be a viable option for large scale, high resolution terrain modeling. Current methods of large scale terrain modeling can be cost and time prohibitive. We present a method for integrating low cost cameras and unmanned aerial vehicles for the purpose of 3D terrain mapping. Using structure from motion, aerial images taken of the landscape can be reconstructed into 3D models of the terrain. This process is well suited for use on unmanned aerial vehicles due to the light weight and low cost of equipment. We discuss issues of flight path planning and propose an algorithm to assist in the generation of these paths. The structure from motion mapping process is experimentally evaluated in three distinct environments: ground based testing on man-made environments, ground based testing on natural environments, and airborne testing on natural environments. Ground based testing on natural environments was shown to be extremely useful for camera calibration, and the resulting models were found to have a maximum error of 4.26 cm and standard deviation of 1.50 cm. During airborne testing, several areas of approximately 30,000 m were mapped. These areas were mapped with acceptable accuracy and a resolution of 1.24 cm.


INTRODUCTION
Three dimensional mapping is an extremely important aspect of geological surveying. The process to do this, however, often poses pragmatic challenges. Outdated methods such as walk-over surveying can create accurate, large-scale models. This type of surveying, however, requires a large amount of time and manpower. In addition, the resolution of these models depends upon the grid size of data points. A small increase in grid size requires a large increase in the number of data points, greatly lengthening the process.
Newer surveying methods can generate accurate, high resolution models, however, they have their limitations. Light detection and ranging (LiDAR) devices are able to make extremely accurate, high resolution models using short wavelength light pulses. These systems, until recently, were large and heavy, making them impossible to use on small unmanned aerial vehicles (UAVs) (Reineman et al., 2009). Although there are now compact LiDAR devices which are within the payload capabilities of small UAVs, these systems are still prohibitively expensive for many institutions. The cost of such systems and the inherently high risk of crashing makes the use of UAV-borne LiDAR devices impractical.
Photogrammetric techniques have been used for 3D mapping as early as 1849 (Birdseye, 1940). By 1904, the United States Geological Survey (USGS) was utilizing photogrammetry for largescale terrain mapping (Burtch, 2006). There are many different photogrammetric methods used for modeling, each having unique strengths and weaknesses. Stereo vision is one such technique used for modeling, however, it is not suitable for UAV-borne, large-scale, terrain mapping. Stereo vision uses the parallax between two images to generate a 3D model (much like human vision) (Matthies, 1992). This requires two separate cameras be-ing placed a known distance apart and being triggered simultaneously. The accuracy of stereo vision models is dependent upon the baseline-object distance ratio (Figure 1(a)). If the object distance is too great (with respect to the baseline distance), the images will not have enough parallax to generate an adequate model. In order to satisfy this requirement, the baseline distance must be increased or the object distance must be decreased. The baseline distance is limited, physically, by the size of the UAV. Therefore, in order to produce adequate models, the object distance must be decreased. This is problematic for several reasons. Low altitude flights limit the ground image area (GIA). Small GIAs (relative to the total mapping area) cause drastically longer flights and larger amounts of data. This increases the time required for data collection and image processing. Gathering data at low altitudes is also problematic for some UAVs due to the reduced flight speed necessary to prevent image blurring.

STRUCTURE FROM MOTION
Structure from motion (SFM) is a technique similar to stereo vision which uses parallax between two images to create 3D models (Koenderink and Van Doorn, 1991). Instead of using two separate cameras, however, SFM uses a single, moving camera. The movement between sequential images creates enough parallax to infer 3D data (Figure 1(b)). The accuracy of these models depends upon a number of variables, however, the primary factors are the camera and lens. These aspects make SFM ideal for UAVborne, large-scale terrain mapping.
Structure from motion has been explored deeply in recent times, leading to the development of a number of SFM software products. Although there are open-source options, we chose a commercial product, 3DM Analyst produced by Adamtech (Adamtech, 2010). 3DM Analyst was chosen due to its capabilities of producing high density point clouds and imbedding metric information International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVIII-1/C22, 2011ISPRS Zurich 2011Workshop, 14-16 September 2011  in the models (Adamtech, 2010). SFM can be split into 5 main processes: camera calibration, data acquisition, feature detection, bundle adjustment, and 3D generation ( Figure 2).

Camera Calibration
Camera calibration requires determining the interior properties of the camera and lens combination (Tsai, 1987). This greatly affects the quality of the model, therefore, it is crucial to perform a precise calibration. We will outline several factors which affect the calibration, issues encountered, and solutions to these problems. Focal length is one of the most important factors when performing calibration. Therefore, the lens should be firmly attached to the camera, focused, and then fixed so that it cannot move. This prevents the focal length from changing due to handling, transportation, and vibrations from the UAV. When collecting images for SFM, there can be a variety of lighting conditions. In order to collect proper images, we must often alter the aperture setting. It is, therefore, beneficial to perform multiple calibrations with different aperture settings. Camera calibration accounts for the distortion occurring in images. Much of this distortion occurs near the edges of the photograph. When taking images for calibration, there should be a consistent number of features near the edges of the photograph (Figure 3). This will ensure that the calibration correctly accounts for the near-edge distortion. We found man-made environments such as rock gyms (shown in Figure 3) to be an ideal location to perform the camera calibrations.

Data Acquisition
Collecting appropriate data is imperative to the success of the SFM process. Familiarity with the SFM equipment and adequately preparing the UAV platform and flight path will facilitate Figure 3: Image of Rock wall at Phoenix Rock Gym before and after feature detection (features marked with red) satisfactory data collection. Recognition of which field conditions necessitate which camera settings (aperture, shutter speed, ISO) will prevent improperly exposed photos. Integrating SFM equipment into the UAV platform should be done in a manner which allows easy access yet protects the equipment from vibrations, debris, and image pollution. This will decrease blurring in images and physically protect the camera and lens. UAV flight path planning will be discussed in more detail in Section 3, however, several general guidelines should be addressed. For the most accuracy and efficiency, images should be taken such that the camera-lens normal axis is perpendicular to the plane of the ground ( Figure 4). Although this does not have to be precise, severe deviations (more than ±15 • ) from this will produce images which are not useable. Sequential images should also attempt to carry a 60% overlap in the vertical direction and a 50% overlap in the horizontal direction. This was determined empirically to work with the 3DM software, although different overlap may be necessary for other software. An increase in overlap will still produce functional images, however, less overlap is prone to cause issues later in the SFM process.

UAV Platform
Our UAV platform, the SR30 ( Figure 5), was purchased from Rotomotion, LLC in 2009(Rotomotion, 2010. This low cost UAV was purchased ready to fly with the specifications provided in Table 1. The SR30 UAV was chosen due to its small size, large payload capacity, long flight endurance, and autonomous capabilities. Although gasoline engines do cause more vibrations than electric motors, the SR30 provides a much longer flight time than similar electric platforms. The standard platform has been outfitted with a vibration isolating camera mount to which the main SFM camera is attached. A smaller video camera capable of live streaming was also installed towards the front of the UAV. The main structure from motion camera is mounted facing straight down in order to minimize angle distortion. The front video camera is positioned at 45 • from vertical.

FLIGHT PATH PLANNING
Parallel line flight paths, also known as lawn mower flight paths, are widely used in aerial photography flights. This path formation allows for consistent image overlap and data collection in a uniform grid. However, these are not the only factors which contribute when planning a flight path. We sought to examine elements which are key to aerial photography and propose a method for generating flight paths.
Terrain mapping with UAVs requires a flight path which accommodates the desired ground sample distance (GSD), proper image overlap, and allows for appropriate imaging. In addition to the aerial photography aspect, UAV characteristics must also be taken into account. This means reducing flight distance and optimizing flight speed and altitude. If the turning radius of the UAV is limited, the flight path should also reflect this and minimize the number of difficult or impossible turns. These factors combine to make manual path planning a difficult task. Due to this complexity, we propose an algorithm which could assist with this planning.
The factors discussed above, while all significant, do not hold equal importance. When using UAVs as a tool for data collection, task optimization must be accommodated over flight optimization (to the extent that it does not cause risk to the UAV). For 3D mapping, the model resolution or GSD is usually the limiting factor. In order to accommodate the desired resolution, we must restrict our above ground level (AGL) flight altitude. Maximum flight altitude can be found using Equation 1. This, coupled with camera specifications, governs the ground imaging area (GIA) as related in Equation 2. Once the GIA has been determined, we can now begin planning the actual flight path.
where: h = AGL flying altitude rx, ry = horizontal, vertical camera resolution (pixels) gx, gy = desired horizontal, vertical ground sample distance (m) θx = horizontal field of view

Path Planning Algorithm
Restricting the flight altitude effectively transforms flight path planning from a 3D problem to a 2D problem * . This then leaves us with the question of how to image the entire mapping area, and which path to travel to do so. The path planning problem can be thought of in two sections. First, the mapping area must be split into a finite number of identical rectangles. This represents the area being photographed. Identical rectangles are used because, assuming a constant altitude and minimal UAV pitch and roll, the ground imaging area of all photographs will be identical rectangles. Second, we must travel to the center of each rectangle in order to take the picture. Doing this in the shortest path available is a version of the traveling salesman minimization problem (TSP). The traveling salesman problem is the task of visiting a finite number of locations via the shortest path (Kruskal, 1956) (Kirkpatrick et al., 1983).
3.1.1 Polygon Decomposition Mapping areas are most often polygons. Those which are not, can be expanded, with minimal image waste, to become a polygon. Image waste refers to the amount of ground imaging area which does not lie within the desired mapping area. This reduces the imaging problem to a polygon decomposition problem. A polygon decomposition problem is the task of reducing a large polygon into a finite number of smaller polygons (Keil, 2000) (O' Rourke and Supowit, 1983). If we assume no vertical or horizontal overlap between images, the rectangles are of the same dimensions as the GIA. Similarly, as we increase the desired vertical and horizontal overlap, we reduce the size of the rectangles proportionally. After determining the size of the rectangles (based upon desired overlap), we begin the polygon decomposition. This is done by arbitrarily starting at the top left corner of the polygon, subtracting a rectangle oriented vertically from the original polygon, subtracting a rectangle oriented horizontally from the original polygon, and recursively calling the algorithm on the two new polygons ( Figure 6). Throughout the decomposition, we keep track of the decomposition history (whether each rectangle was oriented vertically or horizontally), the coordinates of the center of each rectangle, and the accumulated image waste. When the polygon is completely decomposed into rectangles, we can then sort the solutions based upon minimal image waste. Having determined the locations to take pictures, we can then move to the next portion of the algorithm, path minimization.

Path Minimization
This portion of the path planning can be thought of as a traveling salesman problem, to which there are a variety of solutions. In the most basic case, we can use a purely Euclidean TSP solver. A Euclidean TSP solver minimizes the path based solely on the Euclidean metric (Papadimitriou, 1977). This will provide a flight path with the shortest distance. This is ideal for helicopter path planning for which there is no real disadvantage for turning. When path planning for UAVs with turning limitations, however, the algorithm should be modified to include a cost for turning. We utilize an open source TSP solver provided by (Paris, 2008).

Algorithm Results
We have tested this process with multiple polygons in an attempt to test the robustness and accuracy of the algorithm. An image rectangle with dimension 3x4 units was chosen arbitrarily and used for all tests. For all test images, the inner black, solid-line rectangle represents the desired mapping area. The red, dashed line indicates the generated flight path, and the blue dashes show where pictures should be taken (Figures 7, 8, and 9). Starting with the most basic mapping area, we generated a flight path for a large rectangular area. As is suspected, the flight path is very similar to a lawn mower pattern (Figure 7(a)). Moving to a square area, we find a generated path which is slightly unusual (Figure 8(a)). This path, however, has a length of 42 units which is exactly the same as the lawnmower pattern shown in Figure 8(b). Polygons of various shapes and sizes have been tested with several solutions shown in Figures 7(b), 9(a), and 9(b). We also compared the actual flight paths flown while performing field testing in Las Cruces, New Mexico, with proposed the flight path ( Figure 10). Direct comparison is challenging because the actual path had to be split into two flights due to camera issues. However, the algorithm does propose a shorter, alternate flight path. Although there is much room for improvement in the efficiency and robustness of this algorithm, this technique offers a starting point for flight path generation.

Ground Based Testing in Man-Made Environments
A Canon 5D (12.8 MP resolution and 35.8 x 23.9 mm sensor (Digital-Photography-Review, 2010)) with a 20mm lens was used to ground test the SFM process in a man-made environment. We chose to test the process at a local rock gym due to the high number of distinct features and the structure complexity. An area of approximately 100 m 2 was photographed, and distance measurements were taken via manual methods in order to perform accuracy analysis. After generating the model, corresponding distances were measured and compared to the actual distances (Table 2). We found the average error to be 0.85 cm with a standard deviation of 1.50 cm and the maximum error to be 4.26 cm. The results were then compared to the predicted accuracies given by models from 3DM Analyst. Equations 3, 4, and 5 were used, where the planimetric accuracy is the error in the X and Ydirections and the depth accuracy is the error in the Z-direction (Adamtech, 2010). Using the given formulae, we predicted the total accuracy of this model to be 0.30 cm. Although the actual error of the model is very low, it does differ significantly from the prediction. We attribute this to the accuracy formulae not accounting for errors in camera calibration. Ultimately, the model produced was of high quality and accuracy, and it shows structure from motion to be a viable option for modeling.
where: σ plan = planimetric accuracy (parallel to image plane) σ pixel = estimated pixel accuracy σ depth = depth accuracy (perpendicular to image plane) γ = pixel size d = target distance f = focal length b = base distance

Ground Based Testing in Natural Environments
A Canon Rebel T1i (15.1 MP resolution and 22.3 x 14.9 mm sensor (Digital-Photography-Review, 2010)) with a 18-55 mm lens was the first SFM equipment used for our ground based testing in natural environments. The lens was fixed at 19 mm and calibrated. Testing was performed in multiple locations with a variety of success. Data was collected in several environments at International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVIII-1/C22, 2011ISPRS Zurich 2011Workshop, 14-16 September 2011, Zurich, Switzerland   Many of these models suffered due to improper base-distance ratio (see Figure 1(b)) and shrubbery. An improper base-distance ratio often does not provide enough parallax between images, while trees and shrubbery cause false positive feature matching.
The same equipment was tested again in Granite Dells, near Prescott, Arizona. This area allowed for better camera placement and, therefore, a better base-distance ratio. There was also a much clearer line of sight to the modeling target, providing more visibility of natural features. The models produced were of a much higher quality than those of the Camelback test site, however, accuracy analysis is still pending (Figure 11).

Airborne Testing in Natural Environments
Airborne testing in a natural environment was first performed with the Canon T1i camera and 18-55 mm lens (manually fixed at 19 mm). This SFM equipment was tested in Las Cruces, New Mexico, and photographed an area approximately 400 x 100 m. Multiple models were generated (Figure 12), although some suffered from a variety of issues. 1. The variable focal length lens, although manually fixed at 19mm, had small changes due to vibrations which affected accuracy and prevented models from separate flights from being merged. 2. Due to unexpected UAV attitudes, improper flight paths, and camera backlog, overlap between some images was not adequate to produce models. 3. Blurring rendered some images unusable. This caused disrupts in the models and increased error.
These issues were remedied by switching to a fixed focal length lens, improving flight path planning, installing a better vibration isolating camera mount, and using a more advanced camera trigger. With these improvements made, we again tested the UAVborne SFM process. The Canon 5D camera and 20mm lens was integrated to the UAV platform, and the UAV was flown at the Northgate flight field in Phoenix, Arizona. At this facility, a 350 x 200 m area was photographed, with models produced of approximately 70% of the area ( Figure 13). The data collected and models produced were of the highest quality to date. The models accurately reflected the generally flat terrain of the field and the proper location of vehicles, houses, and other distinguishable items.

FUTURE WORK AND CONCLUSIONS
While we have shown that UAVs can be used as a platform to collect images for structure from motion mapping, there are several issues, pertaining to both UAV usage as well as the structure from motion process, which we plan on examining in the future. The path planning algorithm outlined above does offer a starting solution, however, there is much room for improvement. Incorporating turning costs into the TSP solver as well as utilzing a TSP with neighborhoods (TSPN) solver could potentially offer more natural, UAV-friendly flight patterns (Dumitrescu and Mitchell, 2003). Due to GPS error, it is unlikely that a UAV will ever visit the exact location desired. Rather, there is an acceptable region which the UAV targets. In this same manner, there is an acceptable region, or neighborhood, from which the UAV can gather appropriate data. We believe that utilization of a TSPN solver would improve our algorithm and the resulting flight paths. Combining feature tracking with the SFM modeling process could help identify moving objects in images and disregard them in the modeling process. This would reduce false positive feature matching and, therefore, increase model accuracy. Lastly, using multiple vehicles to perform structure from motion modeling could allow for this technique to model rapidly changing systems. Systems which are currently changing too drastically between sequential images, could then be modeled with time, creating 4D models.
We have presented a method for incorporating structure from motion based terrain modeling on low cost UAV platforms. Relative to alternative terrain mapping methods, structure from motion presents a technique which can be implemented cost effectively and can be used for large-scale terrain modeling. With current technological improvements, this process has shown to be a viable option for terrain modeling with much room for growth.