Automated DSM extraction from UAV images and Performance Analysis

As technology evolves, unmanned aerial vehicles (UAVs) imagery is being used from simple applications such as image acquisition to complicated applications such as 3D spatial information extraction. Spatial information is usually provided in the form of a DSM or point cloud. It is important to generate very dense tie points automatically from stereo images. In this paper, we tried to apply stereo image-based matching technique developed for satellite/aerial images to UAV images, propose processing steps for automated DSM generation and to analyse the possibility of DSM generation. For DSM generation from UAV images, firstly, exterior orientation parameters (EOPs) for each dataset were adjusted. Secondly, optimum matching pairs were determined. Thirdly, stereo image matching was performed with each pair. Developed matching algorithm is based on grey-level correlation on pixels applied along epipolar lines. Finally, the extracted match results were united with one result and the final DSM was made. Generated DSM was compared with a reference DSM from Lidar. Overall accuracy was 1.5m in NMAD. However, several problems have to be solved in future, including obtaining precise EOPs, handling occlusion and image blurring problems. More effective interpolation technique needs to be developed in the future.


INTRODUCTION
As technology evolves, unmanned aerial vehicles (UAVs) imagery is being used from simple applications such as image acquisition to complicated applications such as 3D spatial information extraction.Spatial information is usually provided in the form of a DSM or point cloud.It is important to generate very dense tie points automatically from stereo images.Many image matching techniques have been proposed for this purpose (Haala and Rothermel 2012;Jeong and Kim, 2014 ).However, DSM generation from UAV images is still challenging.Unlike satellite and aerial images, UAV images possess only small coverage.UAV images with centimetrelevel ground sampling distance contain many texture-less features such as roofs and pavements, moving objects and repetitive patterns.
In this paper, we tried to apply stereo imagebased matching technique developed for satellite/aerial images to UAV images, propose processing steps for automated DSM generation and to analyse the possibility of DSM generation.

DATASET
To obtain the test dataset, we used the DJI's Spread Wings UAV which is hexa rotor type (Figure 1).

Figure 1. Spread Wings UAV
The image data set was acquired from SONY's DSLR ILCE-7R camera model with a focal length of 35mm and a detector size of 4.88967 x 4.88967 um.The flying height of test area was 180m, which result in a ground sampling distance (GSD) was 2.5cm.In this study, we acquired images including the texture-less areas, such as the bridge and bare ground.The acquired image size is 4800x3200 pixels.Figure 2 show a few images of dataset from UAV.
Figure 2. A few images from the UAV dataset

Aerial Triangulation
For DSM generation from UAV images, firstly, exterior orientation parameters (EOPs) for each dataset were adjusted.Image was acquired by the UAV has position and orientation information obtained from on-board sensors.For UAVs, sensor accuracy was lower than those mounted on the aerial or satellite images.The stability of the platform was also low.Therefore initial EOPs caused serious match errors and poor results.We updated EOPs using some control points automatically generated .We compared each matching result between before and after application of the AT process with same matching circumstances.
Table 1 show the difference between before and after application of the AT process.Figure 3 shows extracted point clouds under the same matching conditions using the EOPs before and after AT process.In these results, it is certain that more matching success points were generated when applying AT, and blunder areas were decreased relatively.

ROI Setting
For generating a DSM, we aimed to use all of the images which lay within the region of interest (ROI) by finding optimal stereo pair from all UAV images.Stereo coverage between two images and convergence angles were checked for this purpose.
If the stereo coverage between two images was too small or the convergence angles were too large or too small, the pairs were not considered.

Stereo matching
If stereo pairs are defined, stereo image matching was performed with each pair (Rhee et al., 2013).Developed matching algorithm is based on greylevel correlation on pixels applied along epipolar lines.Accurate matched points were extracted by applying grey-level correlation repetitively with various correlation windows and selecting best candidates by relaxation.And by using multiple correlation windows, more accurate image matching than the existing area based matching technique is possible.
In addition, by using the pyramid-based technique, the matching stability was enhanced.
Once stereo image matching was completed for all matching pairs, a DSM from each pair was extracted.The maximum patch size was set to 25x25 pixel for experiments.The original images were down-sampled to 1 / 8, 1 / 4, and 1 / 2 of the original size for pyramid matching.Finally, the extracted match results were united with one result and the final DSM was made.At this time, the line-information like the edge in the image was utilized between DSM and it set to the boundary line.And the interpolation work was applied.Matching results and accuracy analysis are reported in the next section.

DSM ACCURCY ANALYSIS
In this section, we discuss stereo matching performance and DSM accuracy for the sample dataset.For experiments, we used algorithms for tiepoint extraction, AT and stereo matching developed in-house as shown in Figure 5.The Difference Map between generated DSM and LIDAR data was made in order to analyse the cause for the error (Figure 8).The numerical value of the Difference Map was indicated with the absolute value of the difference.The table and difference map showed rather large errors compared to the ground sampling distance of the UAV images.The major factors for this error could be inaccuracy of EOPs used and the resolution difference between the generated DSM with the LIDAR DSM used as the reference.
Besides, large errors occurred on the boundary of the bridge, due occlusions caused by bridges.
In order to analyze the major causes of the error in depth, the coordinates of the stereo image corresponding to the coordinate of the DSM was reported (figure 9-11).Through the each results, we found that blunders were from roads with very homogenous texture and the small and thin structure such as the street light poles (figure 10).Holes on road were mainly due to moving objects (Figure 11).And image blurring caused some problems.In particular the degree of blurring was different from images to images.The many holes happened on the moving object and blurring region.To make a more accurate DSM, it necessary to experiment with more images.And effective interpolation and noise reduction technique needs to be developed in the future.

RESULT
In this paper, we investigated the possibility of precise DSM generation from UAV images.We tried to apply stereo image-based matching technique developed for satellite/aerial images to UAV images.We tested stereo matching performance over UAV image pairs and generation of one mosaic DSM over the entire ROI.The accuracy was verified compared with a DSM derived from LIDAR data.Overall accuracy was around 1.5m of NMAD.Proposed method showed high performance even over many homogeneous areas.
However, several problems have to be solved in future.Precise EOPs were one of key issues.It was very difficult to obtain precise EOPs to the level of precise geometric analysis.In some image pairs, the noise and mismatched area were occurred due to inaccurate EOPs.During the mosaic DSM production, some height difference was founded in the boundary area between two individual DSMs.Image blurring caused some problems.In particular the degree of blurring was different from images to images.And effective interpolation and relaxation technique needs to be developed in the future.

Figure 3 .
Figure 3. UAV Stereo DSM (Left: using initial EO parameters, Right: using adjusted EO parameters) Figure 4 shows a few examples of DSMs from successfully matched pairs.

Figure 4 .
Figure 4. Examples of DSMs from successfully matched stereo pair

Figure 5 .
Figure 5. Layout of SW tool used for experiments

Figure 6 .
Figure 6.A DSM generated over entire ROI

Figure 7 .Figure 8 .
Figure 7. 50cm LIDAR DSM of the same area as figure 6

Figure 12 .
Figure 12.Point cloud generated over entire ROI with RGB values

Table 1 .
EO parameters sample

Table 2 .
Accuracy assessment of UAV stereo images