ACCURACY OF MEASUREMENTS IN OBLIQUE AERIAL IMAGES FOR URBAN ENVIRONMENT

Oblique aerial images have been a source of data for urban areas for several years. However, the accuracy of measurements in oblique images during this time has been limited to a single meter due to the use of direct -georeferencing technology and the underlying digital elevation model. Therefore, oblique images have been used mostly for visualization purposes. This situation changed in recent years as new methods, which allowed for a higher accuracy of exterior orientation, were developed. Current developments include the process of determining exterior orientation and the previous but still crucial process of tie point extraction. Progress in this area was shown in the ISPRS/EUROSDR Benchmark on Multi-Platform Photogrammetry and is also noticeable in the growing interest in the use of this kind of imagery. The higher level of accuracy in the orientation of oblique aerial images that has become possible in the last few years should result in a higher level of accuracy in the measurements of these types of images. The main goal of this research was to set and empirically verify the accuracy of measurements in oblique aerial images. The research focused on photogrammetric measurements composed of many images, which use a high overlap within an oblique dataset and different view angles. During the experiments, two series of images of urban areas were used. Both were captured using five DigiCam cameras in a Maltese cross configuration. The tilt angles of the oblique cameras were 45 degrees, and the position of the cameras during flight used a high grade GPS/INS navigation system. The orientation of the images was set using the Pix4D Mapper Pro software with both measurements of the in-flight camera position and the ground control points (measured with GPS RTK technology). To control the accuracy, check points were used (which were also measured with GPS RTK technology). As reference data for the whole study, an area of the city-based map was used. The archived results were referred to image orientation accuracy and to the ground sampling distance of the used images. The results show that the recent development of image orientation methods for oblique aerial images allow these images to be used for high quality photogrammetric measurements.


INTRODUCTION
Since the first major projects in this field were completed (Karbo and Simmons, 2007), oblique aerial images have been perceived mostly as a source of data for urban areas.On the one hand, it has been emphasised that they give a more 'natural' image of a city, one that is easier to understand for a layperson.On the other hand, thanks to the use of multiple viewing angles, they give better insight into the urban environment than classical nadir images, which represent an equally important aspect of the use of oblique photos.For this reason, oblique imagery in recent years has focused mainly on cities and buildings directly.Nyaruhuma et al. (2010) proposed an automated method of verifying the vector dataset of building footprints.They implemented a facades raster matching algorithm that allows for the detection of the demolition site of a building.Later experiments developed the approach to the verification of full 3D building models (Nyaruhuma et al, 2012).The research presented by Vetrivel et al. (2016) is also based on detecting changes; multi-temporal oblique imagery datasets were used to detect damage caused by earthquakes in a city.
A slightly different approach is presented by Nex et al. (2013), whose research focused not on verification but on the extraction of building footprints.They utilized point clouds generated from oblique images using dense image matching in order to create a 2d vector building outline database.A separate popular branch of the use of oblique images for urban areas is the texturing of 3D building models (for example, Kang et al., 2016;Haala and Kada 2010), which are created on the basis of different data (mainly airborne laser scanning).The use of aerial oblique images is also becoming more and more popular in generating the geometrical part of 3D city models (Liu and Guo, 2014;Haala et al., 2015;Frommholz et al., 2015;Tayeb et al., 2016).
A broader use of data obtained by oblique aerial imaging is presented by Rau et al (2015).In their classification of point clouds from dense image matching of oblique images, the following classes are distinguished: Tree, Grass, Facade, Roof, Road, and Other.Thus, they created three dimensional maps of land development.An extended range of use is also presented by Moe et al. (2016).Apart from using oblique images to create building models, these authors suggested using them in different mapping applications in order to decrease the labour input associated with ground surveys.
The development of applications that use oblique images, especially those based on dense image matching or the relative positions/spatial relations of objects in oblique images and other datasets, is possible thanks to the recent improvement of methods of oblique image orientation.Orientation has a crucial meaning for measurement accuracy (Hohle, 2008).Bundle adjustment of an entire image block is the most accurate method for determining the orientation parameters.However, the first attempts to use classical photogrammetric software to conduct the aerotriangulation of aerial oblique images did not provide satisfying results (Jacobsen, 2008;Gerke and Nyaruhuma, 2009).
The problems with classical aerotriangulation methods were grave enough to trigger the development of a number of alternative orientation methods, a review of which was done by Wiedemann and Moré (2012).It is only in recent years that significant progress in this field has been made.Rupnilk et al. (2014) put forward a modification of the classical bundle adjustment method that produces more satisfying results.More recently, further improvements have been made, either in preparing image matching algorithms dedicated specifically to oblique images, as in the work of Hu et al. (2016), or to further modifications of bundle adjustments (Xie et al., 2016).
The latest achievements in the orientation of oblique images are presented in the results of the ISPRS/EuroSDR Benchmark on Multi-Platform Photogrammetry (Gerke et al., 2016).In difficult conditions of the benchmark (inaccurate approximations of camera position from GNSS or weak distribution of GCP and small overlap), it was possible to obtain an accuracy level of a few GSD.Other research experiments (Xie et al., 2016, Ostrowski andBakuła, 2016), conducted on less challenging datasets and therefore closer to real (production) situations, show that nowadays it is possible to achieve accuracy of orientation with oblique images (verified on check points) at a single GSD level.
The purpose of the research presented in this article was to examine how the development of methods and the consequent improvement of accuracy in terms of the orientation of oblique images affected the accuracy of mapping from this type of picture.For this purpose, two blocks of images of Katowice City were used, with an average GSD of 8 cm.The first block was obtained in 2014 and the second in 2016.Digital (vector) based maps of the city were used as the reference dataset.
In the next section (2), methods of photogrammetric measurement used with oblique images are described.Later (in section 3), the conducted experiment was described, including the methodology and data used.Section 4 presents the results of the experimental part of the study.

MEASUREMENTS IN OBLIQUE IMAGERY
The history of the development and use of oblique images is not directly related to photogrammetric measurement (Karbo and Schroth, 2009); this type of image was obtained mostly for evaluative purposes, while, for the purposes of surveying, nadir images were mostly used.It was with the latter in mind that methods of photogrammetric measurement were developed.However, a much more complete method of representation of reality with oblique images led to the development of methods that allowed for at least very simple measurements.

Single Image
With the spread of oblique photography, the company Pictometry International developed and patented a method of measuring using this kind of image.The method, based on direct georeferencing and underlying DTM, is described in detail by Hohle (2008).The solution proposed by Pictometry, on the one hand, allowed aerotriangulation to be left outa process which, as mentioned in the introduction, has, for many years, been a source of problems when processing oblique images.On the other hand, it allowed for measuring threedimensional coordinates using a single image, and thus, it significantly facilitated working with this kind of data.
The mentioned method, however, carries several limitations.The first limitation is that the accuracy of measurements is dependent on the quality of the DTM on which the point of the intersection of the projection ray and the terrain (the measured object) is determined.Another limitation is the fact that georeferencing cannot directly provide accuracy that is as high as correctly conducted aerotriangulation.
A notable development in measuring xyz coordinates for points located on the ground is the use of geometric properties to measure the height of vertical objects in a single image.This kind of measurement is also present in the earlier solutions proposed by Pictometry and was described in detail in a study by Molinari et al. (2014).

Multi Image
One important feature of aerial oblique image datasets is the high overlap between images.With symmetrical-side and forward overlap, a single part of the ground should be visible in more than 20 images for 60% overlap and in more than 40 images if the overlap is increased to 80% (Jacobsem and Gerke, 2016).This strong overlay of images within the block allows for the possibility of performing the measurement on multiple images, which is suitable for monoscopic measurements.Multiple image measurements also utilize another significant aspect of oblique aerial images.Because of the convenient ray intersection angle, the measurement performed on images with a tilt angle of 45 degrees should be equally accurate in both the horizontal and vertical dimension (Rupnik et al., 2015).

Dense Image Matching
Automated measurements from oblique images using dense image matching are becoming more and more popular, and point clouds obtained with this method are used mostly in the automated creation of 3D models for cities.The greatest advantage of oblique images is the fact that they enable the 3D reconstruction of facades, which is not possible when using nadir images.Unfortunately, the processing of oblique images is much more difficult due to the variable scale and low similarity between subsequent images.
Different algorithms, primarily developed for the purpose of matching nadir images, have also been tested for use with oblique images, for example, Semi Global Matching (Gerke, 2009) and MicMac (Nex et al., 2013).New methods have also been developed and modified (Fritsch and Rothermel, 2013;Zhang et al., 2015).So far, however, the accuracies obtained from the matching of oblique images do not equal those that can be achieved with the use of nadir images.We can hope that the ongoing Benchmark EuroSDR/ISPRS, the objectives of which are described in Cavegn et al. (2014), will provide a complete review of dense image matching in the context of oblique images.

THE EXPERIMENT
The aim of the conducted experiment was to determine the accuracy of measurement using oblique images.Both image orientation and measurements were done in Pix4D, which is one of the commercially available software products which provide satisfactory results with oblique images, as shown by Benchmark ISPRS/EuroSDR (Gerke et al., 2016).The measurements were done from many images, that is, the objects were measured monoscopically from at least three images, and 3D coordinates were determined by ray intersection.In order to simplify the experiment, point-like objects such as footprints and corners of buildings were used.Certain problems, such as mutually obscuring objects, dead fields, and an overlay of images, were omitted in the discussion.
Two blocks of oblique images, obtained for the same area, were used.The images had the same GSD but were taken at a twoyear interval, thanks to which it was possible to verify the reproducibility of the results The experiment was divided into two parts.In the first part, the absolute accuracy of the horizontal measurements was verified.Ground flood corners of buildings were measured, and a base map of the city was used as reference data.The second part of the experiment dealt with the accuracy of the distance measurements, for which a base map of the city was used again as a reference to evaluate the accuracy of the measured building footprints.

Test area
The area used in the experiment was the centre of Katowice, Poland.The differences in the height of the topography of this area do not exceed 50 metres (Fig. 1).The height of the area's buildings is typically 15-20 metres, although single buildings with a height of over 70 metres do occur.
For the purposes of the experiments concerning measurements in the XY plane, 4 test fields were selected.They varied in terms of building type and the orientation of the streets.The development in the first field was of an industrial character, the second consisted of residential areas with blocks of flats, and the remaining two had a typically urban character.

Imagery
Both datasets were acquired with a combination of five cameras, IGI DigiCam in a Maltese cross configuration, with a tilt angle of 45 degrees.However, for this experiment, only oblique images were processed.The forward-and backwardlooking cameras were 39 Mpx, and the side-looking cameras were 50 Mpx.The cameras had the same lenses with 100 mm focal length.The platforms on which the cameras were placed were equipped with a survey-grade GNSS A dataset from 2014 was acquired in May, containing 854 oblique images with an average GSD of 9.3 cm and an overlap of 60% within and 30% between the strips.The flight was performed in an east-west direction.The dataset from 2016 was performed in the opposite direction (north-south) and contained 997 images with an average GSD of 9.9 cm.The overlap between the next two images were equal 60% within and 40% between the strips.
The test area was covered by 50 equally distributed (Fig. 1) ground control points measured with GNSS RTK.As control points, easily identifiable objects were selected such as manhole covers or, rarely, road markings.
Aerial triangulation, together with self-calibration, were performed in the Pix4D software (Pix4D, 2016).The workflow was similar to that previously used in Ostrowski and Bakuła (2016).Full resolution of images was used, but the number of desired key points was limited to 25,000 per image.Also, geometrically verified matching was used.
All cameras were laboratory calibrated.However, calibration parameters were used only as precalibrated values during the self-calibration process.The results of self-calibration (Tab.2) are different for both datasets.This could be explained by the two-year difference in time between the test flights and a lack of full stability of the interior orientation parameters for the medium format cameras..
Bundle adjustment for both data sets provided satisfactory subpixel accuracies, which were almost equal in all three directions; the accuracy characteristic are summarized in Tab. 3.

Reference Data
As reference data for the experiment, a digital base map of the city was used.This kind of map is created by a ground survey and stored in the CAD database, which should be continuously updated.The accuracy of the base map, varying according to the object type in the case of a building, the accuracy should be no worse than10 cm.However, the accuracies are defined in relation to the control points of the national grid.Because of this, the accuracy of the base map is limited by the accuracy of the control points used and could be affected by some systematic errors.

RESULTS
During the experiment, this same group of points was measured on both datasets.Measurements were performed in at least two directions.The minimal number of projection rays to the measured point was 3, and the maximum was 14.Most points were measured using 7 or 8 images.Theoretical errors related to the point positions were on the level of one third of GSD for all directions; however, maximal orthogonal ray distances were bigger, and for many points, reached a level of one or two GSD. Figure 2. Histograms of residuals between planimetric coordinates measured on oblique images and derived from the base map for both datasets.

Measurement of planimetric coordinates
In order to evaluate the accuracy of the planimetric coordinates, 118 corners of building footprints were measured.Only points that were visible in both datasets were taken into consideration.Estimated coordinates were compared with the base map.Histograms of achieved residuals are shown in Fig. 2

Measurement of distances
The second part of the experiment focused on measuring distances.Exactly the same 60 linear parts of footprints of building facades were measured on both oblique image datasets and were compared to the distances derived from the base map.Lengths of single distances varied between 3 and 57 m, and the mean length of the measured object was equal to 17 m (with a standard deviation of 10 meters).
Figure 3. Histograms of residuals between planimetric coordinates measured on both oblique image datasets.
In order to perform a comparison with the reference dataset, the following statistical parameters of difference between the reference data and the measurements in Pix4D were calculated: average, RMS, standard deviation, and maximum value (Tab.7).
Histograms of the differences are shown in Fig. 4 and 5.

CONCLUSIONS
The results of the measurements with the use of oblique images presented in this article show that, thanks to the use of modern methods of orientation of oblique images, it is now possible to increase the accuracy of measurements with this kind of imagery.The accuracy achieved might broaden the scope of possible usages in mapping applications, for example, enabling certain cadastral measurements that were not possible a few years ago with the use of technologies of measurement from a single image (Lemmens et al., 2007).
There was no significant difference between the measurement accuracies in the datasets from 2014 and 2016, which proves that the achieved results are reproducible.The accuracy of the measurement (compared with the base map) is on the level of two GSD, both for absolute measurements of planimetric coordinates and for relative measurements of distances.
The visible discrepancy in the accuracy of the established positions between the check points in aerotriangulation and the accuracy of the points measured later from those images might be explained by problems with the explicit identification of the measured point on different images.High values for the maximal orthogonal ray distance obtained during the measurements could confirm this.
The reason for the higher consistency of measurements on both the oblique datasets compared to the accuracy of measurements on the base map still needs to be determined.There are at least a few factors which should be considered: the accuracy of the base map, the discrepancy between the base map and the ground control points which were used, and mistakes in the identification of the measured object.

Figure 1 .
Figure 1.Left: Map of GCP distribution, markers: Red triangle -Ground Control Point, Black dot -Check Point.Right: Digital Surface Model of test area (marked with red outline).

Figure 4 .
Figure 4. Histograms of differences between distances measured on both oblique image datasets.

Figure 5 .
Figure 5. Histograms of differences between distances measured on different oblique image datasets and the base map.

Table 2 .
Interior orientation parameters estimated during selfcalibration in Pix4D.For the same cameras with images taken in 2014 and 2016.

Table 3 .
Results of bundle adjustment in Pix4D for both datasets.

Table 4 .
. Statistical parameters: average, RMS, standard deviation, and minimum and maximum values are shown in Tab. 4. Result of planimetric coordinates measurements.To evaluate the consistency of the measurements on oblique images, differences between the point coordinates, estimated from both oblique image datasets, were calculated.Statistical parameters: average, RMS, standard deviation, and minimum and maximum values are shown in Tab.5; histograms are shown in Fig.3

Table 5 .
Consistency of planimetric coordinate measurements in both oblique images datasets.
Finally, to evaluate the point positioning accuracy, the horizontal residuals (distances) between the coordinates of the same point, derived from different datasets, are calculated.Statistical parameters: average, RMS, standard deviation, and minimum and maximum values are shown in Tab. 6.

Table 6 .
Distances (residuals)between points measured in different datasets.

Table 7 .
Comparison of results of distances measurement measured in different datasets.