STUDY ON PRACTICAL TECHNOLOGIES OF AERIAL TRIANGULATION FOR REAL SCENE 3 D MOELING WITH OBLIQUE PHOTOGRAPHY

The key technologies in the real scene 3D modeling of oblique photography mainly include the data acquisition of oblique photography, layout and surveying of photo control points, oblique camera calibration, aerial triangulation, dense matching of multiangle image, building of triangulation irregular network (TIN) and TIN simplification and automatic texture mapping, among which aerial triangulation is the core and the results of aerial triangulation directly affect the later model effect and the corresponding data accuracy. Starting from this point of view, this paper aims to study the practical technologies of aerial triangulation for real scene 3D modeling with oblique photography and finally proposes a technical method of aerial triangulation with oblique photography which can be put into practice. 1. THE KEY TECHNOLOGIES OF AERIAL TRIANGULATION FOR OBLIQUE PHOTOGRAPHY Through carrying muti-angle sensors on the same platform, oblique photography can capture images from various angles simultaneously, so as to acquire the real 3D scene. Oblique photography have the advantages of highly real effect of model, high precision of model data, short data production cycle and low production cost, which enables it to gradually replace traditional artificial modeling as the main way for 3D modelling (HOHLE J.,2008). In recent years, the technology of software and hardware for oblique photography has achieved rapid development at home and abroad (PETRIE G.Systematic, 2009), and it has been applied in homeland security, surveying and other fields. Compared with traditional aerial photography, the angles of various images have great differences, except for images of under view, images of other angles are not vertical photography, so traditional image matching algorithm based on gray is unfit for the matching of oblique images (Yuan Xiuxiao, Chen Shiyu, 2015). At present, the common matching algorithm for oblique images is mostly based on image features, but the matching accuracy is low. The main reason lies in the wide variation of image angles (Wei Zushuai, 2015). Thus, the present feature matching is the difficult point and key technique in aerial triangulation of oblique photography. That’s the reason why the development of the technology of post processing of oblique photography data is slightly behind the development of the technology of hardware (Deng Fei, Yuan Xiaoling , etc. 2014). If there is no error in the relative relationship between the down-looking camera and the oblique camera, the object square coordinate of the junction point calculated by the collinearity equation should be matched with the point cloud obtained by the matching of down-view image. However, due to the error in the relative relationship, the exterior image of the oblique image derived from the exterior orientation elements and the relative relationship of the down-view image is inaccurate, thereby reducing the accuracy of the elevation obtained by the forward intersection of the oblique image. In particular, when the junction point is located at the edge of the building or the vegetation area, the elevation accuracy of the interpolation is decreased (Feng fei, Feng Jianhui, Zhao Yanyan, 2016). 2. THE INFLUENCE OF IMAGE QUALITY FACTORS OF OBLIQUE IMAGES ON THE ACCURACY OF AERIAL TRIANGULATION At present, the common data acquisition platform of oblique images can be roughly divided into large aircraft, power Delta wings and unmanned aerial vehicles (abbreviated as, UAVs). Usually, large aircraft with a large oblique camera can obtain the highest resolution of the image up to 6cm. Power Delta wings with a medium-sized oblique camera can obtain the highest resolution of up to 3cm. UAVs equipped with a miniature oblique camera can obtain a maximum resolution of up to 2cm. Considering the errors of field surveying and measurement of control points and the errors of matching and adjustment of aerial triangulation, the plane precision of check points is about 1~2 times of resolution and the vertical precision of check points is about 2~3 times of resolution, which has also been verified in the different actual operation area. For example, in a city in south china, we take oblique aerial photography by using rotary-wing UAV with resolution better than 3cm, measuring an area of a total of 16km2. By carrying out the precision test by laying out check points homogeneously, we come to the conclusion that the mean square error of plane precision is 6cm and the mean square error of vertical precision is 9cm. By analyzing the accuracy of aerial triangulation of various test regions, we can draw such a conclusion that the effects of image quality factors, including brightness, contrast and shadow coefficient on the aerial triangulation, are obvious. This is decided by the characteristics that the image matching accuracy is based on degree of significance of image features. If image brightness or image contrast is too small or to large, or if shading coefficient is too large, it will directly reduce the accuracy of aerial triangulation, while the stability of the image poses no significant impact on the accuracy, because general algorithm of aerial triangulation can automatically eliminate the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-119-2018 | © Authors 2018. CC BY 4.0 License. 119 images with large angle differences or neglect the edges of large


THE KEY TECHNOLOGIES OF AERIAL TRIANGULATION FOR OBLIQUE PHOTOGRAPHY
Through carrying muti-angle sensors on the same platform, oblique photography can capture images from various angles simultaneously, so as to acquire the real 3D scene.Oblique photography have the advantages of highly real effect of model, high precision of model data, short data production cycle and low production cost, which enables it to gradually replace traditional artificial modeling as the main way for 3D modelling (HOHLE J.,2008).In recent years, the technology of software and hardware for oblique photography has achieved rapid development at home and abroad (PETRIE G.Systematic, 2009), and it has been applied in homeland security, surveying and other fields.
Compared with traditional aerial photography, the angles of various images have great differences, except for images of under view, images of other angles are not vertical photography, so traditional image matching algorithm based on gray is unfit for the matching of oblique images (Yuan Xiuxiao, Chen Shiyu, 2015).At present, the common matching algorithm for oblique images is mostly based on image features, but the matching accuracy is low.The main reason lies in the wide variation of image angles (Wei Zushuai, 2015).Thus, the present feature matching is the difficult point and key technique in aerial triangulation of oblique photography.That's the reason why the development of the technology of post processing of oblique photography data is slightly behind the development of the technology of hardware (Deng Fei, Yuan Xiaoling , etc. 2014).
If there is no error in the relative relationship between the down-looking camera and the oblique camera, the object square coordinate of the junction point calculated by the collinearity equation should be matched with the point cloud obtained by the matching of down-view image.However, due to the error in the relative relationship, the exterior image of the oblique image derived from the exterior orientation elements and the relative relationship of the down-view image is inaccurate, thereby reducing the accuracy of the elevation obtained by the forward intersection of the oblique image.In particular, when the junction point is located at the edge of the building or the vegetation area, the elevation accuracy of the interpolation is decreased (Feng fei, Feng Jianhui, Zhao Yanyan, 2016).

THE INFLUENCE OF IMAGE QUALITY FACTORS OF OBLIQUE IMAGES ON THE ACCURACY OF AERIAL TRIANGULATION
At present, the common data acquisition platform of oblique images can be roughly divided into large aircraft, power Delta wings and unmanned aerial vehicles (abbreviated as, UAVs).Usually, large aircraft with a large oblique camera can obtain the highest resolution of the image up to 6cm.Power Delta wings with a medium-sized oblique camera can obtain the highest resolution of up to 3cm.UAVs equipped with a miniature oblique camera can obtain a maximum resolution of up to 2cm.Considering the errors of field surveying and measurement of control points and the errors of matching and adjustment of aerial triangulation, the plane precision of check points is about 1~2 times of resolution and the vertical precision of check points is about 2~3 times of resolution, which has also been verified in the different actual operation area.For example, in a city in south china, we take oblique aerial photography by using rotary-wing UAV with resolution better than 3cm, measuring an area of a total of 16km² .By carrying out the precision test by laying out check points homogeneously, we come to the conclusion that the mean square error of plane precision is 6cm and the mean square error of vertical precision is 9cm.
By analyzing the accuracy of aerial triangulation of various test regions, we can draw such a conclusion that the effects of image quality factors, including brightness, contrast and shadow coefficient on the aerial triangulation, are obvious.This is decided by the characteristics that the image matching accuracy is based on degree of significance of image features.If image brightness or image contrast is too small or to large, or if shading coefficient is too large, it will directly reduce the accuracy of aerial triangulation, while the stability of the image poses no significant impact on the accuracy, because general algorithm of aerial triangulation can automatically eliminate the images with large angle differences or neglect the edges of large angle images.

INFLUENCE OF POS ACCURACY ON TRIANGULATION ACCURACY IN OBLIQUE AERIAL PHOTOGRAPHY
In aerial triangulation, the camera's position and pose are used as the initial value of the adjustment, and its accuracy directly determines the efficiency of aerial triangulation and adjustment result.That is to say, the higher the accuracy of POS, the higher the accuracy of exterior orientation elements, the higher the matching efficiency of aerial triangulation, the higher the reliability of adjustment.Based on the actual projects, we find that it is obvious that with or without POS, the efficiency of aerial triangulation is multiplied, but if the POS accuracy is poor, the final reliability of aerial triangulation is rather low.Therefore, in the case of poor POS accuracy, we have to improve the reliability of adjustment of aerial triangulation by some control points.
Usually, POS system on large aircraft is generally highprecision DGPS and IMU system with position accuracy reaching 1m and angle accuracy reaching 0.01°, but POS system on UVAS is common GPS and IMU with position accuracy at about several decametres range and low angle accuracy, and some of them even are not equipped with IMU system.In present project application, it is usually required to acquire oblique image resolution better than 5cm, which requires the selection of a suitable UAV aerial platform for data acquisition.In this case, we cannot obtain relatively accurate exterior orientation elements which can be used as initial values.
In order to improve the efficiency and reliability of aerial triangulation, we suggest the method for calculating exterior azimuth elements which is suitable for practical projects.First, the images of the lower-viewing lens undergo aerial triangulation to obtain the relatively accurate image of the exterior elements.Then, the exterior elements of oblique angle image are calculated by the placement relationship between different camera angles.This method has been proved by practice that it can effectively improve the processing efficiency of aerial triangulation with oblique photography and reduce the production cycle of entire real scene 3D modeling project without increasing the costs of the aerial photography platform.

STUDY ON CAMERA CALIBRATION
Digital camera is the main sensor of aerial photography, and camera calibration is a prerequisite for the high-precision photogrammetry.The accuracy of camera calibration directly affects the matching accuracy of junction points and the entire accuracy of aerial triangulation (Yang Weilan, Hu Haiyan,2017.Calibration of No-metric Digital Camera and Accuracy Analysis.Bulletin of Surveying and Mapping,S1,.
Most aerial photography platforms of oblique photography at present are UAV aerial photography platforms, so the oblique cameras equipped on such platforms are generally non-metric cameras.How to accurately calibrate the camera to meet the needs of photogrammetry, it is worth studying.Due to the fact that cameras on UAV cannot perform camera calibration before each flight, currently, oblique non-metric cameras are usually calibrated using the beam and regional network adjustment method.The so-called camera self-calibration based on beam and regional network adjustment method is to include the camera model parameters as an additional parameter in the aerial triangulation of the beam method area network, and to put it in the overall adjustment calculation of the area network triangulation.Solving parameters of camera self-calibration need a better initial value, and we usually need to compute exterior element at first.In the current high-resolution oblique photography, the number of images in the aerial triangulation sub-regions is often a large number.If these images are used for calibration, the calibration is time-consuming and cannot meet the changes in the parameters of multiple sub-cameras.Therefore, when we process large-area UVA navigation data in actual projects, a small area with diverse resources within a continuous period of time is usually selected for camera calibration.The obtained camera parameters are used for multiple measurements.The aerial triangulation in the area can largely guarantee the success rate of the aerial triangulation in the entire survey area and improve the matching efficiency of the aerial triangulation.

RESEARCH ON AERIAL TRIANGULATION FOR OBLIQUE PHOTOGRAPHY
Imaging matching is the core problem of digital photogrammetry.From the view of oblique photography, because oblique images have complex deformation, there are two strategies to realize matching: One is to use a feature matching algorithm with local invariance, which can resist geometric deformation; the second method is to use multiple images to match at the same time and use information complementary principle to achieve reliable matching.Normally, we only need to use local invariant for feature matching, that is to find similar images for feature matching.However, in case of large areas of water, forestland, desert, etc., multiple images need to be matched at the same time to find as many features as possible to connect the images and to improve the reliability of entire aerial triangulation.
After the feature matching is completed, the image light beam needs to be incorporated into the corresponding projection coordinate system, so a certain amount of control points are required for adjustment.In this process, the distribution of control points also plays an important role.If distribution of control points is uneven, or if there are gross error points in the control points, aerial triangulation will fail.In practical projects, in order to better judge the position of control points, it is usually to perform rough matching on the oblique image first, and then carry out control point stabbing to complete the entire aerial triangulation.

SUMMARY AND OUTLOOK
This paper has studied the practical technologies of aerial triangulation for real scene 3D modeling with oblique photography, and analyzed the influence of the quality of the oblique images on the accuracy of aerial triangulation.The camera calibration, the initial POS calculation, and the key process of image matching have also been summarized for practical production.Besides, it has put forward some rational proposals for the entire flow of aerial triangulation However, from the current production practice, the processing efficiency of large-area high resolution UVA data oblique photography of aerial triangulation is still the direction for further study in the future.