RELATIVE ORIENTATION IN LOW ALTITUDE PHOTOGRAMMETRY SURVEY

Relative orientation is always considered as a key technique, not only in traditional photogrammetry, but also in low altitude photogrammetry. Low altitude images are mainly obtained by general digital cameras on UAV, they have characteristics of small format, large tilt angle and high-overlap in sequence. These distinctions from traditional images urgently call for a new reliable way to recover the relative pose between two adjacent images. For example, better initial values of relative orientation elements are required in the iteration process due to the large roll angle and yaw angle. Also, a more stable and efficient adjustment method should be proposed for the high-overlap images. In this paper, the Direct (D) relative orientation method is firstly used to get coarse values of the relative orientation parameters, then the Conventional (C) relative orientation process is implemented, using the coarse values as initial values in the first iterative calculation. And RANSAC (R) algorithm is finally applied in locating and extracting gross errors in relative orientation. The three steps above form our execution—DCR—to solve relative orientation problem in low altitude photogrammetry. Practical images have been used later to test the DCR method on accuracy and precision of the relative orientation parameters. Our experimental results show that the proposed approach is feasible and can achieve more reliable relative orientation results than the conventional approach.


Introduction
Relative orientation, to recover the relative pose of two adjacent images, has always been and remains one of the most important operations in photogrammetry.A review of the history of relative orientation is given by Wrobel in Gruen and Huang(2001).Generally, two methods of relative orientation have been established: direct methods and iterative methods .Direct relative orientation, based on algebraic geometry theory, requires at least five points when the calibrated intrinsic parameters are given.If the input set of correspondences is very good, then, the direct methods can be used for accurate parameter estimation.However, in practice, the input set of matches contains a considerable amount of outliers.The direct methods, which are linear, are extremely sensitive to wrong matches.The iterative methods or the conventional methods are implemented through performing bundle adjustment.Bundle adjustment is typically started from an approximate configuration of the scene points and camera poses.With a sufficiently accurate initial configuration, the conventional method can achieve the best accuracy.However, with an inaccurate initial configuration there is no guarantee that the bundle adjustment will find the globally optimal solution.
Low altitude photogrammetry survey has been greatly developed in China in the last few years.Its main characteristics are as follows: (1) Without considering human factors by autocontrol flight; (2) Quick and flexible for emergency mapping; (3) High resolution images; (4) Flying under the clouds.These characteristics enable UAV aviation remote sensing information platform to become an effective supplement way to the satellite and manned-plane remote sensing.However, due to the instable UAV platform, low altitude images are usually of small format, large tilt angle and high-overlap in sequence.This brings certain difficulty in automatic processing of aerial triangulation of the image data.
Based on the characteristics of the UAV images and the shortage of the traditional method, we propose a novel solution of relative orientation, called DCR solution, in this paper.This new solution firstly puts to use the Direct relative orientation method and RANSAC algorithm together to obtain the coarse value of the relative orientation elements.Then, we take these coarse values to do iterative calculation in adjustment to get an accurate result.Compared with the traditional method in extensive experiments, the DCR solution is proved to be more accurate and stable.
The rest of the paper is organized as follows.First a simple introduction to the conventional method will be given in Section 2. In Section 3, our new DCR method is explained and in Section 4 several experiments will be performed to evaluate the proposed method in accuracy and stability.

Conventional relative orientation method
According to the coplanarity condition (Eq.2.1) that the two perspective centers and two corresponding image rays lie in the same plane (Mikhail et al., 2001), we could obtain the mathematical model of conventional relative orientation (Eq.2.2): where, q is the y-parallax of relative orientation, , , ″ are three components of the baseline,  , , , ,  d d d d d ϕ ω κ µ ν the corrections of the relative orientation parameters.
In this method, intrinsic parameters of the digital camera and the coordinates in image space of at least 5 corresponding points of the two images should be given.

Direct relative orientation by eight-point algorithm
Direct relative orientation is an important step to reconstruct 3D scene in computer vision.The eight-point algorithm, proposed by Longuet-Higgins,H. in 1981, is a classic method of direct relative orientation.The great advantage of this algorithm is that it is linear, hence fast and easily implemented.
Fundamental matrix and essential matrix are the basic tools for the direct relative orientation.The essential matrix is a metric object pertaining to calibrated cameras, while the fundamental matrix describes the correspondence in more general and fundamental terms of projective geometry.Three well-known constraint equations are listed as follows: where u and ' u are the corresponding points of a stereo pair.
The three equations are used in order to compute the fundamental matrix ( F ) and the essential matrix ( E ).
In eight-point algorithm, a set of eight or more point matches should be known.Thus, the processing for direct relative orientation may be formulated as consisting of three steps.1) Linear Solution for the Fundamental Matrix In equation (3.1) above, u and ' u can be written in term of ( , ,1) u v and ' ' ( , ,1) u v respectively.The row of the equation matrix may be represented as ' ' ' ' ' ' ( , , , , , , , ,1) uu uv u vu vv v u v .Thus, we obtain a set of linear equations of the form , where f is a nine-vector containing the entries of the matrix F , and A is the equation matrix.
If eight point matches are known, then the solution of a set of linear equations is involved.With more than eight points, a linear least squares minimization must be solved.

2) Computing for the Essential Matrix
The relationship between a fundamental matrix F and its corresponding essential matrix E can be captured mathematically, which is K and ' K being the intrinsic calibration matrices of the images involved.Thus, given the intrinsic parameters of the cameras, we can get the essential matrix E from the fundamental matrix F .

3) Result for the Direct Relative Orientation
The transformation and rotation parameters may be acquired from the essential matrix.And a convenient method of doing this is to use the Singular Value Decomposition (SVD).In particular, let , the rotation matrix R and the transformation vector t can be computed by the following equations: For given the first camera matrix It is clearly shown that a reconstructed point X will be in front of both cameras in one of these four solutions only.Thus, testing with a single point to determine if it is in front of both cameras is sufficient to decide which is the real result of the relative orientation between the four different solutions for the camera matrix ' P .

RANSAC algorithm
Random Sample Consensus (RANSAC), published by Fischler and Bolles in 1981, is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers.RANSAC algorithm can estimate the parameters with a high degree of accuracy even when a significant number of outliers are present in the data set.The RANSAC is more formally stated as follows: 1) Given a model that requires a minimum of n data points to instantiate its free parameters, and a set of data points P such that the number of points in P is greater than n, randomly select a subset S1 of n data points from P and instantiate the model.Use the instantiated model M1 to determine the subset S1* of points in P that are within some error tolerance of M1.The set S1* is called the consensus set of S1. 2) If #(S1) is greater than some threshold t, which is a function of the estimate of the number of gross errors in P, use S1* to compute(possibly using least squares) a new model M1*. 3) If #(S1*) is less that t, randomly select a new subset S2 and repeat the above process.If, after some predetermined number of trials, no consensus set with t or more members has been found, either solve the model with the largest consensus set found, or terminate in failure.The RANSAC algorithm contains three important parameters: (1) the error tolerance used to determine whether or not a point is compatible with a model, (2) the number of subsets to try, and (3) the threshold t, which is the number of compatible points used to imply that the correct model has been found.Methods are discussed for computing reasonable values for these parameters in Section 3.3.

Solving the problem
Below we will give a detailed description of the DCR solution.The DCR solution accepts as input the following data: 1) A list L of m matches of the two corresponding images.
2) The focal length (pixels) of the camera and the image plane coordinates of the principal point.
3) The proportion of outliers in all matching points.The DCR solution produces as output the following information: The DCR solution operates as follows: 1) As to get more reasonable corresponding points for relative orientation, 9 grids will be required by equally dividing the bounding rectangle of the overlap area in image planar of the corresponding stereo pairs.2) We pop up each point from one grid randomly, and this will form a set 1 S of 9 corresponding points.3) Using 1 S to perform a least square solution, the coarse value of the relative orientation elements can be computed by the eight-point direct relative orientation algorithm.4) Taking the coarse value as the initial value of the iteration, the conventional relative orientation method is performed to get the accuracy result-RO1.We use the parallax to compute the error for each corresponding point in list L to determine whether the point is compatible with this mathematics model of RO1.Given the threshold q of the parallax, the number of corresponding points which are in accord with the consensus set S1/RO1 can be added up, denoted by 1 n .

5) If 1
n is close to the actual number of the inliers in the list L , 1 (1 ) n m w ≈ × − , the current relative orientation result RO1 is considered as a correct one.The iteration will be stopped and the final relative orientation elements will be solved out by the conventional method again using the 1 n inliers.Otherwise, the above steps 2)-4) are repeated with a new random selection 2 S , 3 S , ….If the number of iterations of the above steps exceeds 9 log(1 ) log( 1) , then the largest consensus set found so far is used to compute the final solution(or we terminate in failure if the 1 2 max{ , ...} n n is less than eight).

Experiments
To demonstrate the validity of our DCR method, we performed the following two experiments with a set of digital low-altitude images with large tilt angle and high-overlap (about 80 percent) in sequence.All images are taken by a pre-calibrated Cannon 5D Mark C digital camera with 5616 pixels × 3744 pixels image format and 6.316 m µ physical pixel size.The focal length of the camera is 5360.547pixels.
In the first experiment, the test data is a stereo pair at the beginning of a strip, and little outliers (about 5%) have been found in the corresponding matching points.The known rotation angles of the left image are  The results of the two experiments in Table 1 and 2 both show that, comparing with the conventional relative orientation approach, the proposed DCR method not only slightly improves the RMSE of both the unit weight and the unknowns, but also notably reduces the times of the iteration.Meanwhile, compared Table 2 to Table 1, it can be seen that the precision and stability of the two approaches are both effected when the proportion of the outliers in all matching points raise to 35 percent.However, the deviation in the result of the conventional method seems even bigger, and result of the DCR method is more accurate.In our further experiments, the DCR method can steadily achieve a reasonable result in condition that there are no more than half outliers in the corresponding points.

Conclusions
In this paper, the problem of conventional relative orientation was discussed and a new DCR approach was proposed.The experimental results show the advantages of the DCR approach over the conventional method in accuracy, stability and efficiency.Using the better initial value for conventional method achieved by the eight-point direct relative orientation, times of the iteration have been greatly reduced.The ability of the proposed approach is highly improved in resisting gross observations by utilizing the RANSAC algorithm.However, due to the shortage of the linear eight-points algorithm and the randomicity of the RANSAC algorithm, more future work are supposed to be further investigated to improve the DCR solution.

[
test data is a stereo pair at the middle of a strip and about 35 percent outliers exist in the matches, which are used to verify the precision and stability of the DCR solution.The known rotation angles of the left image are 0matching points of the first and second experiment is shown in Figure1and Figure2, respectively.In the following experiments, the unit of ϕ , ω , κ is radian, the unit of root mean square errors (RMSE) is meter.

Figure 1 .
Figure 1.The distribution of corresponding points of the stereo pair at the beginning of a strip in the first experiment.

Figure 2 .
Figure 2. The distribution of corresponding points of the stereo pair at the middle of a strip in the second experiment.

Table 1 .
Results of relative orientation obtained by the first experiment data with different approaches.

Table 2 .
Results of relative orientation obtained by the second experiment data.Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia