COMPARISON AND ANALYSIS OF NONLINEAR LEAST SQUARES METHODS FOR VISION BASED NAVIGATION (VBN) ALGORITHMS

A robust scale and rotation invariant image matching algorithm is vital for the Visual Based Navigation (VBN) of aerial vehicles, where matches between an existing geo-referenced database images and the real-time captured images are used to georeference (i.e. six transformation parameters three rotation and three translation) the real-time captured image from the UAV through the collinearity equations. The georeferencing information is then used in aiding the INS integration Kalman filter as Coordinate UPdaTe (CUPT). It is critical for the collinearity equations to use the proper optimization algorithm to ensure accurate and fast convergence for georeferencing parameters with the minimum required conjugate points necessary for convergence. Fast convergence to a global minimum will require non-linear approach to overcome the high degree of non-linearity that will exist in case of having large oblique images (i.e. large rotation angles).The main objective of this paper is investigating the estimation of the georeferencing parameters necessary for VBN of aerial vehicles in case of having large values of the rotational angles, which will lead to non-linearity of the estimation model. In this case, traditional least squares approaches will fail to estimate the georeferencing parameters, because of the expected non-linearity of the mathematical model. Five different nonlinear least squares methods are presented for estimating the transformation parameters. Four gradient based nonlinear least squares methods (Trust region, Trust region dogleg algorithm, Levenberg-Marquardt, and Quasi-Newton line search method) and one non-gradient method (Nelder-Mead simplex direct search) is employed for the six transformation parameters estimation process. The research was done on simulated data and the results showed that the Nelder-Mead method has failed because of its dependency on the objective function without any derivative information. Although, the tested gradient methods succeeded in converging to the relative optimal solution of the georeferencing parameters. In trust region methods, the number of iterations was more than Levenberg-Marquardt because of the necessity for evaluating the local minimum to ensure if it is the global one or not in each iteration step. As for the LevenbergMarquardt method, which is considered as a modified Gauss-Newton algorithm employing the trust region approach where a scalar is introduced to assess the choice of the magnitude and the direction of the descent. This scalar determines whether the GaussNewton method direction or the steepest descent method direction will be used as an adaptive approach for both linear and nonlinear mathematical models and it successfully converged and achieved the relative optimum solution. These five methods results are compared explicitly to the linear traditional least-squares approach, with detailed statistical analysis of the results, with emphasis on the UAV (VBN) applications. * Corresponding author. This is useful to know for communication with the appropriate person in cases with more than one author.


INTRODUCTION
In recent years, the utility of the Unmanned Aerial Vehicles (UAV) has greatly increased in applications such as, surveillance, law enforcement and aerial mapping.Furthermore, UAVs are quickly becoming an integral part of both military and commercial operations.For these classes of applications, accurate UAV navigation is considered a critical element to be investigated.Inertial Navigation Systems (INS) with Inertial Measurement Unit (IMU) and Global Positioning System (GPS) are frequently used in airborne applications.
On one hand, the INS systems are passive systems, which are widely used in navigation and characterized by its immunity against jamming.On the other hand, the GPS aided system is commonly used to correct the navigation error drift caused by using INS alone.However, a GPS aided system has some limitations.When using GPS, the navigation system is dependent upon the availability of satellites necessary for the operation and how reliable the navigation system can be in making decisions about the information received from these satellites.Examples of such situations are the jamming of GPS signal leading the navigation system to not rely on such a signal or, urban areas where loss of satellites is expected (Nilsson 2005).
To achieve robust UAV navigation in such environments, alternative navigation system is employed, such as Vision based Navigation (VBN), where inertial sensors are employed together with low cost imaging sensors which make measurements of the surrounding environment in order to provide the necessary information required for navigation.The main objective of this paper is investigating the estimation of the georeferencing parameters necessary for VBN of aerial vehicles in case of having large values of the rotational angles, which will lead to non-linearity of the estimation model.In this case, traditional least squares approaches will fail to estimate the georeferencing parameters, because of the expected nonlinearity of the mathematical model.
Five different nonlinear least squares methods are presented for estimating the transformation parameters.Four gradient based nonlinear least squares methods (Trust region, Trust region dogleg algorithm, Levenberg-Marquardt, and Quasi-Newton line search method) and one non-gradient method (Nelder-Mead simplex direct search) is employed for the six transformation parameters estimation process.

COLLINEARITY EQUATION AND ITS APPLICATION TO VBN
In a VBN system, given conjugate matched points between the real-time images and the images in the database, the transformation parameters (ω, φ, κ, X PC , Y PC , Z PC ) can be estimated through the mathematical model of the collinearity equation, shown below, using the proper optimization algorithm to ensure accurate and fast convergence for those parameters with the minimum required conjugate points necessary for convergence.
where f, x pc , y pc = camera interior orientation parameters x p , y p = image coordinates X PC , Y PC , Z PC = coordinates of projection center X P , Y P , Z P = object coordinates in ground coordinate system c 11, …,c 33 = elements of the 3D rotation matrix parameters The transformation parameters are estimated through an optimization algorithm using the measured common points in both the aerial image and referenced database (Saeedi 2009).
Once completed, the camera position and attitude will be estimated and consequently the UAV pose can be estimated.
There are two general approaches to investigate the nonlinear least squares algorithms (El-Habiby, Gao et al. 2009).The first one is linearization of the model.This approach is employed by adding corrections to the unknown estimated parameters during the iteration steps.However, in the general photography case without the near vertical assumption, the second approach is used.The second approach is based on gradient method, where modifications are made to the steepest descent approach.

TRANSPORMATION PARAMETERS ESTIMATION BASED ON NONLINEAR LEAST SQUARES
A pose estimation problem can be investigated through the correspondence between the 3D Ground Control Points (GCP) and the corresponding 2D points in image coordinates (Lu, Hager et al. 2000).Gauss-Newton and Levenberg-Marquardt, among other methods such as Trust Region method, are considered as classical iterative approaches for solving the nonlinear least square problem for pose estimation problem (Lowe 1991).
In general, minimization equation for unconstrained nonlinear problem can be described as in equation ( 2 where x = unknown parameters to be estimated (transformation parameters) F(x) = objective function (collinearity equation)

Nelder-Mead Simplex Direct Search Method
In this method, a simplex S of approximations to an optimal point is maintained.This is achieved by sorting the vertices x i to x k+1 based on the objective function values. where x 1 = the best vertex x k+1 = the worst vertex Reflection (ρ), expansion (χ), contraction (γ), and shrinkage (σ) parameters are used in minimizing the objective function (Nelder and Mead 1965).

Trust Region Method
In this method, a quadratic model is described by equation where f(x c ) = objective function at current iteration H = model Hessian At each iteration x c , trust region radius Δ is computed using the following equation: where Δ c = trust region radius Starting with the quadratic model m c and the trust region radius Δ, a trial point x c + s t is calculated such that the model is minimized.Then, the objective function is computed at the trial point and compared to the quadratic model value at the trial point.

Trust Region Dogleg Method
In this method, the search direction is computed over a path s(α) where 0≤α≤α max such that the model mc is minimized over the path s(α).The solution of the trust region problem lies on such path minimizing the objective function.The path is given by the following equation (Kelley 1999): where s cauchy = Cauchy point s GN = Gauss-Newton Point

Levenberg-Marquadrt Method
This method is considered as a modification for the Gauss-Newton method to overcome the problem of the failure of such method when c T c J J is singular.Where, J c is the jacobian of the objective function at the current point.To overcome this problem, a coefficient λ>0 is introduced such that the search direction can be found using the following equation (Luenberger 1984): where d c = search direction Depending on the value of λ, the Levenberg approach will either be a Gauss-Newton one (when λ=0) or steepest descent approach (when λ has big values).

Quasi-Newton Line Search Method
The basic idea behind this method is that approximation of the Hessian matrix H of the objective function is achieved by a positive definite matrix.This positive definite matrix is initially computed by H 0 .As the search continues, the approximating matrix is updated at each iteration step and consequently the second derivative information is updated and improved.The Broyden, Fletcher, Goldfarb, and Shanno (BFGS) update is given by the following (BROYDEN 1970): where

PERFORMANCE ANALYSIS
Solution of the collinearity equation given in equation ( 1) is done using the proposed five nonlinear methods.Simulated data is used such that three situations are realized.The first one is the near vertical assumption, then a general tilted photograph is realized with φ=10° and 30°.The camera focal length is 151.876mm.Three ground control points are used in the tests.Results shown below are the residuals of the proposed nonlinear least squares methods and the number of iterations achieved.

CONCLUSION
Object space transformation parameters estimation was introduced using five nonlinear least squares methods.These parameters are necessary for pose estimation of UAV.Two situations were introduced.The first one is the near vertical assumption and the second one is the general tilted photograph case.Convergence of the proposed nonlinear least squares methods were tested in the situations mentioned before.As for the Nelder-Mead simplex direct search method, it failed to estimate the object space transformation parameters because this method depends on the objective function only without any derivative information.As for the other four methods, the object space transformation parameters were successfully estimated for Φ = 0°.However, when Φ = 10° and 30°, the object space transformation parameters were successfully estimated only by Quasi-Newton line search method due to depending on the Hessian matrix of the Collinearity equation.

Table 1
Initial values for object space transformation parameters