IMPROVING THE UAV-DERIVED DSM BY INTRODUCING A MODIFIED RANSAC ALGORITHM

: The process of finding correspondence points among the overlapping images is called matching. The matching process is one of the fundamental steps in photogrammetry and computer vision with primarily application in 3D model reconstruction. The main limitation with matching algorithms is finding all the correct matches, so-called inliers, and consequently, reducing the incorrect matches, so-called outliers. A number of algorithms have been developed to increase the inliers. One of the well-known algorithms is RANdom SAmple Consensus (RANSAC). RANSAC, however, has a few limitations in terms of the number of iterations, high false-positive rate (outliers), and computational time. To improve RANSAC we are proposing three enhancements steps. The enhancements utilise an Iterative Least-Squares-based Loop (ILSL), a Similarity Termination (ST) Criterion, and a Post-Processing (PoP) step. We tested our enhancements on unmanned aerial vehicles (UAV) images of a forested area. Results show that the proposed enhancements decrease the false-positive ratio (outliers) and increase the number of inliers, with a reduced computational time compared to the conventional RANSAC. This led to more accurate photogrammetry products including Digital Surface Model (DSM).


INTRODUCTION
The process of finding corresponding points between two or more overlapped images is called matching which has many applications in photogrammetry and computer vision. Matching is one of the basic and important steps in producing photogrammetric and computer vision products (Xiong and Zhang, 2009) such as orthophoto, sparse points cloud, 3D model, and digital surface model (DSM).
One of the main photogrammetric products is DSM. In general, the steps involved in producing the DSM are, in order, feature and tie points detection, tie points matching, finding the best matches, image orientation, sparse point cloud generation, dense point cloud generation using the oriented images, 3D surface reconstruction, and DSM generation. Finding the best matches is essential to generate the sparse points cloud, and consequently a precise DSM.
One of the challenges in a matching process is the presence of high number of outliers (false positives) and low number of inliers (correct matches). Outliers decrease the accuracy and reliability of point cloud and DSM products (Lin et al., 2021). Another challenge in a matching process is its huge computational processing, since it is an iterative process (Wang and Chen, 2021), especially when applied to super highresolution imagery with multiple overlapping images, such as those acquired by unmanned aerial vehicles (UAVs). This challenge is enhanced for DSM and 3D model generation of spectrally and texturally similar environments such as treed areas, where finding the correct corresponding points (inliers) is more problematic. Furthermore, cameras (sensors) on-board UAVs are often non-metric and that introduces additional challenges in processing such data, especially in the matching step in comparison to traditional photogrammetric (Raguram et al., 2008).
To decrease the number of outliers, consequently, increase accuracy and reliability, various algorithms have been developed to find and remove outliers before the sparse point cloud generation step. Among many algorithms are Mestimators, L-estimators, R-estimators, Least Median Squares (LMedS), and Hough transform (Choi et al., 1997). One of the well-known methods is the Random Sample Consensus (RANSAC) (Fischler and Bolles, 1981). This method, first, selects an initial random sample (points) to estimate model (collinearity equations or fundamental matrix) parameters, then it evaluates the number of inliers and calculates the maximum iteration number (N).
To date, various modified RANSAC versions have been introduced to improve its performance such as M-estimator SAmple Consensus (MSAC) (Torr and Zisserman, 2000), Locally Optimized RANSAC (LO-RANSAC) (Chum et al., 2003), Progressive Sample Consensus (PROSAC) (Chum and Matas, 2005), RANSAC for Quasi Degenerate data (QDEGSAC) (Frahm and Pollefeys, 2006), Optimal RANSAC (Hast et al., 2013), Universal framework for random SAmple Consensus (USAC) (Raguram et al., 2012), Marginalizing Sample Consensus (MAGSAC) (Barath et al., 2019), Latent RANSAC (Korman and Litman, 2018), and; Geometrical Constraint SAmple Consensus (GCSAC) (Le et al., 2018), and so on. A comprehensive survey of different RANSAC-based methods shows the MSAC has the better performance in comparison to other modified versions in terms of accuracy and computational cost (Fischler and Bolles, 1981;Frahm and Pollefeys, 2006;Torr and Zisserman, 2000). However, it has a number of limitations including the increased false-positive rates of outliers and consequently resulting in fewer inliers, unnecessary high number of iterations, and high computational time. Such deficiencies possibly result from the random sampling process, the presence of noise, and incorrect assumptions of the initial values. This paper proposes a modified version of RANSAC-based methods to address some of the RANSAC limitations by introducing three enhancement steps. These three enhancements are to a) increase the stability and number of inliers using a locally iterative least-squares-based loop (ILIS), b) improve the convergence rate and consequently reducing the number of iterations using a similarity termination (ST) criterion, and c) remove the remaining outliers at the end of the processing loop using a POst-Processing procedure (PoP).

METHODS
Our proposed three basic enhancements improve RANSAC functionality in terms of number of inliers and computational time. Since our proposed method modifies RANSAC, it is important to understand RANSAC-based algorithms in photogrammetry. Thus, the remainder of this paper describes conventional RANSAC followed by our enhancements steps. We will then present the results of applying the enhancements on UAV images over a forested area.

RANSAC in Photogrammetry
RANSAC is an iterative two-step process. In the first step a minimum number of random samples (tie points), usually eight points (Elnima, 2015), are selected to solve the collinearity equations or fundamental matrix between two overlapping images. The interior camera parameters and relative orientations (transformations and rotations) are calculated simultaneously using the collinearity equations or fundamental matrix. Using more points to solve the equations increases the degree of freedom, thus, increases the accuracy and reliability.
In the second step, the model is tested against the rest of tiepoints through a distance function (e.g. Euclidean or Sampson distance) to determine the number of inliers using a predefined threshold. If the number of inliers is higher than a threshold or the number of iteration reaches to a predefined number N, the calculations stop and the final collinearity equations' coefficients are recalculated using all the inliers (Fischler and Bolles, 1981). Eqs 1 and 2 are used to determine the number of inliers (I), the inlier-ratio (e) and the number of iterations (N). ρ is the desired probability, and M is the total number of points (Fischler and Bolles, 1981). (1) where e = inlier-ratio M = total number of points S = selected initial random sample I = the number of inliers ρ = the desired probability of selecting a good sample Our proposed three enhancements to standard RANSAC are depicted in Figure 1. It is worth mentioning that none of the proposed enhancement steps requires additional input parameters from the user.

Figure 1:
The proposed enhancements shows in blue

Iterative Least-Squares-based Loop (ILSL)
As mentioned, the RANSAC-based models randomly select eight points (minimum number required to solve collinearity equations including three rotations, three transformations, two interior orientation parameters (Cx,Cy), and three coordinates (X,Y,Z) for each pair of point)s to find the unknown equations' coefficients and check the model against all the other matched points to find inliers. If the number of inliers is more than the previous set of points, the algorithm updated the inliers. However, what is missing in RANSAC is that it does not consider the inclusion of these early-found inliers to regenerate and improve the model. To include the early-best matches (inliers) in improving the model at each iteration, we propose a Locally Iterative Least-Squares-based (LILS) loop. The LILSloop uses all inliers found in the previous iteration to reestimate the equations and count the inliers again, then, apply a least square solution to improve the model until the number of inliers does not change. If the number of iterations reaches to K or inliers ratio meets the threshold, the LILS -loop stops. The LILS -loop increases the number of inliers, enhances the stability, and increases the convergence rate.

The Similarity Termination (ST) Criterion
The process in the RANSAC-based method algorithms is terminated when N iteration is reached or the inlier-ratio is greater than the threshold. To increase the convergence rate, we define a stop criterion not only to balance the computational time but also to avoid selecting a local optimum. The additional Similarity Termination (ST) criterion considers the similarity of inliers points between two consecutive iterations to terminate the process if the similarity is more than 95%. The ST criteria increases the convergence rate and decreases the computational time.

Post-Processing
To remove any possible remaining outliers (based on the experience, the final results still contain outliers), a postprocessing procedure is introduced to remove any outliers. Once the outliers are removed, the final coefficients are recalculated. This enhancement does not add any computational time, but increases the stability and accuracy.

DATASET AND PLATFORM
We used a set of four overlapping images acquired by a For this study, images were taken by a DJI Phantom Phantom over an area covered by bare land, a single building, road and forests (mostly coniferous trees)3, over Rich's Seashore Dr., Rigolet, Newfoundland, Canada ( Figure. 2). The area is located in UTM Zone 21 N (54°10'21.48" N, 58°26'6.74" W) and mostly covered by of coniferous trees. The parameters of the UAV images are listed in Table 1. Among the acquired UAV images, four different overlappinged images wereare selected over dense, semi-dense, and sparse forestry areas to assess the proposed enhancements.

RESULT
The proposed improvements are evaluated both qualitatively and quantitatively. First, comparing the number of inliers and the relative computational time to RANSAC is done to assess the quantitative analysis. Then, quantitative analysis was conducted by comparing DSM generated after our proposed enhancements and that generated by well-known commercial software (AgiSoft) (AgiSoft PhotoScan Pro). The Scale-Invariant Feature Transform (SIFT) algorithm was utilised to extract match-points for all images (Lindeberg, 2012), in this paper. The proposed enhancements methods were implemented and applied to four UAV image datasets over forested areas (as mentioned previous section) to remove outliers and generate sparse point cloud and the DSM, using The basic collinearity equations (with a normalised 8-point model (Elnima, 2015) ). Table 1 shows the results of the proposed methods compared to RANSAC in terms of computation time and number of inliers for four data sets (1, 2, 3, and 4). The first row in Table 1, shows the number of input match points (containing inliers and outliers) resulting from applying the SIFT to each dataset. Results reported in Table 1, shows that the proposed method outperforms RANSAC in finding more inliers in all four datasets. Furthermore, the computational time is less in three datasets than that of RANSAC.

Qualitative assessment
As mentioned, we also generated the sparse point cloud and DSM using the proposed method and using AgiSoft commercial software to compare the final products. One of the image pairs' results are shown in Figure 3. As shown in this figure, the proposed method resulted in a denser sparse and dense point cloud (more inliers) (Figure 3a, 3c) than that of AgiSoft ( Figure  3b, 3d). The point cloud difference shows the proposed method outperformed the AgiSoft (Figure 3e), and this results in a more accurate DSM (Figure 3f) than that of AgiSoft (Figure 3g). To facilitate the comparison, some areas are indicated in red circles in the two DSMs. These are individual trees and it is clear that the proposed method successfully picked those trees in the DSM while AgiSoft failed to do so.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France (f) (g) Figure 3. The two overlapped images as input data; the generated sparse point cloud using the proposed method (a) and using AgiSoft software (b); the generated dense point cloud using the proposed method (c) and using AgiSoft software (d); the dense point clouds difference (e); the DSM generated using the proposed method (f) and AgiSoft (g)

CONCLUSION
This study modifies the RANSAC method by using three enhancements for improving the performance of RANSACbased methods in terms of increasing the stability, the number of inliers, accuracy, and the convergence rate. The three enhancements include a local iterative least-squares-based loop to increase the number of inliers, as well as the stability of the method, a similarity termination criterion to decrease the computational time, and a final post-processing procedure to increase the accuracy and reliability of the results. The proposed method has been evaluated using the basic collinearity equations. The key points have been extracted using the SIFT algorithm.
The comparative analysis show that the proposed method could find more inliers, with lower computational time, especially in low inlier-ratios. However, it is observed that when the inlierratios is about higher than 70%, the proposed method is slightly slower than the RANSAC, but with meaningfully higher accuracy. Also, the proposed method does not need to tune new parameters or generate a high number of samples. The point cloud and DSM comparison showed the proposed methods can detect and extract more single trees than AgiSoft can do, which is a direct result of finding more inliers by the proposed method. Moreover, the sparse point cloud has more well-distributed points, especially in forest areas than that of AgiSoft. The proposed method is an important step in improving the processing workflow and products of ever-increasing UAV data for DSM and Orthophoto generation for environmental monitoring including forest 3D modelling.