Impacts of stochastic models on real-time 3D UAV mapping

: In UAV mapping using direct geo-referencing, the formation of stochastic model generally takes into the account the different types of measurements required to estimate the 3D coordinates of the feature points. Such measurements include image tie point coordinate measurements, camera position measurements and camera orientation measurements. In the commonly used stochastic model, it is commonly assumed that all tie point measurements have the same variance. In fact, these assumptions are not always realistic and thus, can lead to biased 3D feature coordinates. Tie point measurements for different image feature objects may not have the same accuracy due to the facts that the geometric distribution of features, particularly their feature matching conditions are different. More importantly, the accuracies of the geo-referencing measurements should also be considered into the mapping process. In this paper, impacts of typical stochastic models on the UAV mapping are investigated. It has been demonstrated that the quality of the geo-referencing measurements plays a critical role in real-time UAV mapping scenarios.


INTRODUCTION
Over the past decade, 3D mapping relying on the integration of Global Positioning System (GPS), Inertial Measurement Unit (IMU) and camera is commonly used. The integrated GPS/INS systems are used for direct geo-referencing purposes which allow the determination of the camera platform position and orientation with a high accuracy. The camera is utilised to provide the details of features for imaging. This basically reduces the need of ground control points (GCPs) by eliminating aerial-triangulation, except for mapping system calibrations (Grejner-Brzezinska and Toth, 2004). This in turn reduces the cost and the time for producing 3D digital maps. However, direct geo-referencing which obtains coordinates of ground objects directly using the known exterior orientation elements of photos is influenced by some error sources. These error sources such as interior and exterior orientation elements, errors of GPS, time synchronization and projection centre deviation between GPS and vision sensors and errors of IMU degrade the accuracy and hence the efficiency of aerial photogrammetry (Zhang and Yuan, 2008).
Several studies have adopted a mathematical model which describes the relationship between measurements and unknown estimates (e.g., Wang et al, 2005) to eliminate the errors in aerial photogrammetry. On the other hand, stochastic model represents the statistical characteristics of the measurements. Such a stochastic model is mainly provided by the variancecovariance matrix for the measurements. If both models are defined correctly, optimal estimation can be achieved. The mathematical model has been addressed in different investigations in order to estimate the error sources in image matching, POS (Position and Orientation System) or exterior orientation elements of images (e.g., Zhang and Yuan, 2008).
In 3D mapping using indirect geo-referencing, variancecovariance matrices are often constructed by the error propagation to present the precisions for the position and orientation of the camera, as well as the ground control points coordinates, which are treated error-free in the least-squares adjustments for the tie point coordinates (Marshall, 2012). In the adjustment, it is usually assumed that all tie point coordinates have the same precision. However, these assumptions are not always true due to different conditions such as their geometric distribution, and thus their feature matching environments. Therefore, it is necessary to construct a realistic stochastic model in order to achieve satisfactory geo-referencing and 3D mapping results. In this situation, different weights are assigned for each covariance matrix component which is basically preferred based on the standard deviation (Beinat and Crosilla, 2002). Thus, the covariance matrix is scaled by standard deviation factor (Goodall, 1991).
While stochastic modelling has its roots in statistics (e.g., Rao, 1971;Grafarend et al, 1980), it remains a challenging issue in some real world applications. In the data processing for GPS/INS integration which is the foundation for direct georeferencing, stochastic model has already been investigated in either system level or multi-sensor integration systems level. Several studies (e.g., Wang et al, 1998, Wang et al, 2005 dealt with the construction of variance-covariance matrix under GPS, INS, and other applications. In the same way, some efforts (e.g., Wang and Wang, 2007;Mohamed andSchwarz, 1999, Quinchia et al, 2013) have been directed to improve the stochastic model in GNSS/INS integration systems. Cothren (2005) investigated the reliability in constrained Gauss-Markov model, which dealt with the measurements of the position and the orientation of the camera as uncorrelated measurements. In this study, it was also assumed that the horizontal positioning measurements of the camera have different precision from vertical positioning measurements. While more attention was more paid to investigating the precision of the exterior orientation elements of images, the precision of tie point measurements and the associated stochastic models have not been investigated in details yet.
In this paper, the precision of tie point measurements in stochastic model and its impact on 3D mapping will be addressed. In addition, the properties of the covariance matrix will be examined in order to provide a weight for tie points according to their geometric distribution. The rest of this paper is organised as follows. Section 2 deals with least squares in terms of functional and stochastic models for the UAV mapping, together with a new stochastic model for tie point coordinates. Section 3 presents and discusses numerical results, which will be followed with Section 4 for the concluding remarks.

Functional and stochastic models
The functional model, in general, describes the relation between the measured elements and the estimated elements. In photogrammetry, if the measured elements are image tie point coordinates with Cartesian coordinates, then the functional models could be (Cooper and Cross, 1988): Equations 1 and 2 are known as collinearity equations which show the rigorous geometric relationships between the object coordinates, projection centre and image coordinates of the object. The functional model generally contains non-linear equations and the stochastic nature of the observations, which are considered as random variables, must be taken into account. The non-linear equations can be linearized by using Taylor series. The linearization form of functional model can be expressed as: where l = is the 1  n observation vector; v = represents the 1  n vector of residuals; A = the t n  design matrix indicating the geometric relationships between ground and image coordinates of the object and the projection centre ; x = the 1  t vector of unknown parameters.
The functional model can be an inaccurate description of the relationship between ground and image coordinates of the object due to gross (outlier), systematic (bias), and random errors. The gross errors can be a result of malfunction of instrument or an error in the software. These errors must be detected and removed before the estimation process. The second type of error is systematic errors. Camera lens distortions, light intensity, lack of orthogonality of the comparator axes, and atmospheric refraction are some physical effects that can cause the systematic errors (Cooper and Cross, 1988). The systematic errors cannot be removed, but there are two different ways to reduce their effects. Firstly, they can be considered in the functional model. For example, coordinates differences between the actual origin of the image coordinates and the true coordinates was dealt with as systematic errors and represented in equations 1 and 2 as o x and o y . Secondly, the biases are considered as stochastic effects which are accounted for by random parameters or by a priori correlation between the measurements. The third part of errors is the random errors which are usually unavoidable and bring small differences between the measurements and their expectations. Although they are uncertain, they follow statistical rules.
The stochastic model of least squares can be defined as: where ) (l D = the n n covariance matrix for the measurements Q = the n n co-factor matrix 2 0  = a priori variance factor being assigned as one The symbol P in equation (4) represents the weight matrix with n measurements.
Generally, if the measured values have the same accuracy and are regarded as uncorrelated, then the diagonal elements in Q matrix should have the same values and the off-diagonal elements will be zeroes. However, in 3D mapping using direct geo-referencing, the observations are three types including tie point, camera position and camera orientation measurements which require three types of variance values. The typical form of Q will be as follows: As the number of measurements is usually larger than the number of the unknown parameters in Equation (3), the estimation of parameters and the residuals vector for the measurements can be obtained as (Gruen, 1985): which are the optimal estimates of the unknown parameters and the residuals, respectively, if the measurements are unbiased and the stochastic model is realistic.
The co-factor matrices for the estimated parameters and the residuals respectively, can be written as:

Factored stochastic model for tie point measurements
In the commonly used stochastic model, the variances of tie point observations 2 A  are assumed to be the same. However, this assumption is not always true due to different conditions such as geometric distribution of the features and their matching environments. Based on this fact, the tie point measurements should be given different weights. In this paper, the tie point measurements in x-axis are assigned different weight based on its distance from the x values of the camera position and the measurements in y-axis based on its distance from the y values of the camera position. This will reduce the effect of biases which have not been considered in the functional model. In addition, the proposed method of weighting of the tie point measurements is based on the assumption that the tie point measurements do not have any faulty measurements (outliers). The faulty measurements or mismatches are not discussed here and are treated in separate studies.
The investigation of the impact of the stochastic models on the accuracy of the object coordinates are carried with the commonly used model and the factored stochastic model as discussed below.
The equations used for determining the factor of tie point variance can be obtained as follows: where x F and y F = factor for the variances of the tie points x  and y  = the difference distance between object position and cameras position X and Y = the distance between the farthest object and the centre Thus, the variances of each tie point can be computed as:

TEST RESULTS AND ANALYSIS
Most UAVs have digital cameras for image acquisition and mainly depends on GPS and other sensors such as INS for navigation and geo-referencing. The obtained images in digital form can provide the opportunity to process the data automatically. The GPS/INS systems can immediately deliver the geo-referencing information. The automatic process for obtaining geo-referencing information and for completing images matching gives the ability to generate 3D maps in real time to be used in military, scientific, and industrial applications. However, the quality of the real-time 3D mapping essentially relies on the accuracy of geo-referencing and matching results. So, the precision of the tie point, camera position and orientation measurements and their impacts on the 3D mapping will be analysed in the following sections.

Simulations
Two UAV images were simulated. They have longitudinal overlap and 15, 23, and 23 objects in the overlapped area. The first and the second simulations were completed with assuming all objects have the same height (zero) and the third simulation each object have different height but in the range between zero and 20 meters. Camera position in a local coordinate system and orientation are also identified  In this section, the impact of the precision of tie point measurements on the standard deviations of the estimated object coordinates has been investigated for Mapping Scenario 1 (Figure 1). The computation of the standard deviations for the object coordinates is completed with assuming that we have five levels of precision of the tie point measurements and the precision for the camera position and orientation pseudomeasurements are 6 cm and 0.03 degree respectively. It can be seen in Tables 2, 3, and 4 that the standard deviations in X, Y, and Z axes of the estimated parameters increase with the increase of the tie point precision vales from 0.5 to 0.75 pixels, from 0.75 to 1 pixel, from 1 to 1.25 pixels, and from 1.25 to 1.5 pixels.

The precision of tie point measurements and the standard deviations
From Table 2, it can be noted that the increase in the tie point precision values does not have the same impact on all objects. For example, the standard deviations in X-axis for objects 1, 2, and 3 increase more than 1 cm, whereas the increase for objects 7, 8, and 9 does not exceed 3 mm. Also, it can be seen that the standard deviations in X-axis of all objects in each level of the precision of tie point measurements are not the same. The standard deviations of objects 1, 2, and 3 can be categorised in one group in which the values of the standard deviations almost do not change (the maximum variation is 3 mm). The other objects can be in other groups as follows (4, 5, and 6), (7, 8, and 9), (10, 11, and 12), and finally (13, 14, and 15). From Figure 1, it can be noted that each group lie in the same position in Xaxis.   Table 3. Impacts of tie point precision on the standard deviation in Y-axis for Mapping Scenario 1 Table 4 presents that the (1/4 pixel) increase in the precision values affect the standard deviation of all objects in Z-axis by almost 3-3.5 cm. The geometry impact here is not clearly noted due to the fact that all objects in Mapping Scenario 1 have the same elevation. However, there are small differences because of the effect of the horizontal position. If we see the raise in the standard deviation of object 1 in the cases of 1.25 and 1.5 pixels, it is 3 cm. Similarly in object 8, the increase is 2.9 cm which does not differ significantly.
Tie-point  Table 4. Impacts of tie point precision on the standard deviation in Z-axis for Mapping Scenario 1

The precision of camera position pseudomeasurements and the standard deviations
The impact of the precision of the camera position pseudomeasurements in Mapping Scenario 1 on the standard deviations is presented in Tables 5, 6, and 7. The assumed precision of the tie point is 1 pixel and the precision of the camera orientation is 0.03 degree. The precisions of the camera position are 2, 4, 6, 8, and 10 cm. Tables 5, 6, and 7 shows that the increase in camera position precision increases the standard deviation values in X, Y, and Z axes. Table 5 illustrates the standard deviation values in X-axis under different cases of camera position precision. From Table 5, although the increase in the precision value in all cases is 2 cm, we can see that the increase in standard deviations is not the same. For example, the increase in the standard deviation of object 1 of the cases of 2 and 4 cm precision is 2.3 cm, whereas the increase of the cases of 8 and 10 cm is 4.6 cm. Similarly to what happened in the previous section, the geometry plays a role in the increase in the standard deviation values and in the standard deviation values themselves. For instance, the increase in the standard deviation of object 1 of the cases of 2 and 4 cm precision is 2.3 cm, while the increase in object 8 is 5 mm. Table 6 shows the standard deviation values in Y-axis under different cases of camera position precision. It is clearly seen that the increase in the precision value raises the standard deviation values. It can also be noted that the standard deviation values of the objects (2, 5, 8, 11, and 14) Table 6. Impacts of camera position precision on the standard deviation in Y-axis for Mapping Scenario 1 Table 7 shows the standard deviations for Z coordinates with various precisions of the camera position. From Table 7, it can be seen that the vertical standard deviation of all objects does not considerably change, as they have the same elevation. In addition, we can see that with the precision of camera position of 8 cm, the standard deviation in object 1 is decreased by more than 12 cm compared to the case with 10 cm precision. This is not the same when the precision is changed from 4 to 2 cm. the decrease in the standard deviation in object 1 is only 6 cm. However, the impact of the increase in the precision value (2 cm) in the latter case which is 6 cm can be considered as big deviation that prevents from achieving more accurate results for real-time UAV 3D mapping.  Table 7. Impacts of camera position precision on the standard deviation in Z-axis for Mapping Scenario 1

The precision of camera orientation pseudomeasurements and the standard deviations
Tables 8, 9, and 10 illustrate the impact of the precision of the camera orientation pseudo-measurements on the standard deviations. The assumed precision of the tie point is 1 pixel and of the camera position is 6 cm. The standard deviations are computed under different scenarios of camera orientation precision (0.01, 0.02, 0.03, 0.04, and 0.05 degree). With the increase in the precision values, the standard deviation values increase. Table 8 shows the standard deviations in X-axis for various cases of the precision of the camera orientation. It can be noted that the standard deviations in X-axis is affected by the increase in the precision values of the camera orientation and the distance from the centre of the X-axis. Furthermore, the objects on the same X-axis have almost the same standard deviation. The differences do not exceed few millimetres.  Table 9. Impacts of camera orientation precision on the standard deviation in Y-axis for Mapping Scenario 1 Table 9 presents the standard deviation values in Y-axis under different cases of camera orientation precision. It is obviously realized that the increase in the precision value raises the standard deviation values. It can also be observed that the standard deviation values of the objects 2, 5, 8, 11, and 14 which are close to the centre of Y-axis are considerably smaller than the values for the other objects. Table 10 shows the vertical standard deviations for five cases of the precision of the camera orientation. We can see that the standard deviation of all objects does not significantly vary, as their elevation is the same. In other words, the objects cannot be grouped as it can be done for the X-axis and Y-axis. The increase by 0.01 degree in the precision of the camera orientation can lead to an increase in standard deviation for some objects to more than 10 cm. In Mapping Scenario 2 there were 23 tie point objects ( Figure  2). These objects were utilised to generate tie point measurements. Knowing tie point measurements and camera parameters, 3D coordinates can be obtained through bundle block adjustment. The stochastic model was constructed with the assumptions that the tie point measurements assumed to have half pixel precision, camera positioning measurements have 5cm precision and camera orientation measurements are with 0.01 degree precision.

Orientation
In addition to the 3D coordinates, the standard deviations of the object coordinates in 3D in the Mapping Scenario 2 were computed and presented in Figures (3, 4, and 5). In Figure (3) it can be seen that the standard deviation is the minimum in the centre of X-axis which is camera position in X-axis and increases with the distance from the centre. They are 0.04 m in the middle and increases in the left part to 0.06m then to 0.11 and finally to 0.16 m. On the right, they increase until they reach 0.17m. It is worth to mention that the objects No.

Objects heights and standard deviations
Mapping Scenario 3 was prepared with the same parameters in the Mapping Scenario 2 discussed above, but the only change was objects hieghts which were assigned randomly for the objects. It is important to remberer that the hights lay between 0 and 20 meters. This is because of the fact that the objects close to the boundaries of the images higher than 20 m may not be captured by the camera as shown in Figure 6.   Table 11 shows the standard deviations in X, Y, Z-axes by using the factored stochastic model. By comparing between the standard deviation results for the X-axis with commonly used stochastic model and factored stochastic model, it can be seen the biggest difference does not exceed 5 mm. For the Y-axis and Z-axis, the results are almost the same.  Table 11. Standard deviations for Mapping Scenario 3 using the factored stochastic model

CONCLUSION
In a real-time 3D UAV mapping scenario, the standard deviation of the objects were computed using simulation data assuming different levels of precisions of tie point, camera position and orientation measurements. The standard deviations of the estimated parameters also were computed and investigated using commonly used and factored stochastic model. This paper shows in details the impact of the precision of tie point, camera position and orientation measurements and the geometric distribution on 3D mapping results. It has been demonstrated that the tie-point objects which are closer to the centre between camera positions have smaller standard deviations than the objects that are farther. Also, this study has also presented the impact of commonly used and factored stochastic models on the standard deviation results. The initial results have shown that the factored stochastic model have a similar performance as the commonly used model, while the precisions of the geo-referencing measurements play a critical role in real-time 3D mapping scenarios.
In this paper the factored stochastic model for the tie point measurements was based on the horizontal positions of the objects. However, more complex stochastic models may be further investigated